Pagination

Every search API response contains a pagination section to help facilitate paging through the results and retrieving result counts. Example:

"pagination": {
    "pages": 25,
    "page": 1,
    "next": {
        "offset": 10,
        "limit": 10
    }
  "statistics": {
        "total": 250,
        "total_calculation": "minimum"
    }
}

Using the offset and limit fields in the next object, you can execute a second request to get the next page of results by adding &offset=10&limit=10 to your query (or by setting the limit and offset properties if you're using an SDK). The offset parameter is the number of results to skip and the limit parameter is the number of results to return.

To request results from a specific page, you can multiply the offset by the desired page number minus 1 to get the correct offset value. For example, if you are limiting searches to 10 objects per page, you can jump to the 10th page of results by multiplying the offset by 9: &limit=10&offset=90.

The pages field in the pagination object is the total number of pages available.

Statistics

The statistic field shows the count of results retrieved. The number of results (statistics.total) will either represent an exact value or a minimum value as depicted in statistics.total_calculation.

How statistics Works

Objective uses hybrid indexes by default which combine two powerful search methodologies: lexical search and vector search. Unlike traditional, lexical-only search engines which rely on explicit matches between query and document, vector search instead returns objects in the order of greatest similarity to least. Theoretically, your entire catalog can be returned in order of vector similarity to a query but we limit the initial set of vector results to 250 to optimize for latency.

Result Statistics for Index Types

  • Hybrid Indexes: statistics.total will reflect the greater of vector or lexical matches, up to 10,000.
  • Neural-Only Indexes: The statistics.total will initially be 250 with a "minimum" statistics.total_calculation unless filters reduce the number of retrievable results (see the Dickies Filtered Query example below). You can also retrieve up to the ten-thousandth results.

Statistics Examples

Let's examine some scenarios using a hypothetical Hybrid Index for an e-commerce catalog with millions of objects:

Broad Query: "shirts"

Notice that the total is shown as 10,000 but total_calculation is “minimum” which means there are likely more lexical matches and vector matches in the catalog. This is to be expected from a such a broad query in a multi-million object catalog.

"pagination": {
    "pages": 72,
    "page": 1,
    "next": {
        "offset": 10,
        "limit": 10
    },
    "statistics": {
        "total": 10000,
        "total_calculation": "minimum"
    }
}

Specific Query: "dog toys"

This query for “dog toys” shows there are 620 lexical matches in the catalog but more vector results can still be returned.

"pagination": {
    "pages": 68,
    "page": 1,
    "next": {
        "offset": 10,
        "limit": 10
    },
    "statistics": {
        "total": 620,
        "total_calculation": "minimum"
    }
}

Filtered Query: "pants" with brand "Dickies"

This query shows there are 54 “exact” matches since there are no other objects in the catalog with brand="Dickies".

"pagination": {
    "pages": 2,
    "page": 1,
    "next": {
        "offset": 10,
        "limit": 9
    },
    "statistics": {
        "total": 54,
        "total_calculation": "exact"
    }
}

FAQs

How can I request more than 10 results per page?

Set the limit= parameter to a different value. Read more in the Search API docs.

Can I return more results beyond the page limit?

Yes. If total_calculation is "minimum", you can use offset to return up to the lesser of 10k results or the number of objects in your catalog. Warning: Latency will generally increase as offset increases.

Let's walk through an example of a food-based e-commerce site. Searching for "protein bar", you see a statistics.total of 679 and statistics.total_calculation of "minimum".

"pagination": {
    "next": {
      "limit": 10.0,
      "offset": 10.0
    },
    "page": 1.0,
    "pages": 68.0,
    "statistics": {
      "total": 679,
      "total_calculation": "minimum"
    }
  }

You can use offset to query beyond the minimum. Below we return results after 1000. Notice how the statistics.total goes from 679 to 1100. That is because the statistics.total will show if more results can be returned beyond the objects currently being retrieved.

curl --location --request GET 'https://api.objective.inc/v1/indexes/idx_123/search?&offset=1000&limit=10&query=protein+bar' \
--header 'Authorization: Bearer xxx'    
"pagination": {
    "next": {
      "limit": 10.0,
      "offset": 1010.0
    },
    "page": 101.0,
    "pages": 111.0,
    "statistics": {
      "total": 1110,
      "total_calculation": "minimum"
    }
  }

Will objects with incomplete status still show up in result statistics?

Yes. If an object is searchable, it will show up in the result statistics. Read more about object states.

Was this page helpful?