Ranking Refinement

Ranking Refinement adds a layer of human-like intuition to every Finetuned Index. For every finetuned index, it automatically evaluates results for each search query and assigns a predicted_relevance score of great, ok, or bad to each result, reflecting its relevance to the query. You can access the predicted_relevance score by querying your Finetuned index.

Supported Index Versions:

Multimodal: 2.5.0+
Text: 2.4.0+
Image: 1.5.0+

How It Works

Using relevance judgements, we train a low-latency classifier that categorizes each search result object into "great," "ok," and "bad" categories. This categorization occurs as part of the in-flight request pipeline for your Finetuned Index.

Example Query Output

Let's consider a query for “Animal Onesie” in an e-commerce dataset. Below is a sample of the search results. Notice how actual animal onesies have a predicted_relevance label of “great,” onesies without animal features are marked as “ok,” and non-onesies are labeled as “bad.”

{
  "results": [
    {
      "id": "502880003",
      "object": {
        "detail_desc": "All-in-one suit in patterned fleece with a hood, zip down the front and contrasting colour ribbing at the cuffs and hems. Detachable tail.",
        "product_type_name": "Dress",
        "prod_name": "Animal onesie"
      },
      "predicted_relevance": "great"
    },
    {
      "id": "817124001",
      "object": {
        "detail_desc": "Fancy dress costume in soft pile with a hood and appliqués. Zip down the front with a chin guard, and ribbing at the cuffs and hems.",
        "prod_name": "ANIMAL ONESIE",
        "product_type_name": "Costumes"
      },
      "predicted_relevance": "great"
    },
    {
      "id": "536651002",
      "object": {
        "prod_name": "Bobo onesie",
        "product_type_name": "Hoodie",
        "detail_desc": "All-in-one suit in printed sweatshirt fabric with a lined hood, zip down the front, long sleeves with ribbed cuffs and long legs with ribbed hems. Soft brushed inside."
      },
      "predicted_relevance": "ok"
    },
    {
      "id": "607088001",
      "object": {
        "prod_name": "HAZEL onesie",
        "detail_desc": "All-in-one suit in soft pile with a jersey-lined hood and zip down the front. Long sleeves with ribbed cuffs and long legs with ribbed hems.",
        "product_type_name": "Pyjama jumpsuit/playsuit"
      },
      "predicted_relevance": "ok"
    },
    # More "ok" results 
    # ...
    {
      "id": "720493001",
      "object": {
        "prod_name": "LARA beanie",
        "detail_desc": "Hat in a soft knit with a reversible sequin motif on the front, ears with sequins at the top, earflaps and ties underneath. Fleece lining.",
        "product_type_name": "Hat/beanie"
      },
      "predicted_relevance": "bad"
    },
    {
      "id": "651671001",
      "object": {
        "detail_desc": "Hat knitted in a soft cotton blend with ears on the top and a ribbed hem.",
        "product_type_name": "Hat/beanie",
        "prod_name": "Animal beanie"
      },
      "predicted_relevance": "ok"
    }