Remove Bad Results in Vector Search with Ranking Refinement, Now in Beta

As we’re sure you know, we’re big fans of all things AI-Native search here at Objective. Dense vector search is a foundational part of every AI-native stack, and for a lot of use cases it provides a pretty remarkable step-function over other historical approaches to retrieval. But going from good to great requires control. Every dataset is unique, and every use case is unique. So we’re stoked today to launch Ranking Refinement into Beta, to give you even more control over the relevance of results that are surfaced to your users by surfacing a predicted_relevance field along with each search result for Finetuned Indexes.

If you already have an account, Ranking Refinement is now available on all new Finetuned Indexes you create! If not, go get one!

A Relevancy Dial for Your Search Results

Here at Objective we’re on a mission to make search human. Dense vector search is a modern, AI-native approach to retrieval that has become a cornerstone of every AI-native search stack (like Objective!) due to the flexibility & power it provides. It brings your search app another step closer to feeling human, determining relevancy based on meaning rather than keyword matching. Where keyword matching is a binary, dense vector retrieval is a scale. It works a lot like how we think — considering all of the options, and then ordering them by relevance. If you apply this thinking to colors, “Red” is very relevant to “lava”, “purple” is less relevant, and “blue” is not very relevant at all.

But — our human brains intuitively take this a step further by layering on a stricter human judgement of whether a result is a great, ok, or outright bad match for the original query. And we naturally do this regardless of whether we’re looking at the 3rd result or the 50th. As a result, a search result that falls below a certain relevance threshold can “feel” bad to users because they intuitively identify the threshold where ok relevance turns into bad relevance. That threshold, it turns out, is highly dependent on your dataset, and your users.

Under the Hood

Ranking Refinement automatically adds a layer of human-like intuition to each of your Finetuned Indexes, and surfaces the classifications to your app so you can make the right UX decisions for your data and your users.

Behind the scenes, we use relevance judgements (the same kind of judgements used in Finetuning) to train a low-latency classifier that classifies each search result object with a great, ok, or bad classification as part of the in-flight execution graph.

Those classifications are surfaced in the predicted_relevance field included with each result in the API response. This lets you programmatically decide how to adjust the relevancy constraints at render-time — even making that decision based on behavior of the current user!

AI-Native is About Layers

You’re probably starting to wonder the same thing a lot of people do — why doesn’t vector search just do this automatically? The answer is actually pretty interesting. Dense vector search finds relationships between objects that are sometimes non-obvious to humans, and in some cases are also different than human intuition.

Let’s take a look at a very simplified for-instance. Let’s say your Object Store contains clothing products (and specializes in very sharp looking raincoats, it appears 💅). A search for “yellow raincoat” might return a set of results that looks like this:

While vector search does interpret the “Classic Cove Dog Raincoat” as very related to yellow raincoats, you, with your human brain, intuitively add another layer of judgement. That second result, while statistically relevant, unfortunately isn’t a raincoat for humans.

Ranking Refinement makes that same kind of consideration, and classifies that item as a bad relevance. We know that the implicit context is a search for human raincoats, so even though it surfaced high in the results and was statistically relevant, our relevance can be refined by removing that item from the results you render in your UI for users by skipping rendering items with a predicted_relevance of bad.

If it makes sense for your use case to keep dog raincoats in your search results, though, no problem! Your app can make the decisions that are right for your users & your use case.

You can also experiment with Ranking Refinement right in Console — head to the Relevance > Analyze section of Console, and select two Indexes to compare. For all new Finetuned Indexes, you’ll see an additional configuration option for Predicted Relevance Filter. Give it a try!

Powerful Tools Start with Powerful Controls

You can start to imagine how Ranking Refinement could apply to lots of different use cases — constraining search results that need an added layer of human-like precision, while also allowing you the ability to loosen the constraint for e-commerce applications that sometimes benefit from a more forgiving relevance judgement.

We can’t wait to see what you build with Ranking Refinement!