Reranking

Reranking scores a short candidate list against a query. It belongs after a cheaper retrieval step. Consumers access the selected service as ctx.rerankers.

const answer = await ctx.rerankers.rerank(
  'Which document describes the harbour?',
  candidates,
  {topN: 5, signal},
)

const ordered = answer.results.map(({index, score}) => ({
  document: candidates[index],
  score,
}))

Results are sorted best first. Each row contains the original document index and a score from 0 to 1. Scores are comparable within one call only. topN asks the provider to return only the strongest results; omitting it requests all documents.

Keep the candidate set small enough for the selected model's input limit. Rerankers read the query together with each candidate, so latency and token use grow with both the number and length of documents. Preserve the returned index when joining scores back to application records; the service deliberately does not duplicate document content in its result.

Use the score to order or filter results within that response. Do not persist a universal threshold across providers or models: score calibration is not part of the shared contract.

The project-maintained Jina and Cohere providers share one transport (@dsh-ai-providers/jina-cohere) that calls POST /rerank. It accepts both relevance_score and score response fields. If a compatible server returns raw logits outside the 0 to 1 range, the provider maps every score through a logistic function while preserving order. The DashScope provider calls Alibaba's own rerank API instead, which wraps the query and documents in an input object and answers scores between 0 and 1.

Retrieval pattern

A typical pipeline retrieves a broader candidate set with lexical or vector search, then asks the reranker for the final few results. Keep the original record IDs beside the candidate strings so the returned indices can be joined without relying on repeated text.

const candidates = await searchIndex(query, {limit: 40})
const {results, model} = await ctx.rerankers.rerank(
  query,
  candidates.map(candidate => candidate.text),
  {topN: 8, priority: 'foreground', signal},
)

return results.map(result => ({
  ...candidates[result.index],
  rerankScore: result.score,
  rerankModel: model,
}))

Reject or repair an out-of-range returned index at your application boundary rather than silently attaching a score to the wrong record. The project-maintained providers validate their responses, but consumers of third-party implementations should still treat the contract as the trust boundary.

Operational choices

Use background priority for offline evaluation and index maintenance. Use foreground priority when reranking blocks a visible answer. Pass cancellation through the whole retrieval pipeline so abandoned searches do not continue consuming both search and model capacity.