PDPP

Define Hybrid Retrieval

tasks20/23
Created openspec/changes/define-hybrid-retrieval/tasks.mdView on GitHub →

1. Contract

  • Add reference-contract schema for hybrid retrieval capability metadata.
  • Add reference-contract schema for GET /v1/search/hybrid.
  • Decide first-tranche pagination: snapshot cursor or no cursor support. Decision: no cursor support in v1. The endpoint rejects the cursor parameter with invalid_request and the advertisement carries cursor_supported: false. Clients that need paging beyond limit fall back to the individual /v1/search and /v1/search/semantic endpoints, each of which has a snapshot-honest cursor of its own.

2. Implementation

  • Advertise capabilities.hybrid_retrieval only when lexical and semantic retrieval are both available.
  • Implement GET /v1/search/hybrid by composing existing lexical and semantic planners.
  • Deduplicate by (connector_id, stream, record_key).
  • Return source provenance and per-source score objects.
  • Preserve the existing lexical and semantic endpoint behavior unchanged.

3. Tests

  • Metadata advertisement tests.
  • Happy-path owner-token hybrid search across at least two streams.
  • Client-token grant projection test.
  • Dedup test for record matching both sources.
  • Lexical-only and semantic-only provenance tests.
  • Cursor behavior test matching the chosen pagination design.
  • Cross-surface cursor rejection tests.

4. Dashboard Follow-Up

  • If the endpoint is implemented, update dashboard search to prefer hybrid retrieval when advertised.
  • Preserve graceful fallback to existing client-side lexical+semantic blend when not advertised.

Deferred per task-packet scope: the dashboard should only be touched once the API is green and the change is very small. The blend already works client-side and the advertisement is fail-closed.

5. Validation

  • Run lexical and semantic retrieval suites.
  • Run pnpm --filter @pdpp/reference-contract run check:generated.
  • Run pnpm --dir reference-implementation run verify.
  • Run pnpm --dir apps/web run types:check if dashboard changes. N/A — no dashboard changes in this tranche.
  • Run openspec validate define-hybrid-retrieval --strict.
  • Run openspec validate --all --strict.