A High-Throughput Retrieval-Augmented Generation Backend with Consistent Index Updates and Cache Coherency for Marketplace Product Search

Abstract

Marketplace product search increasingly blends traditional information retrieval with neural retrieval and large language model generation to support richer query understanding, multi-attribute matching, and response synthesis. In operational settings, however, the retrieval-augmented generation (RAG) backend must handle high query throughput while continuously ingesting catalog changes such as price updates, inventory fluctuations, and policy-driven content edits. These changes complicate index maintenance because retrieval quality and user-facing correctness depend on coherent visibility across dense vector indexes, sparse lexical indexes, feature stores, and caches. This paper describes a high-throughput RAG backend design for marketplace product search that emphasizes consistent index updates and cache coherency under frequent, fine-grained catalog mutations. The architecture combines event-sourced ingestion, deterministic embedding generation, hybrid retrieval with shard-local fusion, and a read path that is explicitly versioned by index epochs. Consistency is treated as a systems property rather than an application afterthought: updates are sequenced through a commit log, materialized into multi-index snapshots using write-ahead metadata, and made visible via atomic epoch advances that align caches, rerank features, and retrieval results. Cache coherency is implemented using generational keys, epoch-tagged entries, and selective invalidation driven by product-level dependency tracking. Performance is analyzed through queueing-based capacity models and staleness bounds derived from update propagation delays. The discussion focuses on practical tradeoffs among latency, freshness, and operational complexity in a production marketplace setting.