pathwaycom/serviette

Maintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/pathwaycom/serviette.svg?style=card)](https://bestofgithub.dev/repo/pathwaycom/serviette)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/pathwaycom/serviette.svg)](https://bestofgithub.dev/repo/pathwaycom/serviette)

Loading README…

corpus target	≈ pages	files	chunks (approx.)	indexing time	peak memory (PSS, GiB)	in Qdrant (GiB)
100 MB	52 431	12 969	53 651	44 s	6.86	0.61
1 GB	524 288	240 516	604 299	4.9 min	6.82	1.85
3 GB	1 572 864	841 890	1 888 846	15.4 min	7.18	4.89
10 GB	5 242 884	3 423 359	6 761 214	1.0 h	8.16	16.07
30 GB	15 728 646	9 202 620	20 933 962	3.1 h	10.40	48.57
50 GB	26 214 382	17 083 603	37 472 786	6.0 h	14.65	84.45

measurement	result
gold-article recall — the paper's metric, on the paper's corpus	0.50 vs 0.15 published for the paper's BM25 baseline (0.21 for our reproduction of it)
paired gain from adding serviette to gpt-5, paper's protocol	+5.2 pp over the same model without retrieval (McNemar z = 4.1, 824 questions)
adaptive retrieval in the grounded (context-only) regime	41.3% → 52.8% (z = 7.5) — the largest single effect measured
absolute accuracy (permissive, gpt-5)	73.7% — above every number in the paper, including its 5-step agent (66.0%) and oracle (72.9%)

format	parsed by default with	notes
text / Markdown	as-is
PDF	pypdf — built in; `serviette[docling]` upgrades to Docling (layout-aware, tables)	local, free
Office (DOCX, PPTX, XLSX, HTML, EML…)	Unstructured — built in; `serviette[docling]` widens coverage (EPUB, legacy formats)	local, free
scanned images (PNG, JPG, TIFF…)	PaddleOCR	local, free; install `serviette[ocr]`
audio (MP3, WAV…)	Whisper	when `OPENAI_API_KEY` is set
video (MP4, WebM, MOV…)	TwelveLabs Pegasus — a searchable text description of the video	when `TWELVELABS_API_KEY` is set

Backend	Vector	Reranker · Adaptive · Decompose	MMR	Hybrid (BM25)
DuckDB	✅	✅	✅	✅ in-process
Qdrant	✅	✅	✅	✅ in-process
pgvector	✅	✅	✅	✅ in-process
Milvus	✅	✅	✅	✅ in-process
Weaviate	✅	✅	✅	✅ in-process
ChromaDB	✅	✅	✅	✅ in-process
MongoDB	✅	✅	✅	✅ in-process
Pinecone	✅	✅	✅	❌ — no scan-all API; native sparse-index hybrid is planned

Backend	Test	Real instance
DuckDB	`test_integration_duckdb.py`	embedded — runs everywhere
pgvector	`test_integration_pgvector.py`	`pgvector/pgvector` Docker container
Milvus	`test_integration_milvus.py`	embedded Milvus Lite engine; set `SERVIETTE_TEST_MILVUS_URI=http://host:19530` to run the same tests against a Milvus server (Lite has no growing/sealed segments, so it cannot catch stats-counting bugs)
Qdrant	`test_integration_qdrant.py`	`qdrant/qdrant` Docker container
ChromaDB	`test_integration_chroma.py`	`chromadb/chroma` Docker container
Weaviate	`test_integration_weaviate.py`	`semitechnologies/weaviate` Docker container
Pinecone	`test_integration_pinecone.py`	official `pinecone-local` emulator (Docker)
MongoDB	`test_integration_mongodb.py`	`mongodb-atlas-local` (mongod + mongot, real `$vectorSearch`)

serviette