An illustration tracing a loop from a honeycomb crystal lattice through a stacked layered material, a device array, a circuit grid, and a neural network, with arrows running in both directions.

Literature RAG

The short version

Searching a library of physics papers usually means either keyword matching that misses meaning, or an AI chatbot that invents its citations. This project built a system over 234 physics papers that understands document structure, including equations and references, and answers questions with citations that trace back to the exact sentence in the exact paper supporting them, so every answer can be independently checked.

Role
Initiator and primary builder
Period
2026 (in progress)
DoclingGROBIDCrossRefPostgreSQLpgvectorSciNCLMCP

This project is one strand of Agentic AI for a Physics Lab, the lab-digitalization initiative I started and lead in the Kläui Lab. Its parsing-and-verification philosophy builds on TracEx.

The problem

Klaeui Lab wanted its own paper corpus to be genuinely searchable: not just by keyword, but in a way that lets a researcher ask a question and get an answer traceable back to a specific supporting sentence in a specific paper, rather than a plausible-sounding but unverifiable summary.

The approach

Parsing. Docling plus GROBID and CrossRef, parsed into a custom BlockNode tree with full page, bounding-box, and character-span provenance. 230 of 230 valid PDFs parsed (4 source files were corrupted). An intrinsic evaluation harness mints gold answers from each paper's arXiv LaTeX source rather than relying on hand labelling. That harness drove real fixes: equation-number recovery from PDF glyphs took equation-binding recall from 0.00 to 0.63, citation-style normalisation converted superscript glyph geometry into a consistent [N] format, and a chemistry-formula pass added a precision gate against false positives like "SPIN" or "ISBN" being parsed as chemical formulas. A quantity-extraction feature was built, evaluated, and then removed after a precision audit showed it was net negative.

Bibliography and identity. GROBID plus CrossRef for reference resolution. Reference DOI coverage improved from 37% to 72% (headers improved from 4% to 85%). A deterministic arXiv-to-published-DOI bridge, backed by multi-signal matching (title trigram similarity confirmed by author and year, abstaining by default rather than guessing), produced an intra-corpus citation graph with 605 internal edges.

Storage. PostgreSQL with pgvector and HNSW indexing, live-loaded with 226 unique papers, 16,400 embedded chunks (body and abstract text), and 8,277 chunk-level citation links. An identity-merge bug that had mislabelled 91 of 226 papers is now covered by a live-integration test suite.

Retrieval. Hybrid retrieval combining SciNCL semantic embeddings with PostgreSQL full-text search, fused by Reciprocal Rank Fusion. A live retrieval evaluation built on 100 LLM-generated, adversarially verified synthetic gold questions, with zero human labelling. Query-side keyword extraction took paper-level recall@10 on the lexical channel from 0.03 to 0.72, and hybrid overall recall from 0.90 to 0.94.

Serving layer. An MCP server exposing six tools: search, get_paper, follow_citation, graph_walk, get_figures, and report_issue. follow_citation is the standout: it walks from a citing sentence to the exact supporting passage in the cited paper, fully server-side with no extra embedding calls. Validated live, for example tracing a spin-pumping claim to its cited paper's supporting passage at 0.95 similarity. Every tool returns abstention as structured data rather than raising an exception.

Side tooling. A standalone force-directed HTML paper-graph visualiser, which doubles as a QA check since references with zero citation links flag missed markers.

Process. A sandbox, verify, gate loop: CI runs a full check suite on every push, and a pinned retrieval baseline fails the build on regressions. An independent architecture review midway through the project raised 7 findings, all fixed the same day.

Status

Past prototype stage: CI gate green, pinned retrieval baseline in place, architecture review done with all findings fixed, running on the Laufey workstation. Next steps under discussion include an importance prior (recency, citations, venue), a provenance-aware evaluation metric, and wiring the MCP server into JGU's internal KI-Chat, currently blocked on a GPU-VM hosting request. The scope note for that hosting request targets a full corpus of roughly 10,000 papers, which at roughly 73 chunks per paper implies roughly 730,000 embeddings for a full index build.