TracEx
Scientific papers bury measured values inside paragraphs of prose, and asking a plain AI model to pull them out risks confidently wrong answers. I built a tool that tags every sentence with an ID before any AI is involved, extracts obvious numbers deterministically first, then makes exactly one tightly constrained AI call per paper, and finally re-checks every AI answer against the exact sentence it claims to come from, grading it by how well that check holds up.
The problem
The scientific data is buried in long prose of text. Looking for one parameter from a materials-science paper required going through the PDF and then saving the numerical data by hand. A simple LLM call, or generic AI tools, produce results full of hallucinations. What was needed was a provenance-based data extraction system: one where every extracted number ties back to the exact sentence it came from.
The approach
TracEx is built on docling for PDF parsing, pymatgen for chemical-formula validation, quantulum3 extended with a custom condensed-matter unit registry for number-and-unit extraction, litellm for provider-agnostic LLM calls (across Anthropic, OpenAI, and Gemini), and pytest.
Provenance before extraction. Hierarchical sentence-level IDs are stamped on every parsed sentence and table cell before any LLM call runs, so every downstream claim ties to a fixed text anchor.
Deterministic first, LLM second. Deterministic pre-extraction (regex plus pymatgen plus quantulum3) runs first. Only then does a single LLM call per paper run, using forced tool-use against a JSON schema, with a hard call-budget cap.
Deterministic post-verification. Each extracted measurement is graded A, B, or C by re-checking the LLM's cited sentence IDs against the original text: substring alias matching for sample names, word-boundary regex for numeric values.
Output. One Obsidian markdown note per sample, with YAML frontmatter, materials and properties sections, and a 4-column measurement table with source links and a verification grade, plus JSON artefacts for the raw synthesis, the scored synthesis, and the verification pass.
A refactor in April 2026 cut the pipeline from five to nine LLM calls per paper down to exactly one, while keeping the same provenance and verification philosophy from the earlier design.
The outcome
One public Python project on GitHub, and a concrete builder-side artefact of LLM and AI-agent fluency in application materials, alongside CryoSoft, beyond claims of GenAI workflow design.
The vault's own CV note for TracEx ("TraceX - a LLM powered data extracter-CV.md") is an empty stub file. All content on this page is sourced from the parallel, more detailed Master CV evidence block and project logs instead; see the content dossier for the full source trail.