Comparative Evaluation of Large Language Models for Evidence-Aware Extraction of Antiviral Synthesis Routes from Scientific Literature and Patents

Chemical synthesis knowledge for antiviral drugs is distributed across journal articles and patents and is difficult to convert into machine-readable form. Experimental procedures appear as prose, conditions may be embedded in tables, and reaction schemes can encode structures not fully described in text. Large language models (LLMs) can assist literature mining, but single-shot prompting often produces unstructured step lists, weak traceability, and limited use of information contained in figures. Less is known about how LLM backbone choice interacts with multimodal, agentic architectures for producing auditable multi-step synthesis routes.



We evaluate single-shot (SS) structured extraction on an expert-curated corpus of eleven acyclovir-related documents comprising five journal articles and six patents. Four AvalAI backbones—Claude Opus 5, DeepSeek V4 Flash, DeepSeek Reasoner, and Qwen 3.6 Flash—are evaluated using identical cached text inputs and a common JSON schema. We then compare Opus 5 in SS mode with a full multimodal pipeline (FP) incorporating evidence-ranked page selection, multi-pass extraction, scheme interpretation, reconciliation, and chemical standardization. SS runs produce 22–40 steps, compared with 39 for Opus 5, while the Opus 5 FP pipeline produces 87 steps across nine documents with route graphs, provenance links, and conservative structure resolution.



Because no expert-annotated gold standard is available, step counts are interpreted as representation granularity rather than accuracy. The results indicate that backbone choice affects single-shot extraction behavior, while pipeline decomposition has a larger effect on route completeness and auditability. Two scanned patents remain unresolved, motivating expert-annotated benchmarks and field-level precision–recall evaluation.