Why We Switched Summary-Level Extraction from LangChain to Anthropic's Native LLM

What is LangChain to Anthropic’s Native LLM

LangChain to Anthropic’s Native refers to the shift from building AI applications with general-purpose orchestration frameworks like LangChain to using Anthropic’s native tools, APIs, and built-in capabilities directly. As Anthropic continues to expand its platform with features such as tool use, structured outputs, prompt caching, and agent capabilities, many developers are re-evaluating whether an external framework is still necessary for their use cases. 

How LangChain to Anthropic’s Native LLM is Powerful in Summary-Level Extraction

Our Summary-Level Extraction (SLE) module of the SLR (Systematic Literature Review) platform processes complex clinical research PDFs to extract structured data with visual traceability. When users reported inconsistent traceability and extraction quality, we investigated our LangChain-based architecture and found fundamental limitations with our OCR dependency. This blog describes our migration to Anthropic’s native library, which eliminated external OCR services, reduced latency by 50.7%, and increased accuracy from 86% to 95.6%.

The Clinical Data Extraction Challenge

Challenges in Clinical Data Extraction

Clinical research documents contain critical data across multiple modalities: prose descriptions, statistical tables, participant demographics, and safety metrics. Our SLE module should extract this information with two key capabilities:

1. Structured extraction: Converts unstructured PDFs into validated JSON schemas
2. Visual traceability: Highlights the exact source location of each extracted value within the original PDF

This traceability is essential for regulatory compliance and validation workflows, where clinical data specialists verify that automated extractions match source documents.

The Bottlenecks

Users reported two critical issues during validation:

Case 1: Missing traceability for table data
Demographic information, such as age and sex, is extracted correctly, but without corresponding PDF highlights.

Case 2: Text extraction without visual mapping
The sentences are identified accurately, but the highlights failed to render in the PDF viewer.

These inconsistencies undermined trust in the system and forced manual re-validation, negating efficiency gains.

Architecture Analysis: Why OCR Was Not Working

Our initial architecture relied on a multi-stage pipeline:

Original LangChain + Textract Architecture

Root Cause Analysis

Investigation revealed multiple OCR-related failure modes:

1. Text Quality Issues

• Missing spaces between words (“meanvalue” vs “mean value”)
• Lost special characters (±, μ, %, superscripts)
• Incorrect table column alignment in multi-column layouts

2. Structural Degradation

• Table cells are merged or split incorrectly
• The reading order is jumbled in complex layouts
• Reference citations detached from context

3. Image Blindness

• No extraction from embedded charts or figures
• Loss of visual data representations
• Inability to process image-based tables

These issues cascaded through the pipeline: poor quality of OCR text → inaccurate LLM context → failed traceability mapping.

Alternative OCR Evaluation

We assessed three OCR solutions against our requirements:

AWS Textract (Tables)$10-15Existing integrationSpace merging, symbol loss, poor multi-column handlingMistral OCR (Tables)~$1Low latency, markdown outputBroken LaTeX syntax, complex deployment, structural integrity lossAzure Document Intelligence~$10Stable output, better symbolsMinor hyphenation issues still require preprocessing

AWS Textract (Tables)$10-15Existing integrationSpace merging, symbol loss, poor multi-column handlingMistral OCR (Tables)~$1Low latency, markdown outputBroken LaTeX syntax, complex deployment, structural integrity lossAzure Document Intelligence~$10Stable output, better symbolsMinor hyphenation issues still require preprocessing

While Azure showed improvements, testing showed a fundamental insight: What if we bypassed OCR entirely? Modern vision-capable LLMs like Claude Sonnet can process PDF bytes directly. This realization initiated our architectural pivot.

LangChain to Anthropic’s Native LLM – The New Architecture: Direct PDF Inference

We redesigned SLE around Anthropic’s native library, eliminating the OCR preprocessing stage.

New Pipeline Design

┌────────────────────────┐
│ PDF Input (Base64)     │
└────────────────────────┘
          │
          ▼

┌────────────────────────────────────────┐
│ Anthropic Native API                   │
│ (Sonnet 3.7 + Extended Thinking)       │
└────────────────────────────────────────┘
          │
          ▼

┌────────────────────────────────────────┐
│ Enhanced Traceability Engine           │
│ (Coordinate Mapping Logic)             │
└────────────────────────────────────────┘
          │
          ▼

┌────────────────────────────────────────┐
│ Multi-threaded Execution               │
│ (Parallel Document Processing)         │
└────────────────────────────────────────┘
          │
          ▼

┌─────────────────────────────────────┐
│ Validated JSON + PDF Highlights     │
└─────────────────────────────────────┘

Anthropic Native Architecture – Important Changes

1. Direct PDF Understanding

Instead of using Textract to extract text and pass it to the LLM, we now transform PDFs into Base64 format and send them directly to the Anthropic API. This model views the document as a visual and understands layout, tables, and text all in one go.

This eliminates three layers of potential failure:

• OCR text extraction errors
• Text parsing and cleaning logic
• Coordination between text chunks and original PDF coordinates

2. Extended Thinking Mode

We activated Claude’s extended thinking capability, which allows the model to perform internal chain-of-thought reasoning before generating the final extraction. This proves particularly valuable for:

• Disambiguating table data where column headers span multiple rows
• Cross-referencing values mentioned in text with tabular summaries
• Resolving inconsistencies between different sections of the document

The thinking process is transparent and can be reviewed during validation to help data teams understand how the model arrived at specific extractions.

3. Prompt Engineering for Native Format

We restructured our prompts to align with Anthropic’s best practices, focusing on:

• Clear specification of the required output structure
• Explicit instructions for traceability sentence extraction
• Prioritization of precision over completeness to reduce false positives
• Guidance on handling ambiguous cases (flag rather than guess)

The new prompt format emphasizes exact value matching and verbatim sentence extraction, which proved critical for regulatory compliance requirements.

4. Improved Traceability Logic

We rebuilt the coordinate mapping system to work with Anthropic’s response format. The new engine uses fuzzy matching with position-aware scoring to locate extracted sentences within the PDF, even when there are minor variations in spacing or line breaks.

The system now handles:

• Multi-line sentences that wrap across pages
• Table cells containing the extracted value
• Text within complex multi-column layouts
• Sentences that appear multiple times in the document

Implementation Deep Dive

Challenge 1: Managing Token Consumption

Direct PDF processing consumes significantly more tokens than preprocessed text. For a typical 30-page clinical trial PDF:

• Old approach: ~15K tokens (Textract text only)
• New approach: ~45K tokens (full PDF context)

To manage this, we implemented intelligent chunking for documents exceeding context limits. The system detects logical sections such as Methods, Results, and Discussion and creates chunks that preserve complete semantic units while respecting token budgets. Each chunk includes overlapping context from adjacent sections to maintain continuity.

Challenge 2: Preserving Table Structure

Clinical PDFs contain complex nested tables with merged cells, multi-row headers, and footnotes. We improved our prompts to specifically address table awareness:

The model now identifies table structures explicitly, preserves relationships between values, notes merged cells or nested structures, and references tables by their captions when available. This structured approach to table extraction greatly improved accuracy for tabular data.

Challenge 3: Parallel Processing

To maintain throughput despite higher per-document latency, we used multi-threaded execution. The system processes multiple PDFs concurrently with intelligent rate limiting to respect API constraints while maximizing utilization.

The parallel setup includes:

• Thread pool management with configurable worker counts
• Retry logic with exponential backoff for temporary failures
• Error isolation to prevent cascading failures
• Progress tracking and logging for better operational visibility

Challenge 4: Traceability Coordinate Mapping

The most technically challenging aspect was mapping extracted sentences back to precise PDF coordinates. The new system employs a multi-stage approach:

1. Fuzzy text matching to find the extracted sentence in the PDF text layer
2. Position-aware scoring that considers page numbers and approximate locations
3. Bounding box calculation to determine exact highlight coordinates
4. Validation to ensure highlights align with visible text

This approach handles edge cases like hyphenated words, ligatures, and text reflow while maintaining high precision.

Results and Validation

Accuracy Improvements

We tested the new SLE module on manually validated dermatology clinical trials for Tretinoin efficacy studies from our SME data team.

Key Findings:

• Thinking mode provided a 2% boost over non-thinking, primarily in completeness
• OCR handling reached 100%, completely removing text quality issues
• Sentence accuracy improved slightly, but significantly reduced false extractions
• Order preservation reached perfect scores by using visual document understanding

Latency and Cost Trade-offs

Performance Benchmarks (7 documents, dermatology domain)

Analysis:

• Median latency improved significantly (50-74% reduction) by removing OCR preprocessing
• Maximum latency increased for complex documents that utilize extended thinking extensively
• Cost per document rose by ~65% due to higher token usage from full PDF processing
• Cost-performance ratio: 50% faster processing for 65% more cost represents a favorable trade-off given the accuracy improvements and simplified architecture

The latency reduction came from eliminating the Textract API call and subsequent text parsing, which accounted for 40-60% of total processing time in the old architecture.

Extended Validation Results

After initial success, our data team validated additional articles across multiple therapeutic areas:

Observations:

• Traceability exceeded accuracy (94.1% vs 91.1%), validating our architectural focus on this capability.
• OCR handling stayed near-perfect (99.57%) across various document types and therapeutic areas.
• Lower completeness (73.62%) in broader validation suggests opportunities for domain-specific prompt tuning.
• Sentence accuracy remained consistently high (97.86%), demonstrating strong generalization.

LangChain to Anthropic’s Native – Lessons Learned

What Worked Well

1. Eliminating preprocessing complexity
   Removing the Textract → parsing → cleaning pipeline eliminated multiple failure points and simplified our codebase by ~40%.
2. Model-native capabilities
   Claude’s vision understanding proved superior to OCR + text-based reasoning, particularly for:

• Complex table structures with merged cells and multi-level headers
• Documents with mixed fonts, sizes, and scientific notation
• Special characters (±, μ, %, superscripts) that Textract frequently corrupted
• Layout understanding in multi-column formats

3. Extended thinking for ambiguous cases
   For documents with unclear table references or cross-sectional data, thinking mode visibly improved extraction quality. 
4. Operational simplicity
   Moving from three services (Textract, LangChain, Bedrock) to one (Anthropic API) made monitoring, debugging, and deployment much easier.

Conclusion

Migrating our Summary-Level Extraction module from LangChain + AWS Textract to Anthropic’s native library delivered measurable improvements across all key metrics:

• Accuracy: Exceeded the benchmark >90%
• Latency: Improved by ~50-74% 
• Traceability: 94% reliability in production
• OCR quality: Near-perfect (99.57%)
• Code complexity: -40% reduction

While costs per document rose approximately 65%, the combination of faster processing, removal of OCR errors, simplified architecture, and improved user trust justified the investment. More importantly, this architecture positions our SLR application to leverage future multimodal capabilities without reengineering our pipeline.

For teams building document intelligence systems, our key takeaway is: evaluate whether your LLM can fully replace your preprocessing stack. The cost of external OCR, parsing libraries, and text cleaning often exceeds the token cost of direct PDF inference while simultaneously introducing fragility and maintenance burden.

The architectural shift taught us valuable lessons about cloud service dependencies. By consolidating to a single LLM provider with native document understanding, we simplified operations, improved debugging, and gained access to rapid upgrades as the models advance.

Author’s Note: This article was supported by AI-based research and writing, with Claude 4.5 assisting in the creation of text and images.