ICDAR 2026  ·  Oral Presentation

REPLICA

An Agentic Framework for Visually Faithful Document Reconstruction

1 BharatGen  ·  2 IIIT Hyderabad
* Equal Contribution
Reconstruction — pick a method to compare against the input
Input Document
Reconstruction Output
Fidelity Radar — REPLICA vs. baselines
Green ✓ = axis preserved. TE Text Extraction · LS Logical Structure · PS Physical Structure · VF Visual Fidelity.
Overview

Abstract

PDFs preserve layout, typography, and visual styling — but that fidelity vanishes when converted to the web. Existing tools recover text and structure yet discard physical layout, color, and positioning. REPLICA reconstructs documents into web-native HTML that preserves BOTH semantics AND visual appearance.

While PDF remains the standard for digital documents, it integrates poorly with web-native environments. Existing document-to-HTML systems prioritize text accuracy and logical structure while discarding physical structure — typography, color, and precise spatial organization. REPLICA is an agentic document-to-HTML framework that generates visually faithful, layout-aware, semantically enriched HTML we call FID-HTML. It uses a four-stage pipeline — Segment, Localize, Assemble, Refine — and generalizes across diverse document types and multiple languages. For evaluation we introduce VFDR-Bench, spanning 22 languages and 17 domains with high-fidelity FID-HTML annotations. REPLICA achieves 93% visual fidelity, outperforming strong baselines including GPT-5 and Gemini-2.5-Pro. Notably, REPLICA with Qwen3-VL-8B achieves a 39% overall gain over Qwen3-VL-30B despite being 3.75× smaller at comparable inference time — showing structured agentic design beats brute-force scale.

What's New

Three Core Contributions

Representation

FID-HTML

A high-fidelity, semantically enriched HTML representation that encodes six complementary dimensions of a document.

FID-HTML is a high-fidelity, semantically enriched HTML representation that jointly preserves content, structure, geometry, and visual styling. Unlike generic HTML which merely permits visual fidelity, FID-HTML guarantees it by encoding six complementary dimensions. It serves as both the canonical ground-truth format for VFDR-Bench and the target output of REPLICA, enabling downstream tasks like layout-preserving translation, grounded document intelligence, and semantic chunking for RAG.

Original document

"Facts of Life" — scanned magazine fact sheet (input page).

fid-html · "Facts of Life" (ground truth)

This is the real Fid-HTML source of the page on the left. Click a dimension to spotlight exactly where it lives in the code — flip to Rendered to see this exact markup drawn by your browser (the text is real and selectable).

FID-HTML vs. Other Formats — ✓ full · P partial · × missing

T.E. Text Extraction · L.S. Logical Structure · P.S. Physical Structure · V.F. Visual Fidelity  ·  R.O. = Reading Order, L.H. = Layout Hierarchy.

Benchmark

VFDR-Bench & Metrics

A Visually-Faithful Document Reconstruction benchmark with high-fidelity FID-HTML annotations.

Hugging Face View VFDR-Bench on Hugging Face
Benchmark Comparison — TE / LS / PS / VF coverage
VFDR-Bench is the only document benchmark that evaluates both Physical Structure (PS) and Visual Fidelity (VF).
How It Works

The REPLICA Pipeline

A four-stage agentic pipeline transforms a document page into visually faithful FID-HTML.

Figure 4 — Full pipeline overview. Segment partitions the page into hierarchical semantic regions · Localize routes each region to specialist agents that emit sub-HTML fragments · Assemble orders and merges fragments into a globally consistent DOM · Refine optimizes fonts, restores backgrounds, and iteratively verifies with a reflection agent.
Design Philosophy
Interactive

Pipeline Demo

Pick a document, then step or play through REPLICA's four stages. Intermediate outputs are pre-computed — no live inference.

 Input document
1

Segment

Hierarchical semantic layout detection
Fig. 4 · pipeline map
REPLICA pipeline overview (Figure 4)
Sub-HTML output
Stage 1 · 0
Speed
Use Step to advance one beat, or Play for the cinematic auto-walkthrough. The left map follows the story — drag/scroll it to explore (📷 re-enables auto-follow). Keyboard (while hovering): space = play/pause, ←/→ = step, R = restart, F = fit map.
Evaluation

Quantitative Results

On VFDR-Bench — 5,000 documents across 22 languages and 17 domains.

Metric Comparison — switch metric, bars sorted high→low, REPLICA highlighted
Main Comparison — Table 2

NTED↓ is lower-is-better; all other columns are higher-is-better. Best value per column in accent.

Ablation — Table 3, cumulative components
Cumulative Overall Score (OS↑)
Agent & Tool Reliability — Table 4
Comparison

How REPLICA Compares to Existing Document Parsers

Document parsing tools recover the words. Document reconstruction recovers the page.

Most document parsing and PDF-to-HTML tools convert a scanned or born-digital page into Markdown or generic HTML. They do OCR and document layout analysis well, and they usually recover reading order and logical structure. What they discard is the physical structure — the column grid, the typography, the colour, and the exact position of every block on the page. Open the output beside the original and it is recognisably a different document.

REPLICA performs HTML reconstruction rather than parsing: it rebuilds the page as web-native FID-HTML that preserves content, structure, geometry and visual styling together. The table below is the direct comparison on VFDR-Bench, from Table 2 of the paper.

Document parsing vs. document reconstruction — VFDR-Bench, Table 2
SystemTypeCRR↑NTED↓LPS↑GPS↑VFS↑Overall↑
MarkerPipeline parser0.820.610.190.340.740.53
DoclingPipeline parser0.490.620.170.260.550.38
Qwen3-VLVision-language model0.790.590.220.300.430.47
Gemini-2.5-ProVision-language model0.420.560.140.310.820.47
GPT-5Vision-language model0.790.490.210.430.860.58
REPLICA (ours)Agentic reconstruction0.930.200.730.860.930.86

NTED↓ is lower-is-better; all other columns are higher-is-better. LPS and GPS measure local and global physical structure; VFS measures visual fidelity. Full 16-method table in the paper.

What the numbers mean

Pipeline parsers

Marker and Docling are strong at OCR and text extraction — Marker reaches 0.82 character recall. But local physical structure collapses to 0.19 and 0.17. The text survives; the page does not.

Vision-language models

GPT-5 and Gemini-2.5-Pro produce pages that look plausible (0.86 and 0.82 visual fidelity) yet place blocks poorly — global position peaks at 0.43. They redraw the page rather than reconstruct it.

Agentic reconstruction

REPLICA reaches 0.73 local and 0.86 global position — roughly 3× and 2× the best baseline — while leading on text, logical structure and visual fidelity simultaneously.

Frequently asked

Can I convert a PDF to HTML and keep the layout? That is exactly the problem REPLICA targets. Generic PDF-to-HTML conversion preserves text and some logical structure; REPLICA additionally preserves absolute and relative positioning, font size, colour, text attributes, figures and background.

How is this different from document parsing? Parsing extracts content out of a document. Reconstruction rebuilds the document as HTML, so the rendered output is visually comparable to the input page. We measure that difference explicitly with GPS, LPS and VFS.

Does it handle scanned documents and multiple languages? Yes. VFDR-Bench covers single- and multi-column, scanned and born-digital pages, across many languages and document categories, with expert human-annotated ground truth.

Results

Qualitative Comparisons

Original document (left) vs. REPLICA's reconstruction (right). The right panel is the actual FID-HTML rendered live — try selecting the text: it's real, not an image.

Original
REPLICA Reconstruction live FID-HTML · selectable
1 / 1
Applications

Downstream Applications

FID-HTML unlocks layout-aware document intelligence. Hover an application to zoom into it — click to pin.

REPLICA downstream applications overview
Cite

BibTeX

Raghuveer_2027_REPLICA