ICDAR 2026 Oral

REPLICA

An Agentic Framework for
Visually Faithful Document Reconstruction
Representation FormatFID-HTML → MethodREPLICA → Benchmark & MetricsVFDR-Bench
Raghuveer R1*, Anirudh Srinivasan1*, Venkata Kesav Venna1*, Sreevatsa S2,
Aryan Jain2, Sahithi Kukkala2, Ravi Kiran Sarvadevabhatla1,2
1 BharatGen  ·  2 IIIT Hyderabad   |   * Equal Contribution
BharatGen IIIT Hyderabad ICDAR 2026

What is Visually Faithful Document Reconstruction ?

0 · Text only
1 · + Logical structure
2 · + Physical position
3 · + Visual fidelity
OVERDUE
Meridian & Co.
114 Beacon Street, Suite 400
Boston, MA 02116
accounts@meridian.co  ·  +1 617 555 0142
Invoice
Number INV-2024-0847
Issued 12 May 2024
Terms Net 15
Due 27 May 2024
Bill toNorthwind Traders  ·  Attn: A. Vance
88 Kingsway, Chicago, IL 60604
Description Qty Rate Amount
Consulting services — Q2 retainer 10 150.00 1,500.00
Software licence, annual 1 2,400.00 2,400.00
Support retainer — Apr, May, Jun 3 800.00 2,400.00
On-site integration, 12 May 1 960.00 960.00
Data migration, per record 480 0.25 120.00
Subtotal 4,980.00
Credit — cancelled licence −2,400.00
Sales tax (8%) 206.40
Total due (USD) 2,786.40

RemittancePayment by bank transfer to Eastbrook Trust, account 0042-88190, routing 011000138, quoting the invoice number as the reference. Cheques payable to Meridian & Co. and posted to the Boston address above.

QueriesAny discrepancy must be raised in writing within seven days of receipt. Credited or cancelled line items remain on the statement for audit purposes and are struck through rather than removed.

Meridian & Co. is registered in Massachusetts, no. 04-3729915.  ·  Payment due within 15 days of this overdue notice.
Original — the source page
Reconstruction — rebuilt live, layer by layer
The axes of document reconstruction

Requirements for Visually Faithful Document Reconstruction

Must preserve the four axes of the document
T.E.  ·  Text Extraction

Text

Are the words right?

Characters and words, at block and page level.

L.S.  ·  Logical Structure

Hierarchy

Is the structure right?

Reading order, nesting, tables, lists, formulas.

P.S.  ·  Physical Structure

Geometry

Is the position right?

Absolute placement of blocks and lines on the page.

V.F.  ·  Visual Fidelity

Visual components

Does it look right?

Typography, colour, weight, spacing, backgrounds.

Must be a format the rest of the world can use

Web-native

Machine-readable

Against the checklist
Text ✓ Hierarchy ~ Geometry ✗ Visual ✗ — the same two boxes fail for every system

Limitations of Existing Document Representations

The page carries Text the words themselves Hierarchy headings, tables, lists, order Geometry where each block sits — x, y Style size, weight, colour, rules The format they are asked to write Marker Markdown olmOCR · GOT · Nanonets plain text GPT‑5 · Qwen3‑VL generic HTML NO PLACE TO PUT THEM Markdown · plain text the syntax does not exist generic HTML the syntax exists, but is optional — so the model leaves it out What the system emits Text recovered by every system Hierarchy mostly recovered Geometry never written down Style never written down
The same page, rebuilt
Text ✓ Hierarchy ✓ Geometry ✓ Visual ✓ — one HTML file, rendered by your browser. We call it FID-HTML.

Six Aspects of FID-HTML

Comparison with existing representations

The page carries Text the words themselves Hierarchy headings, tables, lists, order Geometry where each block sits — x, y Style size, weight, colour, rules Existing representations text and hierarchy get through every one of them Markdown · plain text LaTeX · DocTags · hOCR generic HTML · Qwen‑HTML NO SLOT FOR GEOMETRY OR STYLE Markdown · plain text no syntax for a coordinate or a colour LaTeX · DocTags · hOCR partial — boxes or styling, never both generic HTML · Qwen‑HTML styling is allowed, position is never required FID-HTML Text word- and block-level content Hierarchy nested div · li · table · tr · td Geometry position:absolute · left · top · width · height Style font-size · weight · colour · rules, inline required on every element · not optional

Defining Agents and Tools

𝓕A tooldeterministic
same input → same output
no judgment  ·  cannot notice its own mistake
detectorsOCRtable structure reading orderfont fittingbackground
𝒜An agentreasons
decides · checks · retries · overrides
allowed to disagree with what it was given
labelinggroupingrouting specialistsreading orderreflection
one pass · the same answer every time ? picks a tool not good enough → again decides · checks · retries

The agentic framework, stage by stage

1Segment buys Hierarchy · Geometry"How to divide the document into meaningful components?"
Segment stage of the pipeline
  • Ttype, Tdet — digital/scanned classifier; DocLayout-YOLO + IndicDLP-YOLO + Hi-SAM, fused by IoU.
  • Alab — Set-of-Mark labelling; catches mis-segmented and mixed-content regions.
  • Agrp — builds the layout graph; resolves caption–figure binding, list nesting, footnotes.
  • → hierarchical layout tree Tlayout
2Localize buys Text · Style"How to convert each component into equivalent HTML?"
Localize stage of the pipeline
  • Aroute — picks the specialist from the crop, overriding the upstream label when the visual contradicts it.
  • 𝒜spec — six specialists: text, table, image, form, list, equation.
  • Taux — OCR, table structure, TexTAR: invoked only when necessary.
  • → sub-HTML Hv per node · conditionally independent, so parallelisable
3Assemble buys Hierarchy · Geometry"How to assemble components into a coherent structure?"
Assemble stage of the pipeline
  • Tpos — bottom-up merge over Tlayout, preserving containment and alignment.
  • Aro — semantic reading order (≺); spatial sorting fails on multi-column pages and floats.
  • → initial HTML document H · reading-order accuracy 0.98
4Refine buys Style"How to refine structure for visual and structural fidelity?"
Refine stage of the pipeline
  • Tfont, Tbg — non-overflow font scales by bounded search; background restored by CSS layering.
  • Aref — renders R(H), compares it to I, injects targeted CSS; up to 5 iterations.
  • → high-fidelity final HTML · the only stage that sees the render

Walkthrough of the REPLICA Agentic Framework

What is actually in the benchmark  ·  plays automatically  ·  P pause  ·  ⇧← ⇧→ step  ·  click a page to enlarge

VFDR-Bench — 5,000 documents, annotated in FID-HTML

5,000
documents
22
languages
17
domains
100%
expert human annotated
▶ playing

Which metric measures what

Text Extraction are the words right? Logical Structure is the structure right? NTED ↓ · TEDS ↑ · CDM ↑ standard WRR ↑ · CRR ↑ standard Physical Structure is the position right? GPS ↑ · LPS ↑ NEW Visual Fidelity does it look right? VFS ↑ NEW OS mean of 4

Two axes already had metrics

WRR / CRR — words and characters recovered
NTED — DOM against the reference tree

We introduce

GPS / LPS — where every block and line lands
VFS — typography, colour and spacing, judged by a VLM

Both read the render, not the markup.

Overall = the mean of the four

Measured on VFDR-Bench — all 16 methods  · 

Quantitative results

weakeststrongest
  

Ablation studies

Spot the difference

Source scan vs live FID-HTML render

Original scanned document
Original — government notice · Gujarati, tabular
REPLICA — live FID-HTML: real DOM, selectable text, inline CSS

Limitations

Single-page only

page 1 page 2 ITEM QTY TOTAL HEADER LOST break

Font identification

source Meridian ≠ output Meridian bold ✓ italic ✓ colour ✓ family ✗ attributes recovered · family not

Rendering cost

render compare patch CSS up to ×5 single forward pass REPLICA, with refine
Beyond reconstruction

What FID-HTML unlocks

REPLICA as a data generator

Supervision for grounded document models

REPLICA rebuilds any page as FID-HTML text · structure · position · style ICDAR 2026 · Oral GROUNDED DOCUMENT UNDERSTANDING M3Grounder masks as evidence CVPR 2026 DoCoG grounded reasoning ECCV 2026 · ORAL DocLayout-VL the page as a tree ECCV 2026 generates the training data every page labelled by construction and they sharpen every stage better segmentation, reading order, grounding
One paradigm, three contributions

Representation Format → Method → Benchmark & Metrics

Representation Format

FID-HTML

A web-native format for Visually Faithful Document Reconstruction

Method

REPLICA

Agentic Framework with Segment → Localize → Assemble → Refine.

Benchmark & Metrics

VFDR-Bench + GPS · LPS · VFS

5,000 expert-annotated documents, 15+ languages, 3 new metrics.

0.93
Visual fidelity (VFS)

Thank you

Questions?
REPLICA project page QR code
REPLICA
replica-agents.github.io
Grounded document understanding
M3Grounder QR code
M3Grounder
CVPR 2026
DoCoG QR code
DoCoG
ECCV 2026 · Oral
DocLayout-VL QR code
DocLayout-VL
ECCV 2026
CVIT BharatGen
live html · press V for source
Speaker notes
Source of the slide you are looking atV or Esc to close

  


  

    

      
Presenter controls

      

        

          →   Space   PageDown
          Next step / slide
        

        

          ←   PageUp
          Previous step / slide
        

        

          ↓ / ↑
          Next / previous slide (skip fragments)
        

        

          Space
          On the walkthrough: play / pause (arrows still step it)
        

        

          F
          Fullscreen
        

        

          V
          View the source of the current slide
        

        

          N
          Speaker notes
        

        

          O
          Slide overview
        

        

          Home / End
          First / last slide
        

        

          B
          Blank screen
        

        

          ?
          This help
        

      

      

        Presenter remotes send PageUp / PageDown — they work out of the box.