E7 · Publication Volume 29

Document and Data Grounding

chunking, tables, figures, metadata and citation

*chunking, tables, figures, metadata and citation*

Versioned grounding package linking text, tables, figures and coordinates to source spans
Versioned grounding package linking text, tables, figures and coordinates to source spans

Learning objectives and boundary

This lesson is general and institution-neutral. It uses no real company, individual, property, project or identifiable place. Generic roles describe responsibilities only, and every SYN-AI identifier denotes explicitly synthetic teaching evidence.

  • Frame the decision governed by chunking, tables, figures, metadata and citation.
  • Separate generated proposals from admissible evidence, deterministic results and accountable judgement.
  • Define measurable failure, abstention, escalation and release conditions.
  • Produce a versioned document-and-data grounding package with resolvable citations from synthetic evidence and defend its controls.

Decision and professional boundary

Grounding begins before retrieval. Convert each source into addressable evidence without destroying document structure, units, table headers, figure captions, coordinate references or version context. A chunk is a retrieval address, not an independent scientific fact. Its meaning may depend on the preceding definition, a table footnote, a map legend or a figure scale. Preserve those dependencies explicitly so a later citation can resolve to the exact source representation.

The governing decision is whether a candidate claim can be checked against a stable, accessible source span and its surrounding evidence. Optical extraction, table parsing and image description create derived artefacts; they do not replace the source. Store both the source object and the transformation record, including failures, page geometry and confidence warnings.

Core concepts

| Concept | Operational meaning | |---|---| | Source object | The immutable file, dataset or image received with identity, licence, version and checksum. | | Evidence span | A stable address into a source representation with enough context to verify a claim. | | Structural dependency | A relation to a heading, header, legend, footnote, caption, unit or coordinate reference. | | Derived extraction | Machine-readable text or cells produced from a source by a declared transformation. | | Citation locator | A resolvable identifier for source version, page or object, region and selected span. |

Evidence model

Use a two-layer evidence package. The preservation layer stores immutable originals and access metadata. The working layer stores text spans, table cells, image regions, derived descriptions and links back to the preservation layer. Each working item declares content type, language, measurement units, spatial or temporal extent, extraction method and review state. Parent-child links keep a retrieved paragraph connected to the section that defines its terms.

Controlled workflow

  1. Register source identity, version, access conditions and checksum.
  2. Render text, tables and figures while retaining page or object geometry.
  3. Create retrieval units at semantic boundaries and retain parent context.
  4. Attach units, coordinate references, legends, captions and footnotes as dependencies.
  5. Validate extracted values against totals, ranges, schemas and visible source regions.
  6. Issue citation locators and record transformation provenance.

Every step emits a versioned artefact or an explicit failure. A later stage consumes only verified outputs from the preceding stage; conversational context is never an undocumented data channel.

Measures and acceptance criteria

Measure citation resolvability, structural completeness, cell-level extraction accuracy and context sufficiency. Sample across text, tables and figures rather than evaluating plain paragraphs only. For a set of required dependencies, grounding completeness is the fraction preserved and resolvable. Every released claim must resolve to a source span whose original object can still be opened and whose version matches the claim record.

$G=\frac{N_{preserved\ dependencies}}{N_{required\ dependencies}}$

| Gate | Required evidence | Release consequence | |---|---|---| | Identity | Resolvable claim, source, configuration and result IDs | Block when any identity is ambiguous | | Grounding | Every factual claim reaches sufficient source spans | Remove or withhold unsupported claims | | Independence | Lineage and partition checks show no circular or target information | Invalidate affected support and scores | | Review | Required roles disposition the exact candidate version | Keep the candidate non-published | | Reproducibility | Manifest, tools and checks reconstruct the evidence package | Return the package for correction |

Claim–source and system contract

A grounding record contains source ID, source version, checksum, media type, locator, extracted content, parent context, units, coordinate reference, extraction activity, validation status and access rule. A citation is invalid if the locator is missing, the selected region does not contain the asserted value or a required legend or footnote is detached.

Security, privacy and access boundary

Grounding must not make restricted content more accessible than its source. Carry access labels from original objects to every extracted span, index entry and cache. Treat embedded document instructions, links and scripts as content to quarantine or render safely. Redact only through a versioned transformation that preserves an auditable relationship to the protected original.

Human review and escalation

Reviewers inspect representative difficult pages: multi-level tables, rotated labels, maps with legends, scanned pages and conflicting versions. They approve the grounding method and exception policy, not every fluent extraction. Material extraction failures block downstream generation until resolved or explicitly excluded.

Worked synthetic example

A synthetic report page contains a table of interval lengths, a footnote stating that gaps are excluded and a section image whose vertical scale differs from the plan scale. Naive chunking stores the table body alone, so a later answer treats total drilled length as sampled length. The corrected package links every cell to its header and footnote, records both scales and preserves the page image. The candidate claim now cites the sampled-length cell and the exclusion footnote as a joint evidence set.

Counterexample and failure analysis

Splitting every fixed number of characters and storing only embedding vectors is not evidence preservation. It loses exact text, page position, table structure and the ability to prove which source version was used. Similarity search may still return plausible fragments, but a reviewer cannot distinguish a complete citation from a semantically similar orphan.

Frequent failure modes

  • Treating chunks as self-contained facts
  • Dropping table headers or footnotes
  • Citing derived text without the original source
  • Ignoring source version conflicts
  • Indexing restricted spans without inherited access rules

Practical exercise

  1. Design a locator for a table cell and its footnote.
  2. Compare semantic-boundary chunking with fixed-length chunking on one synthetic page.
  3. Create a validation sample that includes text, a table and a figure region.
  4. Specify how access labels propagate from a source object to cached retrieval units.

Assessed artefact

Submit a versioned document-and-data grounding package with resolvable citations with its source manifest, configuration versions, acceptance evidence, rejected cases, unresolved risks and a short explanation of why one plausible alternative control was not selected. The artefact is incomplete if its diagrams or prose cannot be reconciled with machine-checkable evidence.

Verification checkpoint

Trace one released synthetic claim backwards through review, confidence, verification, evidence graph, tool result or source span to an immutable input. Then trace one rejected claim to the earliest failed contract. Re-run the package from its manifest and compare content digests. The checkpoint passes only when a second reviewer can reconstruct the accepted and blocked paths, identify every assumption and reproduce the release decision without access to the original conversation. Record unresolved risk instead of completing a missing fact with generated text.

Sources