E3 ยท Publication Volume 25

Tolerance, Floating Point and Reproducibility

epsilon, rounding, stable ordering and audit output

Learning objectives

  • Explain the decision and evidence boundary for epsilon, rounding, stable ordering and audit output.
  • Design and implement the relevant drillhole data or algorithm contract without hidden conventions.
  • Separate hard release gates from diagnostics, interpretation and authorised review.
  • Produce a numerical policy and cross-runtime reproducibility test suite from synthetic evidence.

The lesson is complete only when the learner can defend the data model, algorithm, tests and release decision. An attractive trajectory or clean interval table without source evidence and executable invariants remains unverified.

This is a general, institution-neutral tutorial with no relationship to any company or individual. All borehole identifiers, coordinates, depths, directions, intervals, values and review events in the lesson are synthetic and must not be used for an operational decision.

Decision context

A numerical policy decides when computed quantities are considered equivalent, how boundary classifications are made and which differences remain visible. One global epsilon is not meaningful across metres, degrees, identifiers and dimensionless ratios. Tolerances belong to specific comparisons and uses. Rounding is a presentation or canonicalisation rule, not a substitute for tolerance-aware geometry.

Write the intended use, consequence of error, required evidence and release authority before selecting a transformation. The same source can be suitable for exploratory display and unsuitable for a released derivative. Fitness is evaluated against a versioned contract and use, not attached permanently to a file.

Core concept

Binary floating-point values approximate most decimal fractions. Repeated arithmetic, different summation order, fused operations and library choices can change low-order bits. Reproducibility therefore records numeric type, operation order, rounding mode where controllable, algorithm version and output precision. Deterministic results also require stable sorting with a total tie-break key, because equal or near-equal depths can otherwise change overlay and compositing order.

Keep received observations, accepted evidence views and derived results as distinct objects. This separation allows corrected evidence or a changed method to generate a new result without rewriting history. Every derived coordinate or interval therefore answers both a scientific question and a provenance question.

Algorithm and data model

Define named comparison policies for depth equality, coordinate equality, angular equality and report formatting. Each policy states absolute tolerance, relative tolerance if applicable, units, scale limits and whether equality affects classification, topology or release. Preserve raw and full-precision computed values; create rounded exports as derivatives. Audit output includes input fingerprints, environment identity, rule versions and hashes of canonical results.

Define the transformation as a pure, testable operation wherever practical. Parsing, semantic validation, evidence selection, numeric calculation and release evaluation are separate stages. Each stage emits structured output and does not depend on interface state, filename order or an undocumented default.

Constraints and invariants

| Invariant | Executable or review test | | --- | --- | | Tolerance policies are quantity- and purpose-specific. | Reject or quarantine any record that violates this condition and record the exact affected identity. | | Raw and full-precision values are preserved. | Evaluate this condition before producing a derived trajectory or interval result. | | Every sort has a deterministic total tie-break key. | Preserve received evidence and create a new version for every correction. | | Canonical output and scientific equivalence are tested separately. | Include the rule identifier, observed value and resolution state in audit output. |

An invariant must survive import, conversion, processing, export and rerun. A failed hard invariant produces no apparently valid substitute. Diagnostic checks remain visible with their threshold, scope and evidence, and require a reviewed rule before they can trigger correction.

Quantitative reasoning

A scale-aware comparison may use |a-b|\le\epsilon_{abs}+\epsilon_{rel}\max(|a|,|b|), but the terms must share the same unit. Circular angles use circular distance; vectors may use angular separation. Stable accumulation fixes input order and may use compensated summation where error analysis justifies it. Validate invariants with tolerance, such as unit-vector norm and support conservation, while separately comparing canonical serialised outputs for exact reproducibility.

Every reported metric includes units, numerator and denominator where applicable, exclusions, comparison policy and evaluation version. Aggregate values are stratified when pooling could hide a local failure. A quantitative diagnostic supports a decision but cannot overrule missing identity, invalid geometry, unresolved conflict or broken lineage.

Evidence and uncertainty

Keep observation uncertainty, interpolation uncertainty, numeric approximation and metadata uncertainty separate. A smooth trajectory can be numerically precise while still poorly constrained between widely spaced stations. An exact interval overlay can still be unfit when a source depth datum is unknown. The assessed result states which uncertainty belongs to the phenomenon, the measurement, the algorithm and the interpretation.

Build an evidence packet containing immutable received records, semantic declarations, validation findings, algorithm inputs and outputs, test results, reviewer decisions and fingerprints. Contradictory evidence remains available. When a required dependency cannot be resolved, return an explicit unknown, conflict or blocked status rather than choosing the most convenient value.

Interfaces and storage

Interfaces transmit identities, units, coordinate and depth references, conventions, value states, versions and lineage beside numeric values. A trajectory exchange includes collar and datum context, accepted station identities, algorithm identity, numerical policy and output coordinates. An interval exchange includes support type, boundary convention and source links. Structured errors identify the record, field, observed value, expected condition and rule.

Store authoritative received evidence separately from reproducible derivatives and disposable views. Indexes, caches and visualisations may improve access but cannot become the only copy of angle conventions, accepted-station decisions or interval lineage. Export round trips verify that identifiers, precision, ordering and missing states survive encoding changes.

Governance and review

Assign responsibilities to roles rather than named organisations or people: evidence custodian, rule author, implementation maintainer, independent validator and release reviewer. A role may propose a correction but cannot erase source evidence. Rule and algorithm changes are reviewed, versioned and evaluated against fixed regression fixtures before they affect a release.

Exceptions are explicit decisions with scope, rationale, evidence, approving role, affected versions and review trigger. They never rewrite a failed rule and never propagate automatically. The host website has no ownership or scientific-authority role in this workflow; it only delivers the tutorial.

Integration checkpoint

a numerical policy and cross-runtime reproducibility test suite
a numerical policy and cross-runtime reproducibility test suite

Read the figure as a reasoning map from preserved evidence through explicit conventions, deterministic calculation, validation and release. Each arrow represents a declared relationship or transformation. Integrate a numerical policy and cross-runtime reproducibility test suite into the evolving synthetic drillhole package, rerun all earlier fixtures and record any changed assumption.

Synthetic worked example

A synthetic interval boundary is computed as 0.1+0.2 in one path and received as decimal 0.3 in another. Exact binary equality fails, but the named depth policy classifies the boundaries as coincident while retaining both raw values and their difference. A second run reverses contribution order and changes the last bit of a composite; stable identity ordering and a declared accumulator restore deterministic canonical output.

  1. Preserve the received records and state the intended decision without correction.
  2. Resolve identities, units, conventions and evidence eligibility; mark every unresolved item.
  3. Run the versioned algorithm and tests while retaining intermediate diagnostics.
  4. Issue accept, reject or quarantine and show how an independent reviewer can reproduce it.

Practice task

Implement the chapter artefact against a synthetic fixture containing one normal case, one boundary case, one invalid case and one unresolved-evidence case. Preserve the received fixture. Produce canonical input, validation findings, derivative output, processing manifest and a short release decision.

Acceptance criteria:

  • Every input identity, unit and convention required by the rule is explicit.
  • The implementation is deterministic under stable ordering and the declared numerical policy.
  • No correction overwrites received evidence or turns unknown into a guessed value.
  • All hard failures block the affected derivative and remain machine-readable.
  • A second implementation or reviewer can reproduce the result from the package alone.

Submit a numerical policy and cross-runtime reproducibility test suite, the golden and adversarial fixtures, exact findings and a limitations note. A screenshot is not sufficient evidence because it does not identify the input version, algorithm or rule configuration.

Common failure modes

  • Using exact equality for computed decimal depths.
  • Applying one epsilon to every quantity.
  • Rounding inputs before trajectory accumulation.
  • Depending on database row order for tied boundaries.

These failures share a pattern: an implicit convenience is substituted for evidence. Diagnose the earliest boundary where the assumption entered, restore the source statement, make the convention or rule explicit, rerun every dependent derivative and supersede rather than overwrite the affected release.

Review questions

  1. Why is a universal epsilon invalid?
  2. How do stable ordering and reproducibility relate?
  3. When is rounding appropriate?
  4. Why test scientific equivalence separately from exact output identity?

For every answer, identify the governing invariant, the evidence needed to evaluate it, the numerical or semantic policy involved and the correct behaviour when the condition fails.

Sources and further reading