E3 ยท Publication Volume 25

Compositing Algorithms

length weighting, residuals, domain boundaries and missing values

Learning objectives

  • Explain the decision and evidence boundary for length weighting, residuals, domain boundaries and missing values.
  • Design and implement the relevant drillhole data or algorithm contract without hidden conventions.
  • Separate hard release gates from diagnostics, interpretation and authorised review.
  • Produce a compositing specification, golden dataset and support-conservation report from synthetic evidence.

The lesson is complete only when the learner can defend the data model, algorithm, tests and release decision. An attractive trajectory or clean interval table without source evidence and executable invariants remains unverified.

This is a general, institution-neutral tutorial with no relationship to any company or individual. All borehole identifiers, coordinates, depths, directions, intervals, values and review events in the lesson are synthetic and must not be used for an operational decision.

Decision context

Compositing changes observation support and therefore creates derived evidence. Before calculation, declare target support length, origin, domain boundaries, residual policy, admissible input states, missing-value treatment, weighting basis and minimum coverage. A fixed-length composite is not simply an average of rows. It is a defined aggregation over the exact intersection between source intervals and target windows.

Write the intended use, consequence of error, required evidence and release authority before selecting a transformation. The same source can be suitable for exploratory display and unsuitable for a released derivative. Fitness is evaluated against a versioned contract and use, not attached permanently to a file.

Core concept

Construct target windows independently within each allowed domain. Intersect every window with source intervals, split contributions at all boundaries, and weight each valid contribution by its intersection length unless another physically justified weight is declared. Missing, censored, rejected and not-applicable values remain distinct. Residual windows may be retained, combined, redistributed or dropped only under an explicit policy that reports the effect.

Keep received observations, accepted evidence views and derived results as distinct objects. This separation allows corrected evidence or a changed method to generate a new result without rewriting history. Every derived coordinate or interval therefore answers both a scientific question and a provenance question.

Algorithm and data model

A composite record links to a target support, domain identity, contributing source intervals, intersection lengths, source value states, aggregation rule and processing run. Store both total target length and eligible contributing length. Never bridge a domain boundary merely to reach target length. If coverage is below the declared threshold, return an insufficient-support state rather than scaling the available value to appear complete.

Define the transformation as a pure, testable operation wherever practical. Parsing, semantic validation, evidence selection, numeric calculation and release evaluation are separate stages. Each stage emits structured output and does not depend on interface state, filename order or an undocumented default.

Constraints and invariants

| Invariant | Executable or review test | | --- | --- | | Target support and residual policy are declared before calculation. | Reject or quarantine any record that violates this condition and record the exact affected identity. | | Contributions use exact interval intersections. | Evaluate this condition before producing a derived trajectory or interval result. | | Domain boundaries are hard unless a reviewed rule says otherwise. | Preserve received evidence and create a new version for every correction. | | Every aggregate reports eligible length and exclusions. | Include the rule identifier, observed value and resolution state in audit output. |

An invariant must survive import, conversion, processing, export and rerun. A failed hard invariant produces no apparently valid substitute. Diagnostic checks remain visible with their threshold, scope and evidence, and require a reviewed rule before they can trigger correction.

Quantitative reasoning

For target window W, eligible source values x_i and intersection lengths w_i=|W\cap I_i|, the length-weighted mean is \bar{x}_W=\sum_i w_i x_i/\sum_i w_i. Coverage is q_W=\sum_i w_i/|W| only when eligible contributions do not overlap; otherwise first resolve or account for multiplicity. Verify support conservation by comparing the sum of contribution lengths with the union of eligible intersections. Report denominator, exclusions and residual treatment beside every result.

Every reported metric includes units, numerator and denominator where applicable, exclusions, comparison policy and evaluation version. Aggregate values are stratified when pooling could hide a local failure. A quantitative diagnostic supports a decision but cannot overrule missing identity, invalid geometry, unresolved conflict or broken lineage.

Evidence and uncertainty

Keep observation uncertainty, interpolation uncertainty, numeric approximation and metadata uncertainty separate. A smooth trajectory can be numerically precise while still poorly constrained between widely spaced stations. An exact interval overlay can still be unfit when a source depth datum is unknown. The assessed result states which uncertainty belongs to the phenomenon, the measurement, the algorithm and the interpretation.

Build an evidence packet containing immutable received records, semantic declarations, validation findings, algorithm inputs and outputs, test results, reviewer decisions and fingerprints. Contradictory evidence remains available. When a required dependency cannot be resolved, return an explicit unknown, conflict or blocked status rather than choosing the most convenient value.

Interfaces and storage

Interfaces transmit identities, units, coordinate and depth references, conventions, value states, versions and lineage beside numeric values. A trajectory exchange includes collar and datum context, accepted station identities, algorithm identity, numerical policy and output coordinates. An interval exchange includes support type, boundary convention and source links. Structured errors identify the record, field, observed value, expected condition and rule.

Store authoritative received evidence separately from reproducible derivatives and disposable views. Indexes, caches and visualisations may improve access but cannot become the only copy of angle conventions, accepted-station decisions or interval lineage. Export round trips verify that identifiers, precision, ordering and missing states survive encoding changes.

Governance and review

Assign responsibilities to roles rather than named organisations or people: evidence custodian, rule author, implementation maintainer, independent validator and release reviewer. A role may propose a correction but cannot erase source evidence. Rule and algorithm changes are reviewed, versioned and evaluated against fixed regression fixtures before they affect a release.

Exceptions are explicit decisions with scope, rationale, evidence, approving role, affected versions and review trigger. They never rewrite a failed rule and never propagate automatically. The host website has no ownership or scientific-authority role in this workflow; it only delivers the tutorial.

Integration checkpoint

a compositing specification, golden dataset and support-conservation report
a compositing specification, golden dataset and support-conservation report

Read the figure as a reasoning map from preserved evidence through explicit conventions, deterministic calculation, validation and release. Each arrow represents a declared relationship or transformation. Integrate a compositing specification, golden dataset and support-conservation report into the evolving synthetic drillhole package, rerun all earlier fixtures and record any changed assumption.

Synthetic worked example

A synthetic domain [0,7) contains source intervals [0,1.5) value 2, [1.5,4) value 5 and [4,7) value 8. Target length is 2 m from origin zero, with residuals retained. The first composite is (1.5\times2+0.5\times5)/2=2.75. The second uses 2 m of value 5. The third combines 0.5 m of value 5 and 1.5 m of value 8. The final [6,7) residual remains a one-metre composite with explicit support length.

  1. Preserve the received records and state the intended decision without correction.
  2. Resolve identities, units, conventions and evidence eligibility; mark every unresolved item.
  3. Run the versioned algorithm and tests while retaining intermediate diagnostics.
  4. Issue accept, reject or quarantine and show how an independent reviewer can reproduce it.

Practice task

Implement the chapter artefact against a synthetic fixture containing one normal case, one boundary case, one invalid case and one unresolved-evidence case. Preserve the received fixture. Produce canonical input, validation findings, derivative output, processing manifest and a short release decision.

Acceptance criteria:

  • Every input identity, unit and convention required by the rule is explicit.
  • The implementation is deterministic under stable ordering and the declared numerical policy.
  • No correction overwrites received evidence or turns unknown into a guessed value.
  • All hard failures block the affected derivative and remain machine-readable.
  • A second implementation or reviewer can reproduce the result from the package alone.

Submit a compositing specification, golden dataset and support-conservation report, the golden and adversarial fixtures, exact findings and a limitations note. A screenshot is not sufficient evidence because it does not identify the input version, algorithm or rule configuration.

Common failure modes

  • Averaging rows without intersection-length weights.
  • Bridging a domain boundary to fill a target window.
  • Treating missing values as zero.
  • Dropping a residual without reporting lost support.

These failures share a pattern: an implicit convenience is substituted for evidence. Diagnose the earliest boundary where the assumption entered, restore the source statement, make the convention or rule explicit, rerun every dependent derivative and supersede rather than overwrite the affected release.

Review questions

  1. Why does compositing create derived evidence?
  2. How are partial source intervals weighted?
  3. What must be reported beside a composite value?
  4. When is a residual composite valid?

For every answer, identify the governing invariant, the evidence needed to evaluate it, the numerical or semantic policy involved and the correct behaviour when the condition fails.

Sources and further reading