E3 ยท Publication Volume 25

Automated QC and Human Review

rules, severity, exceptions and explainable errors

Learning objectives

  • Explain the decision and evidence boundary for rules, severity, exceptions and explainable errors.
  • Design and implement the relevant drillhole data or algorithm contract without hidden conventions.
  • Separate hard release gates from diagnostics, interpretation and authorised review.
  • Produce an auditable QC release package with findings, reviews and exception controls from synthetic evidence.

The lesson is complete only when the learner can defend the data model, algorithm, tests and release decision. An attractive trajectory or clean interval table without source evidence and executable invariants remains unverified.

This is a general, institution-neutral tutorial with no relationship to any company or individual. All borehole identifiers, coordinates, depths, directions, intervals, values and review events in the lesson are synthetic and must not be used for an operational decision.

Decision context

Automation evaluates declared rules consistently; authorised review interprets evidence, resolves conflicts and accepts or rejects residual risk. Neither replaces the other. The release decision must be reconstructed from immutable inputs, rule versions, findings, reviewer roles, exception records and the exact derivative assessed. A pass count alone is not a release argument.

Write the intended use, consequence of error, required evidence and release authority before selecting a transformation. The same source can be suitable for exploratory display and unsuitable for a released derivative. Fitness is evaluated against a versioned contract and use, not attached permanently to a file.

Core concept

A rule returns structured findings with rule identity, version, scope, severity, observed evidence, expected condition and suggested next action. Severity expresses release consequence, not surprise. Exceptions are governed records with scope, rationale, evidence, approver role, creation time, expiry or review trigger and affected release versions. An exception does not change the underlying finding and cannot silently apply to future data.

Keep received observations, accepted evidence views and derived results as distinct objects. This separation allows corrected evidence or a changed method to generate a new result without rewriting history. Every derived coordinate or interval therefore answers both a scientific question and a provenance question.

Algorithm and data model

Organise assurance as deterministic validators, property tests, golden fixtures, integration tests and human review queues. Findings move through open, acknowledged, resolved, accepted-exception or rejected states without deletion. A run manifest fingerprints inputs and outputs and records rule configuration. Dashboards may summarise, but the authoritative evidence is the machine-readable finding and decision package.

Define the transformation as a pure, testable operation wherever practical. Parsing, semantic validation, evidence selection, numeric calculation and release evaluation are separate stages. Each stage emits structured output and does not depend on interface state, filename order or an undocumented default.

Constraints and invariants

| Invariant | Executable or review test | | --- | --- | | Rules emit structured, versioned findings. | Reject or quarantine any record that violates this condition and record the exact affected identity. | | Hard gates cannot be averaged away by summary scores. | Evaluate this condition before producing a derived trajectory or interval result. | | Exceptions are scoped, evidenced, approved and reviewable. | Preserve received evidence and create a new version for every correction. | | Every release points to exact inputs, outputs and rule configuration. | Include the rule identifier, observed value and resolution state in audit output. |

An invariant must survive import, conversion, processing, export and rerun. A failed hard invariant produces no apparently valid substitute. Diagnostic checks remain visible with their threshold, scope and evidence, and require a reviewed rule before they can trigger correction.

Quantitative reasoning

Report counts by rule, severity, borehole, data type and resolution state with denominators. Track affected support length as well as record count so one long invalid interval is not hidden among many short valid rows. Useful service measures include unresolved hard failures, review age and exception exposure, but no composite score may override a hard gate. Regression testing compares finding identities and canonical outputs for a fixed golden dataset.

Every reported metric includes units, numerator and denominator where applicable, exclusions, comparison policy and evaluation version. Aggregate values are stratified when pooling could hide a local failure. A quantitative diagnostic supports a decision but cannot overrule missing identity, invalid geometry, unresolved conflict or broken lineage.

Evidence and uncertainty

Keep observation uncertainty, interpolation uncertainty, numeric approximation and metadata uncertainty separate. A smooth trajectory can be numerically precise while still poorly constrained between widely spaced stations. An exact interval overlay can still be unfit when a source depth datum is unknown. The assessed result states which uncertainty belongs to the phenomenon, the measurement, the algorithm and the interpretation.

Build an evidence packet containing immutable received records, semantic declarations, validation findings, algorithm inputs and outputs, test results, reviewer decisions and fingerprints. Contradictory evidence remains available. When a required dependency cannot be resolved, return an explicit unknown, conflict or blocked status rather than choosing the most convenient value.

Interfaces and storage

Interfaces transmit identities, units, coordinate and depth references, conventions, value states, versions and lineage beside numeric values. A trajectory exchange includes collar and datum context, accepted station identities, algorithm identity, numerical policy and output coordinates. An interval exchange includes support type, boundary convention and source links. Structured errors identify the record, field, observed value, expected condition and rule.

Store authoritative received evidence separately from reproducible derivatives and disposable views. Indexes, caches and visualisations may improve access but cannot become the only copy of angle conventions, accepted-station decisions or interval lineage. Export round trips verify that identifiers, precision, ordering and missing states survive encoding changes.

Governance and review

Assign responsibilities to roles rather than named organisations or people: evidence custodian, rule author, implementation maintainer, independent validator and release reviewer. A role may propose a correction but cannot erase source evidence. Rule and algorithm changes are reviewed, versioned and evaluated against fixed regression fixtures before they affect a release.

Exceptions are explicit decisions with scope, rationale, evidence, approving role, affected versions and review trigger. They never rewrite a failed rule and never propagate automatically. The host website has no ownership or scientific-authority role in this workflow; it only delivers the tutorial.

Integration checkpoint

an auditable QC release package with findings, reviews and exception controls
an auditable QC release package with findings, reviews and exception controls

Read the figure as a reasoning map from preserved evidence through explicit conventions, deterministic calculation, validation and release. Each arrow represents a declared relationship or transformation. Integrate an auditable QC release package with findings, reviews and exception controls into the evolving synthetic drillhole package, rerun all earlier fixtures and record any changed assumption.

Synthetic worked example

A synthetic release contains one conflicting survey duplicate, a two-metre partition gap, three diagnostic dogleg warnings and an approved temporary exception for an image-registration residual. Automation blocks the first two, queues the warnings for review and verifies that the exception applies only to the named image version. After the duplicate and gap are corrected through new source-linked records, the release reruns and passes while preserving the complete earlier decision history.

  1. Preserve the received records and state the intended decision without correction.
  2. Resolve identities, units, conventions and evidence eligibility; mark every unresolved item.
  3. Run the versioned algorithm and tests while retaining intermediate diagnostics.
  4. Issue accept, reject or quarantine and show how an independent reviewer can reproduce it.

Practice task

Implement the chapter artefact against a synthetic fixture containing one normal case, one boundary case, one invalid case and one unresolved-evidence case. Preserve the received fixture. Produce canonical input, validation findings, derivative output, processing manifest and a short release decision.

Acceptance criteria:

  • Every input identity, unit and convention required by the rule is explicit.
  • The implementation is deterministic under stable ordering and the declared numerical policy.
  • No correction overwrites received evidence or turns unknown into a guessed value.
  • All hard failures block the affected derivative and remain machine-readable.
  • A second implementation or reviewer can reproduce the result from the package alone.

Submit an auditable QC release package with findings, reviews and exception controls, the golden and adversarial fixtures, exact findings and a limitations note. A screenshot is not sufficient evidence because it does not identify the input version, algorithm or rule configuration.

Common failure modes

  • A green dashboard hides unresolved hard failures.
  • A reviewer edits source data to clear a finding.
  • An exception has no expiry or affected version.
  • Regression tests compare only total pass counts.

These failures share a pattern: an implicit convenience is substituted for evidence. Diagnose the earliest boundary where the assumption entered, restore the source statement, make the convention or rule explicit, rerun every dependent derivative and supersede rather than overwrite the affected release.

Review questions

  1. Which decisions belong to automation and which to review?
  2. What fields make an exception governable?
  3. Why must affected support length accompany record count?
  4. How can a release decision be independently reconstructed?

For every answer, identify the governing invariant, the evidence needed to evaluate it, the numerical or semantic policy involved and the correct behaviour when the condition fails.

Sources and further reading