E2 · Publication Volume 24

Geoscience Data Semantics, Formats, Storage, Metadata and Provenance

Treats geoscience data as governed entities with semantics, identity, metadata and lineage rather than values alone.

Purpose and boundary of this book

This book is a general, institution-neutral tutorial about designing geoscience data that retains meaning across observation, interpretation, processing, exchange and reuse. It has no relationship with any company or individual. Every project name, identifier, location, value and workflow used in an exercise is synthetic and exists only for teaching.

The website that carries this tutorial acts only as a host; it is not the publisher, a scientific authority, or the curriculum subject. The scientific subject is the portable method described here: identify real-world entities, state observation semantics, preserve uncertainty and provenance, validate contracts and make every release independently inspectable.

The book does not prescribe one product, database or organisational chart. It teaches reasoning that can be implemented with many technologies. Sensitive-data examples describe information architecture and review controls, not legal advice. A qualified review remains necessary wherever law, licence or contract determines an actual use.

Meaning-first architecture

The central claim is simple: data is not just stored values. A trustworthy record combines identity, type, context, value state, validity, quality and provenance. A trustworthy dataset additionally declares its logical schema, controlled concepts, spatial and temporal reference, distributions, access conditions and release history. A trustworthy pipeline preserves those declarations while recording every transformation.

The curriculum follows one control chain:

  1. identify the real-world entity and the question a record answers;
  2. separate observation, interpretation and derived product;
  3. assign durable identifiers and versioned concepts;
  4. model units, missingness, limits, qualifiers and time explicitly;
  5. map a logical contract to fit-for-purpose files, databases and services;
  6. attach metadata, lineage, validation and access policy; and
  7. release an immutable package whose meaning can be tested by an independent consumer.

This order prevents storage convenience from deciding scientific meaning. Physical encodings may change; the declared semantic invariants must remain testable.

Learning outcomes

After completing the tutorial, a learner should be able to:

  • distinguish a value from the semantic context needed to interpret it;
  • design entity and observation models for exploration and geological evidence;
  • manage durable identities, aliases, merges and controlled concepts;
  • represent units, value states, detection limits, quality flags and multiple temporal axes;
  • select and profile exchange formats without confusing encoding with meaning;
  • divide responsibility among databases, immutable objects, analytical stores and caches;
  • author resource metadata, field dictionaries, provenance graphs and processing manifests;
  • version data contracts and demonstrate directional interoperability; and
  • apply classification, licence and access constraints without breaking traceability.

Completion means producing a validated semantic data package, not merely reading the chapters. The final package contains identifiers, concepts, units, metadata, processing lineage, contract fixtures and a release decision.

Prerequisites, notation and conventions

The tutorial assumes the quantitative and uncertainty foundations of A1 plus basic familiarity with exploration records. It does not assume a specific database language or software platform. Examples use concise tables, graphs and pseudo-records so that the same reasoning can be transferred to relational, document, graph, columnar or object-storage implementations.

An entity is a thing whose identity persists; an observation is a claim about an entity or feature of interest; an interpretation is a reasoned conclusion based on evidence; a distribution is one accessible representation of a dataset; a contract is a versioned agreement between producers and consumers. “Required” means a hard release condition. A metric is diagnostic unless its decision threshold is explicitly declared.

Intervals are half-open unless a lesson states otherwise. Units, coordinate references, time-zone offsets, identifier namespaces and concept-scheme versions are explicit. Formulas define diagnostic measures; they never authorise silent substitution, loss or averaging of failed hard gates.

Synthetic teaching dataset

All worked material uses the synthetic “Open Ridge” learning dataset, a fictional collection of projects, sites, holes, surveys, samples, observations, analytical results, model versions and distributions. The name does not identify a real place, company or person. Coordinates are intentionally local and non-geographic, codes are invented, and values are not suitable for operational decisions.

The dataset evolves through the book. Early chapters expose ambiguous values and duplicate labels. Later chapters add entity identities, observation supports, concept schemes, explicit value states, temporal history, format profiles, storage roles, metadata, lineage and policy. This staged construction lets the learner see why each control exists and which failure it prevents.

Every exercise begins with preserved source evidence and ends with a derivative plus a manifest. When information is unavailable, the correct result is an explicit unknown, review state or blocked release—not an invented value.

Learning workflow

Each lesson uses the same repeatable workflow: define the decision, model the meaning, state invariants, quantify diagnostic quality, preserve evidence, design the interface, assign governance, complete a synthetic worked example and produce an assessed artefact. The teaching figure is a reasoning map, not a decorative illustration.

A learner should keep one evolving semantic package. New chapters extend it without erasing prior assertions. For example, an alias is added to an identity registry rather than replacing the source label; a unit conversion creates a derived result rather than overwriting the reported value; a corrected record supersedes an earlier transaction rather than rewriting history.

Peer review should be performed from the package alone. If a reviewer must ask the original producer what a field “really means,” the documentation or contract is incomplete.

Evidence architecture and quality gates

Evidence is organised in four layers. Source evidence preserves received bytes and source context. Semantic evidence links records to entities, properties, concepts, units, supports and time. Process evidence records transformations, parameters, versions and fingerprints. Decision evidence records validation, review, acceptance, rejection or quarantine.

Every release gate asks five questions:

  • Can each required identity and dependency be resolved?
  • Are semantic invariants and value-state combinations valid?
  • Are spatial, temporal, unit and concept references explicit and applicable?
  • Can the output be reproduced or otherwise shown equivalent under a declared policy?
  • Are access and licence conditions satisfied for the requested purpose?

A high aggregate score cannot override a failed identity, unit, provenance or policy gate. Failures remain visible with record paths, observed values, expected conditions and resolution status.

Assessment and completion standard

The assessed product is a synthetic, versioned semantic data package containing an entity registry, observation records, concept scheme, value-state model, temporal history, at least two profiled distributions, storage responsibility map, metadata record, data dictionary, processing manifest, compatibility matrix and access-policy decision.

Acceptance requires exact identifier resolution, valid relationships, explicit units and value states, aligned schema and vocabulary versions, complete dependency fingerprints, passing contract fixtures, documented format round trips and no unauthorised disclosure. The package must also reproduce one derived dataset from immutable inputs.

The final review explains limitations and unresolved uncertainty. A blocked release with a precise reason is a valid engineering outcome; an apparently complete package supported by guessed metadata is not.

Core sources