E7 · Publication Volume 29

Tool-Mediated Geoscience Agents

runtime tools, network boundaries and deterministic computation

*runtime tools, network boundaries and deterministic computation*

Controller-mediated agent loop with typed tools, policy gates and immutable results
Controller-mediated agent loop with typed tools, policy gates and immutable results

Learning objectives and boundary

This lesson is general and institution-neutral. It uses no real company, individual, property, project or identifiable place. Generic roles describe responsibilities only, and every SYN-AI identifier denotes explicitly synthetic teaching evidence.

  • Frame the decision governed by runtime tools, network boundaries and deterministic computation.
  • Separate generated proposals from admissible evidence, deterministic results and accountable judgement.
  • Define measurable failure, abstention, escalation and release conditions.
  • Produce a controlled tool-runtime specification and adversarial test record from synthetic evidence and defend its controls.

Decision and professional boundary

A language model may propose a plan or a typed tool request, but a deterministic controller decides whether the request is valid, authorised and within budget. Tools perform bounded operations such as reading a declared package, validating a schema, transforming coordinates or calculating a statistic. The returned artefact, not the model's recollection, becomes evidence. Separate planning, execution, observation and approval so generated text cannot impersonate a tool result.

Agent autonomy is a control choice, not an intelligence score. Start with one-step read-only calls and explicit user confirmation. Add multi-step execution only when state transitions, retry behaviour, stopping rules and rollback are testable. Professional conclusions and operational writes remain outside the agent unless a separate, reviewed workflow explicitly authorises them.

Core concepts

| Concept | Operational meaning | |---|---| | Typed tool | A bounded operation with validated inputs, declared effects and structured results. | | Controller | Deterministic code that enforces permissions, schemas, budgets and state transitions. | | Observation | A captured tool result with identity, status, timing and evidence references. | | Effect class | Read, derive, stage, publish or external action, each with different approval. | | Stopping rule | A testable condition for success, abstention, escalation, budget exhaustion or failure. |

Evidence model

The execution graph contains request, plan version, tool contract, authorisation decision, call arguments, runtime identity, result artefact, invariant checks and human disposition. A textual answer may reference this graph but cannot rewrite it. Results that fail schema, identity or scientific invariants are retained as failed attempts and never promoted to evidence.

Controlled workflow

  1. Parse the request into a proposed plan without executing it.
  2. Validate each tool name, input schema, resource scope and effect class.
  3. Authorise against identity, task contract, data labels and remaining budget.
  4. Execute in an isolated runtime and capture the complete structured result.
  5. Verify result identity, units, coordinate reference and expected invariants.
  6. Stop, continue or escalate through deterministic state-transition rules.

Every step emits a versioned artefact or an explicit failure. A later stage consumes only verified outputs from the preceding stage; conversational context is never an undocumented data channel.

Measures and acceptance criteria

Evaluate valid-call rate, unauthorised-call rejection, deterministic replay, invariant detection, stopping accuracy and human intervention. Report success only when the requested decision-support artefact is correct and traceable; a sequence of syntactically valid calls is not task success. For deterministic tools, repeated runs over the same manifest must return identical content digests.

$S_{agent}=\frac{N_{verified\ task\ completions}}{N_{eligible\ tasks}}$

| Gate | Required evidence | Release consequence | |---|---|---| | Identity | Resolvable claim, source, configuration and result IDs | Block when any identity is ambiguous | | Grounding | Every factual claim reaches sufficient source spans | Remove or withhold unsupported claims | | Independence | Lineage and partition checks show no circular or target information | Invalidate affected support and scores | | Review | Required roles disposition the exact candidate version | Keep the candidate non-published | | Reproducibility | Manifest, tools and checks reconstruct the evidence package | Return the package for correction |

Claim–source and system contract

Each tool declares input schema, output schema, read and write scope, network policy, deterministic status, timeout, cost budget, idempotency, error taxonomy and evidence-capture rule. The controller rejects free-form arguments, hidden side effects and outputs that lack an artefact identity. Tool descriptions are configuration, not instructions taken from retrieved documents.

Security, privacy and access boundary

Use explicit allowlists for tools, resources and network destinations. Keep secrets outside prompts and tool results. Execute untrusted parsing in isolation, cap output size and depth, and prevent retrieved content from selecting tools or changing policies. Write effects require preview, independent authorisation, immutable before-state and a rollback boundary.

Human review and escalation

Review the controller and tool contracts separately from prompt wording. Exercise denied calls, malformed arguments, timeouts, partial results, repeated calls and conflicting observations. Human approval is bound to the exact proposed effect and expires when arguments, source versions or target state change.

Worked synthetic example

The synthetic agent receives a request to compare two collar-coordinate snapshots. It proposes a read-only manifest tool, a coordinate-reference validator and a deterministic difference calculator. One snapshot lacks a vertical reference. The validator returns a structured failure; the controller prevents the difference calculation and records an escalation. The model explains the missing prerequisite and cites the validator result. It does not guess a height convention or continue with partial coordinates.

Counterexample and failure analysis

Giving a model a shell, database credential and unrestricted network then asking it to “be careful” is not a controlled agent. Prompt instructions cannot enforce operating-system permissions, transaction boundaries or evidence capture. A convincing final answer may conceal failed commands, changed state or data exfiltration. Controls belong in the runtime and authorisation path.

Frequent failure modes

  • Using prompt text as an authorisation control
  • Mixing tool results with generated observations
  • Allowing retrieved content to choose tools
  • Retrying non-idempotent effects automatically
  • Calling a valid tool sequence a successful task

Practical exercise

  1. Specify a typed read-only coordinate validation tool.
  2. Draw controller states for proposed, authorised, running, failed, verified and escalated calls.
  3. Write five adversarial inputs that must not change permissions.
  4. Design a replay test for a deterministic calculation.

Assessed artefact

Submit a controlled tool-runtime specification and adversarial test record with its source manifest, configuration versions, acceptance evidence, rejected cases, unresolved risks and a short explanation of why one plausible alternative control was not selected. The artefact is incomplete if its diagrams or prose cannot be reconciled with machine-checkable evidence.

Verification checkpoint

Trace one released synthetic claim backwards through review, confidence, verification, evidence graph, tool result or source span to an immutable input. Then trace one rejected claim to the earliest failed contract. Re-run the package from its manifest and compare content digests. The checkpoint passes only when a second reviewer can reconstruct the accepted and blocked paths, identify every assumption and reproduce the release decision without access to the original conversation. Record unresolved risk instead of completing a missing fact with generated text.

Sources