E6 · Publication Volume 28

Reference Architecture

client, API, database, files, jobs, cache and identity

*client, API, database, files, jobs, cache and identity*

Logical reference architecture with evidence, control and asynchronous boundaries
Logical reference architecture with evidence, control and asynchronous boundaries

Learning objectives

This lesson is general and institution-neutral. It uses no real company, individual, property, project or identifiable place. Generic roles describe responsibilities only, and SYN-ARCH identifiers denote explicitly synthetic teaching evidence.

  • Frame the decision governed by client, API, database, files, jobs, cache and identity.
  • Model the relevant boundaries, states and contracts before selecting an implementation.
  • Define measurable invariants, failure evidence and a safe release consequence.
  • Produce a viewpoint-based logical reference architecture and boundary catalogue from synthetic evidence and defend its trade-offs.

Decision boundary

Choose boundaries according to responsibility, consistency, scale, trust and failure isolation, not according to a preferred deployment product. The central decision is which component is authoritative for each state and which interactions may be synchronous, asynchronous or offline. A client may present and stage intent but must not become the sole holder of approved evidence. A service may enforce contracts without owning bulk artefacts. A cache may accelerate a representation without becoming an interpretation store. Identity and audit cross every boundary but should not be duplicated as ungoverned local truth.

Core concepts

A reference architecture is a reusable arrangement of responsibilities and relations, not a finished deployment diagram. Use separate viewpoints for context, functional decomposition, information, concurrency, deployment, security and operations. A logical component exists because it owns a coherent responsibility and invariants. A connector has direction, protocol, contract, volume, latency, trust, retry and failure semantics. Deployment nodes map logical components to runtime locations later. This separation prevents a current hosting topology from being mistaken for the enduring system design and makes portability claims testable.

System model and contracts

The teaching architecture contains a user-facing client, contract-enforcing API, transactional metadata and state store, immutable artefact store, background job executor, derived-data cache, identity-policy service and append-only audit stream. Ingestion enters an isolated landing boundary before validation. Published records reference immutable artefacts by stable identity and digest. Long computations become jobs with explicit input versions and output manifests. The client receives representations and job state, never hidden database assumptions. Cross-cutting configuration, secrets, telemetry and schema registries have owners and lifecycles rather than floating between components.

Invariants and acceptance criteria

| Invariant | Test evidence | Release consequence | |---|---|---| | Exactly one governed component is authoritative for each mutable state. | contract test and recorded counterexample | block publication | | Caches and clients never become the sole copy of approved evidence. | replay comparison and digest check | quarantine the artefact | | Every connector has declared contract, failure and retry semantics. | role-based acceptance trace | return the decision unresolved | | Logical responsibilities remain separable from deployment products. | failure injection and recovery record | retain the last verified version | | Degraded states remain explicit to users and downstream systems. | domain review against declared evidence | record an explicit review finding |

Quantitative engineering

Allocate an end-to-end latency budget by measured critical path, but do not add component percentiles as though they occurred in the same request. Record L_{p95} end to end and component distributions with shared trace identity. For a strictly serial chain with independent availability assumptions, a screening approximation is A_s=prod_i A_i; correlated failures invalidate that simplification and must be modelled as shared dependencies. Report throughput, queue depth, payload distribution, cache hit rate, storage growth and recovery objectives per boundary. Capacity claims state dataset shape and concurrency.

Data quality, evidence and uncertainty

Evidence for a boundary includes responsibility conflicts, consistency needs, workload traces, trust differences, independent scaling requirements and recovery tests. A box drawn for future scale without measured or scenario-based need is a hypothesis, not evidence. Record assumptions about network quality, disconnected use, object size, update frequency, geographic extent and review cadence. Maintain an uncertainty register for unknown peak load, external availability and operational skill. The architecture should degrade explicitly: read-only, queued, lower detail, stale-but-labelled or unavailable are different states with different decision consequences.

Interoperability and versioning

For every connector, publish producer, consumer, request or event schema, identity fields, ordering, idempotency, size limit, timeout, retry, error form, version policy and observability fields. Prefer resource and representation semantics over remote calls that expose internal tables. Events describe completed facts and carry source version; commands express requested intent and may be rejected. File exchange includes media type, schema version, reference, unit, digest and manifest. Compatibility tests run in both directions across supported versions. An undocumented shared directory or database table is still an interface, but a dangerous one.

Security and professional responsibility

Place policy enforcement where the protected resource is resolved, not only in the visible client or perimeter. Authenticate human and service identities, authorise each resource action and carry purpose and tenant or project scope where applicable. Trust is not inherited merely because two components share a network. Minimise secrets, rotate them, keep them out of telemetry and bind service credentials to narrow operations. Audit records capture the requesting identity, decision, policy version, resource identity and result. Threat modelling follows data flows and privilege changes, including job workers, caches and export paths.

Operational workflow and observability

Operate the architecture through service objectives, dependency maps, runbooks and rehearsed failure modes. Each component exposes health of its own responsibility, while end-to-end probes test a user decision path. Readiness differs from liveness: a process can run but be unable to serve correct data. Back-pressure prevents overloaded downstream components from accepting unbounded work. Queue age and oldest eligible job are often more informative than raw queue length. Configuration changes are versioned and deployed like code. Recovery exercises remove one dependency at a time and confirm the declared degraded state.

Integration checkpoint

Connect the reference architecture artefact to the preceding volume architecture. Trace one synthetic object from source identity through the new boundary to a reviewed output, then trace one rejection or failure back to the earliest violated invariant. Update the architecture decision record with the chosen option, alternatives, assumptions, evidence, consequences, owner role, review state and triggers for reconsideration. A checkpoint passes only when another reviewer can reconstruct both the successful path and the blocked path without oral explanation.

Synthetic worked example

SYN-ARCH-02 initially connects the browser directly to a shared database and file directory. Review finds no stable API contract, no resource-level authorisation, no job identity and no way to distinguish a cached preview from an approved artefact. The revised logical architecture introduces an API boundary, immutable object manifest, explicit job service and derived cache keyed by source version. A disconnected client can queue a draft intent but cannot mark it approved. A dependency exercise removes the cache; approved records remain correct while previews become slower and visibly degraded.

Practice and assessment

  1. Which component is authoritative for every state shown?
  2. Why is each interaction synchronous, asynchronous or offline?
  3. What happens to the decision path when one dependency fails?
  4. Which deployment choices can change without changing the logical design?

Assessed artefact: a viewpoint-based logical reference architecture and boundary catalogue. Submit the artefact with its source manifest, acceptance evidence, unresolved risks and a short explanation of why one plausible alternative was not selected.

Common failure modes

  • Drawing deployment products without identifying responsibilities or concerns.
  • Treating a shared database as an undocumented integration interface.
  • Letting a cache define interpretation truth or approval state.
  • Assuming network location is sufficient trust evidence.
  • Claiming scalability without workload shape, concurrency or recovery evidence.

Sources and further reading