Data Contracts as Executable Boundary Definitions

Article 2 ended with a placement question:

What does this check need to know, and what should change if it fails?

That question leads straight to data contracts.

A source should arrive by a certain time. A file should contain required columns. A staged table should have one row per business key. A reference value should resolve for the reporting date. A published output should be complete before consumers pick it up.

Those expectations often exist already.

They live in code, runbooks, dbt tests, spreadsheet checks, tickets, dashboards, support habits, incident history, and people's heads. The problem isn't that nobody knows what the data should look like. The problem is that the system doesn't always know soon enough to act.

That's where data contracts become useful.

Not as a vendor pitch. Not as a documentation exercise. Not as a magic fix for bad data.

A data contract helps when it gives the system something it can actually run, at the point where the expectation matters.

The Scope of This Argument

The examples here come from batch and scheduled data pipelines: source receipt, parsing, landing, staging, enrichment, aggregation, publication, and handoff.

The same principle can transfer to event-driven systems, but those systems have different failure boundaries. I'm not giving implementation guidance for streaming, pub/sub, or continuously running event pipelines here.

This article doesn't depend on one contract standard or tool.

A contract might be expressed as dbt tests, SQL assertions, schema checks, Great Expectations suites, Soda checks, JSON Schema, Pydantic models, ODCS, or internal metadata. The format matters less than whether the system can execute the expectation at the right boundary and do something useful with the result.

The Expectations Already Exist

Teams sometimes talk about data contracts as if they introduce expectations for the first time.

Usually, they don't.

The expectations are already there. They just aren't always visible as system behaviour.

A finance user expects a daily file to arrive before a reporting run starts. An analyst expects a field to contain a known set of codes. A downstream model expects a customer identifier to be stable. A reconciliation process expects totals to match within tolerance. A regulatory output expects required sources to be complete before publication.

These are contracts in the practical sense. They describe what one part of the system expects from another.

But they might be scattered across places the pipeline can't act on.

A rule in a runbook helps an operator. A comment in SQL helps a developer. A wiki page helps a reviewer. A dashboard helps someone notice a pattern. All of that can be useful.

But none of it shortens failure latency by itself.

A hidden expectation doesn't fail early. It fails when someone finally notices the mismatch.

Most teams don't lack expectations. They lack executable expectations in the right place.

What a Data Contract Means Here

For this series, a data contract is the explicit definition of what must be true for data to successfully cross a boundary.

That definition is deliberately practical.

The contract might describe:

• shape
• required fields
• data types
• keys
• null rules
• accepted values
• reference resolution
• row counts
• freshness
• delivery windows
• completeness
• ownership
• release readiness

It doesn't have to describe all of those things at once. In fact, it probably shouldn't.

A useful contract is scoped to a boundary.

At source receipt, the contract might care about delivery time, file naming, sender, and expected batch. At parsing, it might care about headers and types. At staging, it might care about keys, nulls, and source-level rules. At enrichment, it might care about mapped values. At publication, it might care about completeness and release state.

The contract isn't just "what should this dataset look like?"

The better question is:

What does this boundary need to know before the pipeline can safely continue?

That keeps the contract tied to system behaviour.

Documentation Doesn't Shorten Failure Latency by Itself

Documentation matters.

It explains intent. It helps people understand a model. It gives reviewers context. It helps someone diagnose a failure at 4pm on a bad Friday.

But documentation doesn't stop a bad file.

A rule in a wiki doesn't reject malformed input. A note in a ticket doesn't quarantine invalid rows. A naming convention doesn't block publication by itself. A contract file that never runs is still outside the operational path.

The system doesn't act on the document. It acts on the check.

That's the line that matters for failure latency.

If a contract says a file must contain a column, but the pipeline doesn't check that until the final output fails, the contract hasn't reduced failure latency. It might help explain the failure afterwards. It might help someone know what to fix. But it hasn't made the failure visible at the boundary where the column first mattered.

It becomes part of an operational control when the system runs it and then does something different.

That change might be small:

• reject the file
• stop one source stage
• quarantine bad rows
• mark a source as not ready
• notify the source owner
• hold publication
• require an explicit decision
• record evidence for later review

Without that consequence, the contract is still useful. It just isn't doing control work yet.

Contracts Belong at Boundaries

Article 2 argued that validation belongs at the earliest useful boundary.

The same is true for contracts.

A contract shouldn't float above the pipeline as a general statement of good intent. It should attach to the boundary where the expectation first becomes testable and useful.

A source receipt contract can check whether the expected file arrived, whether it came from the expected place, and whether it arrived inside the delivery window.

A parsing contract can check whether the file is readable, whether the header is present, and whether required columns exist.

A landing contract can check row count, empty delivery, duplicate delivery, and captured source metadata.

A staging contract can check keys, null rules, basic domain rules, and source-level consistency.

An enrichment contract can check whether source values resolve against reference data for the right date.

An aggregation contract can check totals, balances, and cross-record relationships.

A publication contract can check output completeness, release readiness, and consumer-facing state.

A handoff contract can carry state, evidence, owner, and next action.

The same dataset might pass through several contracts as it moves through the pipeline. That's not a problem. Each contract answers a different boundary question.

The danger is pretending that one contract at the start can answer every question the pipeline will need later.

It can't.

Some expectations only become meaningful after the system has more context.

What a Contract-Backed Control Point Needs

A full data contract is more than schema and basic validity.

Shape, required fields, data types, null rules, keys, and accepted values matter. They are often the first things people expect from a contract, and rightly so.

But late failures often sit outside that narrow definition. They come from missing files, stale data, unresolved reference values, incomplete source sets, unapproved release state, unclear ownership, or missing evidence.

That does not mean every concern belongs in one contract file.

The contract file is usually best at defining the boundary expectation: the fields, types, keys, validity rules, tags, and explicit links to related definitions. Shared field sets, reference data definitions, source bindings, dataset manifests, resolved lock files, approval state, structured logs, routing rules, and release gates can live beside it.

The important part is not physical co-location. It is explicit connection.

A contract can link to a shared field definition. A dataset manifest can name the contracts that belong together. A lock file can record the resolved dependency graph. A source file can bind the logical contract to a physical source. An orchestration rule can say which check blocks which stage. A state table can record approval and run state. A logging layer can keep the evidence.

A contract that references a shared field definition also depends on a specific version of that definition. That version does not have to be written directly into the contract file, but it does need to be declared or resolved somewhere the system can check.

Without that, the reference resolves to whatever the definition currently happens to be.

That's the same class of problem as a hidden expectation. The system looks like it's checking against something fixed. It isn't.

A shared currency code list at a pinned version is a controlled dependency. The same list without a resolved version is a moving target. The same applies to cross-source validation: if a value in one source is checked against a reference set from another, the version of that reference set is part of the expectation.

The model is similar to Python dependencies. pyproject.toml says what the project depends on. uv.lock records the exact versions that were resolved and approved for use. You do not copy every dependency into pyproject.toml, but you also do not run production code against whatever happens to be current that day. Data contracts need the same distinction: declared expectations, resolved dependencies, and a runtime check that the resolved set is the one being used.

Together, those pieces form the control point.

The contract does not need to become a giant file that describes the whole operating model. In fact, that is often the wrong direction. A giant contract file becomes hard to review, hard to reuse, and hard to version.

The better test is this:

Can the system follow the links from expectation to check, owner, action, and evidence?

If it can, the contract is doing useful control work. If it cannot, the contract might still be good documentation, but it has not shortened failure latency yet.

The Same Failure, Expressed as a Contract

Return to the invalid business code from the first two articles.

The source file contains a business code. The raw file has the value. The value has the right type. It matches the expected pattern. Nothing about the file alone proves the code is valid.

The code becomes testable at enrichment, where the pipeline has both the source value and the reference set for the reporting date.

That boundary can have a contract:

Every business code in this source must resolve against the approved reference set for the reporting date.

The executable check finds unresolved codes.

The gate stops the affected source stage or quarantines the unresolved rows. The failure is routed to the owner of the source value, the reference set, or the mapping rule. Publication waits until the affected data has a valid state.

The contract didn't invent the rule.

People already expected the code to resolve. They probably expected it long before anyone wrote the contract down.

The contract made the expectation runnable.

That's the difference.

Tooling Is Still Catching Up

This is where the current state of data contracts gets awkward.

The concept is often clearer than the tooling.

Different tools cover different parts of the problem. Some describe schemas. Some run tests. Some generate documentation. Some track ownership. Some connect to lineage. Some raise alerts. Some help with observability. Some try to provide a portable contract format.

Few tools give you the whole operating model.

That isn't a criticism of the tooling. It's just where the field is.

A specification can describe the expectation. The operational control is the system behaviour around it.

That means teams often have to assemble the operational control from existing pieces: a contract file, a dbt test, a SQL assertion, orchestration logic, metadata, alert routing, release gates, and evidence capture.

You might have to roll some of this yourself.

That's fine.

The design question still comes first.

Where does the expectation become testable? What should the system do if it fails? Who needs to know? What should wait? What evidence should remain?

A simple SQL check that blocks publication might reduce failure latency more than a clean contract file nobody runs.

The format is secondary. The first question is whether the check runs at the point where the failure matters.

Duplicate Execution Is Fine

One common objection is that contracts can lead to repeated checks.

That's true.

A producer might check data before sending it. An ingestion layer might check before landing it. A transformation layer might check before enrichment. A consumer might check before using a published output.

That isn't automatically waste.

Each check might serve a different boundary and audience.

The producer check can catch a problem before transfer. The ingestion check can protect the platform. The transformation check can protect business logic. The publication check can protect consumers.

Duplicate execution can be useful.

Duplicate rule definitions are the dangerous part.

If a file fails a contract and three teams all assume someone else owns the failure, the contract has moved the argument rather than shortened the delay.

An executable contract should make ownership and definition clearer, not more ambiguous.

It should say who owns the expectation, where the rule is defined, who receives the failure, who can act, and what state the pipeline should hold until the issue is resolved.

That's where contracts become operational rather than decorative.

Start With One Implicit Expectation

You don't need a contract platform to start.

Pick one recurring late failure.

Write down the expectation that was violated.

Find the first boundary where that expectation is testable.

Express it in the simplest executable form.

Make the result visible to the right owner.

Make the pipeline do something different after failure.

Keep the evidence.

That first version might be a dbt test, a SQL assertion, an orchestration guard, a file receipt check, or a small custom script. That's fine. The point isn't to adopt a grand contract architecture in one move.

The point is to take one expectation that currently lives in people's heads and put it into the system at the boundary where it matters.

Then do it again.

What Comes Next

Data contracts matter when they help the system notice a broken expectation sooner and do something about it.

That means contracts aren't only about producers and consumers agreeing what a dataset should look like.

The useful version is more operational than that.

The agreement has to run somewhere.

It has to run where the system has enough context to trust the result.

And when it fails, the right people need to know, and the pipeline needs to change what happens next.

That's how data contracts connect back to failure latency.

Article 2 placed validation at boundaries. This article made boundary expectations executable.

The next question is implementation shape.

How do you add these checks inside the system you already have? How do you protect expensive stages without rebuilding the platform? How do you quarantine bad inputs, hold outputs, route failures, and keep evidence without creating a massive new programme?

That's where implementation patterns come next.