Data Contract Standards Between Producers and Consumers

Data contract standards formalize the relationship between data producers and consumers by replacing undocumented assumptions with versioned, machine-checkable agreements. Their value lies in enforcement. A shared specification becomes an engineering safeguard only when its promises are checked and violations can stop a bad change before consumers bear the cost.

Why the implicit model of data sharing breaks at scale

A producer renames a column, changes a type, or quietly redefines what “active” means. The pipeline may load the changed data without an error, causing a dashboard to fail to reconcile or a model to produce nonsense days later. By then, the visible symptom is far from the original change. The dashboard attracts the investigation, although the defect began at the producer’s interface. What looks like an analytical error is an unannounced change to a dependency.

This failure exposes three weaknesses in implicit data sharing. Breaking changes propagate without notification, quality expectations have no consistent definition, and ownership of the resulting damage is contested. Producers assume downstream systems can handle changes, while consumers assume upstream systems will stay the same. As more domains and data products have their own owners, these assumptions grow faster than any team can track by hand. Coordination that was manageable with one Oracle server and one warehouse becomes untenable across distributed interfaces.

Dr. Simon Harrer’s 2026 Cologne talk identifies those interfaces as the root of the problem. CSV on SFTP, JSON on S3, SQL tables on BigQuery, Iceberg files, and Snowflake views all serve critical dependencies. Yet data APIs had no shared specification standard, so each consumer had to bear the risk of interface changes.

What a data contract actually specifies — and what it deliberately excludes

A data contract is a versioned, machine-readable specification of a dataset’s structure, semantics, quality expectations, service levels, and ownership. Automated checks make those promises testable before violations affect consumers. The useful analogy is an OpenAPI contract for a REST API, which describes the external interface without prescribing the service’s internals. A data contract likewise governs the boundary between producer and consumer. It does not dictate how the producer builds or stores everything behind that boundary.

Its structural promises cover field names, data types, nullability, and primary keys. Its promises about semantics explain what the data means to the business, such as net_revenue_usd meaning after refunds, excluding tax and test accounts. Quality rules turn expectations about valid ranges, uniqueness, and referential integrity into assertions. Service levels add freshness windows, delivery frequency, availability commitments, and latency guarantees. Ownership and support identify the producing team, contact channel, deprecation timelines, and versioning policy.

Together, these elements distinguish a contract from a schema file or data dictionary. It is neither a legal document nor a monitoring dashboard, and its requirements are not optional once enforced. It also does not capture all the evolving knowledge needed to use data correctly, including validated queries and tribal join rules. Agents may need that knowledge as well, as a complement to the interface specification.

How the field converged on a single open standard after years of every team building its own

Andrew Jones first wrote publicly about data contracts in April 2021 and described GoCardless’s implementation in December 2021. PayPal described contracts within its data-mesh program in 2022, then open-sourced its Data Contract Template in May 2023 under the Apache License and donated it to the Linux Foundation. A standardization committee, including Dr. Harrer, took out the parts specific to PayPal and broadened the format so others could use it. That work supplied the seed for ODCS.

The significance was not the choice of YAML. Companies had already built their own YAML formats, but those formats were not interoperable. REST APIs had OpenAPI, and asynchronous interfaces had AsyncAPI, while data teams repeatedly had to define their own specification language. As more producers shared data with more consumers, this fragmentation made contracts difficult to move between tools or combine across teams. A shared syntax was useful only if teams also agreed on what to expect.

ODCS v3.1.0, approved December 8, 2025, brought that convergence into a precise, versioned specification under Bitol’s stewardship. The competing Data Contract Specification deprecated itself with the release of ODCS v3.1, reducing the need to choose between rival formats. Bitol is now a graduated Linux Foundation AI & Data project.

What ODCS v3.1 specifies and how its sections map to the producer's promises

ODCS v3.1 organizes the agreement into named sections that connect a dataset’s identity to its operational promises. Fundamentals supplies the ID, name, version, and status, giving the contract a lifecycle that producers and consumers can track. Schema describes tables and columns with their physical and logical types, examples, and data classification. The distinction between physical and logical types is important because storage representation does not explain business meaning. Without that logical layer, the specification says little more than DDL.

Data Quality expresses checks for nullability, ranges, and uniqueness as part of the producer’s commitment. SLAs adds the time dimension through freshness windows, latency commitments, and availability guarantees. Team, Ownership, and Support identify the producing team, contact channels, and escalation paths. These sections connect a failed assertion to both a stated promise and a responsible party. That connection is what a schema alone cannot provide.

Pricing and Terms supports marketplaces for data products, while another section covers data classification, who can access the data, and the policies for how long to keep it. Infrastructure and Servers describes where and how the data is served, and Custom Properties allows organization-specific metadata. The YAML/JSON-based standard has an Apache 2.0 license and is maintained by Bitol under the Linux Foundation, so teams have less need to rely on a vendor-specific format. ODCS sits above transformation-layer tools such as dbt model contracts as the interface specification, with their checks intact.

Where enforcement actually happens — and why the enforcement point determines whether a contract has teeth

A contract that cannot fail a build is just documentation. Checks at ingestion catch missing API fields, type mismatches, and partition changes as the data comes into the platform. This prevents defects from passing through dozens of downstream transformations before anyone detects them. In streaming systems such as Kafka, a schema registry performs a comparable role by rejecting nonconforming messages before they reach consumers.

At the transformation layer, dbt model contracts fail the build when output shape drifts. This catches defects later than ingestion checks, but still before the output reaches dashboards or models. The CI/CD gate moves the compatibility decision earlier in the change lifecycle, before deployment and data movement. Schema checks against the contract can block a merge request that drops a required column or changes a type in a prohibited way. The postmortem action then becomes preventing an unnoticed breaking change, rather than adding another alert at the point where damage appears.

These gates address different places where a promise can fail, so a build check does not eliminate the need to test arriving data. The Data Contract CLI, maintained by the Bitol / Linux Foundation community, supports linting, testing, and CI/CD integration. Such tooling connects the specification to executable checks. The important distinction is between finding the defect at the interface and discovering its consequences in a consumer’s output.

The organizational incentive problem that kills most contract programmes before enforcement matures

The technical architecture is straightforward compared with ensuring producers benefit from maintaining contracts. Producers must maintain schemas, honor deprecation windows, run quality checks, coordinate breaking changes, and answer for SLA breaches. Consumers get data that is stable and that they can trust. Dr. Harrer’s Cologne talk makes the asymmetry explicit, since these duties do not necessarily improve the producer’s own day-to-day metrics. Even a technically sound contract programme can stall because the team doing the work does not get the immediate payoff.

The recommended counterweight is to tie contract-breach incidents to the producer team’s on-call rotation. If a producer’s field change breaks a consumer’s dashboard, the producing team receives the page. Versioning and deprecation policy reinforce that accountability by requiring producers to communicate intent and give consumers a migration window. Without those arrangements, enforcement remains a consumer demand that producers have little reason to prioritize.

Regulation strengthens this organizational case but does not substitute for it. EU AI Act Article 10 requires data governance and management practices for datasets used in high-risk AI systems. Dr. Harrer also identifies NIST AI RMF requirements on data quality and traceability and ISO/IEC 5259 data quality for analytics and machine learning as pressures toward formal agreements. Contracts can provide evidence for audits of whether those obligations have been met. The organizational task remains assigning responsibility for keeping the promises true.

Why agentic AI makes data contract enforcement both more necessary and more achievable

AI increases the cost of an interface that changes silently. When CRM fields are unstandardized or upstream schemas drift, AI-driven summaries can hallucinate, predictive models can train on corrupt inputs, and forecasting assistants can produce confident nonsense. A defect that once left a person to make sense of a misleading result can now feed an automated decision. The contract matters because it defines what the consuming system is entitled to assume. Dr. Harrer’s Cologne talk predicts that this need will make data contracts as ubiquitous as OpenAPI.

Agentic operations also offer a way to enforce those expectations continuously. Data reliability platforms can detect anomalies, manage schema drift, and propose or apply fixes without waiting for a person to triage every alert. The contract supplies the expected state against which that activity operates. An agent can then watch for value drift developing over weeks or unusual combinations of events that precede a pipeline failure. This extends enforcement beyond checking whether a column still has the right type.

CRM and RevOps make the producer-consumer relationship especially concrete. The CRM is the producer, while lead routing engines, scoring models, attribution systems, and AI-driven outreach tools are consumers. As CRM data decays, a structurally valid interface can still deliver less reliable inputs. Continuous enforcement checks that semantic and quality promises hold when the data is used downstream. Without it, deterioration can remain hidden until a forecast fails or a model misfires.

Adopting data contracts in practice: sequencing decisions that determine whether an implementation succeeds

Successful adoption starts with a narrow set of enforced promises rather than an attempt to fill every part of the standard. Before writing the contract, agree that the producing team will be the default recipient of pages for breach incidents. This establishes the responsibility needed to sustain the checks once they find real problems. With that agreement in place, start with automated row-count and freshness checks across all critical tables. These require little configuration and catch the majority of common pipeline failures before more complex quality rules are introduced.

Record the maximum event timestamp after each load and send an alert when the data is older than the SLA allows. Put the first data checks at ingestion, where a missing API field or type mismatch can be caught before it propagates through transformations. That placement shortens the distance between a producer’s defect and its detection. Transformation checks and CI/CD compatibility gates can then protect the interface promise at the other points where it can break.

Use ODCS as the format from the start, even when the initial agreement is simple. This keeps the contract portable and machine-readable as it covers more of the data and its promises. Version every schema change and define deprecation timelines so consumers receive managed migration windows to avoid surprise outages. Add business definitions and richer quality assertions incrementally, giving particular attention to semantics because structural checks cannot catch every expensive misread. A contract enforcing schema and freshness now is more useful than a complete specification that no producer has accepted.

For organizations deploying AI, this sequence has a clear production boundary. Treat enforced contract coverage for the datasets feeding models as a prerequisite for deployment. A model trained on data without an enforced contract is a model whose reliability cannot be attested. The standard supplies the shared promise, but enforcement is what makes that promise a dependable input to automation.