What Is Data Mesh? Architecture, Principles, and When It Actually Works

Data mesh diagram showing business teams sharing data products through a shared platform

Data mesh is a way of running company data where each business team owns its own data and shares it with everyone else in a form they can use right away. The sales team looks after sales data. The delivery team looks after delivery data. A central team is still around, but it mostly builds the shared tools and keeps a short list of company-wide rules.

You’ll see data mesh pitched as a product a lot. It isn’t one. Nobody can sell you a data mesh, though plenty of companies sell tools that help with one. It’s really about who owns what, and that’s why it’s harder to pull off than it sounds.

Below we cover how it works, what the architecture looks like, a worked example, and the part most guides skip, which is why so many companies started a data mesh and then quietly gave up on it.

Data Mesh at a Glance

QuestionShort answer
What is it?A way of organizing company data where business teams own and share their data as products
Who created it?Zhamak Dehghani, who first wrote about it in 2019 while working at Thoughtworks
Main ideaThe people who create data should own it, clean it and explain it
Four principlesDomain ownership, data as a product, self-serve data platform, federated governance
Is it a tool?No. Tools can support it, but it is a way of working
Best forLarge companies where one central data team has become a bottleneck
Not ideal forSmall companies where one data team can keep up with requests

What Is Data Mesh?

Data mesh deals with analytical data. That’s the data behind reports, dashboards, forecasts and machine learning models, as opposed to the live data an app uses to take an order.

Zhamak Dehghani first wrote about the idea in 2019, while working at Thoughtworks. Her 2020 follow-up piece on data mesh principles and logical architecture on Martin Fowler’s site is still the best place to read the original thinking. She expanded it into a book in 2022.

Her basic argument is easy to follow. The people who create data understand it best. So they should be the ones who clean it, explain it and share it, instead of passing it to a data team that has to guess what it means.

A food court is a decent way to picture it. Instead of one huge kitchen cooking every dish with a queue out the door, each stall cooks its own menu. The mall doesn’t cook anything. It handles the electricity, the seating and the health inspections. In a data mesh, business teams are the stalls and the central data team is the mall.

Why Data Mesh Came About

For years the standard setup looked about the same everywhere. Every team’s data got copied into one big data warehouse or data lake. One central data team cleaned it up, joined it together and built reports for the whole company.

That’s fine when a company is small. It gets painful as the company grows.

The central team turns into a queue. A request for a new report can sit for weeks because ten other teams asked first. The central team also doesn’t know the data the way its owners do. If returns suddenly jump in March, a data engineer has to go and ask around, while someone on the returns team would know the reason straight away. Maybe it was a new returns policy, or a bad batch from a supplier.

Then there’s breakage. A developer renames a field in the order system, and a finance dashboard two steps away stops working. Usually the developer had no idea anyone depended on that field.

Data mesh tries to fix all three problems by moving ownership back to the teams that create the data.

Comparison of a centralized data team with a ticket backlog and a data mesh where teams publish their own data products

The Four Principles of Data Mesh

Dehghani described four principles. You can’t really pick one or two and skip the rest, because they lean on each other. More on that after the list.

1. Domain Ownership

The word domain sounds technical, but it just means a part of the business with its own job and its own data. Orders, payments, delivery and marketing are all domains. In a data mesh, each domain team owns its analytical data, and that includes keeping it accurate and documented.

2. Data as a Product

Owning data isn’t the same as dumping tables somewhere. Each team has to package its data so other people can use it without sending a message first. It should be easy to find, clearly explained, accurate and secure. It also needs a named owner, so when it breaks, people know whose door to knock on.

3. Self-Serve Data Platform

If every team set up its own storage, pipelines and security, costs would explode and nothing would match. So one platform team builds the shared tooling. The aim is that a marketing analyst can publish a dataset without learning how the servers underneath work.

4. Federated Governance

Governance is just the rulebook for privacy, security, naming and quality. Federated means the domain teams and the platform team agree on the rules together, instead of one group handing them down. Good setups also build the rules into the platform, so a dataset that breaks one gets flagged on its own.

The four principles of data mesh: domain ownership, data as a product, self-serve platform, and federated governance

So why do they need each other? Picture a company that only does the first one. Every team owns its data, but nobody agreed on a customer ID format, so nothing joins up. Or a company that writes plenty of rules but never builds the platform, so every team ends up filling in forms by hand. The idea only holds up when all four are in place.

What Is Data Mesh Architecture?

Data mesh architecture is how the teams, data products and shared tools fit together. Dehghani kept it loose on technology on purpose, so you can build it on almost any cloud or data stack. Most real setups end up with five layers:

  • The source systems each team already runs, such as the order app or the payment system
  • The data products each domain publishes from those systems
  • A self-serve platform for storage, pipelines, a data catalog, access control and monitoring, usually hosted on cloud server infrastructure
  • Shared rules that the platform checks on every update
  • The people and tools that use the data, including analysts, dashboards, other teams, apps and AI tools

Data products can also be built from other data products. Marketing might combine the orders product with the payments product to work out how much a typical customer spends in a year. Once enough of these links exist, the whole thing starts to look like a net, and that’s where the name comes from.

Data mesh architecture layers: consumers, domain data products, data sources, self-serve platform, and shared rules

What Goes Inside a Data Product

People tend to picture a data product as a single table. In practice it’s a small bundle. There’s the data itself, which could be a table, a set of files or an API. There’s the code that builds and refreshes it. Then there’s a plain description of every field, the quality checks, a promise about how fresh the data will be, and access rules for who can see what.

That promise is often written down as a data contract. It’s a short document between the team producing the data and the teams using it. It covers the fields, their formats, how often the data updates and who to contact when it breaks. There are open formats for this now. The Linux Foundation’s Bitol project maintains the Open Data Contract Standard and the newer Open Data Product Standard, which helps when different teams use different tools.

Parts of a data product: the data, the code, the description, a quality promise, and access rules

Who Does What in a Data Mesh

A lot of confusion about data mesh comes from not knowing who is responsible for what. Here is how the work usually splits.

RoleWhat they doWhat they don’t do
Domain teamBuilds, updates and fixes its own data productsRun servers or build shared tooling
Data product ownerDecides what the product should include and talks to the people who use itApprove every query or report
Platform teamBuilds and runs the shared tools for storage, pipelines, the catalog and accessOwn or clean business data
Governance groupAgrees on company-wide rules and makes sure the platform checks themReview every dataset by hand
Data usersFind data products in the catalog and use them for reports, models and appsRebuild data that another team already publishes

In smaller setups, one person may hold more than one of these roles. That’s fine, as long as everyone knows who owns each data product.

A Simple Data Mesh Example

Say you run an online store with four teams: orders, marketing, delivery and customer support. Your marketing head wants to know which ad campaigns bring in customers who send back fewer items.

With a central data team, this becomes a ticket. Someone on the data team tracks down the ad data, the order data and the returns data. They spend a few days working out what counts as a return, because support and orders define it differently. Then they build a pipeline. The report shows up three weeks later, marketing spots a tagging mistake in the campaign data, and the whole thing goes back into the queue.

With a data mesh, the orders team already publishes an orders and returns product, and marketing already publishes a campaign results product. Both use the same customer ID format, since that’s one of the shared rules. An analyst finds both in the catalog, reads the field notes and has an answer by the end of the afternoon. If the returns numbers look off, the catalog shows exactly who to ask.

Nothing about the data itself changed in that example. What changed is that every piece of it had an owner.

Data Mesh vs Data Lake vs Data Warehouse vs Data Fabric

These four get lumped together, but data mesh is a different kind of thing. Warehouses, lakes and fabrics are technology. Data mesh is about how people are organized around data. So it isn’t a choice of one or the other, and plenty of data meshes sit on top of a data lake.

ApproachWhat it isWho owns the dataBest for
Data warehouseA central store of cleaned, structured data for reportsA central data teamReliable reporting on well-defined business numbers
Data lakeA central store for raw data of any type, kept at low costA central data or platform teamStoring large amounts of raw and mixed data
Data fabricA technology layer that connects data across many systemsUsually central ITFinding and reaching data spread across many tools
Data meshA way of working where business teams own and share data as productsDomain teams, under shared rulesLarge companies where one central team has become a bottleneck

Data fabric is the one that gets compared to data mesh most. A fabric is mainly software that connects data across lots of systems and helps people find it. A mesh decides who’s responsible for each piece. Some companies run both.

Real Companies Using Data Mesh

A few large companies have talked openly about their data mesh work, and their stories show how differently it can look in practice.

Zalando, the European online fashion retailer, was one of the earliest. Its engineers presented their move to a data mesh at the Spark and AI Summit in 2020, with domain teams publishing their own data on top of a shared lakehouse platform.

Intuit, the company behind TurboTax and QuickBooks, published its data mesh strategy in 2021. Its main goals were making data easier to find, easier to trust and easier to publish across a large organization.

JPMorgan Chase described building a data mesh on AWS, where business areas own their data products and a shared catalog lets other teams find and request access to them. Netflix and PayPal have also spoken publicly about using data mesh ideas.

Notice what these companies have in common. All of them are large, with many separate business units and strong engineering teams. That pattern matters when you decide whether data mesh fits your own business.

Tools That Support a Data Mesh

No single tool gives you a data mesh, but most setups use a mix of these categories:

  • Data catalogs, such as DataHub or OpenMetadata, so people can search for data products and read what they contain
  • Cloud data platforms, such as Snowflake, Databricks or Google BigQuery, to store and process the data
  • Query engines, such as Trino, to read data across different systems without copying it first
  • Transformation tools, such as dbt, to build and test data products as code

Pick tools after you know your domains and rules, not before. Buying a catalog first and hoping ownership follows is one of the most common mistakes.

Benefits of Data Mesh

When it’s done well, the gains are pretty practical. Requests stop piling up in one queue, so people get answers sooner. Data quality usually improves, because the people who know the data are the ones fixing it. Every dataset has a name attached. And a good data product gets reused across teams, so the same report doesn’t get rebuilt four times in four departments.

It also copes with growth better. A new business unit brings its own team to look after its own data, instead of piling more work on the same central group.

Why Many Data Mesh Projects Stall

Data mesh was everywhere in 2021 and 2022. Conference talks, vendor webinars, new job titles. Then things went quiet. In its 2022 Hype Cycle for Data Management, Gartner went as far as predicting that data mesh would become obsolete before it ever reached mainstream use.

That prediction has turned out half right. You hear the term less now, but a lot of the ideas stuck around under other names, especially data products and data contracts.

Most of the trouble comes down to people. Ownership sounds clean on a slide, and then someone asks who owns customer data. Sales, marketing and support all touch it, and that argument can drag on for months. Skills are another wall. Telling a marketing team it now owns its data pipelines doesn’t give it a data engineer.

Money is the third problem. A platform that’s genuinely self-serve takes a lot of time and budget to build, and a half-finished one just becomes the new bottleneck. When the shared rules are weak, teams also drift. Each one picks its own tools and naming habits, and you end up with fresh data silos, which is the exact problem data mesh was meant to solve.

One more thing gets overlooked. Owning data is real work, and if it isn’t in anyone’s goals or budget, it slides. This academic review of practitioner reports on data mesh is worth a skim if you want to see these issues described by people who’ve been through them.

Is Data Mesh Right for Your Company?

Our honest take is that most companies asking this question don’t need a full data mesh. It was built for one particular problem, and without that problem it mostly adds overhead. This table should help you place yourself.

Data mesh may fit ifIt is probably too early if
You have several business units that create and use a lot of their own dataOne small data team can handle current requests
Your central data team has a backlog measured in weeks or monthsYour business has few clear boundaries between teams
Some business teams already have their own analysts or data engineersYou have no shared data platform yet
Leaders will back a change that takes years, not monthsYour main problem is poor data at the source, which data mesh won’t fix alone

Landed mostly on the right? You can still borrow the most useful habit, which is treating your most important datasets as products, each with an owner and a clear description. That works fine with a central team.

How to Implement Data Mesh in 7 Steps

The companies that get somewhere with data mesh tend to start small and grow it slowly. Announcing a company-wide rollout on day one rarely ends well. This is the order we’d follow.

Step 1: Find the Problem You Want to Solve

Forget the framework for now and look at where data work actually gets stuck. Are requests slow? Is the same report being built three times? Do two teams bring different revenue figures to the same meeting? Write those down. They’re what you’ll measure against later.

Step 1 of data mesh implementation: a central data team request queue with long wait times

Step 2: Map Your Business Domains

List the main parts of the business and the data each one produces. Base this on how the company really runs, not on how the databases happen to be laid out. For each domain, note who would own its data and who relies on it today.

Step 2: example domain map for an online store with orders, marketing, payments, delivery, support, and product

Image file: step-2-map-domains.webp. Alt text:

Step 3: Pick One or Two Pilot Domains

Go for teams with clear edges, at least one person who’s comfortable with data, and a problem that ownership would obviously fix. Two is a good number, since it lets you test sharing between teams. Starting with six mostly multiplies the risk.

Step 3: two pilot domains, orders and marketing, selected from the domain map

Step 4: Build a Basic Self-Serve Platform

The pilot teams need shared tools for storage, pipelines, a data catalog, access control and monitoring. Ready-made templates for common jobs save a lot of time early on. It helps to run the platform like a product too, with real users giving feedback. If you’re still deciding where the data should live, our roundup of cloud storage alternatives covers the main options.

Step 4: a self-serve data platform with storage, pipelines, catalog, access control, monitoring, and templates

Step 5: Create Your First Data Products and Contracts

Ask each pilot team to publish one or two data products that someone else is already waiting for. Write a data contract for each one, covering the fields, how often it updates, which quality checks run and who owns it. Then add them to the catalog so people can find them without asking around.

Step 5: example data contract for an orders and returns data product, listed in a data catalog

Step 6: Agree on Shared Rules and Automate Them

Pull together a small group from each domain, the platform team and security. Keep the first rule list short. One customer ID format, a naming convention and a way to tag personal data is plenty to start with. Then build those checks into the platform so they run on every update, not once a quarter.

Step 6: shared data rules checked automatically, with one data product blocked for an untagged personal data field

Step 7: Measure Results and Expand Slowly

Go back to your list from step 1. Are requests faster now? Do data products break less often? Are other teams reusing them? Fix whatever went wrong in the pilot before you bring in the next group of teams. Most companies expand in waves over a year or more.

Step 7: metrics to track and a rollout plan that expands data mesh in waves

Data Mesh and AI

This is where data mesh ideas are getting a second life. AI assistants and agents are only useful if they can find the right data and understand it. A chatbot answering sales questions needs to know which table holds the real revenue figure, what each column means and whether the person asking is allowed to see it.

A well-built data product already answers those questions. It has a description, an owner, quality checks and access rules. That’s a big reason companies working on AI have taken a fresh interest in data products and data contracts, even if few of them call it data mesh.

Common Myths About Data Mesh

The most common one is that you can buy it. You can buy catalogs, query engines and platforms that support a data mesh, but the mesh itself is an arrangement of teams and rules.

Another is that the central data team goes away. It doesn’t. It stops being the bottleneck for every report and starts building and running the platform instead.

People also assume it replaces the data lake or warehouse. Usually the storage stays exactly where it is. What changes is who looks after each piece.

And not every growing company needs one. A central team with good habits around ownership and documentation can work well for a long time.

So, Should You Care About Data Mesh?

If your central data team is buried in requests and your company has clear business units, data mesh is worth a serious look. Start with a pilot, not a reorganization.

If that doesn’t sound like you, don’t force it. Pick your five most important datasets, give each one an owner and a plain description, and write down what people can expect from it. That gets you most of the value for a fraction of the effort.

Frequently Asked Questions

What is a data mesh in simple terms?

It’s a setup where each business team owns its data and shares it as a ready-to-use, documented product. A shared platform and a few common rules hold it together.

What is data mesh architecture?

It’s the structure behind a data mesh. It includes domain data products, a self-serve platform, shared rules that run automatically, and the people and tools that use the data.

Who created data mesh?

Zhamak Dehghani. She first wrote about it in 2019 while working at Thought works, and her 2022 book covers it in the most detail.

Is data mesh dead?

The name has faded a bit, as Gartner predicted back in 2022. The ideas haven’t. Data products, data contracts and self-serve platforms are all widely used today.

How is data mesh different from data fabric?

Data fabric is technology that connects data across systems. Data mesh is about who owns the data. The two can work together.

Can a small company use data mesh?

It can, but it rarely needs to. Small teams get most of the benefit by naming an owner for each key dataset and documenting it properly.

How long does a data mesh take to set up?

Plan on years for a full rollout. A pilot with one or two teams can show results within a few months.

Leave a Reply

Language »