Real-Time Data in ERP: Harder Than It Looks

"We want our dashboards to be real-time," the CFO said. The request was casual. As if they were asking for a new color on a report. I've heard this exact request in maybe thirty board meetings now, and I can predict what happens next: the CTO says yes, the dashboard team gets tasked with it, and nine months later they're still wrestling with data consistency problems that nobody anticipated.

Real-time data sounds like a simple problem. Feed fresh data into a dashboard. Display it. The reality is far messier. Most enterprise ERPs—SAP, Oracle, NetSuite, whatever your company uses—were never designed for real-time dashboarding. The architecture of these systems assumes you run batch jobs at night, dump data into a warehouse, and report on yesterday. That worked for fifteen years. It's stopping working now.

The Core Problem

Here's what's actually happening when a CFO looks at a dashboard: they're not seeing data from the ERP. They're seeing data from a reporting layer that sits on top of the ERP. The reporting layer receives data from the ERP through a pipeline. That pipeline has latency.

In older ERP implementations, that pipeline runs once a night. Data hits the warehouse at 2 AM. Dashboards report on today's transactions. Everyone accepts a 24-hour delay. It's fine.

When you want real-time, you're replacing that nightly batch with something continuous. A stream of updates. This is where everything gets complicated.

First, there's extraction complexity. Your ERP system has hundreds of tables. Some of them relate to others. A sales order lives in five different tables across the system. If you pull just one table in real-time, your downstream reporting gets partial data. You have orphaned transactions. You have missing line items. You need to pull related tables together in a consistent way. That's harder than it sounds because the tables are constantly being written to by multiple users.

Second, there's consistency. Your ERP doesn't necessarily write all related data to the database at the same instant. An order gets created. The order table updates. Then the inventory allocation happens. Then the general ledger entry happens. These might be milliseconds apart. But if your real-time pipeline pulls data between step one and step two, you get an incomplete picture. The reporting layer sees an order with no inventory allocation. Dashboards break. Alerts fire incorrectly. Finance reconciliation fails.

Third, there's the infrastructure problem. Nightly batch jobs are simple. You have a window. You run ETL. You're done. Continuous streams need a different architecture entirely. You need message queues. You need stream processing. You need to handle failures and retries at scale. You need monitoring that actually works. Most teams building real-time ERPs for the first time underestimate this by a factor of three.

What I've Seen Work

There are ways to get real-time ERP data that actually work. But they require accepting trade-offs that most organizations don't want to admit.

The first approach is: don't pull from the ERP directly. Instead, build a transactional write layer where your core business systems (order management, inventory, GL) write events to a stream. Every meaningful business event—order created, inventory adjusted, payment received—becomes a message. Your dashboard consumes those messages and stays current. This works. Netflix uses this pattern. But it requires that you're willing to own the event architecture. Most companies using traditional ERPs aren't ready for that.

The second approach is: accept eventual consistency. You don't need data that's current as of right now. You need data that's current as of the last thirty seconds. Or five minutes. This is a huge reframing. Most CFOs resist it because they want to believe their dashboard is showing the truth right now. But in practice, a two-minute delay is almost never a business problem. It feels better because it's faster, but it's actually not that different from batch. You can achieve this with change data capture (CDC) tools that listen to your ERP's database transaction log. This is more practical. Tools like Debezium can do this without invasively modifying your ERP.

The third approach is: real-time for some things, batch for others. Your orders dashboard can be real-time. Your financial reporting stays overnight batch. This is what successful companies actually do. They identify which datasets truly benefit from real-time visibility, and only build the infrastructure for those.

The Cost Nobody Talks About

Building real-time data architecture is expensive. You need specialized talent. You need new tools. You need more infrastructure. You need better monitoring. You need processes to handle data quality when things break at 3 AM.

I worked with a manufacturing company that decided to move their entire ERP dashboarding to real-time. They budgeted two million for the project. It ended up being six million and took eighteen months. Not because the engineering team was incompetent—they were actually quite good. But because once you start pulling threads, you find more problems. The ERP data was messier than anyone thought. The business logic around what "real-time" means was more complex. The edge cases were numerous.

They got value from it, eventually. But the cost was real.

What I'd Tell You Today

If you're an executive asking for real-time ERP dashboards: think about what you actually need. Do you need to see transactions as they happen? Or do you need visibility into the state of your business every few minutes? Those are very different problems.

If you're a technologist tasked with building this: start with understanding your current architecture. Can you implement CDC? Can you write an event layer? What are your actual consistency requirements? Answer those questions before you start building.

The companies winning right now on ERP intelligence aren't necessarily the ones with the fastest dashboards. They're the ones who were honest about what real-time means for their business, and who built infrastructure that matches that definition. The speed helps. But the clarity about what matters helps more.

I think the ERP landscape is shifting. By 2021, expecting real-time data visibility won't be optional. It will be table stakes. The question is how much technical debt and how much infrastructure you're willing to invest to get there.