One Event Backbone Beats Point-to-Point Integration
Count the integrations in a retail estate and the number is always higher than anyone guessed. Commerce to OMS, OMS to WMS, WMS back to commerce, ERP to everything, and a CDP reading from 4 places while writing to 2.
Each one was reasonable when it was built. Collectively they are the reason a new fulfilment partner takes 5 months to onboard, and the reason nobody can answer what the inventory number actually is.
The arithmetic that decides this
Point-to-point integration scales badly for a reason that is arithmetic rather than engineering. With 6 systems there are 15 possible pairs, and real estates land somewhere between 12 and 20 actual links because most pairs eventually find a reason to talk.
Adding a seventh system adds up to 6 new links, each with its own auth, retry policy, field mapping, and failure behaviour. Adding the eighth adds 7. The cost of each new system rises with the size of the estate you already have.
A backbone changes the slope. Each system publishes what it knows and subscribes to what it needs, so the Nth system costs 1 connection instead of N.
The effect is measurable in onboarding time, which is the honest metric for this kind of work. On a multi-tenant retail data platform we run, processing 60M or more events per day, tenant onboarding went from 6 weeks for the first tenant to 4 days for the fourth.
Nothing about the fourth tenant was easier. The connections it needed already existed.
Source of truth is a rule per entity, not per system
The most common failure in event-driven retail is not technical. It is 2 systems both believing they own an entity, which produces an update loop that overwrites itself until someone notices the price flickering.
Write the ownership rule down per entity, and make it granular enough to be useful. Ownership at the entity level is usually too coarse, because inventory quantity and inventory reservation genuinely belong to different systems.
| Entity or field group | Typical owner | Who may write | Who consumes |
|---|---|---|---|
| Product master attributes | PIM | PIM only | Commerce, search, marketplace feeds, CDP |
| Price and promotion | Pricing service or ERP | Pricing service only | Commerce, OMS, search, reporting |
| On-hand inventory | WMS or store system | WMS and store systems | Commerce, OMS, search |
| Reservations and allocation | OMS | OMS only | Commerce, WMS, customer service |
| Order state | OMS | OMS only | Commerce, WMS, finance, customer service, CDP |
| Customer identity | CDP or commerce | 1 named owner | Everything |
| Financial postings | ERP | ERP only | Reporting |
The rule that keeps this honest: a consumer that needs to change an entity it does not own sends a command to the owner, and the owner emits the resulting event. It never writes directly and waits for reconciliation to sort it out.
Where 2 systems genuinely both write, as with inventory across a warehouse and a store network, split the field. Each writer owns its own quantity, and a derived available-to-promise number is computed downstream by a single component that owns the calculation.
Cadence follows the decision, not the technology
Everything can be real time, and pricing everything as real time is how event platforms become expensive without becoming useful. Set cadence from the decision the data supports.
- Real time, sub-second: anything a customer can race. Inventory decrements, reservation state, payment authorisation, cart and checkout state.
- Near real time, seconds to a minute: order status changes, price and promotion activation, product publication.
- Minutes: search index updates, recommendation feature refresh, operational dashboards.
- Batch, hourly or nightly: financial postings, marketplace feeds, reporting extracts, model training sets.
The test is simple. If a stale value causes a customer-visible error or a financial discrepancy, it is real time. If a stale value causes a slightly worse decision, it is not.
Batch does not mean a nightly file drop. Run batch as a scheduled consumer of the same event log, so there is 1 delivery mechanism and 1 set of semantics rather than 2 parallel integration styles.
The envelope is a contract, so write it down
An event backbone with no agreed envelope is point-to-point integration with extra hops. Every consumer ends up writing bespoke parsing per producer, which is exactly the coupling the backbone was meant to remove.
CloudEvents, a CNCF project that reached graduated status in January 2024, exists for this. Its own framing of the problem is the right one: without a common way of describing events, developers write new event handling logic for every event source.
Standardise the metadata that every consumer needs regardless of payload. Event type, source, subject identifier, event identifier for deduplication, and the time the fact occurred rather than the time it was published.
Then describe the channels themselves. AsyncAPI 3.0 gives you a machine-readable description of channels, operations, and messages across protocols, which turns "what does this topic contain" into a document that CI can validate instead of tribal knowledge.
Version the payload schema from the first release. A schema with no version field will get a breaking change in month 4, and by then 6 consumers are parsing it.
Delivery: retries, idempotency, and dead letters
Retries are the default failure response, and retries mean consumers will see the same event more than once. That makes idempotency a requirement rather than a refinement.
Stripe's idempotency design is the pattern worth copying. A client-generated key identifies the logical operation, the server stores the status code and body of the first attempt, and subsequent requests with the same key return the same result rather than performing the work twice.
Two details from that documentation are easy to miss and both matter. Stripe compares the incoming parameters against the original request and errors if they differ, which catches key reuse bugs. Keys are pruned after 24 hours, which sets an upper bound on how long a retry stays safe.
For events, derive the key from the event identifier rather than generating a new one per delivery attempt. Store processed identifiers with a retention window longer than your maximum retry window.
When retries are exhausted, the message needs somewhere to go that is not a log line. Amazon SQS dead-letter queues document the mechanics clearly: a redrive policy sets maxReceiveCount, and a message exceeding it moves to the dead-letter queue where it can be examined and later redriven.
Three operational details from that page are worth adopting whatever your broker is:
- Set maxReceiveCount high enough to survive a transient dependency outage. A value of 1 turns every blip into a dead letter.
- Give the dead-letter queue a longer retention period than the source queue. Message expiry is based on the original enqueue timestamp on standard queues, so a short retention silently deletes evidence.
- Alarm on any message arriving in a dead-letter queue. A dead-letter queue nobody watches is a data-loss mechanism with good branding.
Be careful with strict ordering. SQS documentation warns against pairing dead-letter queues with FIFO queues where message order carries meaning, and the same caution applies anywhere a skipped message changes the meaning of the ones after it.
Ordering and replay are the reasons to prefer a log
Queues and logs are not interchangeable. Apache Kafka guarantees that any consumer of a topic-partition reads that partition's events in exactly the order they were written, and retention is a per-topic configuration rather than a consequence of consumption.
That second property is what makes a log worth the operational overhead. Events are not deleted when consumed, so a new consumer can be added later and read history, and a consumer that processed 3 days of events incorrectly can be reset and replay them.
Partition by the entity identifier, typically order or SKU, so all events for one entity land in one partition and stay ordered relative to each other. Global ordering across the whole topic is neither achievable nor usually needed.
Replay changes how integration bugs get fixed. Instead of a reconciliation script that patches a database, the fix is deploying corrected consumer logic and replaying the affected window.
The first 3 topics, in order
A backbone programme that starts with a platform selection and a 6-month build usually gets cancelled before anything publishes. Start with the topic that removes the most bespoke code.
Order state comes first in almost every retail estate. It has 1 clear owner in the OMS, the most consumers, and the most existing point-to-point links, so replacing it retires the largest number of hand-written integrations per unit of effort.
Inventory comes second, and it is second for a reason. It has multiple legitimate writers, a derived available-to-promise calculation, and the tightest cadence requirement, which makes it the wrong place to learn how your broker behaves under load.
Product and price come third. They are lower cadence and more forgiving of a short inconsistency window, but they touch the most downstream consumers, so the schema versioning discipline gets tested properly here.
Everything else follows once those 3 are stable. By that point the envelope is agreed, the dead-letter alerting exists, and the schema registry has real traffic in it, so each new topic is configuration rather than a project.
One rule to hold through all of it: no consumer reads another system's database. The moment a reporting job or a search indexer bypasses the backbone with a direct query, the coupling comes back and the ownership rules stop being enforceable.
What a backbone does not fix
It does not fix disagreement about what a field means. If commerce and the OMS define order-cancelled differently, publishing that disagreement to 6 consumers spreads it faster.
It does not remove the need for reconciliation. Financial totals still get compared against the ERP nightly, because the reconciliation job is how you find out the pipeline is wrong.
It also adds a component that has to be run well. A backbone with no schema registry, no consumer lag monitoring, and no dead-letter alerting is a single point of failure that 8 systems now depend on.
FAQ
Do we need a backbone if we only have 4 systems?
Probably not yet. With 4 systems the point-to-point links are few enough to hold in your head, and the backbone's operational cost is real. The signal to build one is the fifth or sixth system, or the first time a new integration takes longer than the feature it was built to support.
Should events carry the full entity or just the identifier?
Carry enough for the common consumer to act without a callback, and include the entity version so a consumer that needs more can fetch it and know what it got. Identifier-only events push a read burst back onto the owning system every time something changes.
How do we migrate off point-to-point without a freeze?
Run both for a while. Have the owning system publish events alongside its existing direct integration, move consumers across 1 at a time, and retire each direct link only after its replacement has run clean for a full business cycle including a peak week.
What belongs in the backbone and what should stay a synchronous call?
Facts belong on the backbone. Questions that need an answer before the caller can continue, such as a payment authorisation or a real-time availability check at checkout, stay synchronous. Putting a request-response interaction on an event bus produces a distributed system with worse latency and harder debugging.