8 reasons why you need bitemporal data

There's a question that comes up in every workshop sooner or later: "Do we really need this complexity?" What people mean is bitemporal historization – two timelines instead of one, a few more columns, more logic when loading. It's a fair question.

From projects and trainings, eight typical reasons have crystallized that decide the answer. Three sound technical (late arrivals, corrections, time travel), two business-related (reality beyond the present, retroactive operations), three strategic (decision analysis, regulations, audit). Which of them affect you depends on your architecture – and on who in your company works with the data.

This article is the entry point into a four-part series that takes each of the eight reasons further. Examples come, as always, from the FastChangeCo universe, the fictional cast I use to ground real-world challenges from projects and coachings.

From experience, one preliminary remark is worth making: even when temporal data exists in a data solution, many people don't know why or what to do with it – and that applies equally to technical and business roles. A non-trivial share of those who work with data every day don't even know that bitemporal historization exists or what it makes possible.

The requirements to store data historically are manifold. A frequently named reason is: "Because we need it now…!" – which says exactly nothing. The following eight reasons are the more concrete answers.

---

The first of the eight reasons makes the problem immediately tangible:

Data occasionally arrives out of order

Reason #1: How can data fail to be delivered to the data solution in the correct chronological order?

Put simply, it happens because a data delivery has failed for some reason, or a streaming technology in use doesn't primarily care about the temporal order of the data packets. So this has to be handled in the data solution – a temporal database – so that in the end all data sits in the correct temporal order.

→ Covered in depth in Part 1: When Data Arrives Out of Order – Late Arrivals and Corrections. With a running email-correction example and the transposed exchange rate.

Store the 'reality' for past, present and future

Reason #2: FastChangeCo wants to save its 'reality' for the past, present and future.

This is the case at FastChangeCo, for example, when a department that sells products wants to create prices that should only be valid from a certain point in time in the future. Or when a piece of information needs to 'update' the past – e.g. a retroactive salary change.

→ Covered in depth in Part 3: Past, Present, Future – Data beyond the present (publishing June 10). With Black-Friday pricing and retroactive salary changes.

Corrections of the past

Reason #3: FastChangeCo wants to make corrections of the past.

This is the case at FastChangeCo, for example, when a department has received wrong data or values from external service providers. A later correction delivery contains the correct values. With bitemporal historization, the past can be mapped correctly in time – without rewriting reports from back then.

→ Also covered in Part 1, with the transposed exchange rate: why later corrections must not distort historical reports.

Time travel – ability to query across time

Reason #4: Subject matter experts want to view fact data from different time perspectives (As-Is, As-Was, As-Of, …).

Bitemporal historization separates two questions into two timelines for this – the assertion timeline (technical: when did we know about a value) and the state timeline (business: when was a value valid in reality). This separation is the prerequisite for traveling through time at all without the different views colliding with each other.

FastChangeCo often involves structural changes. When planning for one, a subject matter expert can, for example, look at the sales or profit distribution in the past, in the present, or in a future organizational structure. Or simply track how a customer has moved through the different customer groups: new customer, VIP customer, existing customer, VIP customer, no customer.

→ Covered in depth in Part 2: Time Travel in the Data Warehouse – with the Allen relationship, closed-open intervals, and the mechanics of how time periods relate to each other.

Retroactive operations

Reason #5: The invoice still has to be posted to the previous month.

This situation occurs at FastChangeCo when, for example, an invoice arrives on the 4th of a month with an invoice date from the previous month, and therefore has to be posted to the previous month as well. With bitemporal data storage, this is easily possible without manual effort.

→ Also covered in Part 3 (publishing June 10), with retroactive bookings and HR corrections.

Decision analysis

Reason #6: Traceability of whether a decision was 'right' or 'wrong' in the past.

In some areas of FastChangeCo, decision analysis is very important. Example: a decision was made a year ago, now a year has passed – and no one remembers why this decision was made back then, because the data that led to it is no longer available. With today's data, the decision might have been different.

With bitemporal data, you can travel back in time to check whether the decision was correct at the time – with the knowledge you had then.

→ Covered in depth in Part 4: Trust, Audits and Traceability (publishing June 24). With the regulator perspective and the C-level argument for the investment.

Legal regulations

Reason #7: External influences such as laws and regulations.

When deciding whether bitemporal historization is necessary, laws and regulations play an important role. Especially in the finance and insurance industry, there are many requirements for unchangeable, non-erasable storage of data.

→ Also covered in Part 4 (publishing June 24), with a focus on finance and insurance regulation.

Audit and review

Reason #8: Internal and external reviews / audits.

Data teams have to be able to rely on their data. Bitemporal historization makes it possible to ensure that the correct data is available for every audit. This applies to external audits just as much as to internal complaints – e.g. when someone reports that data has changed. At any time, the team can prove and explain how, when and why data has changed over time.

→ Also covered in Part 4 (publishing June 24), as the core argument for C-level.

---

The "next articles" the original post ended on have meanwhile turned into a four-part series, with each of the eight reasons taken further. The cluster tiles above lead you straight in – linearly from engineering pain to strategic decision, or directly to the reason that's hitting you right now.

Certainly there are more reasons for bitemporal data than the ones listed here. As always, it depends on the specific project, the company, or external influences such as legal requirements, whether additional ones apply.

So long,
Dirk