This time, FastChangeCoTM's Data Management Center of Excellence (DMCE) team looks at the question: what are the business requirements for bitemporal data?
Before the DMCE team takes the effort to historize data with multiple timelines, they need to understand why this is necessary. Which departments, which subject matter experts are interested in historical data and if so, why?
The DMCE team is convinced that the real reasons for implementing bitemporal historization of data are not technical, but business-related.
After long discussions about the real background of the requirements and with the help of Dirk's temporal training, the DMCE team has listed the eight most common reasons for bitemporal historization (or bitemporal database) at FastChangeCoTM:
From their experience, the team is well aware that even if developers, data modelers or users know that there is temporal data in the data solution, many on the other hand do not know why and how they could or should use it in a meaningful way. It does not matter whether the person at FastChangeCoTM comes from a technical or a business background. In addition, many employees who work with data do not even know that bitemporal data exists at all and what possibilities are associated with it.
The requirements to store data historically and to create a bitemporal database for this purpose are manifold. The DMCE team can list many, some more, some less useful. One of their favorite reasons is, "Because we need this historization now...!"
To make this 'request' less frequent in use cases and more concrete, the DMCE creates guidance and introduces it with the one example:
A common request to implement bitemporal historization is that data from certain operational IT systems sometimes does not arrive in the correct chronological order.
This is also the first of the eight reasons for a bitemporal database:
Data occasionally arrives out of order
Reason #1: How can data not be delivered to the data solution in the correct chronological/temporal order?
Put simply, it is because, for example, a data delivery has failed for some reason or a streaming technology used does not primarily care about the temporal order of the data packets. Therefore, this has to be handled in the data solution, a temporal database, so that in the end all data is in the correct temporal order.
Store the 'reality' for past, present and future
Reason #2: FastChangeCoTM wants to save its 'reality' for the past, present and future.
This is the case at FastChangeCoTM, for example, when a department that sells products wants to create prices that should be valid only from a certain point in time in the future.
Or if a piece of information should 'update' the past, e.g. a retroactive salary change.
Corrections of the past
Reason #3: FastChangeCoTM wants to make corrections of the past.
This is the case at FastChangeCoTM e.g. if a department has received wrong data/values from external service providers. A later correction delivery contains the correct data/values. Thus, with a bitemporal database/historization, the past can be mapped correctly in time.
Time travel - ability to query across time
Reason #4: Subject matter experts want to view fact data from different time perspectives (As-Is, As-What, As-Of, ...).
FastChangeCoTM often involves structural changes. When planning for this, the subject matter expert can, for example, look at the sales/profit distribution in the past, in the present, or even in a future organizational structure.
Or simply track how a customer has moved through the different customer groups: New customer, VIP customer, existing customer, VIP customer, no customer.
Retroactive operations
Reason #5: The invoice must still be posted in the previous month.
This situation occurs at FastChangeCoTM when, for example, an invoice arrives on the 4th of a month that has an invoice date from the previous month and therefore must also be posted to the previous month.
With bitemporal data storage, this is easily possible without manual effort.
Decision Analysis
Reason #6: Traceability of whether a decision was 'right' or 'wrong' in the past.
In some branches of FastChangeCoTM the decision analysis is very important, e.g.: A decision was made a year ago, now a year has passed and it turns out: Nobody knows anymore why this decision was made a year ago, because the data that led to this decision is no longer available. It is no longer possible to trace the data on the basis of which the decision was made. With today's data, the decision might have been different.
Bitemporal data, on the other hand, can be used to travel back in time to check whether the decision was correct at the time.
Legal regulations
Reason #7: External influences such as laws and regulations
Laws and regulations can play an important role in deciding whether bitemporal historization is necessary.
Especially in the finance and insurance industry, there are many requirements for unchangeable, non-erasable storage of data.
Audit
Reason #8: Internal and external reviews / audits
For FastChangeCoTM's DMCE team, it is very important that they can rely on their data. Bitemporal historization of the data allows them to ensure that the correct data is available for each audit. This applies to external audits as well as internal complaints, such as data having changed. At any time, the team can prove and explain how, when and why data has changed over time.
Certainly, there are other reasons for bitemporal data besides the ones mentioned. The DMCE team is well aware of this. As always, it depends on the specific project, the company or even external influences, such as legal requirements, whether there are further reasons and whether they apply.
Each of the eight reasons for temporal data I will go into more depth and illustrate with examples. But not this time. More about that in one of the next articles in the series. You should definitely check back.
So long,
Yours Dirk