How do I organize my data?

With 20years of expertise in data technology, Black Tiger is regularly confronted withthe difficulties faced by businesses in their battle with their own data.

This battlecould be summed up as ‘How can I put my data in order?’ to get more value outof it and more freedom in how it is used.

We can reasonably divide the difficulties faced by companies into 4 categories:

·       Data accessibility

·       Traceability

·       Dataquality

·       Makingthe most of it to work efficiently

1.           Data accessibility is the firstchallenge for businesses, due to the complexity of their existing systems:

·       Dataspread across a large number of heterogeneous systems, both in terms ofinfrastructure (public cloud, private cloud, on premise) and technologies (oldlegacy technologies vs. new applications, data lake, BI, etc.), backed up bymore ‘modern’ technologies, document databases, graphs, index engines, managedservices, ....

·       Datasiloed by application, and therefore duplicated

·       Differenttypes of data, from structured to unstructured

·       Problemsidentifying the ‘master data’, its owner and its experts

·       Theidentification, and even more so the use, of data that can be ‘joined’ betweenheterogeneous systems is a very important issue.

2.           Data traceability is a seconddifficulty since, in its absence, and apart from the lack of compliance thatthis generates, it does not allow the company to be in control of its data.Lack of control prevents the company from understanding inconsistencies anderrors and, as a result, slows down the correction process considerably.

·       Onceit has been accepted that transformations, of varying degrees, are necessary inorder to be able to use its data effectively, it becomes essential to keep veryprecise records of these transformations. Not only (in some cases) forregulatory reasons (e.g. GDPR), but also for the very operational needs oferror recovery, rollback, changes to models and rules, etc.

·       Traceabilityis only fully achieved when the transformations carried out during ingestionare effectively usable, in particular by enabling searches, automaticprocessing, etc.

·       Tracingmust also be possible beyond ingestion, once the data has been made availableto other applications.

3. Dataquality is a third difficulty which seems to be obvious to everyone and adiscovery for no-one. It remains a definite problem that companies struggle tosolve.

·       Dataquality is always a challenge that can quickly become overwhelming fornon-specialist companies...

·       It'salways a delicate balance between totally generic issues and totally specificparticularities.

·       Thechallenge for companies is therefore to find the right balance between what isa completely standard element, where it is essential not to go out of our wayto ‘reinvent the wheel’, and what is truly specific to my sector, to mycompany, where we need to focus our energy so as not to get bogged down in anoverly generic approach that does not correspond either to my operationalreality or to my objectives.

·       Thisbalancing act is complicated when you're not a specialist in the field, andwhen you don't have the technology to easily marry these two seeminglycontradictory approaches!

·       Anotherchallenge facing companies, which is always at least underestimated and atworst completely ignored, is the gap that exists between the reality of dataand its ‘idealized’ representation by the business...

·       Finally,another challenge, which stems in part from the previous one, is to be able totreat the mass and the exceptions differently. This is something that is even moredifficult for non-specialists to grasp because it depends on the use case forwhich the data is intended!

·       Aneffective solution must automatically process as much data as possible,reserving the correction of exceptions for business operators. In such cases,it must also be possible, where possible, to reuse the correction made toautomatically process future exceptions of the same nature.

4.           The final difficulty is the effectiveuse of data. At Black Tiger, this can only be solved by providing companieswith a 360° view of suppliers, contracts, partners, products, customers,employees, etc. Such an approach can only be achieved thanks to a technologythat guarantees the 7 most challenging and fundamental data treatments:Accessibility, integration, quality, de-duplication, aggregation, traceabilityand compliance.

·       Thekey concept when it comes to exploiting data is adapting processing to usage.In the world of big data, there is no such thing as universallycorrect/deduplicated/aggregated/compliant data, but as many data sets as thereare use cases! An approximation, a rounding-off, etc. can be perfectlylegitimate for a given use case, but catastrophic for another. Likewise, havingconsent for an email may be necessary for a given purpose, but not at all foranother.

Businessesare confronted by a market of heterogeneous solutions, blurring the path tofollow and complicating the project process.

The sheernumber of offerings on the market drowns customers in a sea of highlyfragmented technological possibilities, making it difficult for them to come upwith an overall, coherent analysis of their problems.

Moreover,technological diversity forces customers to try and solve their problems byadopting several technologies, making the project difficult, costly andsometimes never-ending.

In the end,Black Tiger very often finds itself having to structure all of this in order toguarantee effective data deliverability.