Data and silos: Moving beyond the era of compromises
Today, decision making is becoming increasingly data-driven. The advent of business intelligence, also known as BI, raises the question of how companies should choose their approach.
To become data-driven, companies must make a number of compromises. “How much data do I need to leverage to match my technical capabilities?” “How will I allocate my teams’ time between data collection and analysis?” To add another layer of difficulty, the integration of many IT solutions within the organization has led to the emergence of data silos. In other words, a set of data that can only be accessed by one part of the organization, adding to the time it takes to process the data within the company.
How can we be data-driven despite this data siloing? Should we keep the compromise that forces us to strike a difficult balance? Given the wording of my question, my opinion leaves little doubt.
Between siloed data and the search for agility
- The days of unified enterprise computing are long gone. The explosion of SaaS offerings has had the notable consequence of creating data silos for each new solution added, leading to problems such as a lack of visibility for IT teams on the true number of applications installed or a burdening of processes. Tools optimized to solve specific problems, such as treasury or human resources, are becoming more and more numerous, leading to the unfortunate tendency to multiply copies of Excel files in organizations. As a result, processing costs quickly become exorbitant due to the numerous data extractions and re-injections into other applications.
- Companies can hardly imagine a world without silos: they are currently too deeply embedded in the systems to disappear with a wave of a magic wand. But the problem of sustainable silos continues to create a fine balance between data fragmentation and the search for agile, rapid reconciliation at lower cost.
- On a larger scale, these hermetic silos deprive the enterprise of a holistic view of all its data, preventing it from realizing the full potential of a data-driven strategy. In fact, Gartner reported before the pandemic that 87% of companies have a low level of maturity in business intelligence and analytics, accumulating silos and slowing down their projects. We observe that the trend has changed little since then. But dealing with the existence of silos doesn’t mean we can’t start to weaken them.
How to bypass or break down silos?
- There are tools available nowadays to access siloed data, make queries or even unify silos more efficiently and easily using a hybrid approach. The DataOps approach aims to erase technological boundaries, and therefore silos. Should companies turn to a DataOps approach by promoting collaboration between stakeholders in the data processing chain?
- In addition, there are also several technical solutions capable of virtually breaking down these data silos and releasing the data’s potential to the right people. Data virtualization blurs the boundaries of silos by providing direct access to operational systems in real time, drastically reducing processing times. The data fabric takes this approach a step further by breaking down silos through data capture to cross-reference and connect data.
- While the technological answers depend on the business strategy in place, the level of technological maturity and the use cases, the objective remains the same: to offer a unified view of the data, a secure access that complies with the regulations in place and manipulation possibilities to accelerate the visibility of the data to the business and data specialists.
What if the real problem was precisely this search for compromise?
- Behind the technical challenges, the real problem behind the implementation of data-driven strategies is the acceptance of compromise. Companies find themselves having to choose between good performance on data that is several days old and long waiting times on fresh data. Companies have become accustomed to these trade-offs, yet the technology exists today to bring together the best of both approaches.
Source: JDN, Florent Voignier from Indexima