In green data, the goal is to reduce the mobilization of resources and thus the energy consumption incurred by the use of data. This starts at the choice of data to exploit and also includes governance and of course the choice and optimization of the data platform: data set optimization, ingestion optimization, but also methods to save on compute and storage. Indeed, eliminating redundant compute and storage is a good way to make a data platform greener. Let’s pay more attention to the compute. To lighten the latter, we can use methods like partitioning, bucketing, batch treatment or caching. It’s in that field that we accompany our clients, in the saving of computation and data scans without forfeiting data usage and exploitation by business teams.
How does our solution participate in the greenification of the data platform by reducing its carbon footprint?
In BI, aggregation is a major element to reduce useless computation, which are as superfluous as repetitive. Indexima’s goal is to plan ahead and automate them.
The solution first behaves like a proxy between the data visualization tool and the cloud data warehouse. It analyzes the queries and the users’ usage of the dashboards by breaking down the query patterns. The Analyser, a ML-based internal component, creates an aggregation layer in the form of materialized views in the source base (cloud data warehouse). These materialized views prevent the data from being scanned at each query. Only the relevant data is solicited, and the performance is thus optimized on the level of your tools as well as on the level of your resource consumption, allowing you to save on all fronts.
It’s just like when you buy groceries. It is preferable to make a list ahead of time and to avoid making several trips for each item. With that list, you save not only time, but also gas and car usage, etc. Indexima creates agregation indexes between your data sources and your data apps, thus optimizing your resource consumption without sacrificing your dashboard performance.
Conscious of the issue of data’s ecological impact, we accompany you in the construction and optimization of a data platform that is powerful as well as ecologically and financially sustainable.