Today, companies want to keep data operations simple and appealing for business units. However, with increasing data volumes, data platforms are getting bigger and more complex. That situation has created a need for new experts to manage it. One of these new experts is the analytics engineer.
The analytics engineer is an expert working in the heart of data operations, between data analysts, data engineers/ DataOps, data scientists and MLOps. They clean, transform, test and deploy data to make it available for final users, allowing the latter to answer their questions independently. The analytics engineer has business expertise as well as technical expertise. The need for this combination is born from a gap between business teams and technical teams. Indeed, business teams often lack knowledge about the benefits of technological innovations. On the other hand, technical teams are focused on these innovations, thus getting estranged from business issues. The analytics engineer is here to bridge this gap.
The analytics engineer works on the BI chain as well as on the data science chain.
Regarding BI, they collaborate with data analysts to clean and transform data, optimize data models that are ready for consumption and document the data, thus accelerating the time-to-insight. On the other side of the chain, they collaborate with data engineers/DataOps. While the latter maintains the infrastructure and data movement, the analytics engineer, on the other side of the pipeline, ensures the movement and transformation of the data as well as the optimization of datasets, data models and data products, so the data can be correctly analyzed and visualized by data analysts. The analytics engineer also participates in the choice of transformation and operation tools, on which he has legitimacy.
Regarding data science, the analytics engineer can take on some of the data scientist’s burden regarding simple tasks of algorithm modelization, and help the data analyst on advanced BI analysis tasks. On the other hand, the analytics engineer falls within the MLOps methodology, and thus may use prepared datasets as well as models and algorithms already tested by the MLOps, all to answer business demands and relieve data analysts and data scientists.
For the analytics engineer’s work to truly bring value to the organization, it must be supported by a clear governance policy which precisely defines the roles of the experts to avoid any disruption.
With companies’ willingness to make increasing volumes of data available to increasingly demanding business units, the popularity of the analytics engineer should soar. This new role, halfway between technical and business expertise, can be taken on by a data engineer with an appetence for or training regarding the company’s business issues.