GCP allows Data Engineers to effectively manage the data pipeline with tools like Facets for analysis. It does not replace experts, but offers a centralized environment. Aqsone is exploring its use to evaluate its performance and architectural benefits in its internal projects.
Aqson
Knowledge Lunch July Session
Each month, during a KLS (Knowledge Lunch Session), an Aqsone employee presents to all employees a technical subject on which he has worked or is in the process of training.
For the month of July, Matthieu Vinette, Data Engineer presented to us the advantages that Aqsone could gain from working with the Google GCP platform.
The platform
Indeed, this platform offers Data Engineers the ability to create and manage the entire Data Pipeline, from the ingestion of raw data to the provision of cleaned data to Data Scientists.
For example, GCP provides advanced and interactive data and model analysis tools like Facets (open-source). Facets allows you to visualize your data, characteristic by characteristic, and even allows you to work with images intuitively.
Such tools are a real plus in accelerating the process of data cleaning and de-biasing.
The job of Data Engineer doomed to be replaced?
Absolutely not!
GCP does not spell the end of Data Engineers. On the contrary, it offers them a new environment to understand which centralizes all the tools essential to their work.
We intend to use it as a data science platform in the work of the Aqsone Lab in order to be able to concretely measure, on internal projects, its possible contributions, whether in terms of performance gains or by facilitating the architecture of our projects.
A must see
