From Data Science to Data Engineering to Data Architecture

How is the focus of the industry is shifting from data science to data engineering to data architecture?

Many experienced data professionals today started their data journey with data science projects a few years back.

Initially, they struggled to execute these projects due to a lack of proper foundations.

So their journey into data engineering began when they undertook data engineering tasks to build foundations and infrastructure.

With the hype around data science, organizations spent lavishly on data science talent, hoping to reap quick benefits.

And most often, data scientists struggled with basic problems that their background and training did not address i.e. data collection, data cleansing, data access, data transformation, and data infrastructure.

These are problems that data engineering aims to solve.

And successful data engineering requires to be built upon solid data architecture.

Data architecture is designing the systems in an enterprise to fulfill its evolving data requirements after evaluating the trade-offs.

It makes data systems across the organization flexible where design decisions can be reversible so that data teams can strive for best-of-breed solutions.

Hence we can see that in the last few years, demand has shifted from data scientists to data engineers to data architects.

#datascience #dataengineering #dataarchitecture #analytics

Ankit Rathi is a Cloud Data Technologist, published author & well-known speaker. His interest lies primarily in building end-to-end data/AI applications/products following best practices of Data Engineering and Architecture.

If you have any questions or comments, click the "Go To Discussion" button below!