I wrote an article for TDWI about the ever increasing need for data curators in the worlds of business and IT. You can learn more about it in The Rise of the Data Curator.
One big takeaway from the article is the increasing importance of specialization within the world of data collection and processing:
“As companies become more sophisticated in their use of data to solve their most critical business challenges, they develop specializations in their teams to address each stage of the end-to-end process. Today these roles include individuals who reside in IT – data engineers – as well as those who sit in the business: data analysts and data scientists. In 2018, we will likely see more data curators, a new role that focuses on bridging the worlds of business and IT in terms of data analytics.”
Also, I’ve noticed an increasing trend in the struggles of maintaining efficiency when working with larger and more complex data sets:
“The back and forth between the business and IT can slow down the process [] due to a lack of common understanding of the data and the processing required to make it available for analysis. Data engineers have a deep understanding of the infrastructure and the formatting of the data but not of the data itself. Analysts and data scientists, on the other hand, have a deep understanding of the data but not the underlying systems and tools used to process it.”
