It is 2021 and the “Data Lake”, disparate datasets of organizations, are still evolving with a shift away from on-premises servers to the cloud at an astonishing pace.
Recent studies from Deloitte show the drivers for this are data modernization, cost, and security. The adoption is ongoing for organizations whether government, or businesses large and small.
According to Gartner, in 2020 “83% of the companies will be using cloud platforms and in 83%, 41% will prefer to use public cloud platforms.” In a recent Salesforce Webinar, they detailed how “over 60% of Canadian businesses were forced to accelerate their technology plans” around cloud technologies because of the pandemic.
What does this shift mean for the Data Engineer whose responsibility is this “Data Lake”?
The answer is keeping their certified data engineer skills up to date through learning and by embracing the new technology tools available, whether on-premises or cloud-based.