January 26th, 2016
In 2016 the continued proliferation of enterprise mobile application deployments, enterprise cloud infrastructure adoption, and the maturity of key Big Data processing capabilities will converge to produce a “perfect stew” that drives the adoption of data lake implementations. These new “integration architectures” are necessary to provide the capture, storage, protection and analysis of high-volume, media-rich data in the world of hybrid enterprise cloud environments.
IT analysts and IT industry advisory consultancies corroborate the supporting trends with their findings and forecasts.
Citing just a recent few from the Forbes “Roundup of Cloud Computing Forecasts and Market Estimates 2015”:
- “Globally, cloud apps will account for 90% of total mobile data traffic by 2019, compared to 81% at the end of last year (2014). Mobile cloud traffic will grow 11-fold from 2014 to 2019, attaining a compound annual growth rate (CAGR) of 60%. Source: Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2014–2019”.
- “57% of IT architects and tech professionals are running apps on the Amazon Web Services (AWS) platform today. Rightscale found that AWS, Microsoft Azure IaaS, Azure PaaS, Rackspace Public Cloud and VMWare vCloud Air are the top five public cloud platforms used in enterprises today. Source: RightScale 2015 State Of The Cloud Report”.
- “The more experienced an enterprise is with cloud computing, the more likely they are to rely on cloud-based apps and platforms to boost customer demand strategies (50% likelihood). Source: Mapping the Cloud Maturity Curve - The Economist Intelligence Unit Study (in conjunction with IBM)”.
And while the adoption of distributed cloud-based infrastructure and mobile application deployment grows, so does the capability and maturity of the “Big Data” software necessary to collect, protect and manage the resulting data streams. The adoption of commercial Hadoop distributions (Cloudera, Hortonworks, MapR) and their supported deployments on AWS and MS Azure, along with the emergence of Databricks (Spark) and Datastax (Cassandra / Solr) implementations of their respective Apache projects, portends a promising future for big data processing. (My disclaimer - I recognize that I have barely scratched the surface here in calling out promising examples of open source and commercial vendor offerings and interaction.)
The net of this being, that increasingly mature open source and vendor-supported software is available to any size enterprise to harness value from an increasingly data-rich world.
The drivers for the “4 Vs” of Big Data are in play, and combined with robust growth in mobile, IaaS and PaaS enterprise implementations, there must be complementary revisions to (any) enterprise IT’s data integration and data warehouse strategies. It seems likely that “data lake architecture” will be the banner under which the development and adoption of this next-generation data integration and analysis will be pinned.
(Enjoy the lake, don’t forget your towel.)