Big Data – still sexy, and getting more exciting
By Martin Ellingsworth, President, Salt Creek Analytics
Like a thriving nightclub, a refined wine bar, a smoke filled scotch den, or a corner of a quiet restaurant, everyone has his or her own definition of sexy and exciting.
When it comes to Big Data, the same is true. From factory systems to city traffic grids, from jet engines to genomes, from nano-materials to the Internet of Things, from supply chains to omni-channel customer personalization, everything now comes with data. Watching all these data interact over time, people, places, and things, and then predicting outcomes and next best decisions is sexier and more available than ever before. Now it is seamless, simplified, and synchronized. What is the same in all of the diverse data above is that they require a mix of people, process, and technology to capture the essence and gain the value of the Big Data effect across an organization – where becoming digital and data driven is cultural.
Cheaper, faster, better is where things are hot for technology. Open source software, open data standards, in-memory data management, distributed code execution, and the use of ensembles of algorithms makes it look like everyone is doing big data all the time for free. Like many things in life, there are more people talking about it than doing it, so you should make your own plans to use what you have and leverage all the help you can get.
A primary way to use what you have is dumping your data into a magnetic reservoir and then creating data features and models to simulate your processes and decisions. Once you can mimic the past, you are ready to try to shape your future. Mimicking your past gives you confidence that your data reflect truth, and that trust in data is the start of gaining traction towards a data driven culture.
Leveraging your digital assets is the start of Big Data. New capabilities are required to convert analog assets to digital assets. This is the first place many go. Features from text, images, sound, and even video are being created and associated with people, places, devices, and things, so that profiles and metadata can be used as the building blocks of new classes of analytics. Don’t worry if you can’t build this yourself, many platforms are arising to make this easy and cheaper.
Modern software on commodity hardware has gotten rid of much of the drudgery and repetitious hard work in blending volumes of a variety of data changing at different velocities. New data blending choices strip most of the time and expense out of the front end of every analytic project and put agile capabilities nearer the front lines of business experts.
New reporting and visualization tools, social network mappers, and open source geospatial engines create immense sex appeal in showing big data in action. It is easier than ever to hook up your data to a compute platform on a cloud. It is getting even easier to blend in analog-to-digital assets, third party data, and analytic scores/indices to form powerful insights that you can then watch interact with your business operations.
The waiting and wanting for aha-moment results are still distinctly unsexy. Grinding out all possible interactions to find a one-in-a-million truly new insight means seeing almost a million unsuccessful iterations.
Regardless of the hopes of deep learning, this work takes people with passion and perseverance. The events and moments of Big Data excitement themselves are pre-staged with hours upon hours of planning, permissions, procurement, programming, and processing. Despite all the data, it still takes good business questions to formulate great business solutions.
The Big Data aha moments come from new capabilities in dealing with data and from actionable insights that improve your business. Every stakeholder now has a chance to get what he or she wants if they can create a culture of analytics, which attracts talented people.