I now am a strong advocate to using Big Data as a competitive tool for business. I also I think Australian IT companies can readily create a market for their expertise to give Philippine business the capability to enable them to use Big Data for their business needs. Watch this space.
How to make big data work for you
Big data and analytics have rocketed to the top of the corporate agenda. Executives look with admiration at how Google, Amazon and others have eclipsed competitors with powerful new business models that derive from an ability to exploit data.
They also see that big data is attracting serious investment from technology leaders such as IBM and Hewlett-Packard. Meanwhile, the tide of private-equity and venture-capital investments in big data continues to swell.
The trend is generating plenty of hype, but we believe that senior leaders are right to pay attention. Big data could transform the way companies do business, delivering the kind of performance gains last seen in the 1990s, when organisations redesigned their core processes.
In our work with dozens of companies in six data-rich industries, we have found that fully exploiting data and analytics requires three mutually supportive capabilities. First, companies must be able to identify, combine and manage multiple sources of data. Second, they need the capability to build advanced analytics models for predicting and optimising outcomes. Third, and most critical, management must possess the muscle to transform the organisation so that the data and models actually yield better decisions.
1. Choose the right data
The universe of data and modelling has changed vastly over the past few years. The sheer volume of information, particularly from new sources such as social media and machine sensors, is growing rapidly. But mastering that environment means upping your game, finding deliberate and creative ways to identify usable data you already have and exploring surprising sources of information.
Source data creatively
Often companies already have the data they need to tackle business problems, but managers simply don’t know how the information can be used for key decisions. Operations executives, for instance, might not grasp the potential value of the daily or hourly factory and customer-service data they possess. Companies can impel a more comprehensive look at information sources by being specific about business problems they want to solve or opportunities they hope to exploit.
Managers also need to get creative about the potential of external and new sources of data. Social media are generating terabytes of non-traditional, unstructured data in the form of conversations, photos and video. Add to that the streams of data flowing in from sensors, monitoring processes and external sources that range from local demographics to weather forecasts.
One way to prompt broader thinking about potential data is to ask, “What decisions could we make if we had all the information we need?” Using that logic, one shipping company improved the on-time performance of its fleet by tapping specialised weather forecast data and live information about port availability that it hadn’t realised were available.
Get the necessary IT support
Legacy IT structures may hinder new types of data sourcing, storage and analysis: many were built to deliver data in batches, so they can’t furnish continuous flows of information for real-time decisions.
Business leaders can address short-term big data needs by working with CIOs to prioritize requirements. This means quickly identifying and connecting the most important data for use in analytics, followed by a cleanup operation to synchroniae and merge overlapping data and then to work around missing information.
Such short-term tactics may lead companies to vendors that focus on analytics services or emerging software. New cloud-based technologies may also offer ways to scale computing power up or down to meet big data demands cost-effectively.
2. Build models that predict and optimise business outcomes
Data are essential, but performance improvements and competitive advantage arise from analytics models that allow managers to predict and optimize outcomes. More important, the most effective approach to building a model rarely starts with the data; instead it originates with identifying the business opportunity and determining how the model can improve performance.
Unfortunately, not all model building follows this course. One approach that gets inconsistent results, for instance, is simple data mining, which provides little benefit if managers can’t effectively use the correlations to enhance business performance. A pure data-mining approach often leads to an endless search for what the data really say, but we have found that such hypothesis-led modelling generates faster outcomes and also roots models in practical data relationships that are more broadly understood by managers. Remember, too, that any modelling exercise has inherent risk. Although advanced statistical methods indisputably make for better models, statistics experts sometimes design models that are too complex to be practical. For example, a predictive model with 30 variables may explain historical data with high accuracy, but managing so many variables will exhaust most organizations’ capabilities. Companies should repeatedly ask, “What’s the least complex model that would improve our performance?”
3. Transform your company’s capabilities
Many companies grapple with a mismatch between the organisation’s existing culture and capabilities and the emerging tactics to exploit analytics successfully. New approaches don’t align with how companies actually arrive at decisions, or they fail to provide a clear blueprint for realizing business goals. Tools seem to be designed for experts in modelling rather than for people on the front lines, and few managers find the models engaging enough to champion their use. Bottom line: Using big data requires thoughtful organizational change, and three areas of action can get you there.
Develop business-relevant analytics that can be put to use: Many initial implementations of big data and analytics fail simply because they aren’t in sync with the company’s day-to-day processes and decision-making norms. Conversations to ensure that actions being taken complement existing decision processes are crucial and help companies to reach their ultimate goals.
Embed analytics into simple tools for the front lines: Managers need transparent methods for using the new models and algorithms on a daily basis. The key is to separate the statistics experts and software developers from the managers who use the data-driven insights.
Develop capabilities to exploit big data: Even with simple and usable models, most organisations will need to upgrade their analytical skills and literacy. Managers must come to view analytics as central to solving problems and identifying opportunities – to make it part of the fabric of daily operations. Efforts will vary depending on a company’s goals and desired time line. Adult learners often benefit from a “field and forum” approach, whereby they participate in real-world, analytics-based workplace decisions that allow them to learn by doing.
The era of big data is evolving rapidly, and our experience suggests that most companies should act now. But rather than undertaking massive overhauls of their companies, executives should concentrate on targeted efforts to source data, build models, and transform the organisational culture. Such efforts may soon become decisive competitive assets.
Dominic Barton is McKinsey’s global managing director, based in London. David Court leads the firm’s advanced analytics practice, based in Dallas. Harvard Business Review, © 2012 Harvard Business School Publishing Corp.