Feeling the Beat
by Kate Pavao
Google searches, Twitter feeds, and even Amazon sales ranks produce a lot of data—data that can be used to identify trends in real time and help business leaders get ahead. “There’s a revolution in data about society,” says Douglas Hubbard, author of Pulse: The New Science of Harnessing Internet Buzz to Track Threats and Opportunities (Wiley, 2011). Here Hubbard tells Profit what the pulse is—and how to find it.
of enterprise application spending will be cloud-sourced by 2015. (Source: IDC)
On the pulse: “One common feature in all of the predictive analytics and business intelligence tools is that they’re used in an entirely inward-looking way. They’re for looking at company data. But no matter how big your company is, it’s not nearly as big as the data on the internet. That is where the data about your customers and the economy and your competitors and views on products really resides.
Now that we’ve got a couple 100 million tweets a day, for example, this is a datasource. By using that data, you can actually tell what’s going to come out in a government report or a Gallup Poll much later.”
254 terabytes of data
As of September 2011, the U.S. Library of Congress has collected about —more than the U.S. Census produces in a decade. (Source: Library of Congress)
On privacy: “I’m talking strictly about data that people explicitly voluntarily reveal. Insofar as we are talking about a reduction in privacy, it’s only because people are volunteering it in the first place. When you tweet, you’re meaning to broadcast information to everyone.
Also, some of the data I’m talking about doesn’t even look at the behaviors of individuals. If I notice that all the books having to do with getting out of debt are now selling better on Amazon than they used to, that tells me about a macro trend—it doesn’t tell me anything about who’s buying those books. When I see that the Google searches on unemployment went up last month, that doesn’t tell me who’s unemployed. This is highly aggregated data. We’re looking at the big patterns.”
Number of data terabytes Facebook produces daily —more than the U.S. Census produces in a decade. (Source: Pulse: The New Science of Harnessing Internet Buzz to Track Threats and Opportunities)
On first steps: “Executives should embrace the idea of being awash in data. Bigger companies need to start gearing up on multi-disciplinary teams to maximize this stuff—not just statisticians or people familiar with predictive analytics tools, but sociologists, economists, and mathematicians.
Also, you need to modify internal decision-making processes to fully capitalize on this data. Because if you’re going to use your traditional decision analysis method where you wait until some trend becomes so obvious that it’s already written about in the press—or already apparent to you and your competition—then you might as well wait around for the same government reports and available data that everyone else does. If you’re really going to utilize data like this, you have to think about, ‘How do I change my decision-making process to fully optimize the use of real-time data that gives me macroscopic pictures of the world?’”