New fast data technology narrows the gap between reporting and action.
by Alison Weiss
The world of business is speeding up. Disgruntled tweeting customers want service right now, a competitor has discovered a new, more streamlined way to deliver goods, and your IT department must improve its green footprint—yesterday. Enterprise leaders have always faced the challenge of responding to customers, maximizing new opportunities, remaining competitive, and ensuring decisions are prudent. But now, there isn’t a moment to waste.
New technologies and intelligent devices—from smartphones to smart sensors—are generating more and more data every second. And business decision-makers need to find ways to manage this huge information influx and quickly analyze and act on the information in real time.
Until now, this has been a tall order. In fact, according to a January 2012 survey of 247 executives on data management for business intelligence conducted by the Aberdeen Group, 53 percent contend that too much critical information is delivered too late. Further, a July 2012 Oracle study found 67 percent of executives reporting that gaining intelligence from their data is a top objective. And 93 percent of executives believe their organizations are losing revenue—as much as 22 percent annually—because they are not fully leveraging the information they collect.
Fortunately, there are new strategies for managing the dynamic data moving through the enterprise. To more easily conceptualize this in-flight data, Tony Baer, principal analyst at Ovum, recently coined the term fast data. “If big data is about the three Vs—volume, velocity, variety—fast data really is focused on the second V, velocity,” he says. “It’s a subset of big data focused on acting on the volume and variety of data in real time and conducting analysis to respond in the moment.”
Baer is quick to point out that while the term fast data is new, the financial services industry and Wall Street houses have taken advantage of fast data solutions for years. But the solutions have traditionally been very specialized—with expensive hardware and complex software. However, hardware and memory are both less costly today, and with the availability of low-cost mobile network connectivity and greater processing power, fast data benefits are becoming more accessible. Currently, fast data solutions can be used for fraud detection, real-time credit scoring, and location-based digital signboards. “The cool thing about fast data,” says Baer, “is that it’s not unique to one vertical industry. It’s important to many verticals.”
Paul Daugherty, CTO at Accenture, believes many of the business models generating significant new revenue are dependent upon systems that make decisions about a customer based on user profile, buying history, and real market dynamics—all in a matter of milliseconds. This, he believes, represents the potential for fast data. “The point of data velocity is, how do you take advantage of narrowing windows of opportunity where you can make a difference with a business decision?” says Daugherty.
Customer experience is a clear first option for business leaders looking to take advantage of fast data. Empowered by smartphones and mobile applications, customers are connected all the time. And with social media platforms such as Facebook and Twitter providing new avenues for instant customer contact, opportunities for location-based marketing, and sources for social metrics to mine for customer analytics, businesses have many more opportunities to gain—or lose—customers.
percentage of executives who believe their organizations are losing revenue because they are not able to fully leverage the information they collect (Source: From Overload to Impact: An Industry Scorecard on Big Data Business Challenges, an Oracle research study, July 2012)
“People want information much faster,” observes Hasan Rizvi, executive vice president of Oracle Fusion Middleware product development. “When you’re talking to a customer on the phone, you can’t wait a week to get her data. You need it right then and there and for it to be accurate up to that moment so you can make a much better engagement with your customer.”
As mobile network costs decrease and data speeds accelerate, intelligent devices—including smartphones, smart meters, and even industrial machines—are starting to proliferate, making up an Internet of Things. These smart devices are getting smarter, so they are able to not only capture and sense the information from their environment, but also to share this information very easily from machine to machine (M2M). In addition, having the integrated flow of information from device to IT data center where applications reside is imperative to derive benefits from the M2M revolution. “The data is out there,” says Baer, “and it’s begging to be tapped.”
The key to managing the huge quantities of data generated by intelligent devices is to understand that M2M data has a short shelf life. If you don’t access it soon enough, it loses value. So, if a sensor in a city center is picking up information about a potential traffic accident, it must be acted upon quickly because the data will be completely useless even a week from now.
According to Ed Zou, vice president of Oracle Fusion Middleware product management, the right technology needs to be put into place not only to harvest M2M fast data but also to filter it in some way so that only the most important information is available to be acted on in real time. Otherwise, tens of thousands of devices capturing data every second and all the fast data being sent unfiltered could cause not only a deluge at the data center, but an overwhelming amount of traffic on the network. “Oracle perhaps is one of the first to offer a product with event processing capabilities to enable fast data right on devices themselves,” says Zou.
M2M scenarios may have a definite science-fiction ring to them, but Oracle customers are already benefitting from fast data. For example, some healthcare customers are reducing costs and improving results by transitioning from hospitals to in-home care. Others are using M2M solutions for constantly monitoring medical device data to save lives. In manufacturing, customers are implementing M2M solutions to track the location and condition of assets—ultimately optimizing asset management and improving efficiencies. “The potential is clear,” observes Zou. “The value of applying a fast data strategy has become more apparent across many industry segments.”
The point of data velocity is, how do you take advantage of narrowing windows of opportunity where you can make a difference with a business decision?
In another example, when decision-makers at a utility company adopted smart meters to wirelessly automate meter reading and management, they implemented a fast data strategy. The objective: to better filter and correlate events from smart meter systems to protect back-office systems during peak loads and severe weather events. The fast data solution can generate alarms that help determine when it makes sense to switch from one system to another, as well as streamline the switching process.
Now, the company’s M2M platform can analyze all smart meter alarm events before they reach the outage management system, and the system can also filter alarms associated with known outages—isolating and identifying new events. The M2M solution can also analyze all enterprisewide GPS location reports generated by dispatch crews to identify which crews are closest to the areas requiring field service, shortening response time. M2M technology has enabled the utility to improve overall energy management by reducing the time it takes to identify and respond to a business event.
“It’s a paradigm shift, really,” says Rizvi. “Since there are a vast number of M2M devices sensing and collecting large volumes of data, they must be able to process data that’s happening on the fly, correlate that information at high speed, and then send the right event to the right data center. And now we have the ability to take action quickly with the information when we get it.”
Predictive analytics represent another fast data paradigm shift. Drawing from huge data sets that are current up to the minute, fast data–powered predictive analytics are likely to be much more accurate than those from traditional business intelligence systems. In the case of a manufacturing floor where managers want to reduce machine failure, M2M fast data can predict outcomes based on predefined system events. So if machine A communicates information about an event that is likely to put a part at risk, in real time, machine B gets the message to turn off to prevent the outcome from occurring. “It’s not just predicting what’s going to happen,” says Zou, “but taking action very quickly without latency or waiting.”
There are many technologies that support fast data. Baer lists his fast data ingredients: in-memory data grids and in-memory database technologies, event processing and high-speed transaction engines, data integration, replication and transformation technologies, and real-time and predictive-based analytic tools. Daugherty points to advances in technologies such as Oracle Exadata, Hadoop, and in-memory processing as critical fast data tools.
From embedded Java on devices and Oracle Fusion Middleware to Oracle Applications and Oracle engineered systems in the data center and big data analytical tools, Oracle provides a practical Device to Data Center platform that makes it easy for organizations to collect, store, manage, and efficiently analyze fast data. (See sidebar, “The Go-to List for Oracle’s Fast Data Solutions.”)
Decision-makers will have to decide where they want to begin and which fast data tools best fit the needs of their individual enterprises. The good news is that fast data tools can be implemented incrementally to augment what is already in use for data warehousing and analytics. “Fast data,” Baer advises, “is not a one-shot wonder.”
As with any other business transformation efforts, before making technology decisions, it’s imperative to honestly access an organization’s pain points and identify which business domains would most benefit from fast data. Are supply chains too slow? Are fraud detection problems an issue?
Equally important, decision-makers must also carefully examine how their organizations will actually convert fast data into actionable results. According to Rizvi, this step can be daunting, but it is the key to maximizing the benefits of fast data. “To enable new products and services,” says Rizvi, “organizations need to understand fast data and use it to figure out what their customers want.”
Alison Weiss is a freelance writer in the San Francisco Bay Area and a frequent contributor to Profit.