Big Data in the Cloud

Data in the Cloud

Big Data and Cloud: A Match
Made In Heaven

Paul Sonderegger,
Big Data Strategist, Oracle @PaulSonderegger


Cloud and Big Data are two of the most significant technology developments of recent years; and they complement each other beautifully

Intel

Large enterprises are waking up to the realities of data capital. Data is not just a record of what happened, it’s a raw material for creating new digital products and services.

This means companies are in a race to create unique data capital assets through new mobile apps, wearable tech, and other intelligent, connected devices. Firms are also trying to figure out how to use this data capital to create unique value, whether that’s innovative digital products and services or increased efficiency rivals can’t match.

Paul Sonderegger

Paul Sonderegger, Big Data Strategist

This is what big data is really about – the capture and use of more data in more daily activities as a way to gain competitive advantage. And this emerging competition requires a new kind of computing — in which the cloud plays a crucial role.

But for large enterprises in particular, new big data and cloud technologies must work seamlessly together, as well as with existing technology. Big data and cloud have the potential to be a match made in heaven, but the devil is in the details.

Consider first the technology required to do big data right. Capturing and keeping new kinds of data from smart clothing, location-aware mobile apps, and connected drones requires a family of data management technologies.

 Firms are also trying to figure out how to use this data capital to create unique value, whether that’s innovative digital products and services or increased efficiency rivals can’t match. 

For example, NoSQL databases are great for catching real-time data streams from sensors. Hadoop clusters excel at holding enormous volumes of diverse data and documents cheaply. Relational databases provide rock-solid reliability for transactional data where precision and authority are paramount.

But capturing and keeping data in its original format — what we call data equality — is just the beginning. Companies also need data liquidity – the ability to get the data you want into the shape you need with minimal time, cost and risk. This requires a variety of big data analytics.

For example, data scientists need development environments where they can use their preferred tools like Python, SQL, and R, the open-source statistical package, to crunch new combinations of data. Managers, unlike data scientists, need point-and-click interfaces to explore data mash-ups so they can ask new questions and get new insights instantly. And ubiquitous dashboards must deliver analytics based on data from Hadoop and NoSQL, as well as the warehouse.

 Big data and cloud have the potential to be a match made in heaven, but the devil is in the details. 

So where does cloud come into this mix? As with applications, cloud can be a convenient way to get the capabilities the business wants without going to IT first. This works well for big data pilots and skunkworks projects. Just spin up a Hadoop cluster in a third-party cloud, add in some data and start experimenting.

But how do you take these big data cloud capabilities to production level? The real challenge is to incorporate big data cloud capabilities into overall enterprise architecture.

 The real challenge is to incorporate big data cloud capabilities into overall enterprise architecture. 

For example, an aircraft manufacturer who wants to offer in-flight monitoring of mechanical systems might store the sensor data in a cloud-based NoSQL service so that it can scale the system up on holiday travel days, and back down afterwards. A bank providing algorithmic fraud detection may deliver this capability via the cloud for similar seasonal demand spikes. A retailer relying on SaaS HR and payroll apps will likely run its employee analytics out of the cloud as well.

The answer is hybrid-cloud big data architecture. The airplane maker, the bank, and the retailer all have existing on-premise application and analytic infrastructure.

Oracle’s approach is to create private cloud technology with the same architecture as the Oracle Public Cloud, so companies can move big data management, integration, and analytics back and forth between the two.

 Oracle’s approach is to create private cloud technology with the same architecture as the Oracle Public Cloud, so companies can move big data management, integration, and analytics back and forth between the two. 

Take Oracle’s big data management system. Its main components are Cloudera Hadoop running on an Oracle Big Data Appliance and Oracle Database 12c running on an Oracle Exadata Database Machine. The two are hooked together through super-fast Infiniband networking with Oracle Big Data Connectors and Big Data SQL. This lets you move data back and forth between Hadoop and the warehouse, or query the two as if they were one system without having to worry about where the data is. You can get this same set of capabilities on the same architecture from Oracle Public Cloud. This is the only hybrid-cloud big data architecture that lets you move big data workloads from cloud to on-premise and vice-versa, as needed.

Big data and cloud are two of the most disruptive technologies to hit enterprise computing since the internet itself. Together, they can change the way companies compete, as well as the future of enterprise computing.


This Big Data article is brought to you by Oracle and Intel®.

Intel® and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.