Oracle’s User Experience Thinking Behind Big Data: Endeca Information Discovery
Author: John Fuller, Consulting User Experience Designer for Endeca
What is Endeca Information Discovery?
Oracle Endeca Information Discovery (EID) is a product that performs a complementary role to other enterprise business intelligence (BI) systems by adding capabilities for addressing questions not easily answered by many of those other tools.
Because EID is designed from the ground up around a hybrid search and faceted navigation model of finding things, it enables an ad-hoc approach for users to isolate and present the relevant sets of data using a familiar interface. But beyond the power it affords for working with data in analysis modes, it also has many features reminiscent of portal-like tools in that it enables users to very quickly assemble applications and workspaces by adding charts, tables, summarizations, and the like that will ultimately become the lenses for the insights they're after.
The data that fuel these applications can come from a range of sources—well-defined and modeled enterprise data, jagged or schema-less data such as social content, or completely unstructured blocks of text that you would find in email messages, PDF documents, or files scraped from a document repository. We then bring any and all of these sources together into a coherent experience so that the information can be investigated, ultimately leading to new insights.
Who is Oracle designing Endeca Information Discovery for?
We're trying to help people answer big questions with a tool that combines the kinds of work typically handled by several highly skilled information workers: the people who manage and curate data, the people who build the applications for others to use, and the people who do the analysis work--either defining the problems that need to be answered or doing the analytical legwork of answering the questions.
What we're trying to do is enable individuals to do most--or all--of this work themselves.
That means defining a future where more individuals can go after the tough questions that previously required a small army to deal with, while lowering the cost to do so.
What kind of user research is Oracle doing to understand these users?
To understand what that future might look like, we spend a lot of time talking with the people who occupy these various roles today to understand what kinds of things they do on a daily basis, how their job roles are defined, how self-motivated their projects are, what tools they use, what tools they wish they had, and how they work together with their colleagues to make it all happen.
We started out with a fairly well-defined idea of what we wanted to accomplish with EID and the people we're making the product for. Research about the challenging problems that exist today and that will likely exist in the future gave us a basic framework for the users. However, to really validate what you think you know, it's important to get out into the field to meet real people. To address this need, we did a series of interviews with different people from various roles in the BI space, with a particular emphasis on the analysts who have some portion of their roles allocated to answering self-motivated or complicated questions.
The interviews were framed by a common discussion guide, but we left a lot of time for open-ended conversation to give participants an opportunity to talk about what they really wished they could do but find problematic--either because the tools are complex or don't exist, the data is too big or difficult to wrangle, or the challenges of organizational and human factors are too great.
We've put together a user framework that works at a few levels of detail. First, there's a segmentation model. We use this model to describe the roles of users, roughly based on how their skills, needs, job roles, and activities are divided today. Segmentation models are a really useful tool for defining the boundaries of the space that we're addressing, understanding the different kinds of people that operate within this space, and how they relate to each other. On top of this model, we've built a number of personas. These personas are brief biographies that are composites of the real people who we met in the field.
In our segmentation model, we first divide our users into two groups: the enablers and the sense makers. The enablers are the people who help bring data into an application experience so that the sense makers can investigate the data, synthesize information, and ultimately arrive at the insights.
Within each of these segments, we have further breakdowns that cover the range of admins, managers, information consumers, architects, and distinct analyst types. The personas built on top of the segment model tend to span segment categories. This reflects that we're designing a product for a future where people will be enabled to take on more responsibilities themselves.
The unique problems tackled by big data: the three Vs
I should clarify that EID is not solely intended to help users with big data problems, but there are aspects where I think we really nail it. We all know that data is big, and that it's getting bigger at an alarming pace, but to build a tool that helps users deal with big data, you need to understand the nature of the unique problems that big data presents. We roll these problems up into categories that we call "the three Vs.”
Volume: More records exist today than at any time in the past, and we're creating more all the time. Almost everything that humans, organizations, or machines do creates some kind of a digital trail, and more of it is being captured than ever before.
Velocity: As more people join the digital collective and more of our experiences and activities are funneled through digital experiences, we accelerate the pace of data creation. Ripples travel fast in the digital pond, and there's a desire for businesses to be more responsive in identifying and acting on the opportunities that this new reality presents. By focusing our efforts on a tool that lowers the time and cost for gaining insights from data, we're working directly on ways to help our users address this increased pace of data creation.
Variety: As if all this incredibly rapid scaling of data weren't enough, there's the challenge of unifying it in some way to make sense of it. Data is stored in all kinds of systems, each one with its own formats, schemas, and protocols to get the data in or out. And if you look beyond those challenges of storage and availability, there's the knotty problem of making sense of the written word, and how to summarize large corpuses in meaningful ways. We must understand the people, places, and things that we talk about and the sentiments that we express about them in the richest, most meaningful way possible.
It's this third V, variety, that I think those of us working on EID are particularly focused on when we talk about big data problems, and for a lot of the reasons just mentioned, I think it's the most interesting one.
One of our biggest goals is to enable users to pull in the relevant data--in whatever format it comes--in order to make sense of it, and further, to be able to combine those disparate types of data together so that we can understand broader landscapes of relatedness.
Providing this capability to individuals at a lowered cost is what we think will trigger a sea change when we're all swimming in massively scaled, rapidly accelerating and wildly divergent forms of data.
What is Oracle working on next, related to UX and Endeca Information Discovery?