Oracle Usable Apps | Applications User Experience Simplicity, mobility, extensibility
Oracle’s User Experience Thinking Behind Big Data: Endeca Information Discovery


John Fuller

Author: John Fuller, Consulting User Experience Designer for Endeca
Sept. 10, 2013

Editor’s Note:
John Fuller is an expert in big data and user experience. We asked him to write about his particular experience around Endeca Information Discovery, what he and his team members found during their research, and how they considered the user experience as they built solutions based on those research findings.

Related Links

What is Endeca Information Discovery?

Oracle Endeca Information Discovery (EID) is a product that performs a complementary role to other enterprise business intelligence (BI) systems by adding capabilities for addressing questions not easily answered by many of those other tools.

Because EID is designed from the ground up around a hybrid search and faceted navigation model of finding things, it enables an ad-hoc approach for users to isolate and present the relevant sets of data using a familiar interface. But beyond the power it affords for working with data in analysis modes, it also has many features reminiscent of portal-like tools in that it enables users to very quickly assemble applications and workspaces by adding charts, tables, summarizations, and the like that will ultimately become the lenses for the insights they're after.

The data that fuel these applications can come from a range of sources—well-defined and modeled enterprise data, jagged or schema-less data such as social content, or completely unstructured blocks of text that you would find in email messages, PDF documents, or files scraped from a document repository. We then bring any and all of these sources together into a coherent experience so that the information can be investigated, ultimately leading to new insights.

Oracle Endeca Information Discovery
This example is from an Oracle Endeca Information Discovery health care demo.

Who is Oracle designing Endeca Information Discovery for?

We're trying to help people answer big questions with a tool that combines the kinds of work typically handled by several highly skilled information workers: the people who manage and curate data, the people who build the applications for others to use, and the people who do the analysis work--either defining the problems that need to be answered or doing the analytical legwork of answering the questions.

What we're trying to do is enable individuals to do most--or all--of this work themselves.

That means defining a future where more individuals can go after the tough questions that previously required a small army to deal with, while lowering the cost to do so.

What kind of user research is Oracle doing to understand these users?

To understand what that future might look like, we spend a lot of time talking with the people who occupy these various roles today to understand what kinds of things they do on a daily basis, how their job roles are defined, how self-motivated their projects are, what tools they use, what tools they wish they had, and how they work together with their colleagues to make it all happen.

We started out with a fairly well-defined idea of what we wanted to accomplish with EID and the people we're making the product for. Research about the challenging problems that exist today and that will likely exist in the future gave us a basic framework for the users. However, to really validate what you think you know, it's important to get out into the field to meet real people. To address this need, we did a series of interviews with different people from various roles in the BI space, with a particular emphasis on the analysts who have some portion of their roles allocated to answering self-motivated or complicated questions.

The interviews were framed by a common discussion guide, but we left a lot of time for open-ended conversation to give participants an opportunity to talk about what they really wished they could do but find problematic--either because the tools are complex or don't exist, the data is too big or difficult to wrangle, or the challenges of organizational and human factors are too great.

Oracle Endeca Information Discovery
This screenshot shows an example from an Oracle Endeca Information Discovery Health care demo that helps answer questions, such as which hospital has the most patients and which procedures are most often called for.

We've put together a user framework that works at a few levels of detail. First, there's a segmentation model. We use this model to describe the roles of users, roughly based on how their skills, needs, job roles, and activities are divided today. Segmentation models are a really useful tool for defining the boundaries of the space that we're addressing, understanding the different kinds of people that operate within this space, and how they relate to each other. On top of this model, we've built a number of personas. These personas are brief biographies that are composites of the real people who we met in the field.

In our segmentation model, we first divide our users into two groups: the enablers and the sense makers. The enablers are the people who help bring data into an application experience so that the sense makers can investigate the data, synthesize information, and ultimately arrive at the insights.

Within each of these segments, we have further breakdowns that cover the range of admins, managers, information consumers, architects, and distinct analyst types. The personas built on top of the segment model tend to span segment categories. This reflects that we're designing a product for a future where people will be enabled to take on more responsibilities themselves.

The unique problems tackled by big data: the three Vs

I should clarify that EID is not solely intended to help users with big data problems, but there are aspects where I think we really nail it. We all know that data is big, and that it's getting bigger at an alarming pace, but to build a tool that helps users deal with big data, you need to understand the nature of the unique problems that big data presents. We roll these problems up into categories that we call "the three Vs.”

Volume: More records exist today than at any time in the past, and we're creating more all the time. Almost everything that humans, organizations, or machines do creates some kind of a digital trail, and more of it is being captured than ever before.

Velocity: As more people join the digital collective and more of our experiences and activities are funneled through digital experiences, we accelerate the pace of data creation. Ripples travel fast in the digital pond, and there's a desire for businesses to be more responsive in identifying and acting on the opportunities that this new reality presents. By focusing our efforts on a tool that lowers the time and cost for gaining insights from data, we're working directly on ways to help our users address this increased pace of data creation.

Variety: As if all this incredibly rapid scaling of data weren't enough, there's the challenge of unifying it in some way to make sense of it. Data is stored in all kinds of systems, each one with its own formats, schemas, and protocols to get the data in or out. And if you look beyond those challenges of storage and availability, there's the knotty problem of making sense of the written word, and how to summarize large corpuses in meaningful ways. We must understand the people, places, and things that we talk about and the sentiments that we express about them in the richest, most meaningful way possible.

It's this third V, variety, that I think those of us working on EID are particularly focused on when we talk about big data problems, and for a lot of the reasons just mentioned, I think it's the most interesting one.

One of our biggest goals is to enable users to pull in the relevant data--in whatever format it comes--in order to make sense of it, and further, to be able to combine those disparate types of data together so that we can understand broader landscapes of relatedness.

Providing this capability to individuals at a lowered cost is what we think will trigger a sea change when we're all swimming in massively scaled, rapidly accelerating and wildly divergent forms of data.

What is Oracle working on next, related to UX and Endeca Information Discovery?

  • • We've been working on our processes for communication and collaboration between the teams. We're pulling various ideas from the methodologies out there like agile and lean and have been working to incorporate the best ideas from each.

    • We've given the entire application a visual design facelift. If you haven't seen version 3 yet, I think you'll be excited by what you find.

    • We've been putting a lot of time and effort into a data integration process for the sense-maker segment so that more users will be enabled to pull data into their applications without a lot of outside assistance. This effort has meant really simplifying the process of working with data and presenting options that speak to a wide range of users in terms they'll understand.

    • We've been investing a lot in our data visualizations so that analysts can more easily make sense of the story that the data is telling. On top of that, we're working on ways to make the visualizations more interactive and responsive, encouraging users to dig in, explore, and pursue answers to those new questions that inevitably pop up.

    • We're starting to investigate more about the ecosystems that the tool operates in and how users interact with each other for investigation and storytelling. Finding insights is only the beginning. Sharing those ideas with colleagues closes the loop and makes our insights actionable. Then we can take advantage of this virtual world of big data, not only by dealing with the challenges that the virtual world presents, but by using the insights offered to make a difference in the real world.  |  About Oracle  |  Careers  |  Contact Us  |  Legal Notices  |  Terms of Use  |  Your Privacy Rights