00.100

THE DATA THAT

LIES BENEATH

What you know about your data is only the tip of the iceberg.

Approximately

1/8

of an iceberg's total mass is visible above water.

7/8
1/8

The remaining

7/8

stretches into the deep ocean blue, hidden from view.

THE ICEBERG PRINCIPLE HOLDS TRUE FOR DIGITAL DATA.

Today, organizations know they have a big data problem, but they don't know just how big it is.

Organizations collect data from various sources and use it to understand their customers and grow their business.

This type of data is called structured data.

When a cashier swipes a consumer's credit card at the point of sale, a slew of data is generated and stored neatly in a database for later analysis.

USER LOCATION CREDIT CARD #
Bryant, Matthew 82º 51' 45.9072" S .... .... .... ....
Smith, Brian 64º 12' 3.027" N .... .... .... ....
Berns, Randy 63º 4' 9.0084" N .... .... .... ....
Spectre, Erin 27º 59' 16.4472" N .... .... .... ....
Chen, Jenny 71º 42' 24.9696" N .... .... .... ....
Sanders, Octavio 40º 54' 2.0052" S .... .... .... ....
Takahasi, Rendell 32º 61' 7.1065" N .... .... .... ....
Adams, Robyn 55º 8' 39.8502" .... .... .... ....

Every time a user clicks on a link on a website, data is generated and later analyzed to determine browsing behavior and buying patterns.

THIS 90% IS
DARK DATA.

HOW DOES DATA

GO DARK?

RATING
Legacy system
Archives
CMS
Email systems
File shares
DMS
Backup tapes
Databases

01:

Post-purchase, a consumer speaks his or her mind about a website's lack of mobile-friendliness by filling out the optional "how was your shopping experience" field and clicks submit.

02:

Although structured data is typically filed neatly into tables in databases, unstructured data doesn't always have a defined destination or format. In this case, the company's website is not equipped with the technology to responsibly store the consumer's comments. Why not?

03:

The consumer's rant gets stored in a variety of locations, where it's nearly impossible to analyze.

REASONS WHY DATA GOES DARK:

too much data,
not enough analysis

can only access
structured data sets

data is there but tools
can't make sense of it

JUST HOW BIG IS THE

DARK DATA PROBLEM?

.

GIGABYTES

7.5 SEXTILLION GIGABYTES OF DATA ARE GENERATED WORLDWIDE EVERY SINGLE DAY

DARK DATA'S GROWING RISKS:
THE COSTS TO STORE & SECURE DARK DATA

Data breaches cost

$3.5 MILLION PER BREACH.

If you're protecting your dark data or if you know your data's value, you're less likely to experience a breach.

REVEAL

These are the risks involved in securing and storing dark data.

DON'T BE AFRAID OF THE DARK.

Elena Klau | VP and group director, research, measurement, and analytics at Momentum Worldwide

DARK DATA IS

POWER

"DARK DATA IS ABOUT LINKAGES—THE TIES THAT CONNECT ALL THE DATA POINTS TOGETHER TO REVEAL WHAT'S TRULY GOING ON WITH A BRAND."

Elena Klau | VP and group director, research, measurement, and analytics at Momentum Worldwide

DARK DATA IS BIG DATA WITH

HIDDEN POTENTIAL.

Within your dark data could be undetected fraud... or your next big idea.

WEATHER

Predict the weather.

ROI

Uncover untapped revenues.

SHOPPING

Determine consumer buying patterns.

WEATHER

Predict the weather.

ROI

Uncover untapped revenues.

SHOPPING

Determine consumer buying patterns.

HOW CAN WE BETTER

MANAGE DARK DATA?

We need visibility into the entire process and a 360-degree view of your data. This is not a one-time event, but an ongoing and evolving process.

WHAT DOES IT TAKE TO EXTRACT DARK DATA'S VALUE?

Lessons for data managers:

O1

Consolidate information by merging data sources for easier management, searching, and more.

O2

Embrace technology. Look for a solution that informs ongoing dark data management.

Find the balance between
hoarding and compulsive deleting.

REVOLUTIONARY DARK DATA

USE CASES:

Look to these use cases to see how organizations in other industries are harnessing the dormant power of their dark data.

The Botanical Society of America is digitizing specimen collections to increase biodiversity data and open the door to new scientific discoveries.

LEARN MORE

For the first time in history, basketball fans can dig into 60 years of team and player performance statistics—data that was once only available to teams.

LEARN MORE

Infinity Property & Casualty Corp. realized it had years of adjusters' reports that could be analyzed and correlated to instances of fraud. The company built an algorithm out of that project and used the data to reap $12 million in subrogation recoveries.

LEARN MORE

REVOLUTIONARY DARK DATA

USE CASES:

Look to these use cases to see how organizations in other industries are harnessing the dormant power of their dark data.

DARK DATA DISCOVERIES IN SCIENCE

The Botanical Society of America is digitizing specimen collections to increase biodiversity data and open the door to new scientific discoveries.

LEARN MORE

CONNECTING NBA FANS TO DATA

For the first time in history, basketball fans can dig into 60 years of team and player performance statistics—data that was once only available to teams.

LEARN MORE

FRAUD RECOGNITION

Infinity Property & Casualty Corp. realized it had years of adjusters' reports that could be analyzed and correlated to instances of fraud. The company built an algorithm out of that project and used the data to reap $12 million in subrogation recoveries.

LEARN MORE

LUCIDWORKS IS WORKING TO MAKE DARK DATA USEFUL
AND ACTIONABLE—RETURNING THE RIGHT DATA TO THE RIGHT
PERSON AT THE RIGHT TIME, EVERY SINGLE TIME.



Sources: “IDC Dark Data,” 2014, International Data Corporation I “5 Insights Gartner Wants You to Know About Unstructured Data,” 2014, Data Gravity
“When Data Joins the Dark Side,” 2014, InformationWeek I Data Gravity I “The Dark Side of Dark Data,” 2014, Mindseye Solutions I Search Data Management
Modern Systems I “Bringing 'dark data' into the light: Best practices for digitizing herbarium collections,” 2015, ScienceDaily
“Big Data: Shining the Light on Enterprise Dark Data,” 2013, Hitachi Data Systems SlideShare I Gartner