Who Will Big Data Effect?

Every human on the planet will feel some effect of big data in the coming quarter-century.  Big data will impact everything, including natural sciences, social sciences, sports, entertainment, mass communications, real estate, finance, business, marketing, social interactions  – the list is endless.  Big data will touch virtually everything and everyone.  In the world of business, IT professionals are scrambling to provide the infrastructure analysts need to turn this data into insights. This data can eventually be used for tactical marketing and operational strategies that will produce results that will, theoretically, produce dramatic ROIs by impacting efficiency, results, and profitability.     

What is Big Data?

Big Data is the buzzword du jour, but what exactly is it?  There are almost enough definitions of Big Data flying around out there, that trying to capture them might constitute Big Data.  Definitions vary for scientists, statisticians, business analysts, IT managers, and finance managers.  The ambiguity and inconsistencies of definitions show that we are all scrambling to put structure around the ever-exploding concept.  The Global Language Monitor (GLM) reports that “big data” tops their list of most annoying and confusing technological terms of the decade.  Big data can be structured (such as data stored in a database) or unstructured (such as text comments or audio files). Generally, most definitions focus on big data being an amount of data that stretches to the limit and beyond the capacity of current tools for capture, storage, and analysis. 

A petabyte = 1 quadrillion bytes = 1 million gigabytes = 1 thousand terabytes. 

The Wayback Machine holds 2.4 Petabytes

Interesting Factoid:  The human brain's ability to store memories is equivalent to about 2.5 petabytes of binary data.

When Did It Start?

Little Data, Medium Data, Big Data, Mega Data……… Why the sudden explosion in data?   It’s not really that sudden; it’s just exponential.  Moore's Law (developed in 1965 by Intel co-founderGordon E. Moore) is the observation that the number of transistors on integrated circuits doubles approximately every two years.  This results in data doubling every 18 months (a combination of the effect of more and faster transistors).  Moore's Law describes the phenomena of growth that has been the driving force of technological and social change for more than half a century.   

A younger analyst on a Big Data Linked In Group comment that big data didn’t exist 5 years ago.  Having worked much of my career for big financial companies, I disagreed, saying that big data did exist 20 years ago, but it gets exponentially bigger as we go.  And the tools are advancing virtually overnight, facilitated by open source language and cloud computing and storage.  These advancements are also making Big Data accessible and possible for mid and small companies as well as developing countries.

Where Does Big Data Come From?

If you can imagine it, you can capture it!

Big Data is literally exploding all around us.  When you hear Big Data, most people immediately think of the financial services or retail giants, but Big Data is developing everywhere.  As I was driving to work Monday morning, I pulled up to a stoplight beside a fascinating little compact car decked out with logos and a very large camera system mounted on the roof.  It was a Google Street View car.  That started my brain spinning into thoughts of big data and how amazing it was to tackle capturing images of all the streets in the U.S., much less the world.  It took true vision to even imagine that it could be done and then figure out how to get it done.  In my latest house hunting search, I used Google Street View and Zillow to narrow down my search without even leaving my home.  By the time I did site visits, I had already panned down the street, and all the adjoining streets, on my computer and knew the lay of the land. 

Google Street View (Imagination=Data)

Google Street View (Launched in May 2007)

360-degree panoramic views
Locations on all seven continents – 48 countries

360 underwater views of Australia's Great Barrier Reef (Seaview project) are planned to go live in February 2013.
The Brazilian Amazon rainforest project is scheduled to go live in March 2013.

The latest cars have

15 lenses taking 360 degrees of photos
motion sensors to track position
a hard drive to store data
a small computer running the system
lasers to capture 3D data

Click here to read part 2 of this blog.