Don’t Be A Big Data Hoarder

Rich Brown, Aug 27, 2012

We’ve all most likely seen the hoarding programming on television lately showing everyday people with collected items piled wall to wall, from floor to ceiling throughout their home with barely any room to live.  Unfortunately this too can happen, virtually, to enterprises with no more room to store data because they’ve collected more then expected too fast and are now scrambling to come up with a solution.  The bottom line is don’t be a Big Data hoarder. 


With all this talk about how to handle and process Big Data, how to derive insight and analytics from Big Data and how to store Big Data, there is little conversation around what to do with Big Data after you finally have it.  It’s important before you start handling Big Data that you come with a realistic data retention plan.  This plan has to be both realistic for your analysis needs as well as your enterprise needs for data handling and storage. 

All too often, companies formulate a plan to take in Big Data and begin analysis for customer insight; however, they soon realize they are not only overwhelmed by the deluge of information, but they’re also overwhelmed by the amount of storage capacity that’s required to keep massive amounts of data.  It’s a difficult situation to be in when you finally get what you ask for only to realize you’re up to the rafters in data with no where to walk. 

In terms of the marketing world, BigData is typically obtained in the collection of customer data such as Social Media conversations or detailed customer purchasing behavior over time.  Before your start collecting this type of data, determine exactly how you’re going to use that specific data and come up with a time frame for storing the data where it will remain fresh and relevant for your specific purpose.  Make sure you implement an auto archiving or auto purge of the data once it passes your established time frame for relevancy. 

For most organizations the period of data relevancy can differ on usage requirements, program requirements and also even legal requirements which stipulate the amount of time data must be retained. Make it a priority before you start consuming Big Data to work with your companies internal IT department or your marketing service partner to better help you to understand not only how to use and analyze Big Data but also to determine when is the best time to say good by to Big Data. 



  1. 2 Justin Ware 28 Aug

    Rich, you and I think alike. Check out this video we shot last winter...

    I'd say great minds think alike, but I'm not sure "great" is how I would describe our video... but it sure was fun! Anyway, thanks for this great post.

  2. 1 Rich Brown 28 Aug

    Justin - Thanks for the comment and the link to your video which brings some humor to the challenges of data handling and storage.  When we start looking at BigData the problem can balloon out of control very quickly which is why I always recommend organizations to come up with a plan in advance.


  1. RadEditor - HTML WYSIWYG Editor. MS Word-like content editing experience thanks to a rich set of formatting tools, dropdowns, dialogs, system modules and built-in spell-check.
    RadEditor's components - toolbar, content area, modes and modules
    Toolbar's wrapper 
    Content area wrapper
    RadEditor's bottom area: Design, Html and Preview modes, Statistics module and resize handle.
    It contains RadEditor's Modes/views (HTML, Design and Preview), Statistics and Resizer
    Editor Mode buttonsStatistics moduleEditor resizer
    RadEditor's Modules - special tools used to provide extra information such as Tag Inspector, Real Time HTML Viewer, Tag Properties and other.