What is Big Data?

What if somebody asked you: “What is Big Data?” If you really understand something, you have to be able to say it in one word, and “variety” is the word that best represents what Big Data is. You might assume the defining word would be “volume,” but volume is a relative term—I may have ten megs or ten terabytes of data. So, the biggest difference between “small” versus “big” data is variety. Variety means we have different types of data to consider—and not only data that we have managed so far, which was structured data, meaning it contained metadata (data about the data, e.g., customer name, customer address, account number) which can neatly be put in rows in column and for which names can be given to the attributes or fields.

Whereas, how do you deal with data derived from music? You can use defining terms such as “symphony” or “aria,” but how do you play that music? Or, looking at piece of art—in this case a pastel drawing done by my granddaughter, perhaps either include her picture here, or some well-known painting, then will modify description], how do you express it? You can say that it contains these colors, it’s this particular size, it’s a rendition of a Van Gogh that’s hanging in the Museum of Modern Art in NYC, but that still doesn’t give you the feeling for it, and that is the variety—called unstructured data because it does not contain the attributes that describe it. Description is a big problem with Big Data, when you are dealing with a variety of data that includes unstructured data.

bigdata-variety

Big Data Blogs

Data Storytelling

ct6kga5weaa-u0

What are the main differences between humans and other species? We cook, clothe ourselves, and tell stories. For generations, we have been telling stories which invoke emotions with words. Why should it be different when we tell stories using data?  Story telling is a cornerstone of being human.

Big data is a colossal amount of data arriving at a dizzying speed. It is ferocious in its volume and its velocity. It is a story begging to be told to anybody willing to listen.

10 Skills To Get Best Of Big Data

Big Data technology is new to most organizations and so is awareness of the skills needed to get the best out of Big Data. To “have” these skills overnight is wishful thinking. As a result, in most organizations a large percentage of Big Data skills need to be either learned or recruited, or a little bit of both. Big volumes of data beg for analysis in order to glean correlations and inferences and to prove or disprove hypotheses. These methods point straight to Data Science. In the past, Data Science was practiced only in the academic world. Now, in order to be competitive in the marketplace, every business is expected to possess these academic skills.

Big Data Concepts

big-data-pixabay

  • Here we are with the BIG DATA – which contains the Structured Data (like in Relational databases), Semi-structured data (as with marked up languages) and Unstructured Data (anything and everything conceivable!)! And a lot of it!
  • And of course with any new concepts, implementations, and technology arrive a lot of misconcepts and these misconcepts grow geometrically- not arithmetically! Now all of a sudden we all see that everything is BIG DATA!
  • Too much of anything could be tiring. Big Data seems to be that phrase that is tiring us now. Overnight almost any software as well as many hardware product companies claim brazenly that they are “Big Data” companies.

When can we label the data as Big Data? Let us try to set up a framework of rules when we can call a Data System a Big Data System.