Key Messages
The Big Data Revolution: How Massively Large Datasets Unlock Valuable Insights Into Our World
The potential of big data goes far beyond what we could have imagined before computers and the internet were invented.
When it comes to collecting, recording, and analyzing data, big data offers a wealth of opportunities that simply weren’t available before.
For example, in 2009 Google demonstrated how big data can be used to uncover valuable insights not available through traditional means.
They showed how they could analyze users’ search terms gathered from previous years in order to predict and track the spread of flu virus outbreaks – something that would have taken much longer without their analysis.
Big data also helps us discover new insights and patterns that may not be visible when using smaller datasets.
Analyzing larger datasets provides us with a broader view of events which can reveal correlations and trends that would have previously been impossible to identify.
Google’s example is just one example of the powerful possibilities of big date analysis – revealing insights we could not obtain by analyzing smaller datasets.
The Rise Of Datafication: Unlocking Valuable Insights From Unconventional Sources
Data is increasingly being collected and put to meaningful use in all aspects of our lives.
In Japan for instance, the Advanced Institute of Industrial Technology are using pressure sensors in car seats to measure the distribution of weight that the people’s backsides put on them.
This data can be used to identify individuals so accurately that it can even be used as a security device–the car won’t start unless it recognizes who’s driving it!
Other companies have realized the potential of datafication too, with Apple filing a patent in 2009 to passively measure users’ blood oxygenation, heart rate and body temperature through earbuds and IBM creating touch-sensitive floor surfaces which analyze what different people do while they’re walking on it.
The bottom line is that researchers are now collecting data from sources we previously hadn’t thought of as information and using this data to gain valuable insights into our behavior, allowing them to create innovative products and services.
From the size of our bums to the way we walk, Big Data has become an integral part of how we live our lives.
The Power Of Big Data: How Technology Has Liberated Us From The Limitations Of Sampling
Taking a small sample of data to represent an entire population is a big risk.
The accuracy, veracity and significance of the results depend on how representative the sample is of the whole population.
But with Big Data we don’t have to take such risks anymore.
Big Data gives us access to massive amounts of information, or in some cases, all of it.
This makes it possible for us to examine smaller and smaller segments from our data and make meaningful predictions that are much more accurate.
With a large enough dataset we can “zoom in” on different subgroups in order to get accurate results for any demographic or segment of customers without having to rely on sampling.
In other words, Big Data frees us from the limitations of relying on a small sample size when trying to predict outcomes for large populations.
Nowadays, researchers and companies alike can take advantage of Big Data in order to gain valuable insight into their target markets and audiences while making more reliable decisions on substantial data sets than ever before.
Bigger Is Better: How Big Data Helps Us Live With Messy Data
When it comes to Big Data, vast sets of messier data can often be more useful than smaller, more accurate ones.
This was a lesson that IBM learned in the 1980s when they tried to use a language-translation program; despite their small but high-quality sample of three million sentence pairs, the translation was unreliable on fewer used words and phrases.
Google realized this while trying to tackle the same issue a decade later.
Instead of using high-quality data, they opted for quantity: scouring billions of pages of text across the internet.
And despite the dubious quality of their input, the sheer size resulted in more accuracy than any other system available at the time.
The takeaway? With Big Data, having so much data means that inaccuracies in what you have will have a much smaller effect on overall results.
That’s why huge collections of messier data can often be more useful than those with higher levels of precision.
The Power Of Data: How Big Data Can Uncover Unexpected Correlations To Make Life Better
We now know that, when it comes to gathering data, looking for correlations can be incredibly valuable.
Big data does not always explain why two things are related, but it reveals some pretty useful information nonetheless.
Case in point: a study conducted by IBM and the University of Ontario which analyzed premature baby’s vital signs to uncover potential signs of infection.
Their research found that babies’ vital signs would become very stable just before an infection occurred—somewhat counterintuitive, but invaluable information when caring for these tiny patients.
Big data does not necessarily tell us why two things may be related, but correlation analysis can still provide us with useful insights and leads on potential cause and effect relationships.
Even if we don’t understand exactly why a correlation exists between certain variables, simply understanding that they do is often enough to put them to good use.
The Power Of Secondary Uses: Harnessing The Value Of Data For Maximum Profit
Data collection is often engineered for a particular purpose such as tracking sales records, factory output, website usability and the banking industry.
However, in many cases the original use of the data cannot compare to the value of what it can become with secondary applications.
For example, Swift’s payment data can be analyzed to give highly accurate insights into global economic activity.
Companies like Experian have also taken advantage of viewing old internet searches to gain consumer interests and better understanding market trends.
Moreover, mobile phone companies are collecting real-time location data which can be used to analyze traffic patterns and deliver personalized location-based ads.
These advancements have not been unnoticed – savvy companies and individuals are actively seeking out potential secondary uses for their data collection.
This trend proves that collecting and analyzing data holds great potential for greater value applications beyond their initial purpose.
The Benefits Of Having A Big-Data Mindset In The Data Economy
Having a big-data mindset means that you have the ability to recognize opportunities where data can be used to create value.
It is an incredibly powerful tool, as evidenced by individuals like Bradford Cross and Decide.com who were able to make use of available data sources to create their respective companies.
It also means having the awareness of what data are available and understanding how they can be utilized in order to create something of worth – it doesn’t matter if you don’t have extensive data skills or a plethora of information; if you have the right mindset and awareness, then you will be able to spot new opportunities.
This could range from combining publicly available data on flight times and historical weather records, as Bradford did, or analyzing large amounts of price quotes for products from e-commerce websites – like what Decide.com does – all it takes is the right mindset and awareness.
The potential for creating value through data is out there.
Anyone with a big-data mindset will be able to capitalize on these opportunities before others do – so don’t wait around and miss out!
Combining Data Sets Can Create Greater Value Than The Individual Parts
When it comes to data sets, they often hold more value when combined than they do individually.
Combining data sets allows you to spot trends that weren’t visible before.
An excellent example of this is a 2011 study conducted by a Danish research group which combined mobile phone user data with cancer patient records.
By doing so, they were able to see if there was any correlation between the two—there was none—but the power came in being able to control other factors such as education and income without compromising the accuracy of their results.
The same principle can be applied even within a single dataset.
For instance, Inrix, a traffic analysis company in Seattle, aggregates location data from various sources like car manufacturers and their own smartphone app and turn it into something with much more valuable insights about traffic patterns for customers who are willing to cough up cash for using their services.
Ultimately, combining sets of data can create greater value than the individual parts themselves.
This is why it is an important practice for companies and users alike who want to get more out of their information than what the individual parts have to offer.
Using Big Data To Enhance Services: How Companies Remotely Track What We Do And Use It To Their Benefit
In today’s world, online services such as Facebook are using data exhaust to optimize their products and services.
Every action a user takes on their websites – from mouse clicks and how long we hover over items to the searches we make and typos we enter – is captured.
This data is then used to fine-tune features like autocomplete and spell-checkers, or even tweak the layout of a page to increase engagement.
Companies such as Zynga have done this in regards to gaming; if they find that players quit when they reach a certain point in one of their games, then they will adjust it accordingly.
Meanwhile, other companies like Facebook use user data to show people more of what their friends are doing, making the whole experience much more interactive.
This proves that by collecting user feedback data – either actively or passively – companies can vastly enhance the service they offer.
It also shows how businesses can benefit from having this kind of insight into what customers value most, allowing them to tailor their products better than ever before.
Current Privacy Laws And Anonymization Methods Inadequate For Big Data Privacy
It’s becoming increasingly clear that current privacy laws and anonymization methods don’t offer enough protection to those affected by Big Data.
For example, when companies want to share data collected from users, they often use anonymization – stripping out any personal details – before publishing the data.
But this isn’t foolproof; the more detail present in the data, the easier it can be for people to be re-identified.
Take a look at what happened with AOL and its mountain of old, anonymized search terms: within days of its release, it was already possible for The New York Times to identify one of the users buried in it.
It’s clear that as we move down the big-data road, existing measures become dangerously outdated and obsolete.
Suitable alternatives need to be considered to ensure that everyone is adequately protected when using Big Data.
We Must Never Judge Someone Before They Have Committed A Crime: The Dangers Of Predictive Policing
While big data has helped us to make more accurate predictions about criminal behavior, we must always remember that one should never be judged for something they haven’t done.
This is an important lesson to learn from films like Minority Report, which showcases a dystopian society where people are arrested not for the crimes they have committed, but rather what they are foreseen to do.
This idea of predictive policing can also be seen in many US state parole boards, which use data-based analysis to determine a prisoner’s fate.
Big data is being put to use by law enforcement in order to allocate scarce resources and profile individuals and even neighborhoods based on factors that could potentially predict crime.
In national security as well, similar profiling measures are employed heavily.
However, if misused or taken too far, this kind of profiling can lead to discrimination and issues with “guilt by association”.
We need to be especially careful as these methods of using big data for prediction become more prevalent and target individuals instead of groups – this could lead us down an extremely dangerous route where our freedom and moral choice is taken away from us.
The Dangers Of Becoming Too Data-Driven: Why Beyond The Numbers, We Need Perspective
When we are too reliant on data when making decisions, it can lead to disastrous consequences.
We may be measuring the wrong things, incentivizing behaviors that we do not intend, and trusting unreliable data.
The situation of Robert McNamara demonstrates this point plainly.
As Secretary of Defense during the Vietnam War, McNamara became completely fixated on measuring the enemy body count as an indicator of progress and re-designed his entire strategy around it.
All too late he realized that he was relying on inaccurate information given by officers looking to impress him with reports that were probably falsified.
Being overly dependent on data means that our decisions might be misguided, or even worse, losing perspective and having our actions governed by data that could create more harm than good.
This is why verifying the accuracy and quality of big data is so important to ensure that any decision made based on it is done in full knowledge of its strengths and weaknesses.
Wrap Up
The Big Data Book’s final summary is centered around the idea that data can be used to bring about great value, but must also be properly understood in order to be utilized without consequence.
Big data brings an entirely new perspective and incredible opportunities to individuals and companies who are able to understand this concept.
If used correctly, it has the potential to enhance existing products or services and create new ones.
If used carelessly though, big data can cause us to lose our sense of identity or lead us into a world of data-driven demands, punishments, and controls.
Ultimately, it is important for users to think outside the box with their data and come up with creative ways to extract value from it.
Think of how different forms of data might be combined or integrated from various sources such as online stores or businesses you know – the possibilities are limitless!
Then consider different industries and how they might benefit from this potentially gold mine of information you have access too.
You never know when your creative thinking will turn up a revolutionary idea or product!
