Big data has been on everyone’s lips for several years now, and for good reason. As digital devices and touchpoints proliferate, so does the amount of data we each create. This information may be used to help us better understand customers and customers, make more effective decisions, and improve our business operations. But only if we can make sense of it all.
By choosing the right big data sources and applications, we can give our organizations a competitive edge. But to do that, we need to understand the definition, capabilities, and implications of Big Data.
Big data has already popular applications. From Netflix recommendations to healthcare monitoring, it drives all types of predictive models that improve our daily lives. But the more we depend on it, the more we have to wonder how it shapes our lives and whether we should rely on it so much. While progress is inevitable and something to embrace, the contribution of big data should not be measured by how many companies apply it, but by how it benefits society as a whole.
Defining Big Data and its relationship to Artificial Intelligence (AI)
Big data is not limited to large sets of data. It is defined by the three Vs of data management:
- Volume: Big Data is often measured in terabytes.
- Variety: it can contain structurally different sets of data, such as text, images, audio, etc.
- Velocity: Big data must be processed quickly due to the increasing speed at which data is generated.
As the volume, variety, and speed of data increases, it turns into big data and becomes too much for unattended humans to manage. So we take advantage of artificial intelligence (AI) and machine learning to help analyze it. While the terms big data and AI are often used interchangeably and the two go hand in hand, they are actually distinct.
“In many cases, it is simply no longer possible to solve all problems via human interaction or intervention due to the speed, scale or complexity of the data that must be observed, analyzed and processed. . Driven by AI-powered automation, machines can be infused with “intelligence” to understand the current situation, evaluate a range of options based on available information, and then select the best action or response based on the situation. probability of the best result. ” — Ilan Sade
Simply put, big data supplies AI with the fuel it needs to drive automation. But there are risks.
“However, the tendency to add too much data in AI can harm the quality of AI decision making, so it is important to take advantage of the benefits of Big Data and analytics to prepare your data for the future. AI and to ensure and measure quality, but don’t get carried away with adding data or complexity to your AI projects.Most AI projects, which are mostly narrow AI projects, don’t need big data to deliver their value, they just need good data quality and lots of records. Christian Ehl
Realize the business potential of Big Data
Properly applied, big data helps companies make more informed, and therefore better, business decisions.
“Some examples include hyper-personalization of a retail experience, location sensors that help businesses route shipments for greater efficiency, more accurate and effective fraud detection, and even technologies wearables that provide detailed information on how workers are moving, lifting, or where they are, to reduce injuries and increase safety. Melvin Greer
But this crucial competitive advantage is underutilized because many companies struggle to sift through all the data and tell signal from noise.
According to Greer, five main challenges prevent companies from exploiting the full potential of big data:
- Resources: Not only are data scientists scarce, but the current pool also lacks diversity.
- Data aggregation: Data is constantly being created and it is difficult to collect and sort it from all the disparate channels.
- Incorrect or missing data: Not all data is correct or complete. Data scientists need to know how to separate the misleading from the correct.
- Incomplete data: Data cleaning takes time and can slow down processing. AI can help manage this.
- Truth Seekers: We should not assume that analyzing the data will yield a definitive answer. “Data science drives the likelihood that something is correct,” Greer writes. “It’s a subtle but important nuance.”
Meeting the first challenge is of paramount importance. The only way to solve the other problems is to first create the necessary human capital and provide them with the necessary tools.
The truth Big Data Promise
Data is a wonderful tool, but it is not a panacea. Indeed, “too much of a good thing” is a real phenomenon.
“In my years of working with many companies, I have indeed seen some companies fall into the situation of not using enough data. However, these occurrences pale in comparison to the number of times I have seen the reverse problem: businesses with an overreliance on data to the point that it has been detrimental. The idea that data is necessary to make a good decision is destructive. — Jacqueline Nolis
To illustrate his point, Nolis describes the introduction of Cherry Sprite by Coca-Cola. What motivated the decision? Data. People were adding cherry-flavored “shots” to Sprite at self-serve soda dispensers. So mark one for Big Data.
But as Nolis points out, the very similar-tasting Cherry 7UP was already around — and had been since the 1980s. So the data team might have found the new flavor more efficiently just by browsing the soft drink aisle at the local grocery store. The lesson: Too much reliance on data can get in the way of common-sense decision-making.
Big Data applications: when and how
So how do we know when to put Big Data to work for our business? This decision should be made on a case-by-case basis based on the requirements of each individual project. The following guidelines can help determine if this is the right path:
- Consider the desired outcome. If it’s about catching up with a competitor, investing in something the competitor has already done may not be a good use of resources. It might be better to let their example serve as a guide or inspiration and reserve big data analysis for more complex projects.
- If disruption is the goal, big data can be applied to test new ideas and hypotheses and perhaps reveal other possibilities. But beware of the downsides: Data can kill creativity.
- If a business decision is urgent, “data is still being analyzed” is no excuse to delay it. In the midst of a PR crisis, for example, we won’t have time to mine the available data for information or advice. We need to build on our existing knowledge of the crisis and our customers and act immediately.
Of course, sometimes big data is not only useful but essential. Some scenarios require big data applications:
- To determine if a strategy is working as intended, only data will tell the story. But before measuring whether success has been achieved, we must first establish our measures and define the business rules that determine what success looks like.
- Big data can help process and create models from vast amounts of information. So, as a general rule, the larger and more data-rich the project, the more likely it is that big data can be useful.
Big data may be the hottest topic in technology today, but it’s more than just a buzzword. Its potential to improve our businesses and our lives over the long term is real.
But this potential must be exploited purposefully and in a targeted manner. Big Data is not the commercial equivalent of a miracle drug. We must be aware of where its applications can help and where they are superfluous or harmful.
Indeed, the full promise of Big Data can only be realized when guided by thoughtful human expertise.
If you liked this article, don’t forget to subscribe to the Enterprising investor.
All posts are the opinion of the author. As such, they should not be construed as investment advice, and the opinions expressed do not necessarily reflect the views of the CFA Institute or the author’s employer.
Image credit: ©Getty Images / Who_I_am
Professional Learning for CFA Institute Members
CFA Institute members are empowered to self-determine and report professional learning (PL) credits earned, including content on Enterprising investor. Members can easily register credits using their HGV tracker online.