Augmented Analytics Demystified
What It Means and Why It is the Future of Data Analytics
Gartner Hype Cycle for Emerging Technologies, 2017. Image via Gartner.
Gartner, one of the top marketing research companies in the world, publishes an annual “hype cycle” graph. It describes the status and maturity of all up-and-coming technologies ranging from brain chips to self-driving cars.
In July, Gartner published the 2017 edition of the report (shown above). Notably, the report introduced a new concept called “Augmented Analytics”(“Augmented Data Discovery” in the graph), which they claimed to be the “future of data analytics”.
In the report, Gartner describes Augmented Analytics as:
“an approach that automates insights using machine learning and natural-language generation, marks the next wave of disruption in the data and analytics market.”
While it is a great definition for data experts, it’s not detailed enough for most people to understand what “Augmented Analytics” really means.
So, today let’s talk about what “Augmented Analytics” really is — and why you should care.
To understand “augmented analytics”, we must first understand the problem it solves. That is, we must understand why generating insights from data remains a huge challenge for almost all businesses.
At this point in the business world, everyone agrees that data analytics, like vitamins, is good for a business, and has the potential to drastically increase traction and revenue (if done properly).
However, the problem is that data analytics is not exactly the easiest thing to pull off in the real world.
You see, data is not like a vitamin that you can just pop in your mouth and be done with it.
In fact, data, by itself, is totally useless to your business.
For example, your online data might reveal that your revenue is decreasing by 10% from last month.
But what does that really mean to you? Is this decline an industry trend? Is it because one of your advertising channels is not working? Or is it because of other reasons?
That’s why you need to go deeper into your web analytics, ecommerce, and social media data to uncover what resulted in the decline of your revenue.
Then you need to place those changes into a business context, and identify the ones that you can act on immediately.
Going back to the declining revenue example, you might end up realizing that your social ads are 10% less effective than the previous month, and that you need to hire an agency to optimize your ad spend.
Now that insight is actionable because it connects directly to an action you can take to solve your business problem. These actionable insights are extremely helpful because they serve as a guiding light for what you should prioritize in your business.
However, to go from raw data to insights, you need to go through many technical steps, including:
1) Collect data from multiple sources
2) Clean data so it is ready for analysis
3) Conduct the analysis
4) Generate insights, and
5) Communicate those insights with the organization and convert them into action plans
If these steps sound complex, they are even more complicated to implement in practice, and you need to hire dedicated individuals (usually data scientists or data analysts) to perform these steps for your business.
But here’s the thing.
First of all, data scientists and analysts are REALLY scarce and expensive to hire, making it extremely cost-prohibitive for smaller businesses (100M and below) to leverage analytics.
The McKinsey Global institute projected that the U.S. economy could be short as many as 250,000 data scientists by 2024. And that’s after accounting for the recently growing supply of data scientists from colleges and education programs.
Second, no matter how good data scientists are, they are not business experts.
This means that the executives need to work very closely with the data scientists to make sure the analytical results actually make “business sense,” taking precious time away from the owners’ busy schedule.
Third, in practice, data scientists spend over 80% of their time doing simple mechanical things such as labeling and cleaning their data.
This makes their lives miserable and wastes the investment by the company (read more here).
Finally, data scientists are still human, meaning that their attention span and ability to do repetitive work are limited.
Therefore, a typical data scientist only analyzes a small portion (probably 10%) of your data that they think has the most potential of bringing you great insights. This means you may miss out on valuable insights in the remaining 90% — insights that may be mission-critical for your business.
For these reasons, almost all small and medium-sized businesses (SMBs) we interviewed (and by now we’ve interviewed more than 100) are still in the early stages of analytics adoption, despite a strong desire to leverage their data.
The challenge faced by businesses here is not going away anytime soon. We can’t expect a sudden spur of data scientists airdropped by Martians to solve the talent shortage. Nor can we expect analyzing this data to magically become easier to do for non-technical business people.
That’s where augmented analytics comes in.
What augmented analytics does is to relieve a company’s dependence on data scientists by automating insight generation in a company through the use of advanced machine learning and artificial intelligence algorithms.
An augmented analytics engine can automatically go through a company’s data, clean it, analyze it, and convert these insights into action steps for the executives or marketers with little to no supervision from a technical person. Augmented analytics therefore can make analytics accessible to all SMB owners.
Wait, aren’t those the problems solved by analytical software tools like Tableau and SAS?
This is the common misconception I hear from everyone outside (or even inside) the analytics industry.
Software tools like Tableau and SAS are tools that “support” analysis.
What this means is that those tools make analytics and communicating results easier for analysts in your company.
However, it doesn’t do the analyses FOR you, and it certainly does not eliminate the need of a business analyst or data scientist.
Augmented Analytics, on the other hand, is designed to conduct analyses and generate business insights automatically with little to no supervision, and can be used directly by marketers and business owners without needing the assistance of a business analyst or data scientist.
Therefore, its application is far more advanced and powerful than tools like SAS and Tableau.
On the technical side, tools like SAS and Tableau focuses on offering extremely flexible interfaces. This way, analysts can easily conduct any type of analysis on the platform and beautifully present the results.
On the other hand, augmented analytics focuses significantly more on the end goal of those analyses — insights.
For example, whereas Tableau enables you to create beautiful bar graphs (without telling you what the bar graph actually means for your business), an augmented analytics engine might simply tell you what your data is saying about your business. It may show only the relevant information that led the system reach that conclusion.
Therefore, augmented analytics software is much more focused on creating a knowledge base of business information to detect business trends, and using machine learning algorithms to detect those trends in a company’s data.
So how mature is Augmented Analytics right now?
Short answer, not very mature but it is going to grow VERY fast in the next couple of years.
Personally, as someone who is working on an augmented analytics company (Humanlytics, check it out!), I measure the maturity of all augmented algorithms in three stages, with each stage being significantly more advanced than the stage before it.
Stage 1: Data Preparation and Discovery
This is the stage most of the existing augmented analytics technologies are in. Key players include IBM Watson Analytics, Tableau Insights, and Qlik Sense.
At this stage, augmented analytics algorithms serve as a great complement to existing data scientists or analysts, but does not have the ability to completely substitute them.
Here, the algorithm’s primary purpose is to automate boring data preparation tasks such as data cleaning, data labeling, and data collection.
It may be able to detect some correlations and anomalies in the data, but most of these detections are noise, and data scientists still need to parse out real signals manually.
Stage 2: Signal Detection
At this stage, the augmented analytics algorithm can detect true signals in a company’s data with extreme reliability. However, it is unable to connect these discoveries with business situations or business actions.
Assistance is still needed from data analysts or data scientists to convert those discoveries into concrete business insights, but the time they need to spend on each insight is reduced dramatically.
Many companies will likely reach this stage in 2–3 years.
Stage 3: Actionable Insight Generation
This is the stage in which the augmented analytics engine can directly interface with executives in the company with little or no need for input from a business analyst or data scientist.
A large knowledge base of past business cases will be developed to help the augmented analytics systems connect trends in the company’s data to the larger context of the business. It can then offer concrete action steps based on these insights.
More importantly, the system will be able to track the implementation of these actions and provide additional insights on what the company can do better next time to maximize its operational effectiveness.
Here, the augmented analytics engine is not only a substitute for business analysts, but can also do a lot of things current analysts cannot do.
This stage is definitely a significant leap compared with the previous stages. I believe many businesses will start to reach this stage in 5–10 years.
Get excited about the coming data revolution!
As both a data scientist and an entrepreneur working in this field, I am super excited about the prospect of augmented analytics becoming adopted in the business world.
I like to compare the evolution of data analytics technologies with the evolution of the automobile.
To start, both are extremely complex systems that have thousands of if not millions of parts in them to make them function properly.
However, almost everyone in the country right now can drive an automobile, despite how complex the technology is.
This is because most of the complexity is abstracted away by the technology. Users only need to know the “data” that are relevant to them (eg how to use the steering wheel) to make decisions while driving.
Now, we are progressing even farther to make the task of “driving” unnecessary so that users can simply care about what’s really important to them: moving from point A to point B with as efficiently as possible.
In the data world, we are not even close to the design and technology achievements accomplished by the automobile industry (given we are only 10 years into the data revolution, I am not surprised).
But to quote Gartner, augmented reality is really the future of data analytics because it moves us closer than ever to that vision of “democratized analytics” because it will be cheaper, easier, and better. At last, we will see more and more businesses of all sizes using and benefiting from analytics for years to come.
We are driven by this vision of a future where data benefits everyone, not just the select few. If you are too, join our Facebook community here.