The all-star analyst, powered by AI (Part 2)

In the first part of this series, Noogata co-founder and chief executive Assaf Egozi discussed key drivers and barriers to adoption for using AI/ML to power business analysis. This follow-up article highlights practical ways that citizen data analysts are using the technology to drive value for their business.

Citizen data analysts are on the verge of a huge step forward in productivity. Spreadsheets and visualization tools have served them well for the last few decades but have exhibited shortcomings when dealing with the volume, variety, and velocity of data available to most modern organizations.

A new breed of AI/ML platforms is addressing those limitations, enabling data processing at scale, speed, and with the sophistication required to handle complex analysis. But to extract full benefit from those technologies, analysts need to understand the core capabilities of AI/ML and which types of use cases are most likely to benefit from their application.

Understanding core strengths of AI/ML

Before evaluating different use cases for AI/ML, it is first worth considering the strengths of the technology relative to traditional tools. In that sense, it is important to note that AI/ML algorithms are particularly adept at:

Dealing with big data. AI/ML algorithms excel at processing large amounts of data. Not only do models improve with access to more data, but they are also immensely useful for cutting through large and complex datasets to spot patterns and anomalies, and to cluster disparate data points into useful groups.  

Handling unstructured data. Enterprises today capture a growing amount of unstructured data, which they need to make sense of. AI/ML provides a powerful set of tools to do so – whether that means digesting textual content using natural language processing; extracting information from documents and pictures using image recognition and OCR; or interpreting video or voice streams.

Continuous improvement. One of the advantages of AI/ML over rule-based solutions is that algorithms can be configured to continuously learn from experience, and their performance can improve over time with exposure to more data, even while running in production.

Targeting the right use cases

Across the global economy, AI/ML is being applied in every sector, with applications as wide-ranging as monitoring crop development to medical diagnosis and surveillance technology. 

When it comes to enabling analysts to process ever larger and more varied datasets, the following examples help showcase the strengths of the technology. 

Applying unsupervised machine learning in eCommerce

The world of eCommerce is fluid and fast-paced. Being able to quickly spot market trends and monitor competitor behavior is vital, but doing so is complex. How can an analyst make sense of millions of different potential search terms that consumers use to describe their needs? 

Unsupervised machine learning is ideally suited to help eCommerce analysts with this type of task. A properly designed machine learning model, supplied with the right set of enrichment data, can rapidly sift through millions of search terms and cluster them into topics (those that embody the same consumer purchasing intent). These topics can then be much more effectively studied by analysts to identify trends, map the competitive landscape, conduct price analysis and optimize product positioning.

Applying predictive models to retail location analytics

The saying goes that in retail, as in real estate, it’s all about location, location, location. But how can analysts understand the drivers for successful locations, and identify new high-potential sites for retail outlets; food service; and product launches? Traditional analysis and linear regressions provide some value, but more powerful capabilities are needed to incorporate all relevant data sets into a cohesive model.

Machine learning is ideally suited to help analysts understand how different factors affect store-level and product-level sales performance. This understanding can not only be used to pinpoint new high-potential locations, but also to optimize the merchandising mix of existing retail distribution networks. Machine learning models use a combination of internal sales data and external enrichment data to create a unique “fingerprint” for each location. Connected to a continuous stream of sales data, these models are able to learn from new events – such as store openings, changes in product assortment, promotions etc – to improve both their predictive capabilities as well as their level of explainability. This latter element can be crucial. For example, a sales analyst in a consumer packaged goods (CPG) company will not only be able to provide the field sales team with a targeted lead list for new locations, but also a compelling sales pitch explaining why those locations offer such great potential.

Applying NLP to harmonize data sets

A more generic use case, but one that most data analysts are familiar with, is the need to harmonize data from multiple systems. Common examples include the need to harmonize customer records to identify duplicates and connect entities (helping to establish a 360 degree view of the customer across different business units), or the need to harmonize product catalogs from multiple suppliers or retailers (helping to optimize supply chains). 

Matching data elements from different systems is not a task well-suited to heuristic, or rule-based, approaches. The endless variation in naming conventions is compounded by spelling mistakes and inconsistent reference data, typically resulting in either very low recall (systems that are able to handle only a small portion of the datasets) or low precision (systems that have a high rate of classification mistakes).

Machine learning, usually applying a combination of NLP techniques with classification engines, can deliver dramatically superior results, enabling enterprises to connect data points in a way they never could have before, and unlocking their value. 

Delivering business value

The potential use cases for AI/ML are incredibly broad. However, as organizations look to arm analysts with these technologies, they need to remain cognizant of two key factors. First, as we described in the first part of this article, it is vital that AI/ML technologies are user-friendly, enable data/analytics pipelines to be automated, and integrate with existing tools. Second, it is equally important that organizations target the right use cases and focus on applications that make the best use of the technology’s capabilities to solve critical business problems.