Three steps for conquering the last mile of analytics

Becoming insights-driven is now the ultimate prize of digital transformation, and many organizations are making significant progress toward this goal. However, putting insights into action – the “last mile” of analytics – is still a challenge for many organizations. 

With continued investments in data, analytics and AI, as well as the broader availability of machine-learning tools and applications, organizations have an abundance of analytical assets. Yet the creation of analytical assets should not be the only measure of success for organizations. In reality, deploying, operationalizing, or putting analytical assets into production should be the driver for how organizations are able to get value from their AI and data science efforts.

In a traditional data and analytics continuum, data is transformed into insights to support decision-making. If organizations want to break out from experimentation mode, avoid analytics assets becoming shelfware, and empower front lines to make analytics-powered decisions, they must start with decisions. Then they need to decide how to find, integrate and deliver the insights; and identify data to enable that.

These days, I suspect many organizations would argue they’re doing just that – they’ve hired analytics talent and appointed chief data officers (CDOs) or chief analytics officers (CAOs) to collaborate with business leaders to become more data- and analytics-driven. But many organizations are not seeing the desired impact and value from their data and analytics initiatives and are not able to quickly put their pilot projects into production.

According to IDC, only 35% of organizations indicate that analytical models are fully deployed in production. Difficulty in deploying and operationalizing analytics into systems or applications – and being consumed by downstream processes or people – is a key barrier to achieving business value.

Some might argue that the main focus within analytics projects has been on developing analytical recipes (e.g., data engineering, building models, merits of individual algorithms, etc.), while not much attention, priority or investment is done for operationalization of these assets. This is easier said than fixed. Data does not provide differentiation; decisions at scale do. Applying insights consistently to turn data into decisions will let organizations build a true software-led system of insights to grow and break away from competitors.

How can organizations put analytics into action in a systematic, scalable manner and conquer the last mile? Here are the three key areas where organizations need to pay consistent attention:

1. Understanding Technology Components

The need to streamline and operationalize model management processes requires users to register, deploy, monitor and retrain analytical models. More specifically:

• Register. The centralized model repository, life cycle templates and version control capabilities provide visibility into commercial and open-source analytical models, ensuring complete traceability and governance. It will also promote collaboration among different stakeholders and manage the analytics workflow effectively. Letting organizations store data, code, properties and metadata associated with models enables transparency and shows the real value of analytical assets.    

• Deploy. The deployment step is all about integrating analytical models into a production environment and using it to make predictions. It is often the most cumbersome step for IT or DevOps teams to handle, but it’s essential in delivering value. Ideally, organizations should be able to combine commercial and open source models in the same project to compare and select the champion model to deploy. Depending on the use case, models can be published to batch operational systems (e.g., in-database, in-Hadoop or Spark), on-demand systems (e.g., web applications), cloud, or a real-time system using streaming data.

• Monitor. Once organizations start realizing the value from analytics, the real world does not stop. Scores need to be analyzed and monitored for ongoing performance and regularly evaluate whether models are behaving as they should as market conditions and business requirements change and new data is added. Performance reports can be produced for champion and challenger models using variety of fit statistics.  

• Retrain. If model performance degrades, organizations should take one of three approaches:
  • Retrain the existing model on new data.
    
  • Revise the model with new techniques (such as feature engineering, new data elements, etc.).
  • Replace the model entirely with a better model. 

This requires commitment between stakeholders on which metrics to measure and which will deliver business impact.  

2. Embracing Roles and Behaviors of Different Stakeholders

In order to be successful in the last mile of analytics, a close collaboration between stakeholders with the right skill sets – data scientists, business units, IT and DevOps – is critical. Lack of interest in dealing with deploying and managing analytics into production, leaving it solely to just one team (e.g., IT or DevOps), or not having the right incentives for all stakeholders to communicate will not create value from your analytics or AI initiatives.

For data scientists, developing analytical assets should only be initiated with deployment in mind, while IT or DevOps teams will have to understand the integration requirements, operational data flows and data preparation for model deployment and retraining. The role of business stakeholders is equally important. They are the ones who have to clearly define what benefits are expected from the analytical models and collaborate with data scientists to understand the results after models are put into production and monitor the results on a continuous basis.

3. Establishing a Systematic Operationalization Process

Finally, the only way to ensure the value, integrity and transparency of analytical models is to establish a process for operationalizing analytics. Many organizations have a well-defined process for the analytics development phase of the analytics life cycle. But a lack of process-centric understanding around the model deployment and management phase of the life cycle is an important barrier that needs to be overcome.

A well-defined process, with proper templates and workflow, needs to validate that the model developed using training data is still performing as intended in the real world, and integrating and executing the same model against operational systems or processes. Some organizations make a mistake and stop here. In fact, to fully realize value, the performance of models in production needs to be monitored continually.

It’s no surprise that this last mile of analytics – bringing models into deployment – is the hardest part of digital transformation initiatives for organizations to master, yet it’s critical if they’re going to experience real benefits from AI and analytics investment. To systematically realize full potential from data and analytics initiatives, organizations must involve IT and DevOps early on within the data science project such that operationalizing analytics is not an afterthought; agree on the quantifiable outcomes before building analytical models; and have a clear understanding of the steps, roles, processes and handoffs involved, from data preparation and model development to putting analytics into action.