Commercial Machine Learning Toolkit to Power Digital Business
Commercial AI calls for a cross-road between prescriptive analytics and operationalizing machine learning models.
Introduction
The global Machine Learning (ML) market was valued at around $1.58 billion in 2017 and is expected to reach approximately $20.83 billion in 2024, according to Zion market research. The application of cognitive technologies that leverage the emerging capabilities of machine learning (ML) and artificial intelligence (AI) are being adapted by companies of all sizes. Availability of cloud based GPU’s & TPU’s, open-source libraries, investments from corporates like Google, Facebook and availability of large scale datasets like Kaggle etc, is enabling the application of machine learning within reach for every product development.
Gartner’s CIO Survey found that in the last four years, AI implementation grew by a massive 270 percent. The survey data shows that, with the rise of Robotic Process Automation (RPA), workflows are being augmented with machine learning algorithms for both cutting costs and customer satisfaction. Businesses adapting ML enabled applications are gaining momentum due to the increasing penetration of technology for operational needs and the rising value of business impact. The data-rich nature of machine learning predictions also has been the major driver for digital business seizing the opportunity to derive real-time decisions. What makes Machine learning unique is a holistic, detached view of the domain and automatically recognizing hidden patterns. AI adoption is accelerating and 75% of businesses expected to move to AI powered businesses by 2024. With this, a new trend emerging is “Machine(Deep) learning powering real-time business processes”.
Commercial ML Toolkit
For commercial success of machine learning models, it is important to understand that the commercial applications require the highest degree of functionality and reliability. The success metric is mostly binary, as compared to the incrementally improving statistical metrics. This calls for a high level of explainability and transparency about how the ML model functions. This toolkit explains both the analytics and operational framework to address these needs.
Prescriptive analytics
Traditionally companies have executed business plans, based on historical data. For example, a store manager would decide on a “product promotion” based on previous year data and store manager’s experience. With the increasing availability of real-time information and availability of matured machine learning algorithms, companies are making proactive decisions by means of predictive analysis which uses both historical and real-time trends. However, machine learning also provides an edge beyond predictive analytics for real-time businesses, which is “Prescriptive analytics”, the market which is expected to reach around $1.88 billion by 2022.
More consumer facing businesses are trying to adapt perspective analytics, as it provides actionable recommendation for the possible forecasted outcomes. The greater advantage is that it also performs a “what if” analysis with the given constraints in real-time. Fewer examples includes, a) For a LMS system to recommend additional courses to learn a prerequisite skill, if an employee is struggling to complete a course b)guided selling, by prescribing the right buyer with right content c) better manage store inventory by optimizing real-time based on everyday sales.
A survey-based report by Indian IT giant Infosys titled Human Amplification in the Enterprise found that 98 percent of respondents reported 15% additional revenue for their organizations, by performing AI enabled tasks. In this, the biggest impact came from machine learning, as it assisted in making more informed decisions by providing “recommendations considering business/operational constraints”.
Operational Framework
After this paradigm shift of “Real-time decisions for businesses using machine learning”, AI and ML have become an engineering problem rather than a research challenge. Reliability, scalability and managing complex systems are the focus areas for executing ML models real time. Question is “How do I keep machine learning projects adapt to address market needs as compared to just research experiments with endless iterations?”. Experts think that increased use of commercial AI and ML in realtime will help to accelerate the deployment of models in production [Gartner].
As real-time decisions or recommendations are provided by the computer, the key challenge is to ensure for compliance, transparency and ethics. Thus, “commercial AI calls for a cross-road between prescriptive analytics and machine learning models to build prescriptive models which help in making real-time decisions for the businesses”. Here is the operational framework explaining different techniques and tools to address these challenges.
Explainable AI (XAI)
“Explainable AI” (XAI) is a step towards achieving human collaboration with science and making “Humans at the center of strategy” for building consumer facing products with real-time prescriptive analytics. It’s a set of tools and frameworks, which helps with interpretability and explainability that are crucial for achieving Fair, Accountable and Transparent (FAT) machine learning with no bias. It helps to answer “Should I trust this prediction?”
This emerging field in machine learning aims to address how black box decisions are made by ML models. Some of the simpler forms of ML models like decision trees, bayesian classifiers, logistic regression have got some amount of explainability. The recent research developments have shown progress to bring explainability to more complex machine learning algorithms in the deep learning space. Research labs like DARPA are conducting extensive research around building explainable interfaces for neural nets that produce more explainable models, while maintaining a high level of learning performance.
Over the past couple of years, AI researchers have been developing tools like What-if, DeepLIFT, AIX360, Activation atlases, Rulex, Explainable AI, Alibi, and approaches like Attention, LIME, SHAP , that are enabling practitioners to easily evaluate the quality of the decision rules in use and reduce false positives. These tools also promote explainable AI to be adapted at a scale.
Scientific “Agile” of ML
A new metric which is measured by enterprises is “Time to deploy the first model in production”. A recent survey highlights that 18% of the companies take longer than 90 days and some of them about a year also, to go production. It also highlights that 25% of the ML engineer time is spent taking the models to production. One of the key reasons for this prolonged cycle is, ML is a research intensive environment and it is not deterministic. Another dimension to that is, for research projects failing to prove feasibility, is also a possible outcome. This means that many times you will end up with an undelivered functionality.
An adaptive strategy/process needs to be developed to manage any machine learning projects. This strategy should provide greater opportunity for a lot of investigation, exploration, analysis and tuning, in a continuous way.
The data-driven, open-ended nature of machine learning calls for a faster feedback loop, which naturally lends Agile as a de facto process for machine learning projects. What we need is a “new port of agile” for machine learning.
This ported version of Agile principles, would pick characteristic from a) Integrating this “Scientific method” as an on-going iterative process b) Managing data through the “Data analytics pyramid”, which expresses the business value c) “PERT” based project management
As per the Dresner report, 70% of the R&D departments are most likely to adapt AI and Machine learning for all their enterprise functions. This increased shift towards value-based contracting will push the AI and machine learning platforms to remain responsive and adaptive, which are basic Agile tenets.
3 M’s of ML Orchestration
As commercial use of ML is trending to spike up, firms will be looking to better manage, monitor and maintain the models in production to ensure trust in the deployed AI integrations. Trust and transparency will become even more critical with increasing scrutiny on compliance, data security, and bias. The three “M’s” that needs to be looked at are,
MLOps focuses on operationalizing AI, making the tech/science accessible, applicable, repeatable and automated. MLOps also recommends building a “cross-functional team” where engineers and researchers are embedded in the same team. It also remains as the basic platform to evaluate and enable ML Risk and ML health. Enterprises needs to focus on architecting the MLOps/ CD4ML frameworks which factors in feature stores, model versioning, metadata store, model serving and end to end deployment pipelines. with Open source tools like Kubeflow, FEAST, ONNX, Seldon Core and 5 level maturity models help enterprises to understand their current state and climb up the levels, and thus consider MLOps as a driver of their tangible business value.
ML Health is the functional monitoring, which is used to convey the business model’s performance to the business sponsors. It is critical, as it demonstrates the predictive model performance along with the impact to the product/business. This is achieved by constant metrics evaluation such as accuracy, precision, recall to ensure that they are operating within the expected bounds. A successful communication strategy such as a real time notification platform for drop or increase in model performances is required. For large scale model deployment in production, building a monitoring dashboard, using visualization tools such as Tableau and Qlik are popular as well.
ML Risk does not end with model development. Model drift is expected behavior, when a ML model gets integrated as part of the live application. The model’s accuracy can deteriorate, when there are changes detected to the variables it is trained on. Statistical properties of a)target variable (concept drift) b) Input data (data drift) c)Operational changes (feature drift) can trigger model drift occurrences.
The ideal way to detect real-time model drift is to implement any techniques like Data Deviation Detection, Canary Pipelines, Drift Detection, Production A/B tests, Multi-armed bandit testing(multivariable | optimization) as part of the ML workflow. Another useful approach is, deploying models in “detection mode” as compared to deploying in (intervention mode), so that it helps to demonstrate the drift. Monitoring and Governance tools like Alibi Detect and commercial tools like fiddler, truera are getting quoted as emerging leaders in this space.
As a by-product of consumerization of machine learning applications, comprehensive production governance mechanisms and accountability are critical to ensure that adherence of ML compliance requirements like GDPR, Algorithmic Accountability bill, FDA etc. The expectation is all the stages in the machine learning workflow should be traceable for reproducibility, audit-ability and to assist explainability.
Conclusion:
These focused advancements to bring explainability and transparency to machine learning models is driving the change for ML models to power the digital business in real-time. With stochastic optimisation in prescriptive analysis, a lot of commercial use-cases like online recommendations, intelligent marketing campaigns, credit-loan decision making, fraud detection are paving its way AI enabled digital adoption. Extensive research still continues to bring explainability for more complex mission-critical use-cases like medical diagnosis, autonomous vehicles that need a great level of transparency and explainability.
We also learned that the principles of iterative and cross-functional communication remain more important than ever. Questions around explainability, fairness, and privacy need to be addressed by sustainable machine learning model management and governance structure. The usefulness of machine learning models will be measured in terms of business metrics as compared to model performance.
With these inductive nature of machine learning, it’s clearly evident that the era of AI commercialization is on a path to cross the “VoID”.
