AI first - evaluating the opportunity

Ewen Fleming Ewen Fleming

We’ve looked at the growing number of ways artificial intelligence, or machine learning, can be used. But how should any given opportunity be evaluated? Here are some of the criteria that we have found important.

Exception processing - if exception management is critical but not feasible then deploying a machine learning solution may not be possible.

Explicability - there has been much commentary about the difficulty of explaining the decision-making of deep learning models. It is worth noting that there are models (e.g., random forests) that better lend themselves to explicability and in many cases, perform just as well as (and sometimes better than) deep learning models. The practical preferences of the machine learning community itself are also tending towards facilitating better explicability (e.g., through making use of newer regularisation approaches that explicitly weight fewer rather than more variables). If deep learning is still preferred, approaches exist that permit a degree of interpretability of a given decision by representing complex models with simpler models in the region of the specific decision criteria. Nonetheless, explicability – and the degree to which it can be mitigated – may be a key criterion where models must operate in highly regulated environments.

Data protection regulation - opportunities may need to be evaluated for compliance with data protection regulation. For example, GDPR describes prohibited circumstances and requirements for, among other things, automated decision-making and profiling.

Legal and reputational risk - any biases in the training data will show up in the model. If the training data captures socio-economic characteristics and trends which, although real, are ethically unacceptable, the machine learning model will capture, reproduce and even reinforce them. This potentially exposes the business to legal and reputational risk.

Make v buy - if the level of knowledge or development around machine learning within your business is low, a vendor solution may be a better solution. The machine learning vendor space is, however, still developing. The evaluation criteria you use to determine whether to go with a vendor must be tightly-defined.

Organisational change - determining whether the organisation can accommodate the business change is a critical factor often overlooked. For example, the HR implications of redefining roles.

Fresh developments

Two seemingly benign areas of development are most increasing the scope of how machine learning can be applied to business in the short-term:

1 Growing adoption of Bayesian learning approaches

Machine- learning algorithms don’t always get it right. But if they are able to give a probabilistic measure of their confidence in their output, exception management can be used.

The vast majority of historical corporate deployments of predictive analytics can’t do this.

They typically only provide the most likely answer. This is not good enough in many deployment scenarios.

Setting optimal levels of supermarket inventory requires an ability to trade- off the costs of perishability against the opportunity cost of a missed sale. When the utility of getting a decision right is different from the cost of getting it wrong, classical models can be inadequate.

The growing sophistication and adoption of Bayesian methods – which yield probability distributions over possible outputs to address these classical limitations – is proving key to wider corporate deployment.

2 Reducing the volume of training data required

A variety of methods are effectively reducing the volume of labelled training data required. The ability to train models in simulations of real-world environments has become a critical requirement for models operating at the extreme end of

As the ability to simulate real-world environments improves so too will our ability to train models with ever more modest demands on training data.

An added benefit of the Bayesian approach described above is that it has an in-built ability to incorporate existing domain knowledge into models. This can result in improved performance and shorter training times.

Another important development is in transfer learning, which takes several guises. Simultaneously training models on a range of tasks can result in faster training on any one of them. Transferring pre-trained models from one related environment to another can require considerably less fine-tuning to the new environment than training from scratch.

These developments and others are easing the burden of accessing high volumes of training data. But the most valuable experience a business can have in this respect is implementing a trained model and capturing the benefits. That makes it easier to gain the belief to invest in building further training data assets.

Making it happen

Innovative strategies can lead organisations into strategic cul-de-sacs or lead to unintended consequences. Some newspapers demonstrated this when embracing digital media without apparently considering how this might reduce physical circulation and the associated revenue.

In financial services the 1990s and the noughties saw a number of retail banks create new direct brands with separate infrastructure. They ended up competing for investment with the mother brand, as well as perhaps driving cannibalisation of their own customers. That ultimately led to a capping of expenditure and, for some, either mothballing or reintegration of the new brand – for example, Abbey National with Cahoot and NatWest with Primeline.

Innovation opens new doors and takes us down new paths. However, change generates consequences. When evaluating feasibility, businesses need to recognise that there will almost certainly be unintended consequences as much as those outcomes that we desire.

Machine learning holds much promise, but is in some ways the “new digital” as the unintended consequences of exploiting it are only being experienced for the first time now.

A critical part of successful change is to conscientiously manage the component parts. In the area of machine learning, examples include:

  • Data analytics and MI, particularly on process exceptions and cost
  • Customer journeys and direct feedback on how your customers think and feel as a result of the machine learning interventions. Driver analysis, leading to customer profiling segmentation and personalisation
  • Regulatory compliance and how planned changes may change customer behaviour. The furore over Facebook and Cambridge Analytica should have taught us that organisations themselves need to adhere to the spirit and not just the letter of the law. Regulation may take time to catch up with their policies and guidelines, however that is no defence and will not protect an errant institution’s reputation and market value
  • Organisational redesign. This often centres on the organisation’s structure instead of the capabilities required, to operate effectively in the new world. System, process and people capability should all be considered
  • Culture. Every change initiative ultimately depends on culture. Successful change places this at the centre of things. You need to think of how to get your people, and customers, to embrace, own and ultimately champion the new culture
  • Business processes. These are all too often scrutinised to make them cost efficient. A more discerning approach is to ensure costs are attached to value- adding processes. Cost that does not add value adding should be removed

Ultimately strategy and change is all about execution. Our experience is that machine-learning-driven strategy and outcomes should be approached as part of a holistic change initiative. Actively considering the change implications across all the areas that will be affected or can contribute can open up the scope and scale of opportunities that machine learning can bring to all parts of the business.

In summary, corporates in all industries must now craft a machine learning strategy. Using good judgement will yield the highest return with the fewest adverse surprises. And remember, informed and structured imaginative thinking along with the application of experience-based judgement will be the preserve of humans for some time to come.

This insight builds on an article published earlier this year in Disruption magazine.

For advice and guidance on how to implement, execute or make the most out of your existing AI strategy  contact Ewen Fleming

AI first - going beyond the first steps Find out more