I had a moment of epiphany when a couple of days back I opened the business section of ‘The Hindu’ online and the headline screamed ‘Machine learning, AI top professionals’ reskilling list’. When a newspaper so steeped in tradition starts talking about AI skills I think it’s time to take notice.
Going by the narrative in popular culture AI & ML are either going to bring about world peace by making drugs cheaper, employees more productive and internet safe for everyone or they will destroy world peace with the rise of Skynet and terminators and making almost all of redundant.
How are we, who advise customers in the best use of a technology, to react to such extreme predictions? As always the truth lies somewhere in between the above-mentioned predictions.
Let’s start with a clearer understanding of the words AI & ML. They seem to be used interchangeably in a lot of literature but are actually different in what they mean and are not quite the same thing. To quote a definition from the Forbes magazine
Artificial Intelligence is the broader concept of machines being able to carry out tasks in a way that we would consider “smart”.
Machine Learning is a current application of AI based on the idea that we should really just be able to give machines access to data and let them learn for themselves.
As Enterprises move forward from piloting ML technologies to taking them mainstream some key considerations will make the difference in the success of these projects and in-turn the success/impact on top-line as well as the bottom-line of the enterprise.
More than any other technology in the industry ML is the most heavily dependent on quality and availability of relevant Data. Since, it involves ‘learning’ from the data the algorithm’s need access to data (e.g. customer demographic details, spending patterns in case of the predictive offer or buying details and product information in case of spend analytics). Access to this data via a high productivity development platform like ODP will be a key ingredient to success.
1. In the rush to create the latest and greatest models a lot of enterprises/projects have overlooked the most important aspect of a successful implementation – Go-Live/production environment. How will the models lifecycle be managed? Remember, data is a living entity. It changes and evolves. The models need to change and evolve to stay ahead of the curve and stay relevant. How will we measure the efficacy of a model? Is it still performing and predicting to give the best results?
2. How will the enterprise respond to a regulatory request to explain a prediction made my an earlier version of the model? Do I have the ability to ‘roll-back’ the clock and re-run the model?
3. How will I scale my model? Is there a seamless way to containerize the model and be accessible through microservices?
Last but probably the most important aspect
Seems to me that this is a topic that comes up every few years in the technology industry. I remember the days when techies who understood the Y2K problem and could fix RPG400 programs were in great demand (yours truly being one of them). The skills challenge though for ML is a little different than the usual skill shortage problem.
Let me explain the difference with an analogy from my favourite sport – Formula 1 racing. To win a race you need a brilliantly performing car like a Ferrari AND a great driver like Fernando Alonso who can make the most out of this machine. Today a lot of enterprises are going out and trying to hire the engineer who helped create the Ferrari. What they really NEED is the accomplished driver who can do great things with the Ferrari. Unless you are Google or Facebook you don’t need someone who can design the next best algorithm. You need a data scientist and data engineer who knows when to use which algorithm and how to link the algorithm to get the best output.
Getting the balance of these three ingredients correct is probably the most important task in getting the Machine Learning project purring like Fernando Alonso in that red Ferrari.