Is machine learning still a hype?

Kaggle is the most popular data science community on the web right now. Reading through some of the interviews on popular Kaggle users (or Kagglers as they’re referred to) made me think about one question that the interviewer, Sanyam Bhutani had asked. Do you think machine learning is still overhyped?

The answers to the above question couldn’t be more diverse. There were Kaggle masters and grandmasters mentioning that the statement “AI is overhyped” is overhyped and then there were people on the other end of the spectrum that it was always overhyped and still is despite progress in the field.

Gartner popularly publishes its hypecycle in AI each year such as the one below detailing where in the “Hype cycle” each technology falls.

There is a 10x growth in the number of papers published in AI:

The question “is machine learning a fad” has even moved below in the autosuggest list of Google search:

On a side note, I’m sure the question “is machine learning capitalized” is the burning question on every researcher’s mind right now.

Let’s try to come up with some criteria before we decide whether machine learning is still a hype.

Machine Learning Adoption

Machine Learning First Companies

A lot of us are aware of the progress self-driving technology has made in recent years. AI first companies such as autonomous driving sector, robotics companies, computer vision, conversational AI, et al are the poster child examples for AI driven companies that machine learning and deep learning models in production.

Traditional Sectors being Disrupted

Traditional sectors such as financial services, healthcare, retail that are being disrupted by new-age companies such as

• fintech firms offering neobanking, credit and insurance underwriting on the basis of data
• health-tech companies automating several of the functions performed by workers in traditional healthcare and
• ecommerce firms using machine learning for inventory management, anticipating demand, removing out of stock and over stock, offering highly personalized suggestions, etc making a huge difference to the a company’s balance sheet

The main differentiation for these firms is that they are tech driven making much smarter than their older counterparts.

Traditional Sectors adopting AI

Given how quickly the new age firms are moving, traditional industries have also started adopting AI rapidly in order to survive. Examples include:

• Financial services such as Goldman Sachs, and many more examples here by Arthur Bachinskiy.
• Retail such as Walmart Labs , Lowes, Walgreens
• Manufacturing
• Supply chain management
• Construction

And many many more I’m skipping.

Companies realize it’s hard to productionize models

Gartner predicts that a whopping 85% of the machine learning projects never ship due to a multitude of reasons. Clearly there is a lack of good tooling today for a lot of these problems.

There has been tons of progress would be a very big understatement. Take a look at the State of Machine Intelligence I, II and III by Shivon Zilis.

Now there is a whole conference, TWiMLCon, “focused on the platforms, tools, technologies, and practices necessary to enable and scale machine learning and AI in the enterprise.”

There are also some nascent proof of a successful business catering to the needs of ML infrastructure such as Scale.ai.

Machine Learning now faces mature problems

As machine learning gains adoption in the enterprise, it’s facing a whole new set of problems and standards that a university project is not subjected to.

• AI Explainability: As AI makes its way into industries such as law enforcement, financial services and all, it becomes critical to know where AI might be wrong and why it came up with the result it did. This include the class of problems such as bias detection, making the block box more transparent — the recent furor over Apple Card, where the credit line was offered by an ML algorithm from Goldman Sachs proves the criticality of this problem.
• Reproducibility and correctness: In order to be usable in life critical systems such as autonomous driving, healthcare, machine learning needs to be reproducible and deterministic.
• Security & Privacy: As AI invades more aspects of a business, security has become a huge concern. Take for example computer vision, there are images that can fool an AI algorithm because they might rely more on texture than the features that a human looks at. This way face recognition might prove to be a big security risk. More such risks exposed by students at KU Leuven on how to hide from a security camera with a color printout.

In Conclusion

Whether machine learning is over hyped is a very subjective question. But what is not subjective is the tremendous amount of progress the field has made in recent years. Whether it is the sheer number of production deployments, the adoption by traditional enterprises or the maturity in the type of problems the field faces ahead of it.

Randall Munroe always has it right.

Check out this ongoing Kaggle survey for finding where people are spending most of their time and how you can improve it.