How important is domain knowledge for AI and data science projects?

This audio was created using Microsoft Azure Speech Services

Promises are not always possible to fulfill, even if they come from the bottom of the heart. The inscription in the Statue of Liberty says, “Give me your tired, your poor, your huddled masses yearning to breathe free”.

Similarly, many pure Artificial Intelligence (AI) consulting companies would say “Give me your poor data, yearning to deliver value” and would promise to deliver insights and value to customers. However, those promises often don’t see daylight, and many customers are left disappointed.

What makes AI projects successful?

My strong belief is that we underestimate the importance of domain knowledge which is key at every step when working with data.

Starting with the project selection, domain knowledge together with business acumen obviously play a key role to select the right projects and use cases that will truly deliver value by leveraging and scaling AI.

But it also plays a key role during the project execution.

The data scientist will need to select and prepare the right features from the data set with the appropriate data quality (some aspects will be key, some less, depending on the domain). Without the appropriate selection, the model may never converge or will cost too much of computation power to make economic sense. Without domain knowledge, the data scientist will not have other choice than to take all “potentially significant” features and increase the risk of failure.

In addition to selecting the right features, the data science team will also need to define the success metrics. Here again, domain knowledge seems to me an absolute “must” for two reasons: to avoid aiming for the impossible and to capture what “enough value” means in each case.

Two more success factors are related to an overall culture, strategy and governance of the AI teams:

  1. To scale AI, we should always start with business needs and customer value-driven targets. Here again, domain knowledge is critical.
  2. An agile approach with a multiskilled teams is definitively one of solutions to address the gaps in current deployment of AI.

Choosing the right way and the right focus of managing AI projects saves time and team’s energy, so much needed to deliver tangible results.

How important the domain knowledge is for AI

Explainability of AI models

Then, an important roadblock to adopting AI solutions is insufficient explainability, i.e., the extent to which a model and its decision output can be understood by people. Without domain knowledge, in most cases AI practitioners will not be able to bring any kind of explainability. But, even with domain knowledge, in many cases AI models will not be fully explainable and adoption will be limited by the fear of the consequences of a “misbehaving” model. In order to increase adoption, even without full explainability, a domain expert will have the ability to put the right limits to the AI models to ensure that AI can only improve the current situation without taking any risk to reduce current efficiency level.

What is the impact of AI?

And finally, there comes the time to understand results and analyze the outcome of the models. AI is mathematics: input-in, input-out. Analyzing if they make sense is the role of the domain knowledge experts. They will be able to find, for example, the biases that can tweak the whole outcome.

Bridge to value: how important the domain knowledge is for AI

Giving it all a meaningful purpose

Even though some people still expect magic results from AI, it is not magic. It is also our responsibility as members of the AI community to continuously explain what AI is. As Galileo was saying “The book of nature was written in the language of mathematics”. And mathematics has always helped to model and describe the behavior of the physical world. So to some extent, there is nothing new in AI, it’s another mathematical tool to describe the nature that surrounds us and the physics law of the world.

On the other hand, there is something special about AI, that makes people ask: Can AI help with the big challenges of our time?

If I use Schneider Electric’s domain expertise as example, AI is like a bridge to a greater efficiency. It can boost and maximize our decarbonization and electrification efforts. Accelerating the sustainability gains and finding new solutions to address climate change through data science and analytics makes AI an extraordinary tool at our disposal.

What are your thoughts? How important are domain knowledge and experience in your field?

Join the conversation in social media by commenting here.

Discover how we do AI on

Welcome to our AI Hub

Tags: , , , , , , , , , , , , , , , , , , , , , , ,

Add a comment

All fields are required.