[Podcast] The next big thing in tech: edge AI 

The power of edge AI 

“Edge AI is the ultimate and probably only scalable way to do AI in the real world—collecting, analyzing, and acting on data where it lives” says Evgeni Gousev, Chair of the Edge AI Foundation and Senior Director at Qualcomm. In this episode he talks about the transformative power of edge AI and why it’s becoming a strategic priority for businesses across industries.  

Evgeni shares the evolution of the Edge AI Foundation (formerly TinyML), a global movement that has educated over 100,000 professionals and is supported by companies focused on innovation, including Schneider Electric. Evgeni explains how edge AI is enabling real-time, energy-efficient, and privacy-preserving intelligence directly at the source of data, whether in factories, farms, or smart cities.  

Furthermore, he addresses the challenges of deploying edge AI at scale, including fragmented software ecosystems and the need for skilled integrators. Lastly, he shares use cases that are already delivering measurable impact and offers practical advice for innovators and enterprises who want to start their journey with that technology.

Listen to the AI at Scale podcast

Listen to Evgeni Gousev: The next big thing in tech: edge AI episode. Subscribe to the show on the preferred streaming platform (Apple Podcasts, Spotify) or play the episode on youtube.

Transcript

Gosia Gorska: Welcome back. This is Gosia Gorska, the host of the Schneider Electric AI at Scale podcast. Today I meet Evgeni Gousev, Chair of Edge AI Foundation, formerly known as TinyML, a global organization dedicated to advancing ultra-low power machine learning technologies. Evgeni is also the Senior Director and AI Lead at Qualcomm. His work has earned him numerous accolades, including induction into the microelectromechanical systems and Sensors industry Grabs Hall of Fame and Evgeni also held academic professorships with Rutgers University and Hiroshima University. He holds an MS degree in applied physics and a PhD in solid-state physics. He holds over 60 patents and has published extensively in the field. So we are very happy to welcome you, Evgeni.

Evgeni Gousev: Thank you for having me here, Gosia. Really happy to be here.

Gosia: Yeah, it’s a pleasure. Actually, one source was stating that you have even filed over 100 patents. So do you count? Do you remember?

Evgeni: I think I stopped counting about maybe 10 years ago.

Edge AI Foundation

Gosia: I see. Well, it’s definitely a pleasure having you on our podcast. And the first question that I wanted to ask you is about the foundation. So can you share the journey of Edge AI Foundation from its inception as TinyML to its current state? What is your mission and what are you currently working on?

Evgeni: Absolutely. So formally speaking, the foundation started around six years ago, actually seven years ago in 2018. But the whole TinyML motion started probably about 10 years ago from independent companies working in this field. Back then at Qualcomm, we started the project on ultra-low power vision. So basically the goal was to see if we can run computer vision at extremely low power, like less than one milliwatt. At the same time, we had some activities in other companies too, like Google, Microsoft and other companies started to work in this field. And that was a recognition that devices were getting better, smarter models were getting into the field like convolutional network models and also software was becoming more capable. So that was natural for companies to start playing with this type of capabilities. So we worked on this project at Qualcomm for about four years. We launched our first product in 2017 in computer vision. And that’s when we realized that yes, it’s real. It can be a big opportunity. We started talking to other companies and people from academia and we realized that we really need to build an ecosystem. We really need to build a community here. So that’s where the idea was conceived. It was kind of basically a natural progression of technology evolution. And some leading companies like Qualcomm, Google, Microsoft, ARM, we were already working on this. And then we had our first gathering that was in the Google campus in February of 2018. And we were very surprised that the interest was overwhelming. And for us that was a testimony that yes, this is real. There’s a lot of interest. And then we decided to form an organization, TinyML Foundation that would be leading this movement basically and that’s how it was created. And then over the past seven years, the growth and the journey has been phenomenal. So we’ve been growing in all dimensions. So geographies, number of people, number of companies, number of products, number of business opportunities, number of students in academia. So it’s been really a very, very rewarding journey for us. Just to give you some numbers there, I think over these years, the foundation has been supported by more than 100 companies. We have about 40 companies now who are our strategic partners, including Schneider Electric. Schneider Electric joined TinyML Foundation about three years ago as an active member there and similar companies there. So we have over 25,000 members worldwide. And one number that we are very proud of is the number of people, students we’ve educated in this field over the past two years. More than 100,000, like 100,000, it’s how many zeros, seven zeros, students or professionals worldwide, they took TinyML classes on energy efficient machine learning and applications. So yeah, this just shows the magnitude of this. So that was, again, it’s been like a phenomenal journey and growth. And last year we also made a decision to expand the scope of the foundation into a broader mission to AGI as well. And we’ll talk about AGI in a minute. What AI AGI is and what AGI means for companies and for business. So basically we had an inflection point last year. We expanded our scope and now it was also rebranded as Edge AI Foundation with a bigger mission, bigger scope and bigger community base. So that’s in a nutshell where we are and what we do. The foundation actually drives this ecosystem forward. It connects companies, businesses, academia through various means. So we have several working groups, we have events, we do research projects, we have a scholarship fund to support students, many, many activities there. And people can go to the website agifoundation.org and all this information is there. So that’s in a nutshell where we are and what we do.

Edge AI is taking over the cloud

Gosia: Yeah. So the numbers are really impressive. And if more than 100 companies are participating today and supporting the foundation, can you explore and explain to our audience what exactly is AGI? What are some of the most exciting applications of AGI that you believe are actually changing the industry right now and why these companies are involved?

Evgeni: Yeah, when people talk about AI and people talk a lot about AI these days, it’s it always the default thing is ChatGPT, right? That’s how people think about AGI. There is this default association. If it’s AI, then it’s sort of like ChatGPT. So basically a lot of things in the AI space happens on the cloud, which is OK, that’s you have a lot of resources there in terms of compute and storage and so on. But at the same time, we know that AI on the cloud has some limitations and it’s been voiced by many researchers and companies specifically, it’s not really scalable. People are already talking about the energy crisis, energy gap. And in fact, we know that the cloud and data center construction in the world is slowing down, especially in some parts of the world that have energy deficiencies. And because it is just so much consumption, energy consumption happens when you conduct AI on the cloud, it works fine, but it’s not efficient. It has other problems too, and we’ll talk about some use cases in a moment. What if you need to perform your critical tasks in the field, like helping a remote worker doing his job and there is no connectivity to the cloud? What do you do, right? So I think those are some intrinsic limitations of the cloud. So it’s not efficient. There is quite a bit of latency because you need to access the cloud, you need to get the response back and so on. And there is this dependency on connectivity like in some places that you do there. So that’s where Edge AI and TinyML come to play and come to shine because we are doing machine learning and AI workloads in the physical world or at the boundary of the physical and the digital world. And this is, we believe, the ultimate and probably only scalable way to do AI in the real world, basically collect data and digest data and analyze data and make it actionable where the data lives. That’s in the physical world. Like for example, in the Schneider Electric examples, some of those examples, you have some power plants, you have other manufacturing facilities and they generate a lot of data, sensor data, camera data, people data. And why would you send this data to the cloud and get them back and process them? You just process the data right there. So like, for example, one use case is predictive maintenance. And there are many, many Edge AI and TinyML companies doing this kind of things. So the use case here is you put several sensors around your plant, like for example, it can be vibration sensors, it can be audio sensors collecting some noise levels. It can be vision sensors or environmental sensors. They collect data. And if operations go normal, then you don’t need to send data to the cloud like every second or so. But if there is something abnormal happening or about to happen, and I think when it’s about to happen, it’s even more valuable, then your AI models can predict like, hey, this piece of engine or machine or fan is about to fail and then you do maintenance before the catastrophic event happens, which can be actually quite expensive. And you have some data from companies who do this type of activities and it’s actually quite significant in terms of productivity gain and also overall operational loss. It can be like, for example, the uptime of these machines can be increased up to like 20-30%, which is quite significant, especially if you talk about big industries like oil and gas or energy and so on. So that’s where Edge AI and TinyML come to help. And in the past, you’ve had some plants and manufacturing with the sensors, but there wasn’t really a good way to aggregate, to consolidate and to digest this information. But now with Edge AI and TinyML technologies being available and affordable and deployable. We’re talking about manufacturing one of the use cases here completely to the next level, we’re talking about the digitization of industries. So kind of back to the main point I was going to make, why we are so bullish about Edge AI and TinyML. It really brings this level of intelligence right to the data level and that’s actually quite significant there. I think another important aspect of Edge AI, which I think we get a lot of very, very positive feedback from the customers and end users is data security and data privacy, which again, we live in a very complex world and all these data-related aspects are becoming more and more important. So we see more and more customers voicing concern about data being sent to the cloud and how they’re managed. So there is a big notion and interest from companies and end users to own their data, to control their data.  And that’s again where Edge AI and TinyML help them to do so because in this case, again, there is no dependency on the cloud. People who generate data or companies who generate data, they own their own data, they control their data and they make their data actionable for their enterprise, for their production, for their operations. So that’s why I think again, fundamentally thinking and fundamentally talking Edge AI is the way to go for real world AI applications. Again, for those reasons, it’s energy efficiency, which is scalable, it’s affordability. You’re not dependent on the cost of the cloud infrastructure and we can talk about cost of ownership in more depth here and also data privacy, data security, data sovereignty. I think those are the fundamental things and that’s what drives Edge AI. Edge AI is still a new beast in a way because people have been using the cloud for quite some time. So we expect we already see quite a bit of interest in this space and deployments. And I feel that we’re getting into the FOMO effect, like fear of missing out effect. A couple of years ago, manufacturing and enterprise companies, they were just curious about Edge AI. Yeah, this is interesting. This is cool. We are going to keep an eye on this. And now they’re like, OK, I need to do something about it. And we see more and more use cases coming up and more companies joining this. And it’s actually a great time and great place to be today.

Education and integration: edge AI’s next steps

Gosia: So the benefits are quite clear. But let’s talk about some of the challenges. When we are putting intelligence back to the sensors or closer to the production, as close as possible to the machine, to the operations, doesn’t it mean that we need completely different sensors? Are they bigger? And what are the challenges in deploying Edge AI solutions?

Evgeni: Yeah, I think in terms of sensors, I think there are plenty. I would say there will be probably more sensors in terms of them being smart in a way more digital with some AI inside of the sensors. And this is happening like some of the sensor companies, like leading sensor companies, we’re talking about Bosch, STMicro, Infineon, also leading sensor companies, they’re bringing more intelligence into the sensor itself. That’s TinyML. But I think talking about challenges in this field, I see two and they’re somewhat related. One is if you’re talking specifically about industries like energy or oil and gas and other traditional energy, typically they don’t really have an army of PhD or IT or data scientists, this type of people who can do it. So they basically need to have a layer like system integrator type of companies or companies like Schneider Electric that can help them to deploy their solutions. So that creates a little bit of a time and effect here because you need to have this part of the ecosystem develop like again, system integrators and other ones. So that’s one. And the second part is very broadly speaking is the software part, because in general, if you’re talking about manufacturing and industrial IoT and IoT in general, historically this field has been suffering from being fragmented. So it’s not like you have one software, you have one device that solves all the technology problems. Like in a smartphone platform, you have one platform and then you have 8 billion users who use this, right. So I think in the industrial IoT space, you have plan A, plan B, plan C and they have all different flavors. So it’s not like one-size-fits-all there. So there is this fragmentation that slows things down. So basically we need to have very robust and very scalable software platforms specifically for deployment. So again, the sensors are there, the AI processors are there, accelerators are there, but how do you connect all these pieces of hardware in a seamless way, in a scalable way and also in a secure way? Because obviously there is a concern with AI like what if someone hacks your manufacturing plant, right? That’s not a good thing. So this is probably the next level to make this soft layer and deployment layer like MLOps, DevOps pretty scalable, secure and affordable. And there are many companies working in this field, small and large. I would say this is probably the next level of innovations to make it truly scalable more in the software and deployment. And also there is one more angle here is educating the market because again, in many cases we are talking to traditional industries, they hear about ChatGPT, but they may not hear about these capabilities, what they can do at the edge in terms of technologies and how it’s going to translate into their language, what is there for them if I’m going to use Edge AI or TinyML technology, what is it going to give me, is it going to save my productivity or reduce number of incidents or what? I think so that also needs to be done basically educating the market, building more awareness there.  And that’s by the way, what the foundation does. We have one of the working groups, we call it blueprints, but it’s really showing how do you build blueprints for different industries? How do you deploy it there? So it’s kind of moving from the technology level one level up to system level, deployment level. Those are the challenges I should say. But again, challenges always come with opportunities. Whoever is going to solve these problems on a big scale is going to be a big winner company-wise or start-ups. And then talking about a very short timeline here, I think we’re talking about probably a couple of years until we’re going to see massive, massive growth in this space.

Edge AI surprises with real-world impact

Gosia: Yeah. And as you said, so many companies are already more engaged in the topic. Previously they were just following Edge AI, looking for some examples. Today they are interested, they want to try. And as you said, there is some work that has to be done at the level of operations of systems. Do you have any examples of a use case that was particularly successful that was a surprise to you, maybe in a positive way?

Evgeni: There are many, many use cases. It depends also on the vertical, I think one of the verticals of interest here as we discussed is manufacturing like predictive maintenance, for example. I think with Gen AI, for example, you know, large language models and Gen AI are becoming more popular again driven by the innovations happening with cloud AI. But what is also happening now SLM’s, small language models are becoming more and more capable and popular. And you can run these SLM’s on edge devices and what it means is that you can run some of the generative AI workloads there. I think one of the use cases we’ve done here at Qualcomm, it’s a worker assistant and the problem statement there is that the machineries and in general operations are becoming more and more complex. There is quite a bit of tribal knowledge happens in these fields, but how do you give them the tools to make sure they can effectively and efficiently operate there? So we developed some techniques using generative AI and running generative AI at the edge. And that gives a tool like a physical tool like device for workers to perform their tasks very, very accurately with no human error and much faster than you do it otherwise. And also helps with training new workforce there using this type of things. So that’s one of the emerging trends we see in this field is Gen AI getting into the edge. And by the way, the foundation is going to have an event at the end of May. And it’s also on the website specific Gen AI on the edge. This is the third in the series there. So that’s this use case bringing Gen AI to the edge and doing worker assistant type of operations of all kinds of things. Like for example, if a worker needs to perform some operations on a piece of equipment and if he or she needs to access a precise step-by-step manual on how to do it in a prescriptive way. Then on his handheld device, he can get all the steps like, hey, this is the piece of equipment and this piece of equipment was serviced last time a year ago and now it’s due for maintenance and you need to perform these ABCD type of steps to get it done. And then it gets into the records. So this type of thing, what used to be done previously kind of manually, now it can be automated. And again, it reduces time, it reduces human errors and in general helps this operation. Same is also true for example in warehouses. I think we are talking to some customers in this space and they’re saying that today many operations of warehouses are based on a piece of paper like somebody gets there and makes notes and then who knows what happens with those notes afterwards, right. So there is no history of those things. So that’s again where Edge AI is coming to help them because once you have data, it becomes more data-driven decisions and then, the technology is there. Another use case, very broad use case and very broad vertical is in retail. I think retail is becoming smarter.  It’s like there are a lot of vision-based technologies. There are camera-based technologies to monitor the state of your store, empty shelves, self-checkout lanes, customer behavior, and workflows. How do you do all this optimization there? So that’s a big field. There are a lot of opportunities in smart spaces, smart cities, smart offices, smart home type of applications, bringing this type of technology there. I think, for example, in the smartphone, several TinyML companies have launched products with acoustic detection of some events. Like for example, glass breakage, you could have a sensor at home and if somebody breaks in with the glass, you get a detection and you get an identification immediately that something happens there. Or some video-based technologies. I think the TinyML Foundation, talking about Smart Spaces, had a hackathon with the city of San Jose last year on pedestrian safety. How do you use this type of technology to improve safety, especially in some areas where you see most accidents and some dangerous intersections or some dangerous things. There is a program here in the US and in San Jose, here in the Bay Area, called Net Zero, how to reduce all these fatalities to zero. And that’s again when technology comes to play, the vision-based Edge AI. So many, many use cases, I think. And that’s the beauty of Edge AI and TinyML. Whenever we talk to people in different industries, they always find like, wow, I can use it in my space. Like, for example, in agriculture, I think we have an example in a farm in Arizona, actually it’s a winery in Arizona that uses sensors to improve their water cycle. And by doing so, they can improve utilization of water in the farm by 30%, which is actually quite significant given all the problems we have with water. So this is just an illustration. Every time we talk to people in different areas, then you see this. And then one of my favorite examples actually comes from Africa. We have a TinyML company called Hive. So apparently this is in particular from Kenya. In Kenya, what I learned is that the honey business is a big part of their agriculture economy. They have like 90,000 farms there and the problem statement there is they have about 30% loss in bee families every year because it is disease-related or some human act-related. And what this company developed is they put a small sensor inside of a beehive. So it’s a temperature sensor, audio sensor, and vibration sensor. And by just making this closed feedback loop, they can see what happens inside of the beehive, if it’s healthy or not healthy, if it needs some attention or if someone comes to try to steal it. So you have this kind of very, very basic device, but by brilliant AI and Edge AI technology there you can close the loop and you can really improve these things. This again just an example, completely orthogonal example or another one. Kind of my brain kicks in. It’s like what, 7:30 in the morning? Yeah, another example, another TinyML Edge AI company, they use vision-based technology for salmon for fish farming. So you put small sensors under the water and you can monitor the health of your fish farm, for example. So, and again, you can get this very valuable information how to monitor if you need to include any kind of actions there. So pretty much in every industry and we’re getting in this kind of phase when companies and businesses are becoming aware of this type of capabilities. And that’s why we believe it’s going to be explosive. Like when people learn how to use it and what it means for them. That’s kind of the beauty and the opportunity we spoke about 5 minutes ago.

Problem-solving is the new superpower

Gosia: Yeah, I definitely love all the examples and I’m convinced now that Edge AI is the future for all of these industries that you mentioned. So my last question will be about how can young innovators, how can companies looking into this field start with Edge AI? What advice would you give them?

Evgeni: Well, the number one advice is just jump in. This is an exciting field. You’re going to jump in and you’re not going to regret it. It’s going to be a lot of fun and a lot of rewards. If you’re talking about specific skills and specific gaps, more and more AI tools are becoming automated, basically becoming tools. So focus on learning, not so much tools, but more on how to connect all these different pieces and to solve problems. It is my belief that kind of the future of workforce and the future of education will be more problem-solution based. Not just kind of you learn one piece of knowledge and then you use one piece of knowledge for the rest of your life. That’s kind of in the past. It will be more how do you use all the techniques and how do you constantly learn to solve problems around like problems in your workplace, problems around us. So I think getting kind of in this problem-solution mindset is important basically to be agile, to learn new things and obviously you get some basics in the AI space. And that’s what Edge AI Foundation is doing. As I mentioned, there are already 100,000 people who took some basic level education in this space. And we are developing more material, more certification in this space to help to bridge this skill gap. But in general, I think what is unique about Edge AI compared to just cloud AI? It’s a combination of two things. It’s the machine learning in general and AI and embedded systems, like how do you do it in small devices? So those are kind of the intersection of these two fields. It’s embedded systems and data devices and machine learning and AI that creates this very, very interesting momentum. So basically to sum up, don’t wait, jump in, get your experience in this field and then work on problem solving there. So the problem solving skills are going to become more and more important to be able to connect all these different and sophisticated technologies.

Gosia: OK, thank you so much for all the advice. It was a very valuable conversation. And thank you for staying with us. It’s such an early hour for you. So I’m sure that our listeners will appreciate the conversation and we will share the link to the Edge AI Foundation in case they would like to learn more and follow the courses that are available for them.

Evgeni: Thank you so much. All the pleasure was mine and thank you for having me again.

Like what you hear? 

• Visit our AI at Scale website to discover how do we transform energy management and industrial automation with artificial intelligence technologies to create more sustainable solutions.  

• Listen to the other episodes of the AI at Scale podcast. 

AI at Scale Schneider Electric podcast series continues!

The first Schneider Electric podcast dedicated only to artificial intelligence is available on all streaming platforms. The AI at Scale podcast invites AI practitioners and AI experts to share their experiences, insights, and AI success stories. Through casual conversations, the show provides answers to questions such as: How do I implement AI successfully and sustainably? How do I make a real impact with AI? The AI at Scale podcast features real AI solutions and innovations and offers a sneak peek into the future.