Unlocking AI’s Potential
The most value for businesses will come from domain-specific models, pre-trained on proprietary data, says Taimur Rashid, Managing Director of GenAI Innovation and Delivery at AWS.
In this episode, you can learn how Taimur advices cutomers on identifying successful GenAI use cases, what’s key in the design phase, and how his teams are monitoring the models in the production, maintaining their accuracy over time.
Taimur also shares about a GenAI use case with Schneider Electric, that resulted in considerable productivity gains.
Listen to the AI at Scale podcast
Listen to Taimur Rashid: GenAI and the art of the possible episode. Subscribe to the show on the preferred streaming platform (Apple Podcasts, Spotify).
Transcript
Gosia Gorska: This is Gosia Gorska, and today I have the pleasure of hosting Taimur Rashid. What Taimur brings to this discussion are six years at Oracle, nine years at Amazon Web Services, three years at Microsoft, and two years at Regis as EVP of AI. Currently, he is with AWS as Managing Director of Gen AI Innovation and Delivery. Altogether, it is two decades of experience encompassing leadership roles in product, market, business development, and cloud solutions, architecture, and engineering. His expertise spans big tech firms but also growth-stage start-ups, particularly in areas bridging technology, product, business, and go-to-market. He currently leads AWS Professional Services’ portfolio of Gen AI solutions, building end-to-end AI solutions for customers. He also founded Recursion Venture Capital, a boutique venture capital and management consulting firm. Taimur is a University of Texas alumni with a focus on automated and knowledge-based systems. Welcome to the show.
Taimur Rashid: Thank you, Gosia, pleasure being here.
Leading AWS’s Generative AI Innovation Team
Gosia: Yeah, thank you for your time. And you know, it’s an exceptional episode because we are hosting Taimur directly in our office today as he’s visiting Warsaw. So, we are sitting in the same room and it’s really great to have you here and welcome you in Warsaw. My first question will be about your experience and journey with Amazon Web Services. So, what’s your role right now in the company?
Taimur: So my role right now is leading the generative AI innovation and delivery team. We started the Innovation Center about a year ago for generative AI, and so we’re about a several hundred-person team globally distributed, and we help customers adopt the Gen AI platform of AWS.
Supporting Customers in Building Secure AI Applications
Gosia: OK, that’s really interesting. And how do you evaluate the importance of technology, data, and governance to build AI apps and how exactly are you supporting your customers?
Taimur: That’s a great question. When you look at generative AI, even though in many ways when people look at generative AI today, it’s the tip of the spear. They see these virtual assistants, they see chatbots. But behind all of those simple applications is a very complex set of technologies all the way from the underlying data to the technology that powers that data, the application workflows, and then more importantly, the security and governance that goes around the data itself too. And so, when we work with customers, we always like to start with what’s their use cases, what are their business imperatives. Many times, they’re trying to drive productivity gains, they’re trying to create revenue-generating opportunities for their companies. And so, once we have a good understanding of what their business drivers are, we can work backwards from there and really understand the nature of their applications, their data. We look at data readiness, for example, we like to see, hey, is the quality of the data up to par with the kinds of AI that they want to do. And then we look at security. Security is very important. And once we look at their underlying infrastructure, their data, their application framework, we can then start evaluating what’s the best way to start with an application. And in most cases, what we typically do is we try and understand what sort of security requirements they have. And based on that, we can build an application for them, typically a proof of concept, show them the art of the possible. And once they get that art of the possible, then we can expand it from there.
Understanding Gen AI’s Potential and Limitations
Gosia: I see. And looking at the customers and the use cases you’ve been working on recently, what’s the level of understanding of the potential and limitations of Gen AI? Do you have more customers that are just showing up and asking you, “We would like to start with Gen AI, look for a use case for us,” or do you have more that really come with a particular challenge or opportunity where they see that Gen AI would make sense and you start the conversation at this point?
Taimur: Yeah, that’s a very good question. I would say that when I meet customers, the two most frequently asked questions are what do we do and how do we get started, right? And that is why we invested in the Innovation Center because we bring human expertise into the conversation. And when we meet with stakeholders at the customer side, what we’re able to do is we’re able to identify a set of use cases that they can begin to use Gen AI for, right? And in most cases, people see things like ChatGPT, or they’ll see some other virtual assistant and they want to mirror that as a starting point. And what I’ve seen more often is that is the starting use case, a virtual chatbot. It might be a knowledge base chatbot as well too. And that’s a great way to sort of get started, get the system integrated with proprietary data and now a customer can actually have a conversational chat based on their company data, which then opens up a variety of possibilities for them.
Ensuring End-to-End Security in Gen AI Applications
Gosia: Yeah, that’s a great approach indeed. And you mentioned the security approach, the security side of these discussions. So, what’s the security first mindset and how do you support the security and reliability of AI models for your customers?
Taimur: That’s a good question. Security has been a first principle for all of our Gen AI engagements. In fact, when you look at the history of AWS, we’ve really emphasized two aspects which are so paramount. Number one is operational excellence and number two is security. It could be any application—you may have a front office, a back-office application. These are the first two most important principles that we have.
When we look at security, we like to look at it end to end, and what we typically say is that security begins in the design phase of any application that you’re building, from design through development through the actual integration that ends up happening. Often, what we see is that the tough part is integrating with other systems and making sure that the security policies are truly adhered to on all applications, all levels of the application, in fact.
And now when you’re dealing with multiple systems, when you look at a generative AI application, it’s generating a response based on a variety of different systems. And those systems of record may have very, very sensitive data. They can be combined with unstructured data, which is maybe a little less sensitive, but now the application has to be very aware of authorization and authentication. And so, when we work with customers, we typically have security end to end, from the design phase all the way through the production phase and even after production. There’s a variety of different things that naturally delve into responsible AI, which is a very broad category of things, with security and data privacy being just one part of it.
Checklist for Ethical AI Design
Gosia: Yes. And let’s focus for a second on the design phase. I was curious when looking at some of the architecture of our solutions, how much time and effort do you need to spend to design a robust application so that you actually remove the risk, you mitigate the risk of some potential harm or misalignment of how the application works versus the objective? Part of it is to be able to imagine what kind of questions, for example, the bot should not answer, like a typical question about how to build something harmful to society or an explosive. You basically need to think upfront and list these kinds of questions, list the areas that you don’t want the bot to give an answer to, or you want the bot to respond with “I cannot reply.” So how do you actually design this kind of application? Do you have a checklist for these topics? How does it work in practice?
Taimur: Yeah, so that’s a great question in practice. And I would say that getting it right in the design phase is the most important part because everything downstream from that will then eventually fall into place. When you actually look at the data itself, there are several dimensions that go into the design phase. One thing that you have to think about is, first off, the factual accuracy of the data itself. That’s where data quality comes into play.
As you look at the data quality around that, typically these models are trained with human beings giving the inputs and assigning certain weights to the models. This is where a lot of human judgment gets involved. The first part of the design phase is to approach it in a fair way. Fairness is a very important part of that. The only way you can really get to a point where you have assurance around the fairness of a model is by working with multiple stakeholders. The design phase is a process of a lot of collaboration with stakeholders, truly understanding the nature of the data, and then really saying how you can have the right controls in place whereby there’s no bias in how a model will predict something or generate a response.
Yes, we go through a checklist. Often when we work with customers, they don’t have a checklist. So, we can help and guide them on how to think about fairness, how to think about ethical AI, how to think about enabling the right mechanisms around transparency, and then more importantly, explainability. When a model or a Gen AI application gives a response, how can you actually go and check and see how it got there? That’s a process where you can show the reasoning process of a model. You can have the right traceability back to how a model came to a certain output. That process involves working with stakeholders, understanding the requirements around it, and making sure that the way the model has been trained, you can’t 100% remove bias, but you can mitigate the percentage of bias that goes into a model. And then from whatever you’re unable to remove, you have the right checks and balances downstream in the application to correct for that.
Monitoring Model Drift in AI Applications
Gosia: OK. And then you get the answer that a particular question is not the area of expertise of the bot, so it cannot reply. Yeah, so it’s a good solution. And the second phase that seems quite interesting to me is the operation of the model. So how do you monitor if there are no drifts in the model, if the model is not deteriorating the quality of the answers over time when it’s already in production?
Taimur: Yeah, that’s a great question. Because I always say that getting a Gen AI or any AI application into production is actually the first part of the journey, right? Because after that, you can have a variety of things that happen. The model can drift over time. The data can become outdated, leading to data drift as well. What’s very important is in those production environments to have the right observability capabilities. So, you can actually look at responses, items that are generated, and responses from the users to see if there is the right level of accuracy. It’s really a close feedback loop. As you detect and see that a model has drifted, or the data has drifted, you can initiate retraining cycles and update the data. Having those mechanisms in place is very important and part of the closed feedback loop that we support our clients with.
Gosia: Yes, OK. That’s really increasing the quality of how the model works and you can improve this over time with the feedback loop.
Taimur: Yes
Engaging Customers with AI Education and POCs
Gosia: So, I think it’s time to take a closer look at the Innovation Center that you mentioned because I’m very curious. Looking at the past months or even the past year, there was supposed to be the year of AI practice. This is how it has been announced. How do you engage with customers in terms of education, workshops, engagement, production plans, and how is the Innovation Centre supporting them? And what I’m really curious to know are the use cases, some practical examples, some innovative use cases that you’ve been working on recently?
Taimur: Yeah. So, we started the Innovation Centre in 2023. In fact, it’s been a little over a year now that we established the Innovation Center. What’s very interesting is that its roots actually go back to 2017 when we started what we call the ML Solutions Lab. This was right around the time when machine learning was really starting to become more mainstream with MLOps. Our philosophy around emerging technologies is that they can’t be easily adopted; emerging technologies require human judgment and human capital.
When the Gen AI wave was beginning, we invested in the Innovation Center, starting off with just about 100 or so data scientists and strategists. Now, fast forward more than a year, we have several hundred people globally distributed supporting all of our major regions. Our engagement model is the following: we pair up strategists and scientists with customers that want to experiment with Gen AI or have a clear idea of what they want to do but need guidance around it. We’ve seen that this is where we bring most of the expertise. We’re able to really understand their business imperatives and from that create and identify use cases, help them prioritize it.
We use an approach where we say what is the feasibility of a certain use case based on the readiness and sensitivity of the data. We have a rubric that we apply to identify the first use cases to pursue with a customer. There’s so much demand right now that we get nominations from the field, from account teams. We go through a process of qualifying that so we can get the best experts on those opportunities. The way we engage with them is through proof of concepts (POCs). Our POCs range from four to six weeks depending on the complexity of the use case. Once we’re able to demonstrate the art of the possible, get the customer excited, and show a Gen AI application on their data, we see that immediately there are more use cases that a customer wants to pursue.
In 2023, there was a lot of experimentation with bespoke POCs. In 2024, we’ve moved to more integrated POCs. Now customers have a set of use cases they want to pursue with us. In the process of doing multiple use cases, we can go back to the foundational layers like the data layer, DevOps, and application development aspects of the entire stack. We essentially build a scaffolding layer for them to do not just one POC, but any number of POCs based on that foundational layer.
We’ve supported customers with simple use cases such as a knowledge chatbot or a virtual assistant for call centers. Some customers want to use natural language interfaces to manage their cloud infrastructure. The use cases continue to expand from true generative AI use cases to a combination of classic machine learning models like product recommendations and fraud detection. Even in the retail space, customers want to use the technology for similarity searches, which can help with visual search, product recommendations, or even cybersecurity companies wanting to do anomaly detection at the perimeter of their security.
Gosia: And how much of this work with customers requires additional education? I’m curious to know how many customers come with ideas and you have to stop them because there is some homework that has to be done, like in terms of data management and preparing the infrastructure. How does it look like?
Taimur: There is a process around education. As much as customers come to us wanting to experiment with Gen AI, we also have to educate them. In fact, we constantly have to even educate ourselves because there’s so much advancement going on in the AI space. A new model comes up every other week, right? So, what we try and do is step back and say, what are the conceptual things that we have to educate customers on? We go down to the basics. We don’t even get to the point of talking about AWS services. We talk about what large language models are, how they are built using transformer technology. We discuss retrieval systems and when to use something like RAG, which is retrieval augmented generation. We make sure that we can land some of those concepts just so that customers are educated with all the things that go into building a Gen AI application.
Typically, what we’ve seen is customers want to know about which model to use. So, there’s proprietary models. We offer, for example, the Anthropic models through Bedrock. We also offer a variety of other ones like Meta’s Llama as well as Mistral. There’s quite a bit of education we have to do conceptually, architecturally, and then with our services such as Amazon Bedrock, Amazon SageMaker, and the likes.
Schneider Electric’s Knowledge Chatbot Success
Gosia: OK, so let’s get to the details. If you can give me one example that was particularly interesting for you.
Taimur: Yeah. So why don’t we give an example of Schneider Electric? The Innovation Center engaged with Schneider Electric. There was an opportunity where they wanted to create a knowledge chatbot for their service personnel. You can imagine that it’s a very involved set of things to do. This knowledge chatbot enabled service agents to access information in a very rapid and accurate way.
What’s very interesting about the use case is we had to implement RAG, which is retrieval augmented generation, so that the most up-to-date and factual information could be available in the model generation. When you look at a model, it has been trained at a point in time. But now if you want to bring more relevant or contextual data, you have to implement a RAG system. The Innovation Center partnered with Schneider Electric to build that capability. Responsible AI was very important, being able to explain how a generated response came to be and showing the reasoning process of the model itself.
We partnered closely on both the retrieval system, the responsible AI component, and building the chatbot. From the feedback we got from the team, there was a 40% productivity gain with service agents being able to access information in time and therefore resolving requests in a much shorter timeframe. That was a great example, and we had a great partnership with the Schneider team.
Gosia: Yeah, I’m happy to hear it. Indeed, explainability and the responsible usage of AI is something that is really top of the agenda. What I’m quite surprised with is to see how much effort we put in place to make sure that the tool adds value to the employees but, also gives them additional ways to get information and be educated. Most of the cases of these kinds of bots that we are creating give access to the documentation. So not only do you get the suggestion or reply from the model, but you also get the link to the source of where this is coming from. The person can go to the documentation itself and get confirmation that this is really the source and find more information if they want to educate themselves.
Taimur: Yeah. Enabling that is so important because it empowers the knowledge worker with just so much more information.
Advice for Aspiring AI Enthusiasts
Gosia: Yeah. And we have time for one more question. Maybe we can get a bit more personal. Could you share with us your journey in terms of getting into AI, following all the trends, new acquisitions, mergers, and everything that’s happening with the new models, how different AI companies are developing the space? Any particular advice for people who would like to learn more about AI?
Taimur: That’s a great question. The way I got into generative AI, I’ve always been close to data and emerging technologies. When I joined Redis, one of my charters was to enable an AI business for the company. The advice I give is, no matter how much things evolve within the model space frameworks, everything goes back to data. There’s no AI strategy without a data strategy. That foundational aspect of data becomes even more important with the evolving nature of the different models available.
My journey with generative AI started with data. Redis was a real-time database in memory, and there was so much value to be captured from that. When we added vector similarity capabilities into the actual data source, we opened up a whole new door for generative AI adoption. My advice would always be that data has the most gravity. Everything goes back to it. Even when you look at foundational models today, the future is very domain-specific models. You may have a frontier model or a foundational model, but you may want to distill it down to get it more domain-specific, more pre-trained on proprietary data because that’s where the most value will come for businesses across multiple industries.
Gosia: Yeah. So, the evaluation of potential and limitations is crucial for customers.
Taimur: It is. It’s important to understand what those limitations are because technology will always have certain constraints that you have to work around. That’s where evolutionary architectures come into play.
Gosia: I see. Thank you very much for all the insights and for sharing your story with us and our audience. It was a pleasure hosting you.
Taimur: Thank you so much. It was a pleasure being part of this podcast.
Like what you hear?
- Visit our AI at Scale website to discover how do we transform energy management and industrial automation with artificial intelligence technologies to create more sustainable solutions.
- Listen to the previous episodes of the AI at Scale podcast.
AI at Scale Schneider Electric podcast series continues!
The first Schneider Electric podcast dedicated only to artificial intelligence is available on all streaming platforms. The AI at Scale podcast invites AI practitioners and AI experts to share their experiences, insights, and AI success stories. Through casual conversations, the show provides answers to questions such as: How do I implement AI successfully and sustainably? How do I make a real impact with AI? The AI at Scale podcast features real AI solutions and innovations and offers a sneak peek into the future.
Add a comment