Why Companies Struggle to Implement AI

Welcome to The Did You Know Podcast by Varisource, where we interview founders, executives and experts at amazing technology companies that can help your business save a lot of time, money and grow faster. Especially bring awareness to smarter, better, faster solutions that can transform your business and give you a competitive advantage----https://www.varisource.com

Transcript

Welcome to the did you know Podcast by Varisource, where we interview founders and executives at amazing technology companies that can help your business save time and money and grow. Especially bring awareness to smarter, better, faster solutions that can transform your business. 1.2s Hello, everyone. This is Vic with varisource. Welcome to another episode of the Did You Know? Podcast. Today I'm excited to have Nicola Massrenti  1.1s on the show with us and again, I'm going to have Nicola kind of say his name. It sounds much better coming from him, but Nicola is an expert AI consultant 1.5s for companies startups enterprise who are looking to implement AI for their business. So we're super excited to have you on the show, 

U1

Nicola. Thank you. Thank you, Victor. I'm very excited to be here, so thank you. 1.4s Can you 

U2

pronounce your name for us? Because you said it's so much 

U1

better. Yes. It's Nicola Macedi, 3.2s actually. Your pronunciation was good, so you're almost Italian. 

U2

Great. Yes, I'm definitely going to meet you there one day. So AI is obviously such a it's transforming every single week, it sounds like 1.6s it's amazing. Such a hot topic. So we're excited to go over a lot of questions and get your thoughts, but you have a pretty awesome background. You've been working on AI for a long time before it's even popular. Right. So can you kind of give the audience a little bit about your background? Yeah, 

U1

definitely. So 1.2s I took my bachelor and Master of Science in Milan. In Polytechnic. University of Milan. That is 1.6s a very well known and important university for engineering here in Europe. 1.1s And I started this. 2.6s The university dreaming and looking forward to do machine learning. But while I was close to the end of the master, I realized that was very much focused on robotics, which is cool, but 1.3s my focus was in machine learning and AI. So I very quickly steered the wheel and, you know, tried to work a little bit more in the software side, more in machine learning. So I attended a I mean, my master thesis was about a machine learning model that was guarding a robot because it was predicting the human actions. And then 1.9s that was the time when I realized, okay, I want to do AI is interesting. After that, I worked in two consultancy companies where I really made my bones. I learned a lot because in consultancy in general, you have to wear different hats and it was funny and also very cool. Then I worked in scale up about AI, and now I'm a consultant, I'm a freelancer, and I'm supporting 2.3s my clients in developing and structuring their business for machine learning. 1.5s

U2

Yeah. 1.6s Obviously, like you said, you've been doing AI before it kind of took over the world and is the topic for every single company, really. But what do you think from your point of view? Kind of the virality of AI in the last few months, right? Like, completely just now, every single person I've seen grandmothers talk to AI, right, and thinking it's their grandchildren. It's just like everybody. It's really for everyone. Now, from an AI expert perspective, what do you think? 2s First of all, it's very difficult for us in the field to keep the pace. So it's good that it's getting viral. 2.5s It's very good. But it's challenging because thanks to this virality and these progressional technologies, that's a lot to 1.1s read and to discuss about. I think that all of these so the. 1.6s The virality of tragicity and other technologies are helping many businesses because this is promoting 1.7s what is AI. 1.3s It's raising awareness in many business leaders in what AI can do and the impact. Because to be honest, four GPT. Many business leaders kind of knew what AI is and was but they weren't really aware 1.5s of how much powerful AI can be. So it is lowering the barrier both in awareness and in implementations. 1.1s A drawback is that the virality is increasing a bit the noise. So there are many 3.4s famous people that are talking about AI and claiming 3.4s performance that it is not sure that AI can reach. 1.2s There is a bit of noise in the discussion and in the crowd but I think that is very a very good thing for industries in general and especially 1.8s for the field of AI. 2.8s I also see that this is making many people think that now chat GPT, for example, similar technologies can solve all of the problems. That is not true. GPT and the newest technologies solve many problems, enable many new use cases. But 1.3s they are not the golden key for for everything. And there are foundational concepts 2.4s

U1

in machine learning statistics that are relevant even though and even with GPT. 

U2

Yeah, I think what I see is, first of all, talking to different AI experts like yourself, who truly understand chat GPT or large language model is only one portion or one way of AI, right? There's machine learning, there's so many other things that AI is and this is just a small portion of it. But this part went viral, which brings up the whole virality of AI because I think we figured out, or at least for now. 1.4s People enjoy the way to interact with AI, because AI before was a very complicated, sophisticated thing where experts like you understood it, but not the rest of the people. Right. And I think through Chat, which is how we do Skype or teams, how we text messages, that's how people communicate for the last 50 years. Right? And so now, I think because of the Chat GPT, because AI OpenAI has been around for many years, but even before that, it wasn't going viral. But once Chat GPT came along, there was a methodology to interact with AI. I think that's when it took off, people figured out, this is how we're going to interact with AI now, I guess. And yeah, I think that maybe no, but I agree with you. AI still can't do a lot of things, but the fact that you can interact with it, people can see that instant reaction, like, Well, I typed something and now I get something back. Wow, there's an interaction. Do you feel like that's maybe part of the virality? I guess? 

U1

Oh, yes, definitely. Because 1.6s with Cha GPT and similar other models, 1.2s you are writing, you are talking to this AI. But actually, if we didn't know that on the other side there is an AI system, 1s it would be difficult sometimes to really understand that that is not a human being and is for sure the virality key that helped open AI. And this is really enabling everyone in testing and playing and figuring out what AI can do, because. 1.4s I mean if you take my my mother for example, that is playing with Cha GPT and and you know often tell me hey, Nicola, look how cool is this and that and it is just a matter of a few prompts. If we take nontechnical people such as my mother and ask her mom, please set up a full pay line with data processing transformation in the database just to test this other AI. Another AI that is not chat based. But that would be impossible. So yes, Chat is definitely the reason why 1.6s now there is all of this interest in these technologies but again, Chat 

U2

GPT, 4.3s

U1

it's new in many things but the technologies that OpenAI is using it is not new. 1.6s Yeah so obviously with all that's happening in the last few months with AI, what have you seen as the biggest change in the industry? Not the virality part but just from an expert AI in your circle, what do you think has been the biggest change in AI? Is it the pace of innovation? What has been the biggest change that you see from the inside? 1.3s This is a good and tricky question because many things have changed it 1.3s so for sure now there are many new use cases that. 1.9s Um eight, eight months ago, one year ago were impossible. Now it's like bang. Immediately you get a result for something that a few months ago was impossible 1.7s at the same time. So before 1.7s you need to set up a full team of engineers 1.1s with a lot of skills to experiment and test and try to get a result out of a use case. Now, for some use cases now you are sure that with almost no effort you get a great result. So this is a huge change. At the same time, this is triggering innovation both top down and bottom up because everyone now see that it is possible 2.7s and everyone see that it is just a matter of some iterations, some prompting, some 1.5s research on the best way to ask a model to generate a stunning image, you need to add F four. I don't know why, but that is an example. If you search and test and play with the technology, you see that you get a result. And one year ago it was impossible. In addition to this, the businesses are trying to 2.9s set up the infrastructure and 1.3s their pipelines to support these new use cases and also to be compliant with the privacy regulations 1.2s because that's another problem. So how do I set up my infrastructure to enable the teams in doing machine learning or in delivering this use case? Or how do I ensure that the data from our customers, these sensitive data, are not sent to external providers such as Openi? This is something that is exploding now because this innovation is changing everything. 1.6s

U2

So and that kind of leads very well to the next topic, right? Which is companies or enterprises. I think when something goes viral, it means you see it on the news, you see it on CNN, you see it everywhere. It's not just in your technology world, but you see it everywhere. Right? And so that starts to make companies think about how are we going to implement, how are we going to leverage this technology? G so it sounds good, but they still have a hard time knowing how to leverage it, how to implement into their product. So for the companies that you work with, what are some one or two top use cases that you've seen, where companies are leveraging AI? 1.5s You think, 1.1s yeah, okay, for sure. 1.4s All the use cases that include text, this is a low hanging fruit, but 1.3s NLP, so natural language processing, now, it's very 1.3s easy to develop. 1.1s You can aggregate, extract and transform feedback from users very easily. And you can also structure unstructured data very easily, thanks to GPT. You can, 1.6s for example, 1.5s customer service. You can automate part of the customer service. 3.5s I mean, this is all about 1.3s the newest technologies. You can transform data very easily. And because you have an engine that is the AI, that is very powerful in understanding 2.5s the text. Okay, there is another 2.2s use cases come from the visual models. So 2s they are not as famous as GPT, probably, but the generative models now are creating images and are lowering the cost in generating 2.5s content 2.1s because before people needed to staff photographers. Now it's easier. 2.3s And I'm also seeing that the combination of the two. So text and vision are generating new use cases because you can kind of 1.2s combine the description with an image, especially with the family multimodal models, you can combine both the sources to get something very powerful. For example, you imagine a search engine that want it is not difficult to imagine, but a search engine that want to retrieve 1.4s text and images given a user query. 1.2s You know, these new models are enabling all these use cases that they go from text to vision to video. And there are 1.2s another family of use cases that is really 1.2s starting now. That is about the 3d 1.9s graphics and 3d objects that is again about the new technologies newest model that are generating enabling new use cases for example you can type a beautiful house with garden and these models generate the 3d object of what you just described. 1.3s

U1

This was impossible before, now it is starting to be possible. Yeah 

U2

so you obviously work with again startups and enterprises before so when we talk about implementing AI or leveraging AI, what issues and challenges because you talked about the use cases, right, which is hey, what's possible? But what about the challenges that you typically see when they try to implement AI? 2.2s Is it a technical challenge, is it a business challenge? What challenge do you think companies usually run into when they kind of leverage AI? Yes, 3.3s the main challenges can be divided into four families the first one is about skills because working with AI especially no. The real 1.9s group of skills that you need to work with AI at a professional level. So not only about prompting, but going a little bit further, what is an embedding? How to fine tuning your model, how to evaluate your model, how to design the neural network work. There is a lot after prompting and there is a set of skills that is fundamental to work at a professional level in this field that is about AI but also about the tooling. You need to know how to code, how to use technologies that are made for production because to 1.3s move from your desktop to the production to the customers and 1.3s you need to know how to do that. In addition, there is sometimes a lack. So this is one family. Another family is about the culture business that is not technical. Of course often think that AI is like a magic box. It is not. 1.2s And it's about explaining the process, what they should expect and what they should not expect. They should expect some uncertainty and errors in the predictions because that is part of the process and that if there are errors, those can be addressed. But it's it's not a deterministic process such as other development plans. 1.4s And there is another challenge that is about the infrastructure and the orchestration of the jobs and the processes. That is important because as soon as you have a few customers, you need the infrastructure that is serving the predictions for those customers. And we all have seen the challenges that Openi had with their infrastructure structure because often the model was offline and we couldn't interact with the model. Of course, the volume that OpenAI had to serve was enormous. But at lower scale, other businesses face similar issues. 

U1

And the fourth pillar is about data. Especially if you need to train a model, fine tune a model and work on your business specific data, you need data because AI 1.3s is all about data. Sometimes I see that companies don't have 1.3s well structured and well organized data and that is a blocker. When they want to develop AI use cases. 2.1s It. 

U2

Yeah, no, that was a great kind of insight into the different pillars. And again, I think that's why companies need consultants like yourself and experts, especially when they don't have those expertise in house, even though they think, hey, just implement chat GPT, connect the API, and boom, done. We got magic here. Like you said, it's not so simple. 1s One of the things, again, I'm definitely no expert, but even for me, there are tools like hugging phase, runway, chat GPT, and Throppic. There's so many different 1.3s pine cone and there's different terminologies, there's different tools and language models. 2.3s There are so many, I guess, different tools now, just in a very short period of time, 1.4s is it important for companies to just pick one and just go with it because they all do the same thing? Or how do you, from an AI expert perspective, even for you, right? Are you having to learn all of these different tools and then figure out what fits companies the best, or they pretty much all do kind of similar things. 2.2s So this is another tricky and good question, because this is something that I'm struggling with, that is about 4.3s the number of 1s technologies, models, toolings. This is really getting out of control. 1.1s It's impossible to keep the pace and being an expert in all the different new technologies. So what I'm. 2.2s

U1

What I do in general is I try to 1.2s go a little bit after the marketing and I try to understand the new model, what is using, what is the architecture, the technology, because often 4s AI models, especially now, are basically the same. They just change 1s the data they are trained on and that is a new model. But that is true because the outputs given the same inputs are different, but the engine is the same. So this is something that helps in 1s saying, okay, it doesn't matter if you use A, B or C because more or less is the same. 3.9s It's also true that sometimes 1.7s you need to experiment with different technologies. Now, again, I use Cho GPT as an example because I think that everyone has played with it. But GPT is expensive 1.2s if you make a lot of requests 1.5s with a lot of tokens. But maybe you don't need the biggest machine for your use case. Maybe if you play and test another smaller model. 1.4s Let. For example, Dolly from databreaks that is smaller, it is open source. You can most likely deploy it on your infrastructure, maybe with a bit of effort. With a smaller model, you save money and you have the same output. So it's about exploration and exploitation. You need to kind of be aware of the different trends and keep an eye on where the industry and the research are going. But at the same time I think that it's very important to be pragmatic and kind of saying okay, one is not enough. Most often you need to use three, four, five test, three, four, five different alternatives. 1.1s But if you know what you can discard, this becomes very easy. And I understand that this is something challenging because it's challenging for me that I can understand the technology and the implications. 1.9s But it's especially challenging for the non technical person because 2.1s there are so many news, so many 1.6s new papers announcement that is very difficult to kind of filter out the noise and keep the relevant information. 1.5s Yeah, 

U2

if it's challenging for you, then we have no hope. 1.9s But we I think a lot of users, the virality part is people want the results, right? It's just they don't want to know how it's made behind the scenes and maybe they don't care. But no chat GPT I think has a first mover advantage. I think sometimes that's an important thing just because they were kind of the first one to cause the virality. People don't even know the other companies exist. But for enterprise I think they are looking into these things. But as we finish up a couple of last topics so one is you mentioned security. Obviously it's important for companies. Can you give maybe a few tips or insights from an expert perspective on how companies can better 1.2s secure or look at security or implement security for AI? Any kind of quick tips or thoughts? 1.9s

U1

Yes. So one of the biggest problems I see when exposing generative models to the customers is prompt injection. 1.2s You can make the model say non ethical and statements that are not safe for work and those have an impact on their position of a company. That is definitely something that companies that want to kind of use GPT 1.4s as a gateway with the customers, they need to be aware of this at the same time. So there is 1.2s this side of defense towards the customers that is problematic for some use case basis. The other side is once you have customer data and once you have sensitive data, if you use an external provider. So. 1.9s Openi. For example, how do you ensure that you do not share sensitive data? And that is very difficult. 1.1s You can use other models developed in ours fine tuned on your data where you have your own people to apply. The able and build this training data set. So basically customer send you data you have another model 1.7s that says hey. This part of the text is about name and surname. This is the phone number. This is other sensitive data. You don't want to share this to open AI or to other external services. Make sure to remove those information. That could be a viable 2.8s street. 1.4s Other options are. 2.1s To be honest, they have to be addressed use case by use case. But it's something that it's a challenge for many businesses. Yeah, 2.1s especially this technology is still very new and so, yeah, you kind of get the value, but then you got to understand the potential risk. So to kind of finish Shaw with a fun question, so I gave this story where I mentioned that every time you buy a computer, 1s Windows Computer anyways, the first thing that shows up, it's the Edge browser, right, this Microsoft default browser telling you to use it. And then the first thing every person does when they buy Windows Laptop, okay, is they go download Chrome. Kind of like the first thing that they do. Even though, 1.8s again, it's that first mover advantage in a way where Google has transformed the browser, right? But it's just fascinating because of Chat GPT and OpenAI partnership with Microsoft, microsoft has really transformed themselves really in the last six months of the perception of Microsoft, of their browser, of their Microsoft consumer products. I mean, things that you think for ten years about Google now, you're kind of thinking like microsoft doesn't look so bad, right? And so it's just really fascinating how fast a perception could change 1.2s

U2

behavior. Maybe it takes time, but the question for you is Google again, Google has always been seen as the AI company, right? And they are at the forefront of it. Yet really in the last six months they kind of made a few maybe mistakes or maybe they came out later and so had a couple of disadvantage where now people see Microsoft and other OpenAI as more of that that leader. So just from your expert perspective, you know, love to kind of get your thoughts on 2.4s do you think Google is still kind of able to catch up? Do you think they're still at the forefront, they're just waiting for their time to strike? Kind of what's your overall opinion about the Google and OpenAI and all of these kind of big companies going after this market? 4.5s So I was surprised too, in this lack of responsiveness from Google. I expected a stronger, 3s stronger outcome from the announcements, given the attack. Let's see from my and open AI. But I've been thinking that 1s I think that Google is really in a difficult position, because, first of all, Google 1.6s is the inventor of the brick that is powering, choji, PT. Yeah. It all goes back to 2018 with the Transformer. That is the technology, really the art of the newest large language model. So Google is the inventor. Okay. And Google has always been, since 2018, the leader in research, in the number of publications and the impact. So Google, I think that Google has the technologies. What I also think is that they are careful in what they expose on the Internet and the products and the integrations of the eye in the products, because they now are the leaders in search and the majority of their business is thanks to their positioning in the search industry. Right. They have to be very careful in. 1.6s The way that they change their strategy because 1.3s this has a direct impact in the revenues. And I think that the first iterations 1.1s that Google did 1.1s in releasing the new their generative AI model was a little bit 1.1s and it wasn't bold because 1.5s I had the impression that they released a model with chains basically that couldn't say and couldn't perform at the best. Now 1.3s I think that they realized that it wasn't enough and with the latest announcements they showed the world that they have powerful machines and powerful 1.9s AIS. I don't know how it is going to end because for sure Microsoft being really strategic in being fast and 1.2s saying to the world, hey, if you want to use these new technologies that are a breakthrough in the history of humanity, probably. 2s

U1

We are the providers and we are the only ones that have these new technologies. So I don't know. I don't know. It is going to be an interesting war. I'm preparing map of Corn. 

U2

Yeah. Just as a technologist, when you have competition, you have innovation, right? Because one company is scared of another taking market share and it's just that fear and it's also happening in real time. But yeah, it's really fascinating. I think ultimately the people are benefiting because as they fight amongst each other innovate faster than each other, ultimately maybe the people are benefiting. 2.5s It's a pretty amazing time to be in. I think. So our last question as part of the conversation, if you had to give us kind of that one year to two year horizon view 1s from an expert like yourself in the industry, what do you think are some of the big things you think AI will bring in the next year to two year? Even though every week there's innovation, right? But just big picture, something big, drastic, maybe the next one or two years. What do you think? 

U1

Yes. So I think that the multimodal learning, this trend of. 1.2s AI that can use text and vision and audio and video all at once. I think that this is going to be more and more powerful because this is 1.3s the obvious trend that research is taking and that research should take. I think that we will have more us where we for example, a use case that thinking out loud. 3s There was also a POC from OpenAI. Basically the presenter drafted on paper with a pen how a front end should be and 1.6s

U2

do 

U1

GPT four kind of produced the output. That was the POC first attempt. This is going to be even more powerful. For example, imagine 2s you say. 2.5s I really don't know because it's so new that I have to think about it. But the intersection and the synergies between text and voice and video and audio will be even more powerful. I think also that the 3d generation so models that generate 3d objects are going to get better and we will have you know, I don't know about the Metaverse because this is also about the metaverse. But for sure we will have many ways to consume digital 1.2s environments that go beyond the also think that we will have better 1.3s models that generate better videos. So 1.4s we are seeing now some examples in the latest weeks have shown some examples of innovation in this field. I think that this is going to be even more powerful because other than the research, 1s also the industry is interested in having AI to generate videos, short or long videos but as a tool it's something new that it is going to save a lot of millions to big productions. So also industry is interested. 

U2

Yeah, I think 1.1s in a few years maybe anybody can just write a prompt and now you have a Hollywood video mid. 1s Most things are all possible now and it's exciting. So the last question we always ask every guest is you've seen a lot, you've done a lot. Nicola, if you have to give one advice, whether that's a personal advice or a business advice, whatever you're passionate about, what do you think that advice would be? 3.5s

U1

I think 1s it would be be courageous, just play around with things and do what you and you'll find your way. 2.5s Yeah, 2.4s

U2

I think a lot of people are playing with it, even grandmothers are playing with it. So I think we're definitely hitting on something amazing. 1s But yeah, no. Thank you so much for your time on the show today and excited to partner with you.