AI for Corporate Curriculum Development and Democratizing Education with Anna Talerico

In this episode, host Deep Dhillon converses with Anna Talerico, an entrepreneur with a rich history in education technology. Anna's journey, from bootstrapped startups to her current role as CEO of Corporate Finance Institute, provides a unique vantage point into the changing landscape of education. The two’s focus on key themes like curriculum development, democratizing education, and nurturing curiosity brings to light the evolving dynamics of learning. Anna and Deep then dive into the role of AI, particularly large language models like ChatGPT, in reshaping education. They explore the delicate balance between efficiency and human connection, discussing how technology can enhance but not replace the irreplaceable human aspect of education.

Learn more:

• Anna Talerico, CEO of Corporate Finance Institute
• AI in Education: Improving Learning Outcomes and Enhancing Teaching Efficiency with AI
• Improving Teaching Outcomes and Classroom Mental Health Using AI

Transcript

[Automated Transcript]

Deep Dhillon: Hi there. I'm Deep Dhillon. Welcome to your AI Injection, the podcast where we discuss state of the art techniques and artificial intelligence with a focus on how these capabilities are used to transform organizations, making them more efficient, impactful, and successful.

Anna, thanks so much for coming in. Can you... Maybe tell us a little bit about yourself, your, your background, and how you got started with the Corporate Finance Institute. 

Anna Talerico: Yeah, well thanks again, uh, so much for having me. I'm excited to talk about this today. I, um, am a founder and an entrepreneur by trade. I had a Company that was bootstrapped, so it didn't take any outside investment.

Um, and that was acquired in 2017. It was a marketing technology platform. And after that, I joined a company called Linux Academy, which was a cloud training provider, online training. And, uh, had a pretty crazy couple of years there. We were acquired by our largest competitor and sort of a once in a lifetime experience.

And I left there and went to Arthur Ventures, which was our, they were our investors at Linux Academy. Linux Academy was the first time I had worked in a venture backed organization. It was a fantastic experience. I really got to understand the power of the right venture partner. After being, uh, you know, died in the wool, tried and true bootstrap founder.

I really saw there was this whole other world of what, uh, venture backing could be like, so I joined Arthur ventures as a operating partner to work with our portfolio companies, and that's really my forever home. Um, and, uh, you know, I love being at Arthur ventures as an operating partner, but as part of that role, I often.

Go spend extended time inside of our portfolio. And right now I am the CEO of Corporate Finance Institute. It's one of our portfolio companies. I've been working with the company for a while, working with our, the CEO and the founding team. And when he decided to step down, he, he and the board asked me to join as.

CEO. So this is my second forever home being, you know, here for the long long run as we build an enduring, sustainable brand. But that's just maybe the two minute highlight. Cool. 

Deep Dhillon: Well, that's, that's good to hear. I've, I'm kind of on the opposite end of that where I've done a bunch of venture backed stuff and now started Xyonix doing not venture backed stuff.

So try to bootstrap on our own. Um, but it's, It's, it's nice to appreciate the differences because they're so, they're very different and there's a lot of pros and cons of both approaches. Why don't we get started with you, maybe tell us a little bit about, um, CFI's users, who are they and what are they trying to achieve?

Anna Talerico: Great question. So, uh, CFI, Corporate Finance Institute, provides online training for finance and banking professionals and aspiring professionals. So our customers come to us often to gain new skills, so they might be able to get a promotion or work. work more effectively in the role that they are in.

Oftentimes they might be coming to us, let's say right out of college, and they feel like they need more practical skills as they're starting to enter the workforce. They have the theoretical skills, but, but they need the actual kind of practical skills to jump right into the roles. And sometimes people come to us as career switchers.

They've been doing something else and they want to break into finance or banking and They're looking to shore up their skills, so that's kind of a we have a pretty broad customer base in terms of summer season professionals looking to just stay current and brush up. Others are looking to do career switching or just get ready.

And the interesting thing about CFI is, Super global customer base. We have customers in 150 countries all over the world, which will be interesting as we start to talk a little about our first AI application and how we're thinking about it. But, um, yeah, super global audience, which is a blessing and a curse, right?

We know how do we serve so many different markets in a way that helps them become finance and banking professionals. It's a unique opportunity for sure. Yeah. 

Deep Dhillon: So what are the, what are some of the key use cases? I imagine you have a whole content, uh, aspect to the business where you've got a bunch of content, but what, what makes CFI different than a normal kind of online learning place?

Do you have like curriculums? Do you have instructors? Are they your instructors? Like, are there, is there a specific coursework people are following? Is it unstructured content display that they're navigating, consuming discussion forums? What's, what's there basically and then what's different and what are the key use cases?

Anna Talerico: Yeah, love it. All of the above, right? We have a programming that is training. Let me step back training. Traditionally, it's quite episodic, right? We're not usually training all of the time in our lives as professionals. We go through different phases of training. Um, certainly CFI is there for that portion.

We have curriculum, you know, courses that people go through, um, you know, just like you'd see in a traditional, um, learning, even, you know, electives and different types of, um, courses, but in requirements and things like that, we have a certification component. So we have five different certifications that are quite rigorous to go through.

They require a big investment. So our customers can choose to go through the certification pad. Or they can choose to kind of design their own curriculum or go through some of our preset courses. But then we also are really with our customers through the life of their professional journey. So we have, you know, members only programming, we have community and networking and things like that, that are sort of there during in between the episodes of training.

Um, and you know, just even micro lessons, right? So we might learn in a course around about how to create a three statement model. But then we might have, um, a pro, you know, micro lesson on how to troubleshoot your three statement model that you're actually using two years later as you're, you know, struggling with one, for example.

So we try and be there for those, uh, training moments, but also in between those training moments as well to just enhance how somebody's working. 

Deep Dhillon: Are your users professionals on their own that have the relationship with you, or are they coming in through their companies who have the relationship with you?

Anna Talerico: Both. We really started as B2C, so it was a lot of individuals, and we launched our B2B business about two years ago. Um, and so now it's a nice equal, you know, mix, but it started as individuals coming and wanting to just, you know, change their lives or, um, again, get a promotion, land a job, that type of thing.

And now, um, we have a lot of business, business customers that are buying on behalf of their, um, their employees, which is different, right? As individuals training tends to be episodic, but as. Corporate citizens. There tends to be more of an expectation around ongoing learning and development. Um, it used to be that the organization expected that of employees.

But now we're hearing more and more about employees. Once those ongoing learning and development moments from their employer. So it's kind of interesting to see cyclically how that's changed. Um, but you know, you asked about our different how we're different. And today, our differentiation is really around how we think about the curriculum.

Finance and banking is steeped in tradition. It's almost always done in person, even still today. So CFI was the first to market with a full, robust course catalog that delivered everything entirely on demand and online, which to you and I as people that live and breathe in tech doesn't sound that groundbreaking, right?

But it was a totally different delivery method for organizations and for individuals that gave them a chance to have this kind of world class training that normally would cost. So much, you know, cost prohibitive for individuals and for smaller organizations, it gave people access to that type of training when we launched, but then second to that, you know, in terms of our course curriculum, all of our instructors have worked in the sector and so our content is not born out of academia or theoretical knowledge.

It's designed. From sort of how to get people role ready from people that were in the role. So we leave the theoretical stuff to the academic setting and we focus on the practical. Hands on skill development that somebody needs to actually do the job, but we're looking for a first year analyst to sit down on day one at their job and know exactly what they need to do and how to do it.

Or a career switcher who's never been in finance to feel confident in the skills that they acquired to help them go pursue the roles that they want in finance, even if it's not their background. So for us today, It's not so much about our technology platform and delivery, but more how we deliver it.

Certainly that will evolve over time. 

Deep Dhillon: So you, you mentioned instructors, um, do you, that, that you have, do you also have advisors, you know, where people have like a one on one advisory kind of a relationship, or is it really oriented around the coursework and people, um, kind of progressing through a course?

Anna Talerico: Yeah, so today it's really mostly I would say around the coursework. We have our community where people can tap into, you know, their network of other learners and certainly some of those learners kind of can act as mentors. But we also have program specialists, which are people that help our learners through Their learning journey with questions about the material, just like you would have a, you know, mentor or a tutor, our program specialists serve that role for our learners today.

Deep Dhillon: And are these, um, instructors and program specialists, are they? Employees of yours, are they part of like, uh, an external community and they're like freelancers or something, or are they doing it just out of their, you know, for their own reasons, just for, you know, out of the goodness of the heart, like, or for, for, you know, for gaining street cred or whatever, like, tell me a little bit about that.

Anna Talerico: Yeah, so they're all employees of CFI. So our program specialists are full time employees. And obviously they need to be they need to deeply know all of our content so that they can help learners, you know, as our learners encounter questions or need help and support. And then our instructors are all full time employees as well.

Some of those instructors will tap into people that are specialists in something to teach a course or to help develop course materials, um, somebody who's still working in sector, but that's always done via the conduit of our full time, um, instructors. 

Deep Dhillon: So I'm going to change the direction a little bit because this is our show is of course all about AI.

So I imagine, um, you know, if we rewind three or four or five months ago, you saw a ChatGPT and you're like, wow, okay, this stuff is amazing. It generates a bunch of incredible content. Is that what we're talking about with respect to AI? In your organization, and are we specifically talking about bringing efficiencies to your instructors and your program specialists and optimizing how they interact with their, their students and with your students?

Or are we talking about something else? Both. 

Anna Talerico: So I think that there's the learner journey and their experience and how we support them through that with AI. And then there's the other side of how we create. Courses and curriculum and how that might be supplemented and enhanced through a I in the future.

Go flashback to a few months ago and ChatGPT, you know, really slow. We knew right then that this process of education was forever changed. We also knew that there was no way to predict what that might look like, right? We could sit around and try and predict it, but we knew that. It was one of the things we had immediately was like, we're not going to out AI, so like, we need to just lean into this and know that we're not going to have all the answers and be open to the fact that this is going to change how people learn and how material is delivered.

We need to be open minded about that, not fight it, but also not sit around to spend time trying to predict where it was going, because it's for us, impossible to predict what this looks like in a learning experience, you know, two or three years from now. 

Deep Dhillon: So one of the things that you know that I see a lot as a kind of an emerging idea.

There's like a bunch of students in a room and and you know professor at the front lecturing there's that old older model and I mean it's still prevalent, of course. But then there's also this model of, you know that maybe you have later on in your career when you have folks working for you that you can just kind of have conversations and ask them a bunch of questions and pepper them with questions and you learn quite a bit that way most of us have been.

Kind of through that evolution, find that having experts all around you, you learn at time. I'm curious if you're thinking of, you know, these large language models as a vehicle to construct kind of content for digestion and like reading and listening or whatever, or if you're also thinking about students interacting and going back and forth and talking and like pulling on information and, you know, in that chat like modality.

I mean, I feel like your ChatGPT probably more than anything has. As just kind of redefined how we think about how valuable chat actually can be in terms of learning. Do you find that to be the case? Or are you thinking about something different? It 

Anna Talerico: starts with chat, for sure, for us. We are, we have a chat experience right now that's in beta.

It goes live in early July. Um, and so that was our first step is like let's launch a simple chat bot that's been trained on our content so we are confident in the responses given and how it's interacting with our learners because You know, we all know right now You can't always be confident in the material that you're getting in some of you know, some of the element, um large language model Interfaces like chachapiti.

So first let's train a chat. Um experience like on our Content itself. So that as a learner's in a course that they can ask clarifying questions about the material, so it's not yet a tutor, but it's certainly I imagine will be more tutor like right now. It's, um, you know, asking questions about the content and clarifying and we know that that's already something that our learners need because we have program specialists and that's what they're doing.

So we're looking at these chat experiences Extending the scale of our, uh, program specialists. And rather than having to kind of come outside of the learning experience to ask that question, now you can ask that question in the learning experience, so it's more fluid and immediate. But what we're most excited about with our first launch with this chat experience is, yes, it's great that anybody can ask a question and we can keep people in the learning experience.

They don't have to go out to a program specialist and disrupt their learning flow. But what's more important for us on this, Launch is that you can ask and receive answers and some somewhere between 25 and 50 different languages. And so we're excited. First step for our international customers because we have such a big global audience.

So now all of our content is in English, all of our courses. So now we believe that it's just going to accelerate the learning of those students who aren't feeling fluent in English, that they'll have somebody, something, somebody, you know, in a period right there with them, where they can be, um, you know, communicating about the material in their native language.

So that's the first. Big thing that we're excited about. 

Deep Dhillon: Need help with computer vision, natural language processing, automated content creation, conversational understanding, time series forecasting, customer behavior analytics? Reach out to us at Xyonix.com. That's X Y O N I X dot com. Maybe we can help.

How do you assess the efficacy of the generative output here? And how do you guard yourselves against blatant ad libbing, which we know happens with these models, and are you passing, are you using maybe the output primarily as input to your existing humans, and then they, as suggestive input, and it makes them a little more efficient in the response generation, or are the answers going straight to your students?

Anna Talerico: They're going straight to the students, but we keep a very close eye because that's our biggest concern. Is it returning back the right types of answers in an understandable way, you know, that's not hallucinating. Um, and so we've really over the last two months, that's what we've been kind of pressure testing.

Um, and then as we took it out to our first set of beta customers, keeping a very close eye on, um, is this a good experience that's You know, enhancing the learning journey. So we just have to keep a close eye on it right now. And we, this is obviously going to change over time, but we've been pleasantly surprised at the chat experiences that our customers are having and the, you know, accuracy of the information that's being returned by the chatbot.

But we also know we can't take our eye off that at all. 

Deep Dhillon: So do you have some kind of statistically meaningful measurements that you do on the generated responses and are those like, you know, human directed ground truth measurements where they're where you've got your actual instructors and maybe program managers that are that are generating correct answers and then And then you're looking for, um, like a scoring on that.

And you're able to like kind of weigh in or, um, are you using maybe models directly like GPT for, for example, to actually score it results or some combo or 

Anna Talerico: yeah, today it's not anything where we're talking like statistically and reporting on it, it's more been like. Eyes on the questions, the responses, and even just our team questioning, but it's hard to predict what actual learners are going to question.

But, you know, even before we took it to the first set of learners, like us, just, you know, kind of hitting it constantly with different forms of the questions, questions that the program specialists Get a lot, you know, running that through the models to make sure those basic questions that we hear all the time are being answered correctly.

So I think if you ask me that question in a few months, we'll be like, here's how we're tracking the data and reporting on it. But today, it's just been like all hands on deck, all eyes on this and making sure that we've got subject matter experts looking at the responses and feeling that they're high 

Deep Dhillon: quality.

Yeah, like one of the, one of the challenges that, you know, that we're seeing in this area is that you have, you have various levels of creativity controlling that, um, that you need for the model output, right? Like on the one end of the spectrum is, you know, you're looking for very close question to question in your knowledge base alignment.

And if they're very close, then you only answer with prescribed text. A little bit looser than that is you, you sort of look for those. 10 or so, um, you know, reference material answers, and then you rely on the LLM to kind of ad lib and massage the alignment and interpretation of that information, but you're really staying very true to that core content.

And a little bit more than that is these LLMs, of course, in order to get as good as they are, especially cause you're talking about 25, 50 languages or so. You know, they're trained on the world's knowledge. So, um, is you're relying on information outside of your curriculum, outside of your control reference material base.

And at that point, things get, um, fraught because they often in this. Often the answers that you get when you allow it to go outside of the narrow box are a lot better, but you're also away from your materials. So I'm curious how you, how you're addressing that problem. 

Anna Talerico: Addressing it by debating it. When we first said, look, this is what we're going to launch as our first step.

And it's going to be contained to the course experience that you're in. We said, it's only going to be trained on our material. And as you start to then play with that, you see the limitations and you start to say, well, Maybe if we went to let it bring in outside content, it would be a more wholesome, wholesome, I should say, you know, more wholesome learner experience and a more, you know, complete picture.

And then you say, well, that's, that was too far, right? That's not information that we actually, um, can feel like we put our stamp of approval on. So I don't, we don't have an answer for this yet. I think that when I think about a more fulsome experience. For the learner. It makes sense that it needs to pull an outside content.

When I think about something that we all can feel good about and say this is corporate, you know, this is Corporate Finance Institute. It needs to be our content. And so we don't have the right the right answer yet. This is something we've been playing with. And even just that, you know, we were going to go live with here in a couple of weeks that when we come out of the beta experience.

I can't tell you exactly if it's going to be only our content or not. 

Deep Dhillon: Yeah, I mean, this is a, this is a challenge for everyone, right? Like people are trying to figure out how to navigate this because, um, I don't know if you, did you get, I don't know if you trained your own LLMs up or fine tuned on top of something, um, or fine tuned on top of like large models outside, maybe 

Anna Talerico: tell us open AI.

Yeah. We use that. Okay. 

Deep Dhillon: And are you using GPT four or three, five or yeah, four. Yeah. If you're using GPT four, then, um, Then you have a bunch of other issues right like cost is very significant for what you're doing right like each each response to send like a handful of reference materials over to GPT for have it reason on it.

I mean, and then if you want to like start controlling the prompting so that your editorial guidelines can be kind of instituted. Then you're not, you're not necessarily able to do that in one back and forth. You might need a couple. So latency becomes an issue because everybody knows those guys are like on their knees, barely able to keep the service up.

So, um, so that's an issue. And then, and then it knows a lot like GPT-4 is. Has more breadth of knowledge than anybody that either of us know, but it might not answer the way you want. Usually the fault is that it goes, it blathers on and you know, that might not be how your people want to learn. And so you have to like oftentimes you have to have like very dynamic prompting to be able to like wrangle that thing into a box.

I'm curious how. How you're finding that. 

Anna Talerico: It's first thing it's interesting that you talked about the latency. That was the thing we didn't know how dialed in we'd be able to get. And the first sort of things that we were playing around with were too long. Like there's just, there's no way that we could sit and have a learner sit and wait for 40 seconds to get an answer back.

We just knew that right out of the gate, that was not going to work. So we would, you know, we're, we're happy right now with the response time that we're seeing the cost. Is I'm really glad you brought it up because I don't think enough people are talking about this, right? There's a in our sector and education big classes with what Khan Academy is doing and do a, you know, um, do a lingo and everything.

But the reality is those some of the fantastic experiences that they're creating with AI. They're very costly and they're passing. It could be 20 

Deep Dhillon: cents for one response. Yeah, 

Anna Talerico: absolutely. So, you know, you look at how transformative maybe the Khan Academy tutor is going to be. Your curriculum development, you know, AI is going to be.

But it's going to be costly and cost prohibitive. It's not a sort of, um, democratizing education at that point, right? Or you look at Duolingo, you have to really pay to have that sort of Duomax experience. And I don't think we're, we're talking about that 

Deep Dhillon: enough. Yeah, I mean, I think what's happening is, um, we went from a world where in the machine learning AI world, where we spend most of our energy just trying to get a certain level of efficacy, and at least with respect to like generative text.

We went to a different place where there's this giant beast in the sky. It's not always there when you want it, it falls over. It costs a lot, but it's pretty shockingly great. And now a lot of the energy is like, okay, once we've figured out what we want, how do we get it to be cost effective? And so there is.

You know, running on GPT 4, there's running on GPT 3.5, and then there's like taking some class of frequently asked questions and like routing those all the way down, maybe into your, into your private LLM, your, your, uh, Flon T4, whatever hosted model, maybe even below that, where you're doing even older school stuff, which, you know, in this day and age is like literally five months old, but, um, where, where you're, you know, you're directly matching against your knowledge base and.

all of those latter things get a lot cheaper, but perform a lot worse. And, and a lot of those former things perform amazingly well and cost a lot. And so a lot of what's happening now is like, The engineering and intellectual effort is going into like pulling this stuff down so so folks can like yourselves control your own destiny and just make your own decisions.

Pull things into your own heart, you know, maybe not in your own physical hardware but into your own cloud environments or whatever. And I'm curious how you're thinking about that problem because like a lot of folks are just blatantly using GPT 4 as training data. The licensing isn't that obvious. Like it seems like you can, as long as you're not trying to do anything general.

And then it seems like you're even in a better place if you're taking it to get training data to run on a different lower level open AI model. Cause it's like, well, either way you guys are getting the cash. I don't, it seems like they're playing along. Cause they could have put guardrails up for that. Um, I don't even know like exactly where that line is, but I'm curious how you're thinking about it.

Yeah, I don't 

Anna Talerico: even know that I know how we're thinking about it right now. When we went back to, again, go roll back to four or five months ago, when, okay, ChatGPT, here it's exploded overnight, even though it didn't, you know, for the world, it did, uh, we're talking about things that have not been overnight, but now suddenly it's in everybody's hands and we're thinking through how it's going to impact education and learning.

For us, it was so basal and boots on the ground. We have to start developing internal expertise on this. Like today, we're not going to go get it, you know, externally, we need to create our own AI. Experts, I've used that word loosely and have our team have experience with them. And it was like, let, let's just start, right.

Let's go get open a, um, AI's a p i, let's think about what's the most utilitarian useful thing that we can launch the, the fastest and go on for the, it was just like boots on the ground. Like we, if we waited six months, we waited a year, we knew that we would. Feel caught off, if not actually be caught off because I think a year from now, what our learners are going to expect from us from an AI kind of experience is going to be radically different than it is today.

And certainly. Even think about, you know, if our average customer, let's say, is 26 years old, right? Well, what a 26 year old is going to expect from us in three years is going to be so different than what they expect from us today. So, It wasn't even about anything other than go get it and start and like use our staff.

We don't, we didn't even think we could go get people that would help us externally. Right. Everybody was scrambling for those resources. So it was very kind of like boots on the ground move right now, you know. Yeah, 

Deep Dhillon: no, we see that a lot. I mean, we, um, shameless self promotion, but yeah, we're definitely those people that you can lean on to, to, to get the expertise.

And, you know, and frankly, like what happens when you fully control the data and run everything in house versus what happens when you go totally outside and just run with open AI and like, how is that going to evolve over time and how does that impact strategy? When all of your target users know they can go straight to chat GPT and prompting gets a lot better or less necessary.

What does it even mean to have this, um, distinct, um, materials that you're presenting? And so I'm curious about that. Like for a business like yours, you've got, um, so everyone has now access to this gigantic reasoning and intelligence engine at this point. Right. And what happens when the general broad brushed intelligence reasoning engine is so good.

That with just a few lines of prompting, it's going to probably, I'm guessing when you look at your, even your instructors, it's probably doing better than them a lot of the time. And, um, and that's not at all to say that your instructors aren't great or you're, you know, I'm sure they are, but, you know, we have really bright folks that, you know, we use to like do narrow tasks that GPT 4 just outperforms them left, right, and center.

So we've sort of had to reassess like. What is it that our humans are even offering that is better or more valuable than GPT 4. In a lot of these narrow cases, we're like, it's nothing. So we're not having them do that anymore. I'm curious where, where you're seeing, you know, that kind of thing. And, and how do you ultimately compete against an ever improving monolith in the sky that knows everything about almost everything and reasons.

Anna Talerico: I love this question. I mean, I Back to things I was saying four months ago. That sounds so absurd now. Like I remember saying, well, you still need to know what questions to ask. Right. Well, the reality is we're all getting really good at how to ask the questions to get the information back. Like that. 

Deep Dhillon: Such a stupid Yeah.

You just ask Chad four, you're like, what do I ask you? What, what should I ask you? You know? So, you 

Anna Talerico: know, just shifting so, so rapidly that goes back to like this. The day one mindset of like, we don't know where this is going. And it's hard to predict for us. We just have to stay open and be coming from a place of yes.

It's so conceivable that we're going to blink. And AI is creating syllabus, curriculum, the content, right? The structured learning, which it doesn't do so well today, right? Today, it's like all the information. Yes. But what about the structure of the learning and the progression? It's going to be doing that.

We can have a. AI generated, I always forget the word for it, you know, person, you know, that's giving this material just like a human would, a human like experience with a, um, AI that feels like a trainer, a teacher, a tutor. So that's all going to happen. I think. Today I think about, I think there's a lot of learning that can happen without human interaction and without engagement, and especially for self directed learners.

But I also know that human connection is a foundational part of how we become motivated. We learn and how we retain information, we're not all self directed learners, but you know at all. And we also learn through storytelling and the structured delivery, the nuanced delivery of that information that helps accelerate how we connect the dots and come to meaning and understanding.

And I think that this like teaching in a way that helps people connect dots and come to meaning and understanding and have context and nuance. Absolutely. I will deliver on right, but you still are motivation to learn. So come from connection with, 

Deep Dhillon: I think that's a, I think that's a really important point that maybe it doesn't get talked about enough.

I feel like the most important job a teacher does, you know, is help motivate right like it's, and as much as. You know, like my Google calendar, my Chat GPT, like any social expectations, those bots might synthetically create to have of me are not going to really resonate with me because at the end of the day, I know they're a bot.

But we all know that like a lone founder struggles because they don't have to do the thing, right, we all know like, like students without somebody else relying on them like in a group assignment. may or may not do the thing. Um, and, and, and whenever there's like social expectation around you, our innate need to like rise up and like help other humans out and not, um, disappoint, doesn't really map to robots.

It's one of the few places. So I'm curious, like, What, what does that mean as far as product goes? Like, does that mean taking your instructors and maybe your, um, you know, your program, uh, folks and like extending the reach of their social expectations in some way, you know, like, where somehow they're like, Connecting tighter with the students and they're like, you know, cheering them on and sure, maybe it's bot generated, but it looks like it's coming from them and and and maybe there's sporadic one on one real life interaction.

Like, what does that mean as far as product? 

Anna Talerico: Absolutely. Like that's happening today already. You know, our subject matter experts spend time every day in our community, like our actual instructors connecting with students, right, answering questions, telling stories, giving advice and mentorship. And yes, a certain part of that could be done by AI even today, but we very much view that as part of the role of the subject matter experts and how it will evolve.

I think too, there's this You know, I can ask you today, tell me about the day in the life of an investment banker, a first year investment banker in as much detail as you can. And it will, and it will be accurate and less hallucinating. But I'm not going to get that same story when I asked somebody who was in investment banking for 20 years that was in that role.

It's going to be a different kind of story, right? Different fabric, different nuance to it. So I don't, again, we'd be foolish to think we're not going to get all that from AI, but I still think there's this other layer that we get from human interaction. So yes, that the role is as our subject matter experts will absolutely change.

Deep Dhillon: One of the things I notice when I see, when I talk to folks who aren't, you know, in the tech world about, you know, chat GPT, for example, you know, I'll, I'll give them the tool. And the first of all, they're just like, wow. But then, um, there's like this massive blank slate problem. Like humans are not great at like knowing how to initiate conversations, right?

Like people who have these very successful careers. Like one of the things they're great at is just talking to pretty much almost anyone and being able to just ask questions. Like any great executive, as I'm sure you know, is great at asking questions. So what do you do? And like just digging in and asking and asking and asking.

Most people are horrible at this from what I've seen and they don't even know what to ask or how to even use the tools that are staring at them. And I'm curious, like, like, I know that like ChatGPT, there's a full expectation and, and all the LLMs. I don't want to over index and check GPT because this happens with everything.

All these machine learning AI systems have this expectation that you initiate the interaction. And when it comes to learning it. It feels like if you go to a middle school math class, I'm pretty sure that the students don't pull to learn, right, like the teachers up there pulling to get these middle school kids to do stuff.

The teacher is the one asking the questions, and then the students are mostly ignoring or spacing out or not paying attention. But it seems like we got to get better at getting people to care to learn. Which is what you're talking about the motivation, we also have to just get them better at like being intellectually curious, and then we have to get them better on acting on that intellectual curiosity.

Anna Talerico: Totally natural curious number one natural curiosity is my number one skill that I look for in people, just that innate natural curiosity and some people have it and some people don't and how do we awaken it and others that You know, when you think about in a world where now not only do we have all the information that we could ever need, now we can actually ask questions in a conversational way, get to that information.

And then, you know, you think about, well, what do we need to teach people? We need to teach people how to. Ask the right questions. Think critically about the questions. Think critically about the responses and the answers. Like what we need to teach people has to change, right? That how do we get people to initiate that with the right context and the right Frame and ask the question in the way that's going to get the right response.

I think that's going to become a skill and I don't know how to tap into it when people don't have it naturally, it gets to intrinsic motivation, right? I mean, just a little side, when we were talking kind of about. The motivation of the human element of, you know, wanting to please a teacher, wanting to please somebody, or the reward of, you know, the gold star.

I mean, tech already does that, right? That's the streak. That's not even AI. Think about why does Duolingo have the average daily users they have? It's the streak functionality. And it's like tapping into this competitiveness, and we want to, you know, maintain our streak. Well, what's interesting about the streak is, There comes a point of flips.

Now you're not doing the streak because you're still interested in learning Spanish. You're just doing the street to check the box, right? So we've lost. This one 

Deep Dhillon: resonates a little bit too much with me. We'll take a little detour for a second because I got, you know, as a full disclosure, I'm a shareholder in Duolingo, but And I love Louis Vaughn on, I've loved him for years.

Um, you know, that kind of godfather of, of human, uh, annotation and, and, and, and human and human computation. But I had a 740 day streak and do a lingo. I don't know what happened, but somehow I lost it. Right? Like I just lost it. I can't remember what happened. And I thought, Jesus guys, they gave me like one day, but I was like out of pocket somewhere.

They gave me one chance and that was it. And then they never gave me another chance, and I've never touched Duolingo since. And I'm like, seriously, people, like, somebody inside, I know these guys, you have, you have really bright, you know, engineers and, and, and computer scientists, like, someone's got to analyze this data and say, we're going to give you this one last chance to buy your Streak bag.

But make me feel not bad about it. Um, right, 

Anna Talerico: exactly. Cause now you have chain, right? They can't make you feel bad about it. That's not true. So let me ask you something. Cause I, I just, I'm fascinated by this. At some point, maybe it was streak day 700. It was not about the learning. It was 

Deep Dhillon: totally shifted. I got obsessed with this.

Stupid streak, you know, it was just, I mean, like I would, um, I had to, I just started learning other languages because, I mean, I started off with Spanish and then I found that like, okay, if I'm going, you know, into another country, then I can switch. So then I, so I'm like, okay, well, I'm no longer going on holiday to Mexico, but we're going to Italy like in another, you know, few months.

So I'm going to switch to Italian. And then I got obsessed with Italian for a while. And then, then, uh, I was like, you know, I guess I should, you know, being from India, I'm fluent in Punjabi, but I've never really, um, because I'm raised here, never really read and wrote Hindi. So I'm like, Oh, well, I guess I keep the streak going.

I'm going to switch to Hindi. So I did that. Got really obsessed for a long time and then I lost my streak and now I just, now I have to like get up to speed on Portuguese because we're going to Portugal but I don't care because I don't have my streak and it's never going to be as good as 740. No, you burned.

They burned me and I just think like, man, that's, that's a problem. I don't, I, I I think part of the reason it switched for me with Duolingo is I think they fail in a few really key areas. The thing about language is you have to write. If you don't write, and I mean physically write, like, like I'm sitting in this meeting and I, you know, we're chatting.

I have my notepad. I have notes. I can't read them. I'm positive you couldn't read them. No one on earth can probably read these notes, but it's like me wiggling my pen on this paper. somehow gets it into my head. They don't have that. And I think that's crazy. That's like insane to me. And, and then the other thing is they're not doing anything like conversational outside of the cutesy little icons.

I feel like it's almost like they've over gamified it because, you know, they're on wall street now. They got to. Report the engagement numbers, like everything comes down to who's paying the five or 20 bucks a month. And it's been over gamified, where it's less about language learning now, and it's just about like video gamifying this thing.

Yeah, that's so true. That's sad, because I know Louis Vuitton is not about that, but I think that's what's happening, so. Definitely. As I understand it, you can pay something more, and you get the actual version that talks to chat GPT 4, and you can actually converse. Um, so that might be the next way I get back into it, but 

Anna Talerico: you should try it.

I think it's called duo math and it's stuff. I've seen it. It's pretty 

Deep Dhillon: amazing. There's been languages that I've learned where I'm in country. I am immersed. I have a tutor that spends three hours a day with me. I have social expectation, which I think is massive, right? You know, we had this tutor when we were, um, filling in on a, on a, on a language in, in, uh, in India, she'd come every day.

But we knew she was coming. And if we didn't do the work from the day, this is my wife and I, if we didn't do the work from the day before, then you just felt ridiculous. So you just, so it's that feeling that has to like materialize. And the same thing happened to me with guitar. Like when COVID hit, I got obsessed with, uh, I'm like, okay, I'm going to just.

take the pandemic to learn guitar. Um, and I got somewhere, but not very far. And then after the pandemic ended and stuff, I'm like, my wife gave me, um, um, guitar lessons with this amazing guitar teacher nearby. And going back to your motivation comment, this guy is better than any teacher or instructor I've ever seen.

Simply getting me excited about what it is I'm learning, like forget about the ticket. I mean, he's great at all this other stuff too, but the thing that's like really differentiating is, you know, when, I don't know if you've ever played guitar, but like when you're first learning, there's just this insanely boring stuff you have to do.

Cause you have to get your hand to move in weird ways that it doesn't move. And it's painful and annoying and super slow going, especially if you're learning on an acoustic and. He had this way of getting super excited because I could play the same chord we've been working on for three weeks, but it suddenly sounded clean.

And, and then he would like analyze the heck out of it and get, get all excited about it and then go home. And all of a sudden this activity that just seemed dumb and not to the point, cause I like, I just want to go in, you know, with my musician friends and just boom, like play and jam together. And instead I'm stuck on this boring chord thing because.

He's like, we got to unlearn all this bad habits you picked up during your two years of COVID playing. Um, I don't know. I think, I think there's something really key there where we have to ask ourselves, what are the machines never going to be good at? And maybe we need to start focusing on those 

Anna Talerico: things.

A thousand percent we do. And we can use the guitar analogy. Is there going to be a world where you can have a virtual AI simulated, you know, instructor who's hearing what you're playing and giving you very real feedback. Yes, like everything that that that instructor could do, but they're going to be an AI world where you can show up and jam.

No, like, and get that energy from other humans in the room making music together. Like, no, right? So let's not pretend that AI might not be, you know, we'll never be able to teach you guitar in the same way. But AI is not going to make five humans in a room feed off of each other's energy and spontaneity of, you know, making music together, like So I think you're right.

That's what we have to start focusing on, or what are the things that AI can't create for us? 

Deep Dhillon: Have data, have a hypothesis on some high value insights that if extracted automatically could transform your business, not sure how to proceed. Bounce your ideas off one of our data scientists with a free consult.

Reach out at Xyonix.com. You'll talk to an expert, not a salesperson.

Yeah. I mean, the question I've been thinking a lot about, and I'm curious what your take is. I'm increasingly convinced that with this AI revolution, intelligence and, like, superb reasoning are commodified. Right? Like, people, everybody now has access to, they don't have to make stupid decisions, they don't have to reason like crap anymore.

Even if they're dumb. Like, they can really, they have the ability, and it doesn't take a lot to get first class intellect and reasoning. Maybe not. You know, 160 IQ level, but we got fives on almost all the AP tests, amazing scores on the LSATs, like it's pretty crazy right now, but what happens in a world where reasoning and intelligence are commodified, like what else?

happens. Cause I think it's too, it's too easy to just say, well, you know, humans like existence is pointless. And like, you know, and, and, and, or we get sidetracked with the work conversation, like throwing all that stuff aside. I don't think that's true. Like what I'm finding is like, personally, my work has gotten so much more interesting.

I don't have to spend. six hours writing a proposal. I spend 15 minutes because I get GPT 4 to write it for me and then I just torque around the edges and boom it's out the door so I can go do something interesting. What 

Anna Talerico: I hope, and I just may be naive, but what I hope the emphasis shifts to is soft skill.

And interaction and engagement is there a way that the impact on our output is such that now what's more important is, is how we interact and, you know, being in the room, uh, and the human connection, perhaps now there's more time and more emphasis, more time for it, um, and more emphasis and importance placed 

Deep Dhillon: on it.

Yeah, it's like, um, it feels almost like creativity is more valuable. Yeah, because creativity is still, um, like that raw analytical horsepower that it took to come up with reasonable plans and reasonable approaches. Anyone can get that now. And the more you interact with, you know, the systems, the faster you learn so that you're, it's, it's almost like I see my like.

My thinking going up a level like it's, it's like, you know, how like 20 years ago, like I remember I think it was 1990 1995 was like 94 was the first year that I got a laptop for work. And so that was like the first time that my work info went home with me, and I had internet not just at work but at home, that was sort of connected in a way.

And so so at that point. Anytime anybody asks, you know how like you're sitting around and somebody's like, so, you know, do grizzly bears have two homes or one home, you know, like the conversations came up. And that was the first year that I stopped engaging in that kind of conversation like I stopped caring because I, because my brain said the answer exists.

The answer is easy to find. 2007, the iPhone comes out. I'm like, the answer is even easier to find. I don't care anymore. Like that stuff doesn't matter to me anymore. And I'm wondering, like, is that's what, that's kind of, I'm seeing that happen with me with chat GPT 4, where somebody is talking to me about something and I could spend a lot of intellectual energy to understand it in the moment.

Or I could have a different conversation at a higher level, knowing that I can go home and have a conversation with GPT 4 to fill in all the gaps and like, and like, I don't know, I feel like, I'm wondering, like, what is the equivalent of that? Like, I don't care about the one hump, two hump conversation about bears.

Anna Talerico: Yeah, that's an interesting question. I know exactly what you mean, but I don't know what that looks like. You know, I said that like my hope that is that. human interaction and thought skills becomes more important. I think the flip side is it is really naive because so we can simulate human interaction right now with AI, right?

We can have somebody to chat with all day long, um, and you will walk away feeling that was human. Like I end my day with Chats GPT as my thought partner. And I feel like thanking it. I feel like giving it a gift. Like it's already stimulating these human like feelings, right? So it's even a bit naive, probably, my bleakest outlook of just thinking about, you know, the soft skills and the human interaction getting more highlighted.

It's tricky stuff. We've, 

Deep Dhillon: this has been a totally awesome conversation. I really appreciate it. But I'd like to end with, you know, a five year projection, take it where you will, but what is the world of learning like for banking and finance professionals, five years out, assuming everything that you're doing now works out the way that you see it now and you're pursuing it.

I know we've covered a lot of the uncertainty, but let's assume that your vision, like pans out. What does it mean? What does the world look like for those professionals? 

Anna Talerico: Well, I think that number one is some of the jobs that we train for today are going to look fundamentally different and maybe some of them, um, will be different jobs.

I think there's certain roles in finance and banking that that will be supplemented pretty drastically by AI. So I think that when I think about teaching and learning. In five years, learning will be totally shifted. I think how course material, how the structured delivery of content is largely going to be driven by AI.

And so the human layer on top of that is likely around the storytelling, the real world, um, of the experience of being in the roles that, that, uh, People work in and will be much more like you said, kind of coach and mentor and cheerleader like and, you know, guide, then they will be creating material and curriculum and things like that, we can't take away I still just no matter what.

I think you can't take away storytelling. I think you can't take away the, the nuance of, of working in roles, right? And what that's like, that that's going to be hard for AI to, to replace. 

Deep Dhillon: Well, thanks a ton. This has been, uh, this has been really fun. Um, so thanks so much 

Anna Talerico: for having me. Yeah. Thank you.

Really enjoyed it. 

Deep Dhillon: That's all for this episode. I'm Deep Dhillon, your host saying check back soon for your next AI injection. In the meantime, if you need help injecting AI into your business, reach out to us at Xyonix.com, that's X Y O N I X dot com. Whether it's text, audio, video, or other business data, we help all kinds of organizations like yours automatically find and operationalize transformative insights.