The Semantics of Search | Brett Matson, Funnelback

In this episode of the Discussing Stupid podcast your host Virgil Carroll has a conversation with Brett Matson, the Managing Director of Funnelback Search Technology, about search, semantic search, knowledge graphs, artificial intelligence in search, and the related leverages and pitfalls. 

Brett shares that there are several big things coming in the area of search, after reaching a plateau and stagnation for a few years. Some of these are paradigmatic (understanding the search has multiple purposes, not just page rankings,) and some include changing the approach (i.e. modifying queries,) changing presentation of results (modules,) knowledge graphs, semantic search (detecting the intent of the query,) and many more. 

Brett explores how some of these new technologies are intrinsically more attractive to people. For example, knowledge graphs are (or can be) visual and show relations between entities in them. Thus, they are much more intuitive in contrast to being faced with a wall of data to choose from. The point is, of course, making content a lot more intelligent and hence more useful, by treating it as a product and perfecting its delivery. 

In Brett’s opinion, during the next decade we will see a real proliferation of smart tools that will help users and companies perform a significantly better search. Virgil and Brett also discuss how it is very important to avoid making too many connections when using a new, powerful technology that can do that. This can bog down the whole organization, its data servers, employees, and finally, users.

Virgil’s conversation with Brett is very rich and diverse, so make sure to listen to the whole episode and pay close attention to what he had to share. 

Links:

Episode mentioned:
Future-proofing your experience delivery strategy with Intelligent Content by Kate Skinner.

Siraj Raval's YouTube channel is full of short, fun videos that teach all different aspects of machine learning and AI.

Coursera's Machine Learning course, one of the original online machine learning courses, delivered by leading AI researcher Andrew Ng.

A great article by Sebastien Dery discussing the challenges of knowledge graphs. 

Subscribe for email updates on our website:

https://www.discussingstupid.com/

Watch us on YouTube:

https://www.youtube.com/@discussingstupid

Listen on Apple Podcasts, Spotify, or Soundcloud:

https://podcasts.apple.com/us/podcast/discussing-stupid-a-byte-sized-podcast-on-stupid-ux/id1428145024

https://open.spotify.com/show/0c47grVFmXk1cco63QioHp?si=87dbb37a4ca441c0

https://soundcloud.com/discussing-stupid

Check us out on socials:

https://www.linkedin.com/company/discussing-stupid

https://www.instagram.com/discussingstupid/

https://www.facebook.com/discussingstupid

https://x.com/DiscussStupid

Transcript

Announcer: 0:00

Note. This podcast does not discuss nor endorse the idea of discussing stupid ideas because we all know there are no stupid ideas. Hello, and welcome to discussing stupid. The podcast where we will tackle everything digitally stupid. From stupid users and the crazy things they do to stupid practices and the people who use them. We'll explore the stupid things we all do and maybe even come up with a few ideas on how to do things better. And now that I got your attention, let's start discussing stupid.

Virgil: 0:37

Hello, and welcome back to the broadcast of the podcast. My name is Virgil Carroll, and I'm your host and principal human solutions architect at HiMonkey. Well, spring is in the air, and everybody is gonna be going outside searching for new things and great weather. So I said, why not talk about search? And the ecosystem around search is getting so much bigger, and there's so many new things coming out with the way artificial intelligence and the way semantic relationships are going to really empower us to have better searches both internally and externally to our organization, that I thought it'd be really great to bring an expert in this area in and have a discussion about what semantical search in particular, kind of this whole knowledge graph type concept that we have, how all of our content is connected and how that's really going to play together to give us really the search of the future and what that really means. So joining me today is Brent Matson from Funnelback Search Technology, And Brett and I are gonna have a conversation and talk about what semantical search really is and how it can be leveraged in our organizations and some of the pitfalls we have to avoid as well. Well, Brent, thanks for joining me. I really appreciate you taking time out of your busy schedule, especially you and I trying to coordinate our schedules from the US to Australia. But, you know, about a year ago, I talked with one of your colleagues, Ben, and we kinda talked about artificial intelligence and search and how that's kinda having a big effect. But as you and I were just talking about it, it's kinda crazy how much search has gotten so much bigger in just the last couple years where it was around for a very long time, but it's all of a sudden gotten this emphasis that isn't out there. So before we start, if you wouldn't mind just, maybe giving yourself a brief introduction.

Brett: 2:31

Sure. And and thanks for having me on your podcast. So I'm the managing director of Funnelback. We're an Australian based search engine technology company that's now starting to explore other areas like content insights and knowledge graphs. We're originally a team of engineers and researchers that came out of the, Australian government research organization called CSIRO. And, we spun off from there and then we're acquired by a company called Squiz, and now we're sort of very much focused on developing a world class search and insights platform. And I guess we're primarily in Australia, and also do business in the UK and more recently in the US as well.

Virgil: 3:07

Great. So let's kinda start where I just was saying that, you know, you and I were just talking beforehand about just how much more has come around the search industry over the last couple years and just how much it seems to be changing in a very fast pace. What are some of the big things that you're seeing right now that you're seeing that are really kind of affecting this change, and, how are you guys responding to that?

Brett: 3:31

Yeah. Yeah. It's an interesting question. So back in the earlier days, I think I I did and a lot of other people had a arrogant view that the only thing that was needed for customer experience when it comes to search is good quality ranking and good quality ranking can solve everything. And so we we sort of went down that path for a number of years and then it sort of became clear that, and it's kind of obvious when you think about it but search is more than one use case. There's many different reasons for why people are searching and what they're searching for and sometimes you know that something exists and sometimes you don't know whether the thing you're looking for exists and and sometimes you need all the information on a particular topic or you just need to be pointed in the right direction and then you'll find the rest from there. And so then we had sort of more technology come about like faceted navigation and auto completion and best bets and all these things that were sort of designed to cover off those use cases that strict sort of keyword searching didn't cover off so well. And then I I felt like search stagnated for a few years. We sort of we'd reached this plateau where we didn't really know where things were going from here. Like, search quality was good on the web with, services like Google and was getting better within the enterprise and the technology hadn't evolved for for a number of years. And then I think there was, another realisation which is that even search with all of its bells and whistles wasn't covering off a number of really important use cases, particularly within the enterprise but also on organizational websites and and that's really this problem where when somebody searches, you're you're really asking them to get a bit creative around constructing a query and and nobody really wants to get creative when they're looking for information. They just want the information they're looking for. And that process of forcing the issue of having to create a query and then the risk of not finding what you're looking for becomes a bit of a mental hurdle in people who are doing searches and that causes them to sometimes not use the search and and want to use the browse instead because they see that as more of a defined pathway to to finding information. And I think what's starting to happen over the last couple of years is solutions to this problem where you're looking for information but you don't quite know what to search for. People are concerned, Am I going to be too specific in my search and get no search results or am I not being specific enough? Am I going to get too many search results? And there's a guy at Apple who talks about this sort of saying that even just the paradigm of having search results at all is outdated now because I think his name is Sebastian Derry. He talks about how people don't wanna scan a search results page. That's like reading a novel a page out of a novel. It's too much to have to do to wanna quickly find information. People expect to receive information in in sort of organised modules where we know things about these modules, whether it's a person or a project or a document and knowing how these modules fit together. And so the 2 sort of streams that I see really happening at the moment are sort of branching into knowledge graphs. And and knowledge graphs is really just kinda like the browser equivalent of a search engine in a way. It's it's sort of saying, you may not know the keywords to use, but if you have a a mental model of what you're looking for, their knowledge graph gives you named relationships to follow. And those named relationships allow you to do reasoning as well, which is another interesting area. But the other area that I'm seeing more and more of is sort of, the neural network based information retrieval where, you know, search engines are very good at matching a keyword to a document and and there's relevancy ranking algorithms that traditionally have used statistical mechanisms to do that ranking. But when neural networks are really powerful is they're good at not just understanding, say, the intent of a query, but understanding the parameters within the query as well. So for example, if you were to go to a university website and say you want a academic who knows about biomedical engineering, in the traditional sense, you would just type in biomedical engineering and you would get the ranked set of search results that that correspond to that topic. And then semantic search came around and in semantic search, you might have a more structured view of the query where could type in, yeah, a fielded search over people and I want people who have the skill set field set to biomedical engineering. I think with neural networks, neural networks have a a really good way of being able to detect the intent of the query. So I'm looking for somebody with a specific skill set is the intent, and then the parameter within the query is biomedical engineering. The neural network takes that out of the query, and then you can use that structured representation of what the person's looking for to then formulate the right query, whether it's a database query or a search engine query or something else to produce an answer to the question. And both of those technologies, the knowledge graphs and the neural network based question answering, go really well together as well because once you've got the knowledge graph, you can also use the knowledge graph to answer questions using neural networks. So I think it's the answer is is in the data structures and it's also in the algorithms being used to interrogate these data structures.

Virgil: 8:08

Yeah. I think you bring up a good thought thereof. I think this, you know, where we hear everybody every day talk about AI in this and AI in that when when fundamentally, it's exactly what you said. It's it's these different components and how these components interact. And another big piece of that is, you know, kind of this natural language processing that we're looking at these days as well. I wanna be able to go into a search engine or really whatever it is, and I wanna say, well, I need to find x y z, and it needs to be able to translate that. And I'm assuming that ties a lot in with kinda how neural networks work. And from that side is kinda gauging that intent, and that seems to be a very big way that the search world, especially, is moving is trying to more less rely on people to be better searchers and more trying to understand not only their patterns, but the patterns of others and take a more educated guess. Is that a fair way to say it?

Brett: 9:08

Yeah. That's right. Yeah. That's exactly right. Yeah. And and it's interesting too, like, just going back to what you're saying at the start about the neural networks. I've seen a lot of cases where there's an immediate assumption that neural networks are the new technology so therefore it's better and we should be doing it. But there's a really quick way to sober people up on that which is, well, let's get a quantitative test going where we can test the accuracy of the system and test the neural network against a statistical algorithm. And what we found in our early experimentations was it's really hard to beat a statistical algorithm if you're just looking at doing search. But if you've got specific use cases in mind, then neural networks will outperform them. I think it's about understanding the task you're trying to solve and understanding the customer or the user and, yeah, joining the dots on that.

Virgil: 9:52

Well, and that's kinda where it always starts is it always starts with what people are trying to do and understanding that piece is not. So so when you start talking about these neural networks and, you know, kind of all these pieces, I mean, it's not like you just turn those on and it starts happening. What are some of the things that you have to work with organizations to kind of understand to be able to do to get these pieces. I mean, I think that's always one of the problems that we always run into, especially in the world of search is everybody is looking for that magic bullet, just that flip the switch and something's gonna happen. And all of a sudden, my god, my people are finally gonna find what they're looking for and always get to the right resource. But we know, and probably most of those people do know as well, is that that's just not reality. There's other things that have to go into it. So as people start going down this path of looking at more of these cognitive technologies, what are some of the things they have to start doing to make that happen?

Brett: 10:50

Yeah. Okay. So, and and this goes back to just what you're saying as well. So the first thing we do and and have been working with some of our customers on is identifying what the user or the customers are looking for. And typically, we would look at 3 different sources. So one is being a search company, we can go and look at the the search analytics and see what people are searching on. Often, there's there's some form of searchable knowledge base as well, and this could be tied away in a separate product somewhere else, but people often type in longer questions to a knowledge base, hoping that it's gonna match one of the knowledge base articles. So that can be a more informative source of the more complex problems that people are trying to solve or information they're looking for. And the third thing, depending on the organisation, can actually be call center logs as well, and that's where you get those really detailed this is my situation and this is what I'm hoping to do or solve. Where we'd start is by putting together a list of intents. So what are the intents that these users have in terms of what they're looking to what information they're trying to access or what problem they're trying to solve? And then quite often, you sort of get that straight out principle where you've got a large portion of the overall queries coming in are represented by a relatively small number of intents, and so you can cover off quite a significant portion of the overall sort of customer query load with just a a small number of intents and then focusing on those initially to build out training sets to train the neural networks. And and there's some really good tools these days for being able to really efficiently specify a, say, a rules based system for how the training data should unfold, and then there's tools that'll take those rules and expand those out into quite large sets of training data. So some of the issues around neural networks that have been around forever, which is having large amounts of high quality training data, are being solved to some degree anyway. So from there, we can build these neural networks that produce the ability to to understand the intent of the user, and then probably what we'd do is make sure that we're actually evaluating the quality of those neural networks to make sure that they're answering correctly. And in the cases where we do it, try and outperform the the search as well because otherwise, you could just put a search engine in, which is a bit easier.

Virgil: 12:54

Yeah. I mean, you know, one of the things you and I talked about a while back when you were first sharing with me funnel backs, knowledge network or knowledge graph, sorry, I said knowledge network, didn't I, was around this concept of semantical search, something that being in the world that I've lived in, I've known about for a very long time, not only from the aspect of search, but also from the aspect of how sites are architected, how everything is. Everything's got a relationship. You have to understand those relationships. But one of the interesting premises that I think I brought up during that discussion was, what about not only having that from inside, so you're crawling a website or you're crawling an intranet or you're crawling an enterprise file share or something like that, but that's not really where it's supposed to stop. It's about what happens when you crawl all 3, plus you have, you know, a CRM and since you have this data over here, and you've got your enterprise ERP over here, and you have all this information there. And to me, that's really where the power of these systems start coming in and saying, okay, so I need to find somebody who is going to bring some type of expertise on a project I'm working on. And it's like you said before with biochemistry example, you would, you know, typically, you'd search on biochemistry, you'd have it there. From the standpoint of saying, this is my type of project, being able to actually traverse all these different systems, have an understanding about it, understand that this type of project is like that type of project, which is like that type of project. And this person has had conversations about that, and that person has spoke about it, and this person over here has been the sales support for sales over here on this particular type of project. All that information could come together and say, hey, guess what? Brett is actually the person you wanna talk to around this project because looking at all this information, this is what it's figured out and knowing your intent is to get somebody to work on on the project. Brett is gonna be the person that's probably gonna be the best to help you. Now I know I said this is my Shangri La. Is that where you see this world going?

Brett: 15:09

Yeah. Yeah. Definitely. So I I think what you've just spoken to there is kind of the, you know, the promise of semantical search 10 years ago was that finally we'd have smart search engines that understand humans and and what they're wanting and will use sort of more advanced statistical algorithms for producing the answer to a query or a question. And it hasn't really panned out that way, and I think part of the problem is that try as you might, within an organization where you've got all those different types of content, each one of those different repositories has a different different, like, ecosystem of ranking evidence that goes along with it. So an intranet has links like the web does and it has site structure. And so there's a bit of information there that you could use to determine the importance of individual pages over other pages. But then as you're saying, when you combine that with, say, a large file share or a big database, you're also incorporating in large amounts of information that don't have much in the way of ranking out evidence. And quite often, depending on the query, there's even no real concept of relevance because you might have 10,000 records that all equally relate to your query. So how are you going to rank all those database records alongside your intranet pages? And it gets really messy really quickly. And I think the the nice thing about combining search with knowledge graph, so we're not talking about getting rid of search, is that the knowledge graph can almost act it can act to answer questions but it can also act as an assistive interface where it sort of provides you with a conceptual view of the organisation and how it fits together. And from there, the human can go, ah, okay. I now know where I need to go to find what I'm looking for. Or if you present it visually, as you were saying, like, it becomes immediate who the expert is that you're looking for. So the knowledge graph itself may not know, but the human, when looking at it, can quickly infer what they're looking for. And so I I find it really interesting the way that you can sort of yeah. Sometimes search works really well. You can if you know the keywords and you know what you're looking for and go and find it. But what we're seeing probably more often is doing a search, getting the search results, and then the knowledge graphs appearing alongside the search results. And people's eyes immediately go over to it because it's quite visual. And from there, they can click into the knowledge graph based on on, say, one of the search results and then start browsing from there. So the search gets you part of the way there and the knowledge graph gets you the rest of the way there. And going back to what you were saying before is I think it's those relationships that exist that is what gets the Knowledge Graph so much power. Like, naming a relationship means that you can start at one point, look at what relationships exist to your other points in the Knowledge Graph, pick the one for the problem you're trying to solve or if you're if you're trying to reason something else. And, you know, an example of that might be that John Smith's mentioned on this document, but is that John Smith the customer or John Smith the employee? And so by hopping through the knowledge graph, you can quickly see that this document is much more closely related to John the customer than John the employee. So you can sort of reason out problems that that you couldn't answer with traditional search.

Virgil: 18:05

Yeah. I could actually see that even going a few steps further because when you were talking about it, it went in my mind. Like, what happens if I, you know, I'm talking to somebody in conversation and I use some kind of name or I use some kind of acronym for a project or, you know, a company or, you know, I work with a lot of government in government. Love their acronyms and love to use them for everything. So how do you you know, from there, when when you start talking about knowledge graph and you start talking about having these semantical relationships, how do you get over that when you don't have those clearly defined parameters behind things to say that this is absolutely that?

Brett: 18:45

Yeah. So one of them is is, I guess, what I was talking about before where you can use the graph to try to understand the distance, I guess, in terms of number of hops between different points within the knowledge graph. Another way is that we'd encourage our customers to often when you've got really acronym heavy organisations, for example, they might have an acronym dictionary. And so if you feed in that acronym dictionary so that each acronym is a point in the knowledge graph and then links off to every where that acronym's been mentioned, and that might involve meetings that have taken place and projects and documents. Then very quickly by clicking on that node in the knowledge graph, you can get a really complete view of what is this thing. Oh, this acronym represents a project. Here's everything related to this project. Or if it's somebody's name, you can really get a quick sense of everything this person's touching within the organisation, the projects they're working on, the customers they're working with. So it goes back to that idea of there being no unstructured content because any kind of a a reference to a named entity within a document is structure that exists within that document. And so, yeah, the knowledge graph itself serves to to create structure where it doesn't exist within unstructured content.

Virgil: 19:51

Yeah. I can see that. I mean, you know, you think of it, everything technically does have some kind of structure to it. I mean, even non threaded discussions or just even this conversation here, it has some type of structure. There's some type of analytical piece to that about how we're structuring our conversation, how we're structuring our phrasing, me asking you questions, you responding, those kind of things all have kind of structural components from it. So it sounds like, you know, where these systems can be very valuable. There's still a fair amount of work that people are gonna have to do to really get these systems to work the way they want them to.

Brett: 20:28

That's right. Yeah. So you you've still got all that extract, transform, load type issues around getting the data into the system. And that's the question we often get at the start of every project. How much work do we need to do to get our our data into a state where we can sort of make optimal use of it? And, yeah, to some extent the answer is often, really how long is a piece of string or, you know, the more you do, the better it'll get. And I sort of read an article on this actually yesterday and, maybe this isn't the right podcast to be talking about intelligent content, but it really clicked with me the idea that, you know, going back a couple of decades, there was a lot of excitement about the semantic web and pretty soon the web's gotta be fully structured in a way that machines can process the web in the same way that humans do and everything will be just tightly linked and automated. And it never really took off because semantic web is really hard because it takes a lot of work because there's a lot of metadata and it goes out of date really quickly. And so that sort of area went flat for a while, and it sort of come back a little bit with sort of tagging notations like microformats and that sort of thing. But this article that I read, it's from somebody called Kate Skinner, who works as the digital solutions manager at Westpac Bank here in Australia. And the article is, all about how intelligent content is the idea of making content more useful by treating it like a product. And treating it like a product in this sense sort of means making it reusable and reconfigurable, and so it's going back to what you were saying before. Yeah. The process of how do you how do you make your content more amenable to be used by machines and this goes to scaling content so that it doesn't become a roadblock. So when you've got, say, you might have 10 digital channels, websites, intranets, off-site advertising, apps, chatbots, whatever. If you're having to create content specifically for each of those channels and then keep it updated over time, then the amount of work involved just becomes enormous and so teams have to scale back and and sort of do things in a more basic way. But if you design content from the start so that it is reusable and so when you produce a a chunk of content, it might be a paragraph or it might be a sentence or even a word, and create that content with the understanding that it might end up on the website or it might be on on the Internet or it might be within an app or it might come back as the response from a chatbot or it might become a node in the knowledge graph, then what you can do is put structure around that content that describes that the meaning and purpose of this content and then allows that to be and I can see, actually, another part of this is making it format agnostic. So it goes out to all the different channels, and that little snippet of metadata tells it this is how this content should be displayed in this channel, and this is the meaning and the purpose and maybe the audience. And so you obviously can't do that at a large scale, but if you can start to build those processes into your content and into your data, then I think over the next 10 years, what we're gonna see is a a real proliferation of sort of smart tools that can grab that data and use it in all kinds of ways. And so it just got me thinking about yeah. I often think about the technology, but I I probably spend less time thinking about the content and even less time thinking probably about the people side of this. So how do we create a culture where people are creating content with a long term view and a view of having this content behave more like a product? I just thought it was was fascinating.

Virgil: 23:44

Yeah. That is interesting, and it's hilarious because when you brought up those old semantical sites and I've just been sitting here, of course, listening intently, but also side note, racking my brain trying to remember. I actually remember, one of those sites, and I actually used it a lot many years ago to find new authors to read. I love to read, and I love to find new authors. And Amazon, of course, didn't have quite its recommendation engine that it does today. But back then, I would search on my author, and it would pull up like a, you know, a web of how this author is related to other works, their books, and how these books relate to other books and other authors and that kind of stuff. And I probably found 5, 6, 7 different authors that I started reading their entire series from something like that. But I mean, that to me is what, you know, Semantic Web is about. That is really what it is. And that's really to me is what knowledge graph and a lot of these technologies are supposed to be. It's supposed to find those threads, kinda pull those threads, and have those threads help inform us how to get somewhere else in there. So, I mean, you know, this is kind of what I consider the future state of the berry picking search pattern. Click one thing, go through the next, go through the next, go through the next, and eventually, you get to where you wanna be. But from an enterprise, I mean, I think that's such an interesting challenge. And as we were getting ready on here, I'd said I just spent the entire day putting together a new talk on Microsoft Search, of course, Microsoft's newest platform around their 365 kind of bringing a ubiquitous search experience and connecting everything together through their Microsoft Graph API system and all that kind of stuff. But part of that is really there where as I dig more into this technology, I see the excitement around it, and I can see where the value is and how this could actually make life better. I also see that flip side where it's like, okay. Now and I'm gonna go specifically thinking about 365. I mean, I have a small company and a small organization, so we don't have a lot of stuff anywhere. But we still in in our SharePoint environment, we have 20, 30, 40000 files inside there. Emails throughout the years, I have every email that I've ever written inside my 365, which is a 20 plus years of emails, and that's just one person out of everybody. We use Teams. We use all these different pieces. And I think, oh my god. All of a sudden, I'm gonna do a search, and this thing's gonna present me with 10,000,000 data points that I have to choose from. So how does the technology really get around the fact that if we start saying, oh, look. This works. So then we start adding more and more and more content, and we start adding more and more repositories and creating more and more relationships. How do we keep this technology really from getting out of control and maintain some semblance of still getting people to that subset of data that they need?

Brett: 26:42

Yeah. Yeah. So this is this sort of brings into play the whole user experience side of things, doesn't it? So I remember back in the early days of search, we had some people that would be quite concerned at the fact that Funnelback was producing 4,000,000 search results, And I'd say, well, you don't have to look at all of them. You can just look at the top 10. But what you're talking about is that there's this new paradigm now, and so it's not that you're just looking at 10 search results. You you might have a thing here that's connected to 30 other things, or 30 other types of things. And each each one of those types of things might have a 100 to a 1000 things that it it's actually connected to. It's inherent almost in knowledge graphs that they just grow massive really quickly, if you can find those relationships and keep adding data. And I was reading about the, you know, the competing sort of technologies, your data lakes versus data warehouses and and knowledge graphs at the moment seem to be sort of in fashion very much. And part of the reason for that is that they as a data structure, they cope really well when you've got you might have large amounts of information as a whole, but you might have some parts of that knowledge graph being a bit sparse and, a bit messy and then other parts that are are really dense but really clean. And so from a data structure point of view, it supports all of that. But then the challenge is from a from a user interface point of view, how do we simplify this? And so, yeah, there's been some really interesting reading on that as well, around how to expose just enough of the graph to make it useful to humans so that they can sort of see where to go and navigate from there or or answer a question. And when we built Knowledge Graph into Funnelback, we saw that there are other companies doing knowledge graphs and enterprise knowledge graphs, but they stuck to these sort of traditional bubble chart interface that you'd imagine when you think of a knowledge graph. There's a circle over here that has a line connected to this circle. And when we started down that path, we sort of realized very quickly that this is cumbersome. You have to zoom right out to actually get the structure of the thing. But then once you're zoomed out, you can't see what anything is so you gotta zoom back in again. And how do we make this work? And so, yeah, putting effort into an interface where the sort of design requirements or or principles were that it had to be intuitive for anyone to understand and had to be visual and engaging. And so far, the feedback has been positive that it doesn't look like a graph. It looks almost more like a traditional search, but you can sort of rather than drill down into search filters, you're exploring this never ending web of information. But I think you raise a good point. So what what happens when you're looking at datasets 10 times as large or a 100 times as large or a 1000 times as large? Is that gonna hold true? And, I think we're we're still yet to see. But what I'm hoping, I guess, is that we get some really good UX designers to come and take on that challenge.

Virgil: 29:19

Yeah. I mean I mean, that is the thing. I mean but, probably, that's been the thing with search all along. I mean, search technology has always gotten better and it provides better results, but the reality is is that people still have a hard time finding it. And it's amazing because I can't think of the number of organizations that I go in, and they use such a simplistic interface like Google and they love it. Yet if your interface is anywhere that simple, it can't be the same. And there's this entire mentality. Years ago, I long, long time ago, when I was first getting into business, I worked with Jacuzzi and that and Jacuzzi, you know, makes Jacuzzis and that. Well, Jacuzzi is a brand, and there's a lot of different brands of Jacuzzis out there. And one of the things that really struck me and and kinda really influenced, you know, frankly, a lot of things they said, Well, we like that our brand is also used as a description of what you're trying to buy because if you go over here to this other brand, you're using Jacuzzi. Well, it's kind of the same thing, you know, in this world. It's like Google is kind of the standard, but they expect you to be a lot different from it. And there is this concept of internal search that kind of gets really worked around. And and it goes for public website as well, but I think I see this more on an internal search side where you see organizations just struggling and, you know, a lot of these search engines. Well, yeah, it surfaces content, but, fundamentally, from the most basic standpoint of a search engine, a search engine's about words. And, you know, if you have this page that's really important over here that uses the word you used to search 5 times, then you have 487 documents that use the word 10,000 times among them, you don't have the strength. And so one of my hopes about these kind of technologies like neural network and this graph technology and that is that in understanding the intent, otherwise, understanding that overall, these people think that these pages are more important than the others or they tend to be looking for this from its learning and then understanding those relationships that we'll be able to get around that. It's still not gonna take away our need to still have good content, But the reality is, again, take websites out of this and really just look at internal enterprise search. And the reality is when you start talking about 4, 5, 10,000,000 pieces of content, you sit there and say, well, you gotta clean up your content. It's almost a laughable statement. I mean, you know, I deal with those situations all the time. I'm I'm working with a customer right now that just has 40,000 documents that people need to search with. But to talk to them and say, by the way, if you really wanted to do this the absolute correct way, you'd go back and rewrite all 40,000 pieces of content to to better. It's just unrealistic. And so where in the past, we used promoted results, we used refining, we used tuning, and all that kind of stuff. It really sounds like this is the way the search is going, is that search is gonna continue down this path of trying to outguess us. Is that fair to say?

Brett: 32:27

Yeah. I think so. And with your point about the, yeah, you can't rewrite all your content, and this goes back to what we're saying about the semantic web as well. Like, nobody's got time to sit there and remanufacture content. And and the idea behind the semantic web, if you look at sort of maybe a more modern approach to it would be you might have a document that talks about a project, and that document names people, project officers, but maybe also people on the client side and meetings that took place and deadlines and dates and the rest of it. And so that traditional view of, well, if you put all of that into metadata in the document, then the search engine will understand. And then you have to do all these amazing things in terms of discovery of that document. And then if you repeat that 40,000 times, you'll be looking really good. And, obviously, that's never gonna work. But I think what does work quite well is if you've got different classes of content. And so what I've seen sometimes is organisations that run a lot of projects, they have documentation related to the projects. And so certain documentation needs to be very heavily fielded and that often goes into project systems. But then you might have 40,000 files sitting on a file share. So the nice thing about these modern approaches where you're sort of saying, well, exactly what you said, it's not reasonable to think that we're going to add structure to all of this content. But with the knowledge graph, we can automatically identify named entities anywhere within the 500 page Word document sitting on that file share along with the other 40,000 documents on that file share. Then you can bring them into the knowledge graph. And then I think what you just said about search is true, but also I think a way of cutting through all of this noise and complexity is context, I think. So looking at the user and what they're trying to do. And so if somebody's in the staff directory, for example, being able to click on a person in the staff directory and have their note in the knowledge graph pop up and start browsing from there versus somebody who's using the project system and might wanna know everything about that particular project, or they might be in the CRM because they're about to meet with a customer. So click on that customer name, and, again, the knowledge graph pops up. So, yeah, so the the idea of segmenting content and bringing context into it as well, I think, helps to to cut down on that problem.

Virgil: 34:37

That is at least until the technology grows to the point in which you ask it for something, and it says that's not really what you want. You want something else, or I'm gonna tell you what you want, and we have well, we'll just say we've watched a lot of movies like that in there. So what ends up happening there? But I think you make an interesting point there because that actually, you know, you knowing that Microsoft is kind of the other big part of my world. That's very much where they're going as well is really from the standpoint of Microsoft Search is kind of this search experience that consistent and is pulling from all their data sources no matter what application you're inside. And now Microsoft right now is very much focused on its eco own ecosystem, which is both advantageous because it's so big, but also, you know, they're eventually gonna have to bring NVIDIA's other systems, and they're gonna have to look at ways in which people can integrate it there. And I really do. I agree. I think that that's gonna be one of the things that, especially inside an enterprise needs to be looked at is how can we get that same experience across all these different things and not just limit our scope to, well, I'm searching this system, so this is really all the information I'm gonna get is because I'm looking in the system. Albeit, there is also the contextual side of it, which is I'm in this system and I'm doing this search. Therefore, there's probably that contextual intent that most importantly, I wanna find something in that system, but there may be these other tangial pieces of information and other systems that may relate to that. And, yeah, I always thought that, you know, from a standpoint of findability, that's always one of the biggest things is because a lot of people look for the same piece of information differently because there's so much context in their own world and the way they look at things, the way they do things, and how it all works. And when you have a very one dimensional search process, when you have a a very one dimensional findability or navigation process or anything, you're really just kinda getting to the point where you're being successful for a very small part. And I really do think that, especially knowledge graph, has such an interesting context in there because it's basically saying, I may start in the completely wrong place, but that doesn't mean I didn't have a good intent. Therefore, if I know that intent, there may be something that pops up that lets me follow those threads down to where I really needed to be even though I'm not really sure how to get there.

Brett: 37:14

Yeah. Yeah. That's that's right. And actually, something you just said a moment ago about different users working in different contexts. One of the nice things about the knowledge graph side of things is that you can actually not just help users with discovery, but you can learn a lot about the organization by looking at the knowledge graph as well. And so for the different users working in different context, if you start to look at the, say, the employees in an organization and start to track the relationships they have with other people and projects and documents. Being able to get an understanding of the context of a user through studying the graph and seeing how the graph changes over time can be quite interesting And this is sort of future stuff for Funnelback. But looking also at how information's consumed. So you've got a new employee. Are they building relationships with other people in the organisation by going to meetings with them? Are they producing content? Are they engaging in projects and doing things like sentiment analysis across that? And then also looking at is the content that they're producing being consumed by other people in the organisation? And so there's this fascinating heartbeat that you I think you can get across an organisation if you analyse that structure because the knowledge graph really is just a representation of the organisation and see how that changes over time. And so this goes into the idea, that Gartner often talks about with the fact that search engines are evolving into insight engines because if you can use them to look at the data and understand the data, then you can start to understand the organization and the people in it too.

Virgil: 38:42

Yeah. If you could see the smile on my face from that, could it's amazing how many times I go into search scenarios, and and I say, you know, what's your data look like and what have you learned from your analytics, and they've ever never actually looked at that. And, again, this is probably not the appropriate place to bring this up in my podcast, but you know you need to build it exactly like you have that functionality in your accessibility out of the way you can look there and say, look. After you refine this engagement, the engagements have increased by this much over time, and you can map that actually over time. I agree. I think that's a really important thing because I think one of the things is is even from the analytical side, you know, in utilizing the systems is a lot of times we can see data that tells us something's wrong, but we don't have a good measurement to know that once we try to put in a fix or try to do something to take care of what's wrong, whether that actually gets better over time. I mean, you know, and it's not always as simple as saying, well, this search returned result didn't return results, and now this search returns results. Because, again, looking in that, it's a much bigger story than that.

Brett: 39:46

Yeah. That's definitely. And what you reminded me then as well, like, the accessibility auditing is one example of being able to understand, you know, the nature of the content that's being produced by the organization, and there's many business reasons why you'd want to have accessible content. And I mentioned sentiment analysis before, so there's almost a bit of a creepy side of this if you were to start to perform automated analysis over all of the content connected to an individual in the organisation, and this is not something that Funnelback has done. But, yeah, starting to get, I guess, a measure of the sentiment of a person and even one of the areas that we're looking at, not in relation to knowledge graph but in relation to search, but being able to detect the bias in a particular document or in in the case that we're looking at in job ads. So if certain words are being used that will attract men to apply to the position over women, say, for example, then being able to detect that and track that change through an organisation yeah. I think it is it's sort of almost like a convergence over the next few years of, yeah, searching inside engines and and knowledge graph, but converging with organizational network analysis because, really, what we're doing is integrating the organization and creating a conceptual representation of everything going on. And then, yeah, then I think the next logical conclusion is you're gonna start to look at that and draw conclusions about the organization.

Virgil: 41:03

Well and that's, where the world is going. I mean, we went digital. Therefore, we've gotta have something like that from the digital side. So, well, Brett, I really appreciate you taking the time and working through the time difference with me. I know you've probably got a busy day ahead of you, and I am going to wind down for the night now, the day before. But thank you very much for joining me. And can you tell me if people wanna learn more about kinda semantical technology and and that kind of stuff in neural networks and that? Do you have any good resources? I mean, besides FunnelBack, which we'll obviously have on here, are there other good resources that you'd recommend that people check out?

Brett: 41:42

As an entry point, I've actually, found some great YouTube channels, which I probably struggle to remember the names of them off the top of my head. But, yeah, when you're getting into these sort of areas and it's new concepts that you're trying to understand, I often find that going and checking out some of the instructional videos on YouTube that can animate what's going on with these technologies, get that sort of representation sorted out, and then, yeah, going off and reading some articles. Coursera as well has, a lot of fantastic courses that go through this. And as somebody who struggled through university because I didn't like sitting in lectures for a whole hour and listening to somebody talk at me, the idea that you can engage in in a video a short video and then do a quiz and then learn a little bit more. I love that sort of interactive learning. So, yeah, my my tips would be YouTube and Coursera.

Virgil: 42:26

Well, I'll tell you what. I'll promise everybody that's listening that I'll get those links from you if you can come up with them, and I'll make sure that they're part of the podcast there so that we could share those as well. But, again, thank you very much. And, of course, if somebody wants to learn more about you and Funnelback, how might they be able to do that?

Brett: 42:44

Well, our website would would probably be the best start. That's, funnelback.com. Funnelback is a name that came from the the crossing of 2 Australian spiders, the redback and the funnel web. And, thankfully, this spider doesn't exist in real life. It only exists in our company name. But, yeah, that's that's where you'll see all the information about Funnelback.

Virgil: 43:03

Oh, perfect. Yeah. I don't actually think I knew that story. Well, again, thank you very much, Brett, and have a great rest of your day.

Brett: 43:09

Thanks for having me. It's been fun.

Virgil: 43:14

Well, I hope you had enjoyed the discussion that Brett and I had around semantical relationships and how that's really gonna affect the search of the world. I think it's such an interesting topic and I think it's one that is gonna really play a role in when I really think about what it means when we start talking about artificial intelligence and machine learning and all those different components there. This really is where it's going is really understanding how that information comes together and how it really works. So now before we go, I'm going to mention the last piece that is the stupid buzz. This is where I kind of talk about a buzzword that irritates me. But I decided today, instead of talking about a buzzword, I was gonna talk about kind of a portion of the word, otherwise, something that has been used to describe other words, and that is the word micro. And so one of the things that is happening a lot out there in discussions around digital marketing and also in internal discussions around IT infrastructure and all that is this term micro, where now we need to talk about something small by saying things like micro interactions or microservices or microarchitecture. And what does that really mean? Well, really it just means small. It means a thought. I always think of like a microinteraction as something like how a rollover happens on a button or maybe how the feedback comes back on a login form or something. It's basically those small subtleties that we're taking care of where microservices more about having this one little particular piece of functionality be pulled out so that it can actually participate in some larger processes or across lots of processes. But overall, it's just one of these kinds of weird senseless things that don't make a lot of sense. And it's just us repurposing a word because we got bored of the way we used to describe it. So now you know that when you hear micro X or anything else like that, you're basically talking about something we've already discussed in the past, but now we just got a new cool way of saying it. So thanks again for joining me for the podcast. I've really enjoyed having you. Next time, we're gonna continue on this search series and talk a little bit more about some of the offerings that Microsoft has, especially coming out of their 365 platform and how they're kind of integrating with Bing and some of the things that that's really gonna bring not only to the enterprise, but also to the public search side of the world as well. So thanks again. And if you haven't already, please feel free to subscribe. You can do it through Apple Podcasts. You can do it through Spotify, through Stitcher, through all the popular mediums. We'd love to have you as listener as we continue down this journey together in that. And if you have any follow ups, feel free to reach out to me. You can reach me via email at me atdiscussingstupid.com. That's me at discussingstupid.com. Until next time we do, feel free to start discussing stupid on your own.