The Impact of AI on Freight Procurement: Transforming Operations Through Automation | Episode 53

Hi, everybody. It's Patrick Escolas with the Banyan Technologies Tire Tracks, our presentation of the look of impact of AI on freight procurement with part of our mini-series. And I have with me Roger Boza, the Chief AI officer over at CloneOps.ai. Roger, how are you doing today?

 

Good and thank you, Patrick, for having me here. It's a pleasure to be on the show with you.

 

Is this when you tell me that this whole screen and picture is just AI created and this is nothing of what you look like?

 

We're heading in that direction, but I can tell you, we got an assistant right here zooming that. He's going to be pretty soon taken over by meetings and responding as if it was me.

 

I like that. I like that a lot. Let me start by, as we're looking at AI specifically within logistics execution here, from a pure AI standpoint, or even just background standpoint, Roger, who are you? How did you get into AI and what is CloneOps.ai for anyone that hasn't heard of them, or maybe has and just doesn't know exactly where you sit?

 

All right, so let me tell you a little bit about my background. I have a bachelor's and a master's in computer science with a track on software engineering.

 

You definitely know how to tell people to turn it off and on again.

 

Yeah. Usually, if something is not working, my recommendation is turn it off, power it back on, hit it with a hammer. If that doesn't work, then call IT.

 

That's right.

 

At the moment, I'm actually finishing up my PhD on artificial intelligence, both computer vision and natural language processing.

 

Where do you get that from? Is it from Miles Dyson over at Cyberdyne University? That's a Skynet T2 reference for anyone that isn't that nerdy.

 

No, it's my university here in my hometown, Florida International University. They're not very well known for AI, but they do have a track and a department in there. In fact, they also have a department called the Applied Research Center, which focuses on doing AI solutions for the industry. Not necessarily this one, mainly the Department of Energy and Department of Defense. That's how I started doing a little bit of my work. Actually, for the past seven years before joining CloneOps, I was doing AI automation and workflows in the nuclear industry.

 

Wow. Okay. A lot to unpack there. One of the things that, and thank you for joining us, because a lot of the people that I'm talking to for this mini-series already have a freight background, but probably don't have the AI background in the chops that you have. First, I want to start with, how is there a difference, maybe at that master's level of computer science versus the PhD into AI? Are you just looking at prompts all day? Or, what are we getting into that makes this so in depth? What do you love about it?

 

Yeah. There's quite a big difference between masters and PhD work. At the master's level, somebody is very specialized. They're really good at doing implementations. They understand the theory. They can develop applications for it, but they don't necessarily understand completely at a very foundational level, the inner workings of artificial intelligence.

 

Got you.

 

They know the pillars, right? Artificial intelligence is composed of a couple of things, like linear regressions, which is mathematical models for that, analysis, deep learning, machine learning. They get exposed to a couple of the methods, the ideas, the algorithms in there. But once you transition to PhD, you select a very specific problem. Usually, you try to advance the field forward. You basically research for two to five years, your proposed solution.

 

As we go in and I'll have some logistics-based questions, CloneOps.ai, where are they positioned? What are you doing now? Because like we talked about, you're not necessarily in the freight execution piece of it, but more of business. Where do you play? Where is your niche?

 

Correct. Yes. We focus on business communication.

 

Not logistics.

 

No. But it is in the logistics. Business communication from the point of view of you can imagine, you got a broker, you have an influx of calls, or an outflux of calls, depending on what scenario we're talking about. Let's say, you had a carrier sales rep doing great negotiation and load booking. On a typical day, they get a load, they'll go to a load board, they'll post it online and immediately, and I’ve seen this in person, as soon as they posted online, you get five, 10 calls, one after the other.

 

What your CSR is going to do, he's going to pick up the phone for the first one, right? He's going to start talking, trying to negotiate a rate. Hopefully, the goal is for the first carrier to call to actually negotiate the rate. Most of the time, that doesn't happen. You waste three, four minutes talking, negotiating. You hang up, you go and look at your caller ID, you're going to call back the second person that called to see if you can get some capacity in there. He’s angry, he’s frustrated, because you didn't pick up. He might have found another load, called that broker. It's just a back and forth, right?

 

Eventually all the stars align, you get somebody, they'll move the load for you. Here's where CloneOps comes in, we're using agentic AI. For this scenario, negotiation and load booking, you're going to get a virtual agent to respond to phone. No more missed calls, right? It will respond for everybody calling in. It will verify you, to see if you're approved with them.

 

Right. If you're a real person. Yeah.

 

Yeah. Yeah. I'll talk a little bit more about that, because we're doing something from fraud with technology, we're calling voice ID.

 

Okay.

 

We'll talk about that in a little bit.

 

Yeah.

 

The agent picks up, right, starts negotiating with you. If it comes to an agreement, it books the load on your behalf. If it doesn't, we have the capability to lock that as long as we have integrations with the TMS you’re using to load that capacity in their systems.

 

Now, and that's awesome, both from A, integrate with Banyan and TMSs like that. Please offer more value. That's a very selfish and biased approach. As a salesperson, I've lived that life, not even on the carrier side, where I got to answer every phone call that comes in. Doesn't matter if I know the number, doesn't matter if I've been calling people. Every number that comes in is an opportunity for a commission or a deal that I might just be waiting on that call. That speaks to me a lot. I think that that's not necessarily – sure, there's an application within logistics, but that's almost across the board anytime you're going to have phone conversation interactions that way.

 

Now, tell me this, and before we get back into more logistics, I want to go back in a little bit into your expertise and AI. Now, at the PhD level, are we talking just more of these learn language models, or are we starting to get to that point of actual computer sentience, or where that line is going to be crossed soon? What does that look like from a sci-fi nerd to a logistics freight guy to Roger, who is getting this information at the highest cutting-edge level?

 

I love that question. I heard one of my favorite keywords, which is sentient AI.

 

That's right.

 

Yeah. My opinion, we're still quite far from sentient AI. Now, we have our entire artificial intelligence framework, revolves around probabilities.

 

Okay.

 

All the models are trained on the data set. Whether they learn a function to predict the value correctly, because we have some labeled information, we know the input, we know the output, we can get some mathematical models in there to model that distribution. Where machine learning and deep learning kicked in is instead of us having to tell it what to pay attention to, what to learn, what features to extract, deep learning and machine learning extract that information at a basic level by themselves.

 

Okay.

 

We're not telling it – I will give you an example. Let's say, we had an image, and we want to know if this image is of a cat and a dog. It's one of the simplest Hello worlds in the field.

 

Right.

 

Normally, as a human being, you're like, “All right, I know that's a cat, because I see the fur, I see the pointy ears, it has a long tail, is very small.”

 

Right. But to your point, they're both four-legged, they both have tails, and they both have ears and are equally about the same size. From a pure just, if I'm doing bullet points, they have so many in common that's up to the perception to define.

 

Agreed. In fact, if we actually give a human being about 10,000 images of cats and dogs, they'll get roughly 80% correct.

 

Okay.

 

Some cats are going to start looking like small dogs, and that –

 

Those dogs that you’re keeping in the purses start looking like cats.

 

Yeah. Yeah. Deep learning and machine learning really have an advantage there, because we're not telling it what features to pay attention to. Basically, the way that we train them is here's 10,000 images. I'm going to let you know which one were cats, which one were dogs, and it goes through an iterative process. What it does is during those iterations is if it gets it right, nothing changes, if it gets it wrong, it goes back and it updates the neural network.

 

It learns, right?

 

“That's how learning happens.” It's updating, tweaking the knobs in the computational neurons in the network.

 

How would something like that, which it seems impressive, apply to logistics? If we're telling the difference between cat and dogs, what kind of things could that, here's 10,000 shipments. Here's what's a good one. Here's a bad one. Now, I want you to go find rates for the shipment that I need to move tomorrow and tell me if it's a good price. Where does that have application? Maybe that's not it. Maybe that's just the low-hanging fruit. How do you see it as a person who I'm sure ideas are blooming left and right of where you can fit AI in just about everything that we do?

 

Great question. I got a really good example in the field of computer vision, which is what I was describing. You can document classification, right? You might get an email, you might have a racon, a bill of lading, an invoice, a receipt.

 

Might get an email, get a million of those emails for sure. Go on. Now, you're talking my language.

 

During the order of hundreds. What you want to do is you want to understand what type of document that is, right? Because if it's a receipt, I want to extract certain information for it. That's where another part of AI kicks in, which is called optical character recognition. That way, we can pull out contextual and textual information out of images. You still need to know what type of document it was before, so that you have a baseline of what you want to extract.

 

It's not learning on the fly from nothing. It's got to have a core there to begin with. Then from that, it can build upon it.

 

Correct. Yes. That's another key point of machine learning and deep learning, which is called knowledge transfer. Normally, you start with a generic data set, right? That's a whole bunch of images, or receipts, bill of ladings, proof of deliveries for document classification, but it doesn't have to start with documents from the industry. It can be in general. What you do afterwards, and this is called fine tuning, or knowledge transfer, what you do is you have this base model. Now you give it the data appropriate for the industry, or for the scope, and it learns now to specialize within those documents.

 

Yeah. That brings a question, or maybe a concern. If we're feeding it all this information, does it have to be, and I think this is one of the early debates, or friction points of AI, how do you get it all this information from anywhere you can without taking other people's proprietary information? How do you feed it with real enough data that – because the public space only has access to so much to grab and do with what they want. Just about every 30%, 40% of it has someone's IP, or trademark stamped to it. That's really what you want it to know, though, because that's where people are investing and putting private dollars into. How do you differentiate? How do you marry those two trains of thoughts? I got to protect my information, but I also need this AI to have as much information as it can, or it's not going to be as worthwhile.

 

Yeah. The first way to tackle that is stay away from public models, right? An example here is the general OpenAI ChatGPT model.

 

Don't drop my entire share drive of my company files into ChatGPT and tell it, “Hey, you're my new bot. You're going to do work with me.”

 

Yeah. It's going to work amazing. It's going to do what you want, but you just gave OpenAI the right to use that data to –

 

You're going to have an HR conversation coming up pretty soon.

 

Oh, yeah. You're going to run into something more issues as well in there. What you can do is you can grab offline models, which means they'll run locally. Some models are small enough for you to actually do that, the typical computing power that companies have. Other models are too big. ChatGPT is way too big to run locally. You need a cloud service for that. Well, you can request a service and it gets pulled off of their production environments, it goes into your own clusters. Whatever data you put in there now is privately yours. It will not be used to train models that are open to the public, or shared across the network.

 

When – Yeah, please.

 

Found a little bit on that, because there's a couple of tricks, right?

 

I would just say, that was pretty simple, but I know you've got some new ones to it. Please.

 

Let's say that you don't have enough data, because I put – so, machine learning and deep learning is all about data. The more information you have, the better the networks learn.

 

Better it's going to be. That’s right.

 

The more they can generalize, the more robust they are. Let's be honest, right? A lot of times, you don't have a lot of data. You might be a startup company. You might have only been in business one or two years. You might only have 500 documents, instead of 50,000 documents. There's another technique is called data augmentation. What that does is it's another AI model, but this one is a lot smaller. What it can do is it can learn the data distribution of your 500 examples and it generalizes, it generates from within that data distribution, it generates new samples for you. What you can do now is you can use this new samples and train the network that you want.

 

If you've got the data, you can use it all there and use it to great effect. If you don't, it doesn't mean you’re SOL. You can use AI to pump that up. Then as you get more data, continue to build upon that to get better probabilities, like we talked about at the end of the day. Okay.

 

You're probably familiar with this. Have you heard of the service called Midjourney?

 

I have not. Tell me about that.

 

Midjourney was a service. I mean, they were pretty cool. They were one of the first ones to actually impress me. You can give it a sentence, or a paragraph and it would generate an image based off of that. We got a text to image. Let's say, that I want to start classifying –

 

Like a good-looking bald podcast host, right? That's insane to see how close it –

 

We’ll use you right now. We use you right now as our prime example. Midjourney is a generative process of taking text to image. If I was limited in some information, I can use one of those service, generates hundreds and thousands of images and use them now to train my model, fine-tune my model, make it a little bit better. Gaining some time so that I can acquire original ones and merge everything together.

 

Then, you're also, to your point, you're not going out and grabbing somebody else's protected data. You've generated all these to start that. Okay. That makes sense to me, which is great, because I assumed most of what you said would go right over my head. I'm happy about that. Now, we talked about AI within logistics and on the conversations back and forth on the rate, getting emails on the racon and being able to do it from there. What aren't we talking about in logistics space that we could be doing with AI?

 

I know that you're talking about next steps or, “Man, what if we did this?” What could logistics and freight be doing with AI that we haven't really gotten there yet, or no one's been creative to start thinking it through?

 

I think there's been some conversation about what I'm going to say, but it's definitely where I see the industry going.

 

Okay.

 

That is AI talking to AI.

 

Okay.

 

From our examples and our pilots, what we noticed is we got a virtual agent representing a use case scenario, or a broker. Now, the carriers are going to get a whole bunch of calls. They get frustrated, they're picking up the phone, they're talking to virtual agents, and they have a need now to have a virtual agent to respond for them. At that point, now you have a virtual agent on one side and the broker, you have a virtual agent on the other side, the carrier, and now you just have AI talking to AI.

 

At the end of the day, the email gets to the right person to say, “Hey, load this up, or get a truck out here.”

 

Correct. Yeah. There's three main communication mediums. You got text, you got email, and you got voice. A lot of those transactions happen across all three. You might get a phone call to start negotiations. You might get a text back, because you couldn't respond right away, or track and trace, here's my update, this is where I am. Then at the end, you might get through email, the bill of lading, the proof of delivery, whatever else closer. You got three modes of communication. What AI can do is bring them all together, so that is aware of the whole context from a general point of view, and engage in communication. At this point, communication is just the exchange of information back and forth. You don't really need voice at that point.

 

Now, and that makes sense. Here's where I like that I'm talking to you, because I want to talk about what could be a negative repercussion from that, because I see the benefits and how that could work. Now, we have AI talking to AI. Isn't the fear that they develop a language that we can no longer decipher, because they find it more effective to talk in that language than the language we've taught them?

 

Yes. I think you got the hint there where I said, voice is not needed anymore, right?

 

They’re talking ones and zeros and blips and blips. I'm going, “Man, this is the weirdest Morse code conversation I've ever heard.” Yeah.

 

That's where I was heading, right? All communication is going to be digital. At the end of the day, it’s ones and zeros. Normally, conversations, and in this case, communication back and forth, follow a protocol, is a standard way for two agents, or two mediums, right, or two different endpoints to communicate. They agree on how the exchange of information happens. That protocol right there might be AI to AI. Like I said, it doesn't have to be voice. It doesn't have to be human readable language. It just takes more information to store it and display it. It's going to be simply the ones and zeros that communicate back and forth.

 

I still think that you need AI, because you have the benefits. You want somebody to represent your own interest. That will be your AI. you're communicating with another party. They have an AI to represent their best interest. Now, communication just happens in between.

 

With that, and I do get that and I like it. Now, will they'll be in this world where the AI are talking to each other, there's going to be a receipt somewhere, right? Either on the local server, or on a cloud of this conversation. At the end of the day, someone goes, how did these two systems get to the point of, “Hey, we're going to do it for $300, and I'm going to have a truck there at 7 a.m. tomorrow.” Is there a ledger someone can refer back to? Again, because it's logistics, a mistake is going to happen somewhere along the line and be like, okay, where do we circle that and make sure we don't do that in the future?

 

Yes. I agree. Normally, that happens at the observation level. Meaning, you might have a dashboard that is collecting, aggregating all of this information. It might be by event ID, or it might be by conversation ID. There's always a log, a transcript of everything that has happened between all parties in the communication.

 

Right. Okay. That makes sense. I think there and much to the theme of logistics and AI, that's where you'd want a person. You don't want AI watching the two AI for oversight, much like, I don't want me and my sports team to be the one telling the ref what to do. Because inherently, I'm extremely biased and I have no reason to necessarily do the right thing for all parties.

 

That's actually something that happens with large language models. They get to be a little bit biased. Biases introduce either when you give it the data to learn, or the way that you structure the neural networks and the outputs when you introduce another human bias. What ends up happening is it's going to make predictions that favor that bias.

 

When you say bias when it comes to AI, is this just, I'm going to lean to make these guys be my good guys? Or is this almost a bias in perception that, hey, if whenever you turn left, it's wrong, you should always turn right?

 

I think it's both. I'll take it back to the example of cats and dogs, right?

 

Yeah.

 

If I gave it 9,000 images of dogs and 1,000 images of cats, that model is going to be way better predicting dogs than cats.

 

Yes. Okay. It's almost a data bias.

 

It's a data bias. Yes.

 

Okay, that makes a lot of sense. How does somebody, say that I'm trying to use AI in logistics and I want it to be as accurate as possible, how do I avoid a data bias?

 

You try to make sure that your data is balanced. Let's take it back to that document classification as an example, right? Maybe we gave it 10,000 receipts. We gave it 100 BOLs and we gave it 5,000 PODs.

 

Got you.

 

AI will always make a prediction. Usually, there's a probability attached to it, or some sort of confidence as well, if the engineers and the models can support it. What happens there is you're always going to get an output, right? If you just say, “Hey, I got AI. I'm going to turn my back and I'm going to let it run by itself,” it's going to make errors. Now, the errors, right? That those wrong predictions aren’t –

 

And humans never make errors.

 

Oh, not at all. I wouldn't say that.

 

No, no. Please, continue your point. I'll remember that for my next question.

 

You start getting errors. You always need, and that's a philosophy we have here at CloneOps.ai, always have a human in the loop, not for the repetitive task. Let the AI handle that, because that's what they're exceptionally good at, but an exception handling. Things are going to go wrong. You just need a way to catch them.

 

I'm smiling, because that's an amazing thing. That's what we talk about with the use and the real function and end day goal of a TMS, or software in general. 80% to 90% of it is the nothing has happened different. It is the bread and butter, and then you still always have to have that 10% to 15% that, hey, the world is on fire. This didn't get where it needed to, so I got to make sure I'm paying attention to that. The only way I can do that is knowing the majority of things is being handled by a technology somewhere. Otherwise, I got to have way more people involved than I probably have the payroll for.

 

Yeah. What you want is some analysis in there to help you, as the human in the loop to know when things are going wrong. In our case, for our conversation, we have a conversation score. That's a metric that is composed of three different analysis. We will analyze the text itself, because going from voice to text is a transcription, so I have to see.

 

Right. You make sure you see what errors could happen in step one, right there.

 

Yeah. What I'm doing is, is this conversation at the text level going positive, neutral or negative?

 

Correct.

 

That's a sentiment analysis. We'll take it a step further. How is the mood of the conversation? In this case, it's a customer, but it's a carrier, it could be a dispatcher.

 

Yeah. Someone saying, “Yeah, I really like working with you,” or someone's talking about all the expletives about why they didn't like what happened. Yeah.

 

Is the mood in the conversation at this point in time, is it angry, happy, sad, frustrated? That relates to somebody dropping an F bomb in the middle of a conversation, because they got frustrated.

 

Right.

 

That doesn't capture everything. We have a third metric and there's the voice itself, the audio. We analyze the audio to see if you are screaming, whispering, if you're rushing through your words. Are you speaking normal? What we do is we combine all three of those into a single number. This number is going to be between zero and a 100. We’re going to let you know how that call is going. Now, whether something goes wrong because the virtual agent made a mistake, maybe it had grown data because somebody entered gibberish in the TMS and we pulled it from there. Or maybe they're just not understanding each other, or they're not agreeing. That should drop that conversation score and we can capture that with the sentiment, the mood and the tone of the voice.

 

What you want to do is you never want to turn your back on the system and just walk away and let it work. It's going to work maybe 80%, 90% of the time with the scenarios. But that conversation score allows you to now monitor the calls, sort, or filter by it. If a call is going bad, here's where the human intervention comes in. Go ahead and take over that call and mend the situation.

 

Now, that makes a lot of sense to me. It sounds like, those that are doing all of these fine details of execution in your world, or where AI is involved are going to pivot to more of a management role, but not necessarily managing people, but of either managing AI, or of the rules and the analysis that's happening there. You can do more, but you're still there, you're still at the dashboard, you're still at the controls as it will.

 

Yeah. That's usually an organizational structure that changes, right? AI is not just replacing jobs. It's a tool. AI is a tool. It should be more effective. It lets you have higher throughput. It lets you scale very fast. Now, you need new roles to do additional things, because you're engaged in AI.

 

It's not coming for your job. It's very much the automation in the manufacturing lines, where you might need to pivot from putting something on and off the line and figure out how to manage these robots doing it, or how to build the line itself. It's not necessarily taking your exact job, but you may want to look at how you can be the one managing the new technology.

 

Exactly. Yeah. Humans, we’re very good at creative thinking and problem solving. Usually, with AI –

 

We also create those problems. We're really good at that.

 

Yeah. Yeah. We can invent a problem and we can drown in half a glass of water.

 

As growing up, someone always used to said, “Patrick's really good at talking himself out of trouble. He probably talked himself into trouble, too.” But, yeah.

 

I can see that happening. Yeah.

 

Yeah. I like that call out and at this, where humans are creative and at problem solving, let me go back to this thought ahead on errors. We're replacing some of the human interaction, and some of the human execution, which inherently, has errors in it. The AI and all of the language models in the different AI communication are also going to have errors. Is there a different level of acceptable errors they were going to perceive, or by the nature of the technology, it's always going to be less, or it's always going to be more?

 

In my experience, once you have good AI models, the error rate is a lot lower. That's part of the process, because we would train models, compare them to a bench line, where we know how humans perform, right? Let's say, the humans were performing 85%, 86% accurate on this benchmark, we were aiming for something higher than that. AI usually takes us to the mid-high 90s.

 

Okay. When you say good, does that mean it's been doing it long enough? What is it, having it in place long enough, or having the right amount of data? Or is it the actual engine that it's built on? How do you define a good AI versus an AI that's not good enough? That’s another great, hard question, I'm sure. Yeah. Not an easy yes or no. Right. It's why they pay me the big bucks.

 

That's usually a threshold that is set either by the engineers based on business insight, or an entire industry. I'll put in a little bit of my background, right? Coming from the nuclear side with the Department of Energy.

 

See, you can't have bad AI when nuclear is involved, right? The opening, or the window of error has to be minute comparatively to a lot of other industries.

 

Exactly. That's what I was going to say. In that industry, those solutions are considered critical systems. You're aiming for 100%.

 

Right.

 

Now, here's the caveat. AI is never 100%, because it's probabilities, right? What's the probability of me predicting a good outcome?

 

Even at 99.999% still has that .0001% that lives out there. Right.

 

At that point, you usually want a fallback mechanism, or you want some explainability to catch and have the AI explain as to why make that prediction, or that decision. For most of the industries, they're not critical systems, so you don't push them that high. It's a diminishing return what you get out of AI. Let’s say, you have a model and it was already at 95% accurate. Getting it to 96% might take you, let's say, a couple of weeks. Getting it to 97% might take you a couple of months.

 

There's an exponential demand for time and effort once you hit a certain point, is what I'm hearing. It's almost this classic, perfect is the enemy of good, right? Or better is the enemy of good enough, because if you get it to a certain point, that's as good as you're going to get it, and have a human being there to be ready for it, because otherwise, you're investing in other five, six years and quadruple the resources you did just to get to this point. Okay.

 

I'll give two examples, so that we can put some context into this.

 

Please.

 

Let's go back to the document classification. If I were to design this, right, and you are my “boss.” It's like, “Hey, Roger, we need this to work as good as possible. I want this to be perfect.” I'm going to come back to you and I'm going to tell you, no, let's be real. Are you okay if the AI classifies 97 proof of deliveries correct out of 100? Are you okay with three of them being wrong? We will catch it with a redundancy system, and now we have the human intervention. We go in and we manually fix those. Are you okay with that? If you tell me yes, then I'll set that as the criteria for the models. That would be this accuracy.

 

If you tell me no, then we're going to have another conversation where I might have to explain why going with higher accuracies are going to be difficult. Maybe we can push 98, or we can push 99, but that's where the engineering comes in.

 

Roger, I think you just hit on a point that comes up with everybody I talk into logistics, is that no matter what technology you put in there, whether it's basic automation, integration, or now as we're talking some complex features of AI, it's that nothing can get done, or get done the right way without a human understanding of what that goal is and that relationship there. Because like you said, Roger, make it good. Okay, I'm going to need you to define what good is. Does that mean one out of three? Does that mean one out of 10? Because without that, your good is all right, here's 50-50 ball. It is not the same as what they – And if you don't have that conversation and that relationship, it doesn't matter. It's not going to work.

 

What we want is an objective metric. Something that has no opinion on it, because A, this is good. That's just an opinion, right?

 

It's really tough to do. You're talking logistics. Everybody has an opinion and they're all going to tell you whether you want to hear it or not very loudly. No, so this objective base. Yeah, go ahead, though.

 

I'll give you the other example, because the one that I explained is for computer vision. But it also happens with natural language processing. In the case of CloneOps, where we have voice AI for conversations, what we would do is here’s an agent, here's a use case scenario. Let's go back to the rate negotiation and load booking. I'm going to give it 100 test cases. If it successfully negotiates on 97 of them, or 98 of them, I'm going to say, this is good.

 

Correct.

 

That criteria right there and I think that's where companies and decision makers, they have to sit down and have a conversation about it is, what is our acceptance criteria for this to work? What are the KPIs to know that this is how we want it and we will mitigate the rest?

 

I like that. I like that. It's almost calming to know that we'll still be needed as a species during even most of the robot takeover. Jokingly, but to a more serious question for you is, as you're going through this PhD, as you're delving into this, both from a professional and what sounds like a personal curiosity and passion side, what excites you the most about what AI can do? On the other hand, with the devil on the shoulder, what is your biggest concern of what happens when we get AI to a certain point?

 

Yes. I think I can answer that based on how I got interested and my curiosity for AI.

 

All right.

 

I was playing video games, and maybe you heard of this one, World of Warcraft.

 

Hold on. I am an original Vanilla player from 05.

 

Right here. Yes, sir.

 

I am now playing anniversary edition. We're going to exchange contact information after this, because nobody needs that nerd dump. Go back, just loving that you're a wild player.

 

I started right at the end of Vanilla, going straight into the Wrath of the Lich King. What happened was I jumped into the game by mistake, it caught my attention, started playing it. It was consuming a lot of my time.

 

Oh, yeah.

 

To get the next resource, to get a better pair of boots, or a better garment, or a better sword, a better shield. I have to play weeks. I was like, “You know what? I got programming skills. Let me try to automate that.” By the way, there's a difference between automation and AI. My first attempt was, let me try to automate what I'm doing, so that it can play by itself, because I want to go do other things, right? I don't just want to play. I want to enjoy the end game. Getting over there, I don't want to spend weeks trying to get it done. Let me automate this. Within a couple of minutes, it looked like crap. I was like, all right, I can tell that that is a robot. Somebody's going to see me. Somebody's going to report me.

 

I’m getting kicked off. I'm breaking the terms of service. It’s easy to see from a mile away. Yeah.

 

I started looking a little bit more, how do I make my character move, fight, collect, behave more natural as a human being, as I would be playing it? I tried with equations. I tried with models to replicate what I was doing. That is the essence. That is the nature of AI.

 

Ah, I love so much that the core of your entire AI professional career comes from World of Warcraft. We will definitely be talking about this outside of this podcast.

 

For sure. I went back to education to learn now the foundation of how to actually do AI.

 

That's awesome.

 

One of the difference at the PhD level is I don't necessarily just have to go and use a ChatGPT model, and I should say, a large language model. I don't have to use what's out there. Or for computer vision, I don't have to use the models that are out there, open source. I can create my own. Usually, what I want is the smallest model, so that it doesn't consume a lot of resources, both computationally and –

 

Natural resource-wise. Yup.

 

Yeah. You want the smallest model as possible, but that it gives you the accuracy that you want. Now, what's available out there open source, usually meets one out of those two.

 

Okay.

 

Here's where I come in, and this is one of my roles at CloneOps. I will develop the new technology, if what's available out there is not worth it business-wise for us.

 

Right. If it ain't broke, don't fix it. If there's nothing there, let's come up with something.

 

Yeah. I don't like reinventing the wheel. I don't like shooting a bazooka to kill an ass.

 

Right. Right. With that understanding where you come from, what's that highest peak? What's the lowest low of what AI could do both ways?

 

The highest peak is considering it as a tool, it gives me my time back and the others.

 

That’s the only thing we can’t buy more of.

 

When we have video games, what we can never get back is time. That’s my very thing about AI, it gives me time back. In my case, I usually will use that so that I can spend more time with my family, or doing other activities that I like to do.

 

Heck, yeah. All right, what's the flip side?

 

On the opposite side, what scares me the most is when one of these models make a prediction, and the model itself is not explainable. We have no idea why it made that prediction.

 

This jump in logic, or probability that we can't track.

 

Yes, that's the problem. At the basic level for AI, you can have something as simple as a decision tree. If it's the weekend, and it's sunny, and I don't have to work, I'll go to the beach. If we end up at the beach, we know exactly why, because it's a decision tree, we can follow that path.

 

Right. If A, go to B. If B, go to C. It's like, the geometric proofs that they made you learn, and you're like, “Why am I writing in a math class?” Yeah.

 

Exactly. Yeah. With large language models and computer vision, which usually, you need neural networks, you have millions and billions of parameters in there. Each one of them is like a trace, is a jump, is an if.

 

Yup.

 

From the human context of this, look, I have a hard time understanding 20 conditional statements as to why we made a decision.

 

Let alone a hundred, let alone thousand, let alone millions and billions. Yeah.

 

We can always go back and say, well, clearly, it predicted this, because we gave it this input, and here's everything that happened. If I ask now, put it in human terms, what actually happened, we're like, “Hey, no, no.” We're stuck there. That's the part that worries me a little bit when some of these models make predictions, and we have no idea why, because it also ties to another concept. GPT, it stands for generative pre-trained transformer. That first word right there, generative, it means that it can generate new data.

 

It's creating.

 

I don’t want that. It can create. You want that. If not, things are going to sound very robotic, is going to sound like pre-recorded calls, or pre-recorded arguments. You want it to generate for you.

 

Sure.

 

But as soon as we give it the ability to generate new content, that means that we're introducing randomness, and that's where explainability starts being hard.

 

All right. Okay. Roger is against entropy. All right. No, but yeah, no.

 

Don't quote me on that, because my thesis is in entropy. That’s a whole another topic and we might need five series on a podcast to cover entropy and information theory.

 

No, it's a great call out. But I think that's a very important call out and fear or concern there. Because again, it goes back to almost one of the first things we talked about that even as those AI could be talking to each other, there's a receipt somewhere. You can follow the logic. You can understand where it came from. The second you’re – and it's in a black box and you can't see that, that's something to be concerning. Whether from a business sense, or a morality sense.

 

I think you have been reading, or hearing about deep learning, because you said another keyword, black box. True enough, that's actually the term that we use for the model itself, the architecture between the input and the output, was in the middle is a black box.

 

That's just AI, man. I mean, that's just sci-fi, as far as grow and up reading the greats. I think one of my last questions will be, as you're looking at these cutting-edge technologies within AI, as you're messing with things in your PhD class that most of us probably can't comprehend, are ideas like the Asimov rule of three being inputted, or these the Wild West? For anyone that doesn't understand, these are basically the three rules of science fiction robots is that one, you can't harm humans. Two, you can't harm yourself. If you have to harm yourself to save a human, you're basically doing that. What parameters are you putting in? Is it as simple as three rules? Or is it as big as a Bible, or a dictionary that someone's got to load up into? How are you, for the lack of morality and human perception, how do you encompass that into a technological piece?

 

Yeah.

 

Yeah. I know you have no one way answer on this. I knew I had to ask a real hard one for you, because you're way too smart for me. I had to stump you and make it tough for you a little bit.

 

What I can tell you is that that's actually two fields. One of them is computational science. The other one is philosophical.

 

Okay. That makes sense. That makes sense. That's where I waste a lot of my time is the philosophical. Ask my wife, “Why are you thinking about that? That has nothing to do with us.” I don't know how not to.

 

Yeah. From the computer science side out of this, yes, we can always put in – so, code when we write program, syntax, script, it's deterministic. Meaning, given the same input, when you execute it, it will have the same output, as long as –

 

That’s the computational side.

 

- you don’t give put randomness in it. From a programming point of view and an AI point of view, if you have a system that is deterministic, yes, you can put those three rules, right? Human life is precious, preserve it and help it. It's like a two-sided sword here.

 

Sword. Yeah.

 

Yeah. If we're talking AI and we're given the capability to generate, the first question is, can you generate a new set of rules that can trump, or overthrow its original programming?

 

Right. Like, the first rule has to be, you can't mess with this. Then the fourth rule also has to be, you still can't mess with this. This is it. No, okay. That's awesome.

 

AI is not really good at rule-based.

 

No.

 

Especially if it's generative, right?

 

Trust me, I've gotten in a fight with Copilot and ChatGPT. Yeah, I know. I know firsthand.

 

Yeah. Good, because it's a contradiction, right? If I tell you to follow rules, you will not be able to generate new information. If I want you to generate new information, you have to skip some of the rules. Some people get clear and they tell me, it's like, “Well, Roger, can we just have a rule that says, generate new information?” Well, that would invalidate the hypothesis at that point.

 

Oh, my God. Roger, you just found a weakness in my parenting. I want them to follow the rules, but I want them to be creative enough to break them, so I don't understand that they're breaking them, so that I know they can have street smarts. Oh, man.

 

I think you get enough. That's why it's really hard to answer, because even we as human beings, and you gave a perfect and beautiful example, by the way, because I run through the same thing. I have a three-year-old son and a seven-month-old baby girl.

 

Yeah. Congratulations.

 

For my son, thank you. For my son, I want him to listen to me. If I say this, you better do this.

 

But I also want you to think on your own.

 

Yes. I want him to be independent. As soon as I introduce that, letting him make his own mistakes, then now he has the ability to do what? Ignore what I'm saying.

 

Okay, and I think that's awesome. Now, as we're coming to an end here, because like I've said a few times, we could keep going and nobody else would want to listen, but us. As I have this platform and as people listen and watch, what is something that Roger, or the Chief AI Officer of CloneOps, or just a guy going for his PhD in AI, or Roger off the street, what do you have to say to the world? Borrow my soapbox. Here's two, three minutes, and even if it's shorter than that, it's fine. What do you have to say to anybody listening, or watching that you want to make sure you get your point across?

 

Yeah. AI is not theoretical anymore. If we go back 10 years ago, where it's like, oh, AI. AI this, AI that. Well, we’re at the point that AI is everywhere we are. It's on your phone. It's on your smart devices. Even my fridge that I bought about six months ago has AI inside. If I'm putting in a milk jug, it's going to tell me, you have two of these ones in here. If I'm at the grocery store, I can check the inventory. I know that sounds funny, but I can check the inventory of what's in the fridge.

 

I'm imagining a space odyssey moment where it's like, “Roger, you don't want to drink that milk anymore.”

 

You can check. When you first put it in there, right? If you know the expiration date, I’m going to stop right there.

 

That’s awesome.

 

AI is all around us, right? My opinion, there's no need to be afraid of it, because it's not the sentient AI. I think we're really, really long away from that. We will have to make some changes fundamentally to how we compute and execute on computers, which that's a completely different topic. It's a tool. It's here to help us. It makes us more efficient. When it's used correctly, we get the most precious thing as a resource back to us, which is time.

 

That is powerful. Roger, thank you so much for all of your information, your depth of expertise, and just being someone that understands the nerding out of being a World of Warcraft player. I really appreciate your time today. For everybody watching, thanks for joining us for another presentation of the impact of AI on free procurement that we're doing for a mini-series. Engage with me, subscribe, watch, listen, find me in Azeroth. Thank you for being here. We'll see you on the next one. Roger, really appreciate it. Thank you. Have a great one.