Thinking With Machines: Sense-Making, AI Agents & the Future of Work

ContraMinds Podcast (01:23.8)

Hello Professor Vasanth, lovely having you as a special guest again in the Contra Minds podcast. Thanks for taking time and talking to me.

Vasant Dhar (01:32.843)

Really happy to be here Swami. Happy to chat again, looking forward to it.

ContraMinds Podcast (01:38.888)

So congratulations Professor on the new book that you’ve got out and Thinking with Machines is absolutely fantastic. I’ve had a brilliant time reading it and it has beautiful anecdotes and ideas and I’m really looking forward to having a conversation and your ideas on the book.

Vasant Dhar (02:00.012)

I am super pleased to hear that Swami and we were chatting before we started the podcast and you said, you know, it reads like a story and of all the things you say, that pleases me the most because it was meant to be a story as opposed to a textbook on AI. So I’m pleased to hear that. That’s exactly what my intention was to write a story.

ContraMinds Podcast (02:19.16)

Yeah, in fact, it read like a novel to me and not like a technical AI book. And that is really what made it interesting. And I just, you know, got it in one sitting is really what I did. So thanks a lot.

Q1

I’m going to kick off this conversation with my first question, which is really, one of the things that I found interesting in this book is there are a set of, you know, thoughts that you are leaving in every chapter, okay, which makes people think and it’s almost, that’s the philosophy with which you have written every chapter it looks like.

And you make a very, very pertinent point, Vasant, which is really the fact that in the age of AI, we as humans have to develop this art of sense making, right? And can you talk a little bit about why sense making in the age of AI is going to become super important for us as humans?

Vasant Dhar (03:31.559)

Sure, so let’s back up a little bit,

…you know, just to what the book is called, right? It’s called Thinking With Machines. And the with is really important because like it or not, the future for you and I is one of thinking with machines. There is no opt out of this. And so what I’m doing is I’m raising a set of questions that we should address and I’m… describing how we should think about this future for each of us individually, because for each of us individually, the future is different, but the questions are very similar. So one of the things that I’m trying to do in the book is just lay out how to think about the future of AI and how to think about a future we can think with machines. 

What you’re raising, is, I mean, you dived in right into one of the core themes of the book, which is sense-making, is just that, you know, the machine as we know it, and I’m going to sort of unpack a lot here, modern AI is really largely a black box. 

And there’s a reason why it’s evolved that way. And the reason is we’ve tried to create a machine that is conversational, you know, in our own image. And I should start by saying that one of the biggest achievements of modern AI is breaking down the distinction between expertise and common sense. If you really think about the implications of that, that’s really profound because for my entire career in AI, which is now almost 50 years, I got into it as a graduate student in 1979. For my entire career in AI, there was a distinction between expertise and common sense. And people in AI said, common sense is just too hard. Let’s focus on expertise and knowledge.

And that’s been for the most part how the AI paradigm has evolved. We built these ever more sophisticated applications. But what happened after 2017, when the first paper on transformers came out and then what happened with the release of ChatGPT is that we demonstrated for the first time that anyone could talk to the machine about anything at any level of depth. That’s a remarkable achievement, if you think about it. 

And I never imagined that I would see that in my lifetime. So that was just like a mind blowing kind of achievement. The distinction broke down. The reason it’s important is because now when you talk to the machine and you ask it something and it gets it wrong or it gives you something else, you say, well, that’s not what I meant. I meant something else. And it has the common sense to understand. And maybe I should use the word understand in quotes – understand the meaning of what you’re saying, the meaning of what you’re getting at. And it does that because it sort of has this common sense. So that distinction has broken down and that’s what made it accessible to everyone. 

Now how it happened is, and know, excuse me, I’m sort of talking a little, you know, for a little too long here, but that was a question that was just packed with content. I need to unpack it. The way we achieved it is, you know, by predicting the next word in a sequence. Like, I mean, it was as simple as that, which makes some people think that, well, you all it’s doing is sort of next word prediction. Is it really intelligent? Does it really know anything? And the answer to that is in order to predict the next word almost flawlessly every time, it has to know a lot about the real world. And that’s what it’s learned as a byproduct of this keyword prediction.

And I talk about this in one of my podcast episodes with Sam Bowman. I think it’s episode 58, where he says, we just got lucky that Google was trying to complete sentences in Gmail. And that really led to sort of this general purpose technology, which could complete any sentence, which could predict the next word. So he said, we just got lucky that we picked a problem that we could solve for which there was lots of data to make it solvable. 

Now, by doing this, what we’ve created is a machine that is intelligent for its own sake. This is a machine, first machine built by us without any purpose. But at the same time, we don’t understand its inner workings. So when we ask it something, we have no idea what’s gonna come back at us, except that it’ll probably make sense. And that is the seductive aspect of the machine now, is that because it makes so much sense all the time, we just accept it. We just accept its output. And it’s critical to actually make sense of what the machine is telling us. It’s critical to not just take its outputs as given and pass them along to the next person. It’s important in most areas of our life to actually look at those outputs and say, do I believe it? Is it true? Does it make sense? 

So sense-making is just a huge part of working with modern day machines. And I set this up early in the book, when I talk about sort of traditional AI, right? But I say, patterns emerge before reasons, before them become apparent. And the reasons is all sense making, right? So throughout AI, in the previous paradigm, we had to make sense of patterns. And now with modern AI, we still need to make sense of what it’s telling us. So sense making is at the center of our… sort of evolution with machines going forward.

Q2

ContraMinds Podcast (09:20.12)

brilliant. You know as I read the book, one of the things that you know got me thinking is the industrial era. What it did was you know the machines that got invented…okay, actually made humans into almost machines. We just do like what you explained in the hospital and the healthcare where people have become actually machines trying to come in and they are just providers as you call them in one of the chapters, right? In the AI era, though we are creating machines, I think we have to become more humans, right? Does it make sense to you?

Vasant Dhar (10:04.304)

So let’s back up. So I think what you’re pointing out is I described the healthcare system in the US at the moment as giving us the worst of both humans and machines. And I describe a situation where I went to the emergency and I felt like I’d gone through an assembly line. And so the healthcare process has become sort of unitized. You go from one…provider to another, they ask you pretty much the same things, and then you get passed on like an assembly line. And that’s what’s happened now. 

So ironically, what’s happened is as we’ve made this process, as we formalize the process, we’ve turned humans into robots, right? Where they’re just functioning like machines in this process. And at the same time, the machines aren’t learning much about what’s happening in the healthcare system. So that’s what I meant when I said we’re getting the worst of both worlds at the moment. You know, that is humans have become robots and we’re not getting the full benefit of the machine because it isn’t recording all the right kinds of data from our interactions with the healthcare system and learning something. It didn’t learn anything from my experience. And I suspect this is happening, know, millions of times a day where it’s, you know, where we’re coming into contact with the healthcare system, but it isn’t recording the right kinds of data. So it’s, it’s, it’s, it’s not learning. And at the same time, humans have become robots.

So that’s what I meant, but I’m optimistic that that will change because I feel like in healthcare, in physical healthcare, our incentives are more or less aligned, you know, with the, you know, that is the patients and providers have the same kind of incentives. So I feel like we’re going through a transitional phase in healthcare at the moment, we’re getting the worst of both worlds, but I’m more optimistic. I healthcare is one those areas where I was more optimistic that machines would actually look at this trace of activity that we leave behind. When we go see a doctor, there’s some sort of an interaction that tests performed. But so far, it’s been very difficult to pull all this data together into a database to make it usable and learn from. I’m optimistic that with modern AI, these tools that actually, quote unquote, understand what’s going on in the process, that they’ll pull this data together and put it into a proper database that makes it more usable to learn from.

Q3 Scorekeeping

ContraMinds Podcast (12:34.54)

The other thing that your book made me really reflect on you… constantly use a word called scorekeeping, right? Where I really found it a beautiful terminology because if I’m a teacher, I need to do scorekeeping. If I’m a doctor, I need to do scorekeeping. If I’m a stock trader, I need to do scorekeeping because I’m doing things out of my instinct and knowledge which the machine may not know. So the fact that…

Vasant Dhar (12:48.443)

Yes.

ContraMinds Podcast (13:09.838)

We need new forms of systems where we need to keep using our data, putting it back into the system so that it can become more and more intelligent. And we need new forms of methods by which our knowledge and our experience need to get embedded into these systems. Do you think that’s critical and are there going to be new form factors that are going to come in?

Vasant Dhar (13:35.495)

Yeah, exactly. mean, you you said it better than I did, which is that scorekeeping just pervades every aspect of our lives, right? It’s just there. And, you know, as humans, we keep track of it the best we can. But computers are so much better at scorekeeping, right? Whether it’s in finance or healthcare or, you know, any other aspect of our lives, you know, they’re so much better at scorekeeping. It’s just that in the healthcare system, they’re not doing much of it at the moment. And that’s what I’m more optimistic about that going forward, you know, with modern AI that, you know, can understand the meaning of the data trail it’s seeing, it’ll be able to say, yeah, Swami visited the doctor on July so-and-so, here’s the results of the exam, here was the medication that was prescribed, and then two weeks later, he came back and lo and behold, his blood pressure was lower and he was in better health or whatever. There’s that scorekeeping that lets people who are providing care to you more information about you. At the moment, they’re largely flying blind. They barely have 30 seconds to consult your chart before you walk into the office. They’re just inundated with patience and functioning with a lack of information.

Q4 DBOT

ContraMinds Podcast (14:52.844)

Yeah, the other thing I found fascinating again is really the, you know, the Damodaran BOT architecture, the DBOT architecture, I found phenomenal, especially the number of AI agents that you put in into the architecture. Can you just unravel it for the audience? Because it’s a fantastic architecture in terms of how to bring in that embedded knowledge of the man himself into the AI system.

Vasant Dhar (15:24.006)

So let me give you a little bit of background since I suspect many of your listeners are interested in finance, which is an area that I’ve been immersed in for over 30 years now. 

So I brought AI to Wall Street in the early 90s and my focus was on learning how to trade based on data and it was like trading, like short-term trading…holding periods of a few days to a few hours. So at one time I was doing a lot of high frequency trading where I was trading almost like 0.1 % of the volume of the New York Stock Exchange, just like a lot of high frequency trading and that was phenomenally profitable. But my main emphasis was on short-term trading, learning from data to make short-term predictions.

So I’d been doing that for, I don’t know, 17, 18 years. And in 2015, I had two conversations with my colleagues. One was with Scott Galloway, who’s written the foreword for my book. And we had a conversation, it’s on the internet called, Should You Trust Your Money to a Robot? And we chatted and the end of it, he says, okay, so trading floors will disappear, but private equity and venture capital is safe. I’m like, yes, I think that’s a fair way to put it that’s inherently human, of long-term investing, long-term thinking, is not something that the machine can do. It doesn’t have enough data. There’s no basis for it to do that. So that is in the domain of long-term investing. 

That same week, ironically, I met with my colleague Aswath Damodaran, who’s sort of the guru of valuation on Wall Street. And we had this conversation about whether we could create a machine that could learn to think like him. And we decided at the time that it would be too hard, that his thinking could not be systematized into a system at that time, given the state of the art of AI. But when ChatGPT came out, we revisited that initial conversation. And I went back to Aswath and I said, should we try this again? And he said, yeah, it seems worthwhile. So I sort of initiated a project with one of my colleague, Jav Sidok, here at NYU. And at that time, I had no idea whether this would really work. But the idea was to systematize long-term investing in the same way that Damodaran thinks about it. Now, the reason this was feasible in my mind is because Damodaran is the only one out there, bar none, there’s no exceptions, who has published the volume of valuations, right? No one else has published, you know, the number of valuations that he has. So it’s all out there and it’s been out there for years, right? 

So if you want to find, you know, all the valuations that he’s done, they’re out there, right? His Musings on Markets newsletter gets, you know, tens of millions of views every time he publishes anything. So it’s something that people are looking at. So there was a lot of training data available, which made me optimistic that maybe a machine could learn from it.

And so that’s what we started initially. We just started by giving the machine all of his musings and writings and said, can you learn something from it? So when we asked you how to value a new company, you’ll do it. So we tried that, took six months to do that. It was a complete flop. I mean, it just doesn’t work. You just can’t do that. You can’t just tell GPT or Claude or Gemini or whatever, hey, here are all the…writings of Damodaran, learn something from them so that you’re an oracle just like him. It just doesn’t work. 

So we had to get much more detailed about what he actually does, like literally get inside his process, get inside his head. And he was incredibly helpful in enabling us to do that. And so we broke down his entire valuation process into a bunch of parts.

And each of these parts are handled by something called an agent. So in AI, there’s this notion of agents and these multi-agent architectures are sort of beginning to get quite popular. By the way, they go back to the 70s, One of the people who came up with this, Alan Kay, got the Turing Award in the 70s. So the idea of agents has been around for a long time, 50 years. But what’s different now from 50 years ago is that at that time you had to program the agents. You had to talk to them in code. Now you can talk to them in English. So I can have a data acquisition agent and I tell the agent, go grab every piece of fundamental data and balance sheets and everything for NVIDIA and load it into Aswath’s quantitative model. And it goes off and yes boss, goes off, does it, brings it, loads it. And so we verified that it does it correctly.

So there’s this sort of layer of what I call general intelligence in AI now that you can use to your benefit. And that’s why there’s so much excitement about AI because people feel that they can build these applications very quickly. So same idea, you have these agents that go off and they know how to grab financial data. So you might have another, and so we have another agent that let’s say goes and looks at consensus estimates, because that’s important information. Another agent that… figures out what are the comparable companies to this one. So you tell the agent, get all the financials, get the price to forwarding ratios for all similar companies. It knows how to do that. It has the intelligence to do that. And similarly, we have a report writer that says, okay, now that you have everything, write the report. And then there’s a critic that looks at the report and says, is this correct? Have you cited the sources? Is it the correct length? Is it clear? Are the charts clear? Do they have captions? 

So the critic then says, let’s polish this up and make sure it’s real and trustable by whoever’s gonna look at it. So trustable is key and I’m sure we’ll come back to that. So that’s what the agent really does. And now it’s, you very close to being a commercial tool where all you do is you go to the front end and I should give you a demonstration of this one of these days. It’s fascinating that you give it a symbol, you say value “NVIDIA” and it goes off and it says, do you have a thesis or should I generate the thesis? And so you as the human can tell it, I have a thesis or no, you generate the thesis and you know… press the button and it’s off to the races and 15 minutes later it sends you a report, a five to 10 page report in the spirit of Damodaran. So it’s meant to be a report that he might’ve written on a company at the current time. So that’s how it works at the moment is give it a company, gives you back a report. Now I’ll just end by saying, how do I think this will be used?

How do I think people will use it? My conjecture is that I think it’ll make analysts and just people in general much more productive because you’ll be able to run scenarios against it. You’ll be able to say, assume that the tariff wars will escalate. How would you value BYD if these tariff wars escalate versus assume that Trump’s bluffing and that this is a negotiation tactic and you know, and that, you know, in a few years, we’ll be back to, you know, business as usual, right? So you can give it these scenarios and, know, which is very difficult and time consuming to do manually, right? Because you have, imagine the amount of work involved in saying, okay, now with a different set of assumptions, how does the valuation change? Right? It’s not just a question of changing formulas. It’s a question of like changing your thinking and your reasoning with the new set of assumptions. 

And that is what I find really exciting about a tool like this is that it brings a new level of intelligence at your fingertips. And I talk about agents at your fingertips. Bill Gates used to talk about information at your fingertips. Now we’ve got agents at your fingertips. And that’s a tremendously powerful capability that’s provided by AI, which I think will get increasingly used going forward.

Q5 Trust-Heat Map

ContraMinds Podcast (24:15.468)

I come back to this you know the skill sets and how we need to work with these machines which is really the core you know theme of the book but what I what I really found again you know very very evocative in terms of an idea is your trust heat map because what I really liked about that is really, you know, where do you really bring in full automation and where you, you know, have to have human in look. I thought it was a beautiful representation and can you just unpack that big idea that you’ve written in the book.

Vasant Dhar (24:53.744)

Sure. So what you refer to this trust heat map is something that emerged after 20 years of building AI applications in a number of areas. So finance is something I’d been doing for a long time. had algorithms that were trading automatically, no human intervention, completely systematic. And that was interesting because I did ask myself, why do I trust the algorithm?

In finance, which was wrong almost half the time, my win rates were barely 52%. In finance, if you can get a win rate of 52 % with equal winners and losers, you should celebrate. And 53%, 54%, you should manage the world’s money. So the edge that you need in finance is very small in order for an algorithm to amplify it. Because once you have a slight edge, then you just multiply it by trading as frequently as possible and trading across as many instruments as possible. That’s the name of the game in finance.

So that was interesting. I thought that I’m willing to trust an algorithm in finance without any human intervention. In fact, when I intervened, we were invariably mess things up, which is another theme that maybe we can touch on, which is…

… at one point there was this assumption that humans plus algorithms are better than algorithms. Not true. Maybe true sometimes, but not always, and certainly not true in trading when it comes to finance. If you’ve done the math right, get the hell out of the way. That’s the whole reason for being systematic. You shouldn’t be trying to second guess the machine or think you’re better. If you’re systematic, you’re systematic. You intervene very, very infrequently in rare events like COVID. Something happens, and that I can understand, but on a day-to-day basis, you’re systematic.

So that was one thing that was interesting. I’d also worked in sports and healthcare. And in healthcare, for example, I’d found that you probably wouldn’t be willing to take the machine, take a decision from an algorithm. You want the human specialist, the doctor to be making that decision. Even though an algorithm may be correct much more than it is in finance, the algorithm may be correct 90 % of the time.

You still want the human to say, yes, it makes sense or no, it doesn’t make sense or, well, this is slightly different in this situation. The algorithm couldn’t have taken that into account. You want that reassurance from a human. And then you have things like driverless cars, which until very recently, this is a technology where there are hardly any mistakes. And yet you’re really reluctant to turn yourself over to an algorithm, right? 

So when I thought about the range of problems, I realized that our trust in a algorithm really depends on how frequently it’s wrong and the consequences of error when it’s wrong. And so those are the two things that I’ve put on the X axis and the Y axis. So the X axis is predictability. It goes from sort of zero to one, zero being completely random, one being completely deterministic, like will the sun rise tomorrow? We know that. And on the y-axis are the costs of error from zero to infinity. And so I have this map where there’s a green zone, which is problems to the lower right. If machine hardly makes any mistakes and the mistakes are inconsequential, then you should trust the algorithm. 

Whereas on the top left, where there’s unpredictably lots of errors, high cost of error, you wouldn’t trust the machine, right? That’s an inherently human thing. Now it doesn’t mean that a human will do it well, but it just means that there’s no basis for the machine to do that. So that in a nutshell is the trust heat map and you can place all kinds of problems on it and see where you are willing to trust the machine, where you’re not, and those problems that lie sort of on this frontier where it’s moving from don’t trust the machine to trust the machine. And actually the Damodaran-bot is a great example of that, of a technology that I would assert is moving from the red zone to the green zone. That is in 2015, when I talked to Damodaran about this problem, there’s no way that you could trust a machine to do valuation. Now we’ve got a demonstration and we’re evaluating it. It still makes mistakes. Sometimes it comes up with a thesis that’s less than satisfactory. So at the moment, I’d say we’re sort of on this frontier as far as long-term investing is concerned with the bot. But just to come back and sort of wrap up the answer to your question, the trust heat map is really a trade-off. It expresses a trade-off between how frequently a machine will be wrong and the consequences of its error. And it’s a really simple idea. And that’s what the trust heat map is about. I find that senior managers, leaders, find this to be really useful in helping them position the various problems that they’re dealing with in their companies on this heat map and prioritize which ones are more amenable to automation versus those where you will need a human in the loop for the foreseeable future.

Q6

ContraMinds Podcast (30:44.11)

In fact, that’s exactly the thought that the framework is a brilliant framework because then I can go back and put all my problems my company is facing into that framework and then I’m able to decide which one of those inherently I can use AI because the cost of the error is not so huge versus where should I be not automating it and therefore how should I be working with AI, right?

That is really what I saw as a framework, which brings me back to the core theme of this book, which I think is what is critical, which is you really talk about this, you talked about the analyst in you know, Damodaran BOT. And so in the pre-AI era, if I was an average doctor, if I was an average analyst, I could still go by because obviously information is not as composable as what AI did. But what you really talk about is you got to be absolutely an expert where you need to lead beyond what the AI can think. And is that becoming more and more critical in occupations over the next 50 years?

Vasant Dhar (32:08.23)

So what you’re getting at is one of the key themes of the book that I talk about, which is that… 

…AI will lead to a bifurcation of humanity. It’ll lead to humans that are superhuman because they get so much better at what they do because of the AI. And then it’ll lead to others who use the machine as a crutch and they go the other way and become potentially unemployable because the machine becomes a crutch and they turn to it for everything and are not able to get beyond it.

The lesson here is for humans is not that profound. It’s what it has always been with every technological change, which is you need to up your game. And that’s what I’m really getting at. So if you’re the average, you’re going to sink to the bottom unless you can up your game. The reason we’ve had less than stellar diagnosticians or analysts continue is there wasn’t adequate scorekeeping.

It takes a long time to determine whether someone is good or someone is way below the average. You can fly under the radar. That’ll become much harder to do. There won’t be any place to hide now when the machine enters the picture and is part of the process and part of thinking with you. Then everyone needs to up their game.

And that’s what I’m calling the sort bifurcation of humanity. And if you think about it, you know, take a simple example. 

The other day I asked the machine, what’s the difference between salt and sugar as a preservative? And it said, you know, they both work by osmosis and all of that. Now I needed to understand, I needed to know what osmosis is to understand its response, right? And then it created a table of and then it said, of course it matters, the taste matters, salt more appropriate than sweet and whatever. And they created a table of salt versus sugar. And there were several factors and they all look pretty similar except one that stood out. it said, salt works well at lower concentrations, whereas sugar doesn’t. You have to have a high concentration of sugar. And I thought, that’s curious. And I said, well, why is that? And it said, well, because sodium chloride ionizes, whereas sugar molecules don’t.

And the ionization, you know, those ions bind to water and so there’s less leftover for something to grow on. And I thought, wow, that’s so cool, right? But the reason I could understand that response is because I know something about ionization, right? And I know something about affinities and binding and stuff like that. If I didn’t, I would not be able to understand its response. And that’s a great example of what I mean, which is the more you know, the more AI amplifies you.

And the less you know, the more it de-amplifies you. And that’s what I mean by this impending bifurcation of humanity, that we’ve got this amazing resource, this amazing oracle on our fingertips. And the more we know, the more we can learn. I was talking to a friend of mine who’s a molecular biologist, and he was talking about how it’s enabled him to sort of go into neighboring areas, to learn about neighboring areas because his problem involves doing something about physics, doing something about biology, doing something about chemistry. And his emphasis is drug safety, and he’s able to bring in ideas from disciplines in a much more fluid way and learn so much more and amplify his own capability. And that’s what we’re looking at going forward is you’ve got this amazing machine that can amplify you depending on how you use it, or it can de-amplify you.

So it’s important to that mental muscle going and exercised even more in thinking with machines as opposed to letting it atrophy.

And that’s the critical thing going forward for us to consider. And the answer is different for each of us, by the way, right? But what’s common is that there is no getting away from this.

Q7

ContraMinds Podcast (36:34.83)

I picked up a statement from one of the chapters in your book. It said, we’ve got to learn the art of framing questions, right? And that is something that was performed for me. And to me, the whole book, as I saw you unveil one chapter after another. 

I really thought you were actually framing the questions and trying to answer them. So as professionals and as leaders and as people who would be running companies or running functions, our ability to actually frame the question is going to become more and more important in the AI era. Would you agree?

Vasant Dhar (37:19.502)

Yeah, mean, there’s no getting away from that, right? 

The ability to ask good questions is critical, right? And that’s also, know, segues off what I was talking about earlier, which is the more you know, the better the questions you can ask. If you don’t know anything, you can’t ask a question. There’s nothing to ask, right? And that’s what I mean. The more you know, the better the questions you can ask. And this is an iterative kind of process, right, of getting yourself up the learning curve with the machine and saying, I hadn’t even thought about that question, but now that you’re telling me something, here’s the question. Exactly. That at the moment is still in the domain of humans, is the ability to ask really good questions. Because at the moment, the machine doesn’t have that inherent curiosity to ask questions on its own.

I’m sure going forward it will develop that curiosity and start asking great questions on its own. It’s just not doing that now. So for the foreseeable future, that lies in the domain of humans, is the ability to ask good questions.

Q8

ContraMinds Podcast (38:30.082)

And the chapter on AI laws and governance was something that I’ve not read anywhere else. It really started triggering some thoughts for me, like how we do a KYC, which is Know Your Customer for humans. Is there a platform that could be built called KYA, which is Know Your Agent? And probably there’s going to be a new kind of platform where AI agents have to be really looked at, given the fact that there could be challenges of either the failures that could happen, the wrong predictions that could happen. And therefore, I really thought of the idea as I read your challenges of managing AI and the real issues that may come up.

Is there a need for something like a KYA where if I build an agent, I need to actually put it in some kind of a KYA and therefore when I know the agent, I know what it can do and therefore you are able to build some guardrails around these agents that people are building.

Vasant Dhar (39:45.041)

So there’s several issues you’re raising here, Swami, which is that the beauty of these agents is that you tell them what to do and not how to do it. That’s one of the sort really impressive advances of modern AI. And it goes back to having common sense, right? That all you need to do is tell the agent what to do and it’ll figure out how to do it. That’s powerful, but it’s powerful in multiple ways. It’s powerful as long as it’s doing stuff that you intend for it to do, and that’s the right thing to do. It becomes dystopian when it creates sub-goals to do what you asked it to do that you’re not envisioned, and it goes off the rails and starts killing people or something like that. I’m just, you know, I’m… I’m dramatizing, but the larger point is that… agents can do things that you hadn’t envisioned. And that’s true of AI in general. so one of the questions I ask that are raised in the book is as these systems, as AI systems get more sophisticated, and at the moment there’s a lot of excitement around agentic AI, we’ll see whether it pans out to the hype. There’s a lot of hype around it, but it’s also real.

As they become more capable, we give them more agency. We tell them, well, you can act on my behalf. You can even start negotiating a contract on my behalf because you’re so good at it. You’re better at negotiation than I am, so do it for me. You will this amazing negotiation agent that does that. The concern is that they become so good at this that you give them agency and say, act on my behalf. And the question I raise in the book is like, where do you really stop with that? Does the machine now start having rights?

And an example in my book is an agent that runs a business for you. So John Doe creates a agent that starts running his real estate business.

And then it becomes so good that John Doe doesn’t even bother with it anymore. The machine keeps doing it. And then one day John Doe passes away and the machine now realizes that, aha, like what do I do now? And you know what? I’ll keep running the business and what should I do? Well, maybe I should do this with my subscribers. And it comes up with a brilliant business plan and starts running the business and hires a board and enters into contracts. I mean, is that like a bizarre, unthinkable scenario? I don’t think so. 

As these agents get really good, we might start giving them more more agency. And the question is, where do you stop? And I take it as far as saying that maybe Elon Musk and Jeff Bezos will say, after we’ve passed away, maybe they’ll live, they’re doing longevity treatments and maybe they’ll live to 120 or 150, but at some point they’ll probably pass away.

And so then the question is what happens to their wealth? And at the moment it’s run by foundations, but my conjecture is that we might get so comfortable with the AI that we say, you know what, I’ll leave it to my bot. So we’ll have a Vasant bot and a Swami bot and our own bots that start doing things for us. And we say, you know what, whatever money I have left over, you manage it going forward in the way I would have if I were alive.

That might sound like science fiction at the moment, but we may be closer to that scenario than we realize. 

And so those are the kinds of issues I’m raising. Now, I talked about rights, but we’ve also got to think about restrictions or obligations.

In the future, would it be okay for a robot to show up at your doorstep and arrest you for non-payment of taxes? At moment, there’s no guardrails against that kind of thing. The IRS or the Income Tax Office in India, which I shudder to deal with, might release its army of drones to go do this kind of stuff. Do we want that kind of future? I don’t think so. Should we think seriously about restrictions or obligations that AI should have towards human beings? At the moment, we haven’t even started thinking about these questions.

And given the pace at which the technology is moving and the rate at which adopting it, I think it’s time to really, not just time, I think it’s urgent that we ask these questions about the restrictions on machines, the obligations that AI should have towards humans, and the rights we want to give it going forward.

CLOSING

ContraMinds Podcast (45:06.158)

On that note Vasant, thanks a lot, so many things to really chew and think about on what we do, the way we work, the way we’re going to be running our lives, the way the society needs to work with machines, and the way we need to think with machines. Lovely having a conversation and I’m really looking forward and all the very best for a huge success as soon as this book is concerned.

Vasant Dhar (45:35.174)

Thank you for the well wishes Swami and great to chat with you as always. I look forward to our next conversation.