HAL Transcript: Rishov Chatterjee on Who Answers When an AI Agent Takes Patient Records

This is the full transcript of the HAL: Who Signed Off On This? episode with Rishov Chatterjee, AI engineer at Curology and former Director of Data Science and AI at Integra Connect. The show is on Spotify and YouTube. For the highlights, read Who Is Responsible When an AI Agent Takes Patient Records? A Healthcare AI Builder on BAAs, Biometrics and Cost.

The transcript follows the published episode. It is lightly edited for readability: repeated words, false starts and short acknowledgements are removed, and names that the transcription software misheard are corrected. Timestamps mark where each turn begins in the episode.

Transcript

Chris Sowa (00:00): Every AI model that touches a patient record was built by somebody. Somebody tested it, somebody said it was ready, and somebody signed off on it. Most of the people on this show are the ones who sign. Today’s guest is one of those people who makes the key decisions of how AI can operate in large enterprise environments, including medical and public safety environments. And the stakes moved this past week. The Australian Prime Minister told his country that an AI agent had walked into their Medicare portal back in July and pulled out data. The company that built it noticed in August. Australia was told in September, I can tell you what, nobody signed off on that. This show is HAL: Who Signed Off on This? Short conversations with people who have to sign for AI, how they decide what to approve, where they insist a human still looks, and what they say. When they need to see more before they trust a machine with more. HAL is the computer from 2001, the AI that went rogue. This show is about the people who make sure that never happens on their watch. I’m Chris Sowa, co-founder of RedactSure, an AI controls, governance, and privacy company, and I’m your host. My guest today is Rishov Chatterjee. He spent years in this environment, both in the area of public safety as well as in the medical profession. And he has been everything from a principal AI scientist to in charge of the whole AI operation. Rishov, welcome to the show. I’d like to be I’d like to ask you just quick give us a little bit of an introduction to your background and what you’re focused on now.

Rishov Chatterjee (01:52): Absolutely. So I’m Rishov. And I am an AI engineer and also a data scientist, currently working in the intersection of e-commerce and personalized skincare, with a brand a very popular brand Curology. And I’ve been transitioning from the overall like healthcare space with a primary focus on personalized oncology as well as value-based care. So I’ve worked with insurance companies. I’ve looked at the hospital side of things during my tenure at City of Hope, and as well as working with a lot of private practices as well as large hospital systems as partners with Integra Connect. And I’ve also worked with insurance companies as well, such as like Anthem. And also United Health Group and a couple others. So a lot of them as partners with Integra Connect when I was a director and a principal applied scientist there. So in terms of what I’m focusing on now, I’m interested in a couple of different things from agent point of from agentic AI’s perspective, which is not only trying to solve the key problems that businesses want, but trying to make sure that they’re solved in a safe manner. Cause as Chris pointed out, right, that a lot of these agents can sometimes be hard to control. So that’s why it’s really important to kind of steer the ship away from them back to the human in the loop to make sure that things go the way that we expect them to. But yeah, happy to be here.

Chris Sowa (03:22): Thank you. We’re excited to get into this conversation. And I think that because you’ve not just been doing this for a little while, you’ve been doing this for a long time, you’ve seen the evolution of AI and these the ch the difference with LLM models. And so my question to you is, maybe going back to when you were first getting started with similar medical models to the environment we’re at today. What are the differences if there are any in terms of how people need to think about patient data and what they need to sign off on in that process and the controls in place? How is it different in an AI agentic environment, if at all, Rishov?

Rishov Chatterjee (04:00): Absolutely. That’s a fantastic question. So starting with the earlier stage things, let’s say in the past like decade or so, a lot of times the adoption for AI and healthcare usually started with machine learning or predictive models. So these predictive models would take some input data that you can get from either an EHR or you can get from, let’s say, patient management systems, things like that. You would essentially be able to get this information and find patterns in that data to be able to come up with decision making that could be discrete, such as saying, like, hey, will this person have breast cancer or not? Or will this person have a tumor in the next like two, three years? Versus let’s say more continuous outcomes, such as you know, what’s like the potent the propensity for this person to. To need a certain treatment or before they become like a high-risk patient, right, for a certain condition or things like that. So that’s what we usually started with. The problem with these models is that they require a lot of training data. And because patient dynamics can change in a hospital, even like month to month, there could be a chance that whatever data we trained on becomes stale really quickly. So a lot of times that these models they tend to drift and they become less reliable over time. And so that’s really what happens on the development layer. And then when you try to integrate it within the existing systems that doctors use or insurance companies use, there needs to be a very easy way to access with their system. So that’s where like the integration piece becomes a another challenge because you want to make sure you do it in a way where everything is like secure and not necessarily completely disparate from how they run their operations. And that’s always been a huge bottleneck for adoption of solutions for a while. How it changes in today’s context, I would say, is now we’re making less decisions from single models. And we’re now we’re focusing on what we call as frontier intelligence, which means you have one model that is pretty much learned a lot of information about the world through lots of data that comes around publicly on the internet. And the idea is that this information should be enough to provide inference that tells you a lot about decision making about diseases, diagnosis, et cetera. And these are easier to integrate because they’re essentially API calls. But the problem is that you still have to provide a lot of the sensitive context as an input to these models. So a lot of times there’s concerns about. How much of that data gets retained? Because now all of a sudden you might be sharing patient level information with third party providers. So that’s why there’s a lot of push towards extra HIPAA compliance and SOC2 compliance to make sure that we’re de-identifying the ability for the patient’s information to not be seen in that retention policy.

Chris Sowa (06:58): In interesting. So are you mainly focused on you know within the environments that you’ve worked on in the past, kind of API driven AI and what versus the some of the agents that actually navigate different screens, like actually would you know go out and try to d capture information out of an EHR system, for example, on a different screen. You know, if it’s a Citrix environment with you know maybe OCR it or whatever.

Rishov Chatterjee (07:26): So I have interacted with both types of systems. And one of the things that I would say is that even with a lot of like modern EHR systems these days, they also use APIs, right? So for example, when we look at something like Epic, right? So when I was at City of Hope, I spent a lot of time looking at Epic systems and one of the aspects around Epic is that they also have their own API for being able to interact with other systems, right? So they had they and one of the things that they had to do was they had to make sure that they can make their resources for patients and medications and everything, encounters. They wanted all this stuff to essentially be made available in a very interoperable way with other systems. Right. So that’s why we have this organization called health level seven right which created the HL7 messaging standard and then now that has become much better with API compliance through something called FIRE, right, or FHIR, which is fast healthcare interoperability resources. So by being able to get all of those API responses within that grain, now it’s much easier to integrate with other systems. And that pretty much ties to both EHRs and also various applications.

Chris Sowa (08:39): Well, that’s great. And now one of the things that I focus on is kind of time to outcome. Right. And a lot of times, many businesses, that’s time to how much money we can make or save using something. In your world, it’s even bigger than that, right? It’s like delays could lead to maybe not somebody not getting the care they need and potentially having a bad health outcome and maybe even dying. So, you know, the time to action is very high. At the same time, you have to be really safe with the data, which as you mentioned, slows things down. Can you give us a sense on a project type? You don’t if you can disguise it a little if you need to. I don’t want to get burning out any secrets, but how long does it take to get through some of the due diligence in terms of make making sure the data is safe and protecting those clients’ PHI?

Rishov Chatterjee (09:35): That’s a very good question. So I’ll start with saying, let’s say if you’re getting all this information from an EHR, that’s where this will become a big concern. Right. Because a lot of times we would use some information retrieval type technologies, such as named entity recognition. And these days with agents, they’re able to also add tools that allow you to extract text from scanned pages like PDFs, et cetera. So with all the information that comes out of it. Now the idea is you’re trying to take all this information about the patient, their diagnosis, when they were admitted, when they had their first treatment, and we’re essentially modeling a patient’s journey. So this patient’s journey now can be in it set up in different ways, where we can put them into like a graph, we can put them into what you said is a time to event model, which we usually refer to as like survival analysis. So with any of these types of setups, the key thing we have to remember is instead of trying to directly identify the patient based on some of the attributes, we have to think of how do we constitute PHI to begin with. So whenever we think about PHI, there’s really two types of standards we think about. One of them is what we call as expert determination, where you have someone that really has tons of years of experience with. Understanding what makes information identifiable. And that person does an audit that checks to see, okay, from the data that you have, is there anything that could potentially identify the patient? Right. So that would be one perspective. The other perspective would be to use something called the safe harbor method, right? Where we essentially say you have 17 different elements which constitute identifiable information for a patient. So based on that. The idea is that you would use an agent or another model to essentially identify from what you have extracted, does it fall under those 17 attributes? If it does, then that’s going to be your signal that you have to fix it, right? You have to clean it up, scrub it. And once that’s done, then you can use the remaining information for modeling. And usually what ends up happening when you do this is A lot of the signals that we use for, let’s say, time to event modeling, it’s less likely to be affected by the identifiable information. Where the issue comes about more is the scarcity of data if you’re dealing with, let’s say, hospitals that don’t have a lot of patients, such as like private practices.

Chris Sowa (12:04): No, that’s great insight. Now there’s a question I usually ask on this show, and I’m gonna couch it a little differently because of your expertise. I usually ask somebody, you know, if somebody were have transferred out a hundred thousand dollars out of your organization, who would be responsible and who would be the person on the hook for that? In your case, I’m gonna go a different direction. I’m gonna say, you know, if somebody s lost 10,000 patient records or to or s or an AI agent took 10,000 patient records and took them out of the organization, who in your organization would be responsible for that?

Rishov Chatterjee (12:41): So that’s a very good question. So let’s say that if the organization happens to be a business associate, right? If they’re a business associate with, let’s say, a hospital or an insurance company, then technically the extraction of those medical records would be a violation of HIPAA, right? So that’s why it’s like one thing that’s really important, right, about HIPAA is that it only applies to covered entities, right? So the covered entities would be. Hospitals, insurance companies, and business associates. So assuming that the organization that I work for is a business associate of a hospital or an insurance company, then usually what ends up happening is the risk is given to the personnel that owns the data. So in this case, like if the data is an EHR related information, the EHR is probably coming from the hospital, which is doing a collaboration with the business associate. So that means technically the medical legal risk would be in violation of the hospital. But at the same time, because the issue was warranted by the business associate, the first thing that would happen is we would have to follow the 30-day notification rule, right? Which is to say that, hey, we violated this. So now we have to make sure that we let the media know and the press, and we also have to let the hospital know that this happened. And the blame will actually end up going to the hospital because it’s their information that the EHR was originating from.

Chris Sowa (14:12): So I have we haven’t yet shared it with our listeners, but there was a guest that we I had on last week, and he his point, he’s actually running one of these medical centers. He said, well, sometimes I hear from different companies that they want to sell us back our data. And you know, because they’re liable for the data, as you pointed out, his contention is that they fully own the data already, and that anybody else that uses that data. Really ought to come to them and get permission and pay for it. I don’t think he used the word pay for it. I probably added that part. So give me a little on that. But you get the idea. What do you think about that? If you’re going to hold me accountable or my AI agents, you know, AI just break in and steals data and they throw it out, should I should they be the ones that get credit for that data? Because the data is an asset.

Rishov Chatterjee (15:00): True. So it really depends on a couple of factors, right? Because even though the data is originally coming from the healthcare provider, the most important aspect is what did we actually do with that data? Did we keep it the same? Did we transform it in any way? Right? Because usually what happens with the business associate’s point of view is that if they are going to sell the data as an asset, then they can’t just sell whatever they’ve already extracted from the healthcare organization. They have to do a lot of transformations in their pipelines to say, instead of looking at this data set from the perspective of here’s like X patient, Y patient, it’s really more about saying, here’s all the information we have about how these medications can lead to adverse events if they’re used together or things like that. So Conceptually, the data would have to change and it would have to be completely deidentified, right? In order for that to become something that is very different from what we originally started with. But technically, if you are selling it and you have a contract as a business associate, you technically are bound, right, by law to be able to chat with the healthcare organization to get permission to sell. So that’s essentially what would happen. But in terms of let’s say agent based liability with creating these data sets or let’s say dispersing it to other systems, technically it would be in violation of the contracts that the business the BAA agreement that’s created between the business associate and the organization.

Chris Sowa (16:35): I always say BA agreements are critical and we need to have them. But if you have to actually use them, something was bad. Right? Some something broke in the process. Okay, so just you know, something you mentioned I think is relevant. I had a conversation with a data engineer in Europe a couple weeks ago, and he talked about you know information they were getting from call centers, so telemedicine information, which has obviously become a lot more common in the last s decade or so. And that swapping out account names and n things didn’t cut it because of the biometric voice print. So I wanted to kind of get you because you mentioned the identifiers. How do you deal with things like that? Let’s say the other data, the contextual data, which could potentially also be a digital fingerprint, if you will, quite literally in this case with a voice.

Rishov Chatterjee (17:27): This is a fascinating question. I’m so glad we’re talking about this. Because I think one of the things that you mentioned is that speaker was from Europe, right? So what’s interesting about Europe is that in the European Union, they have this law called GDPR, right? So the global data protection regulation. And so what’s interesting about GDPR is that, they look at data overall in a much more comprehensive way, right? Rather than just saying that we have protected health identifiers, which are pretty much those 17 attributes that we were discussing with the safe harbor method. So that means that there’s actually no clause under HIPAA that talks about what do we do in the case of biometrics. And this is something that I have also been concerned about for a while because data is becoming more and more multimodal as we start to see progression with agent development. And so as you mentioned, the voice of the agent is absolutely an identifiable signal. It’s just that we if you actually didn’t de-identify that, it wouldn’t necessarily be considered as a HIPAA violation. Right. But if you do make your solution available around the globe, then you would have to adhere to different laws. Like in your in Europe it you would have you would fail the GDPR bar if you don’t de identify it there. But then there’s also like another example, which is when it comes to let’s say in countries like China. Right. So China has this framework called PIPL, in which they actually have a specific clause which is focused on biometrics. And they say that if you do not have these biometrics de-identified, then you absolutely cannot pass the PIPL bar. And so there’s a lot of like fragmentation around the globe around these regulations, where some are very like I would say loosely oriented towards the standard identifiers and then there are many that are now more progressive and actually thinking about saying that, data is a lot more multimodal and universal. So we have to go beyond just that typical scope with text, right? Which is what we usually get from these EHRs. But what I would say is that I’m also doing some research currently at Curology right now. And this is also part of the talk that I gave at DataCon LA recently, where I’m looking at privacy preservation for biometrics from visuals. So for example, when you have images of a patient, let’s say you have a selfie and you’re trying to look at their, let’s say their acne or things like that, you can still determine who the person is. So rather than just having the agent only look at that image, what if we can already use other models to say, hey, the acne is actually on the left cheek of the person? So if it’s on the left cheek, then why not just create a segment where you’re only focusing on the cheek? So that way you can’t determine who it belongs to anymore. So these simple things like this, right, could be enough to be able to pass that bar. With voice, it’s a little bit different because you have to look at things like fingerprinting and a couple of other aspects to ensure that it can’t that the voice can’t be found in a certain like public database or things like that. But definitely there are lots of ways to deal with that, right? Where you can potentially go into transcription, right? And when you go into transcription, then you delete the audio recording as part of meeting retention bar. So there’s lot of different ways you can go about solving.

Chris Sowa (20:47): No great. And you mentioned the LA Datacon. I’m just curious, what’s the vibe there? And there’s all there’s a lot of buzz about rogue AIs. Almost every day, it seems like to me at least, there’s a story on this. That you know, is they talking anything about that? What are the what are less if you had to pick kind of the most two common most heard conversations that you heard, what w what will those conversations have been?

Rishov Chatterjee (21:16): So one of the major conversations I’ve seen is the rapid development of agentic AI replacing the way that we do analytics. Now that’s one thing I’ve seen has become very commonplace now, where a lot of times you would have a lot of experiments that would be done by data scientists or data analysts, but now a lot of those experiments are now being automated and even reasoned by these agents, right? And so a lot of times, the personal judgment around which approaches are best and how to evaluate those approaches, that’s starting to be diminished. And that kind of shows like when you look at some of their talks, right? Like they’re not being extremely exhaustive with everything that’s available in that skill set. But then the other conversation that I would say is also quite interesting is actually related to a specific talk that happened. And the talk that occurred was related to these prediction markets such as Kalshi or Polymarket. And what we’re noticing there is it’s actually easy to rig the system with some of these markets such as sports, like when you go in the polymarket. And what’s interesting is that because it focuses on anonymity, it’s really difficult to find culprits that are easily able to cheat the system. And so even though we have algorithms that can help you to identify what are the wallet transactions that are leading to that, no one’s taking any action about it. So that was the other major thing that happened there. But in terms of rogue agents, didn’t really hear that much about it at Datacon.

Chris Sowa (22:50): So if you in your world that business is business as usual, nobody’s worried at this point. That’s really interesting. Okay, now from a modeling perspective, you know, you’re you talked about how people are using LLMs and how the analysis has changed, but I’m hearing from a lot of people in technology that their job has changed too. What I mean by that is the way that they actually operationally go about things. So, I need to set up a new environment, a new data environment, on AWS, an S3 environment. So I’m gonna use an AI agent to set that up. Or, you know, I need to create a couple of different programs to do analysis or whatever, and I actually have a you know three or four agents writing that code for me and I’m supervising them. Have you seen this yourself in your own work with your colleagues? What’s happening in that area, Rishov?

Rishov Chatterjee (23:50): So I’ve seen this at different levels. And what I would say it comes down to is familiarity with those different areas of technology, right? So let’s say if you have some colleagues that are very good at analytics, but they haven’t really been involved in cloud environments, then they wouldn’t know how to spin up those things. So what happens is that these agents are now providing the accessibility in a much simpler way. Right, where they’re saying that yeah, we can easily connect to AWS using some MCP servers or skills right that these agents are pre-configured with to be able to do all these things. But of course, one thing that everyone also neglects to mention is the hidden costs of doing this, right? There’s a lot of expenditure that goes into every tool that gets called and every service that has a response that interacts with the agent. So all of these expenses, even though it might seem insignificant for one person doing it, when the whole organization starts doing it exponentially increases. And this has led to significant budget issues for a lot of organizations where they’re even starting to rethink like how they do planning for their financial year, where they even have to consider controlling budgets to top tier models to only certain dollar amounts as cap limits per month or even on a quarterly basis. And a lot of times it even affects like the number of headcounts that they have, right? To be able to maintain all these expend expenditures.

Chris Sowa (25:21): No, I think you’re right. So it’s a it’s amazing that you know, on the one hand, you can have you know four eight four agents or eight agents writing code for you, but there is a corresponding expense to that. And you know, touched on kind of frontier models, you know, there’s also a lot of low lower cost models and some pretty good Chinese models that are out there as well. I know you know, so we within my organization, we can operate with any of those models. And so, you know, what’s interesting to see because we do work also in confidential data spaces, we wouldn’t recommend a school system, for example, be connected to a Chinese model. Now, I know there’s lots of arguments for and against that. Obviously, there’s a is a major cost because you’re talking about cents versus dollars for a task. What are your thoughts on that? So I want to put it in three categories. Let’s call it the frontier models, which I know there’s only some stuff that just they can do at this point, but the let’s call it the dumber models are getting smarter and smarter. There’s the open models, and then there’s the proprietary foreign models in specifically. So those, you know, I and there’s probably a lot more categories, but just for sake of the show, those three categories. How do you th how are you what are your thoughts on that? Are there some models you say? For corporate reasons or public relations reasons I don’t use these or because of real security risks I don’t use these. Yeah.

Rishov Chatterjee (26:50): A couple of things that come to mind. For one, when we think about a lot of these open source models, we also have to think about the number of parameters they have. Because usually the more parameters they have, the more likely they are capable of being able to do tasks with minimal iterations, right? So let’s say if you are downloading these models on a local machine or even on an enterprise machine, which allows you to download it if they do have the permissions to do so, then one of the things that comes up is do I even have the amount of VRAM and the GPU that’s necessary to be able to even use that model to begin with? Right. So that’s a very important point because w whenever you are trying to use these models, yeah, we might be able to potentially run them, but if they stream the response really slowly because we just barely met that hardware requirement, then I can imagine from a user experience point of view, people will be frustrated when they try to use it over and over again just to save costs, right? They’ll say, I could have probably gotten this whole thing done with maybe one or two calls of Claude or OpenAI, right? That’s probably the argument that they’ll make. And so that’s one problem. And then the other issue is that let’s say that if we’re not allowed to delegate these on the local machines, but instead we have to have an actual served deployment, very similar to how we interact with APIs like OpenAI or Claude, then all of a sudden we’re paying for the infrastructure, right? We have to pay for the scaling and all those things. And all of those expenses can actually equate or actually be very similar to what you see with. The frontier intelligence. Cause yeah, we’re not pushing it to their servers, but we’re now utilizing a lot more cloud compute. And when you horizontally scale that across the organization, it can become expensive very quickly. So that’s one issue I would say with the open source when it comes to accessibility across the organization. But if you’re delegating it to let’s say certain portions of the organization that say, hey, this group of team members all have 128 gigabytes of GPU RAM, then absolutely, you can download these local models and have as much fun as you want with it, save a lot of cost. But again, you’re still paying for all those machines that have a hundred twenty eight gigs of RAM. So there are trade offs, right?

Chris Sowa (29:10): It’s never really a perfect you’re touching on something I think is important too, and that is this idea that who gets to use what models and how much do they get. So does it almost becoming a sign of status that says, you know, I get to use the frontier Fable models and you know, these other groups only get to use, you know, whatever, Kimi, the Chinese model, because what they work on is less sensitive and less important. Your thoughts on that. You’re just and this is just an informal thing, rather than your organization, your colleagues, the larger group of people that you’re you’ve been talking.

Rishov Chatterjee (29:47): So that’s an excellent point, right? So regarding status, I would say the it could potentially be indirectly seen as status for sure. But what’s interesting is that the status framework is being decided by the organization, right? Because the organization is the one in charge of all these departments, right? So they’re thinking like whose work is actually important versus who’s not. And that’s the part where there’s a gamble. Right, because we don’t necessarily know how hands-on the people at the top are in executive leadership when it comes to knowing what everything how everything is dispersing like throughout the entire organization. So that’s why I would say the status could be attributed to those who are deemed lucky, if we will, for lack of a better term. But definitely I would say when it comes to the access to models, what would always be best is if there are policies or guidelines that are put in place that say, let’s actually educate others on how to use the lower models in a more effective way. Because if you actually teach others how to do the prompting and the strategy, maybe even do hands-on workshops with those other teams, then even if people do have access to higher frontier intelligence, I think the jealousy would be less if everyone is able to get their work done with the lower. And plus they would probably be seen more kindly by the organization because their overall expenditure is less than the people using the ones with higher status.

Chris Sowa (31:16): So you’ve worked with both insurers and folks providing the medical care. There was an article a few weeks back in the new York Times about this AI war between the insurers and medical providers increasing costs, and we just talked about costs. I based on some of the numbers I’ve seen, at least initially the insurers seem to be winning insofar as the number of denials, if you will, for coverage are up. Compared to previous years. And that doesn’t mean they don’t get paid, but money delayed is money lost because obviously you know money has value and with interest rates rising, you know, that becomes even more true. So what are your thoughts in terms, and you actually are also run an a I didn’t I thought you still run the a nonprofit as well ChanR. So what happens to the organizations that can’t afford the AI models to do battle, if you will, to compete in this environment. Have you seen that in any of the nonprofits you’ve been talking to or so even some of the health deliverers that are struggling with this new reality?

Rishov Chatterjee (32:25): Absolutely. Yeah. The this is a very fascinating point that you brought up. So when it comes to some of these issues with like denials, right? And also the capability for these nonprofits to be able to overturn denials, right, by creating appeals and things like that, the capabilities become less for the nonprofits for sure. Because they have less access to being able to see the documentation that’s coming from. The insurance company analyzing that and being able to draft evidence to overturn those denials to begin with. Right. And I and I’ve seen this firsthand you know, working at Integra, where I actually had to build a solution like this for doing appeal generation for a revenue cycle management group where we were focused on where we were focused on emergency medical care. Right. So let’s say if you had someone that’s on let’s say basic life support or even advanced life support, they essentially would have to be responsible for their bill because the insurance company felt that the place that the ambulance took them to, meaning the facility where they were receiving care, the facility that it was traveled to apparently was not like the most nearest appropriate facility. Like that’s what they would deem as the argument. But then they would never actually give you the justification for that, right? They would never show you on a map, hey, here are all the facilities and here’s why that’s the nearest one.

Chris Sowa (33:49): And if an AI made the decision, it’s a black box in some cases and they might not even know, right?

Rishov Chatterjee (33:55): So when you have all these situations, a lot of times the best way to solve it is just provide the evidence yourself. And then that way, they’ll probably be less likely to refute it. But as you mentioned, it does cost money because you have to use APIs for figuring out that geographical structure. You also have to use a lot of LLM costs to be able to draft the appeal and get the evidence to line up with your points. And so all these things at scale, when you’re doing this for hundreds of thousands of denials, that’s really when it becomes very expensive. And so that’s why I would imagine that for these nonprofits, right, unless they have really generous donations, such as City of Hope, for example, like you would probably have to rely on local models and seeing what’s the best you could do with local models at a short at a smaller scale. And then from there you have to seek partnerships with companies that do have access to a lot more infrastructure to be able to do this. And you maybe even get funded by some of these like companies that are in the middle for so for example like with emergency medical care let’s say you have a partnership with like global medical response or even AMR then that would help you a lot to be able to get a joint funding to be able to solve those problems because that would also provide value to those companies so then they can also get their EMTs and everyone paid as well.

Chris Sowa (35:20): I think that’s the right answer. It’s gonna be, I an i in a an evolving battle. We’ll be watching that one. So we’re coming up to the close to the end of the show, but before we do, you’ve been working on something really cool at Curology, and I just wanna give the viewers a j a chance to hear a little bit more about it. So if you don’t mind, just share a little bit more about what you guys are up to and what’s the vision and how you’re leveraging AI to solve great things.

Rishov Chatterjee (35:46): Absolutely. So Curology is a personalized skincare brand, but they also sell various skincare products over the counter as well. We recently got into a partnership with Walmart and we’re also moving our e-commerce into Shopify. So one of the major things that we’re doing with AI is we’re essentially trying to target a particular segment of customers, which are usually very young folks that are less likely to go to a website and have consultations to moving more towards like a mobile app where this mobile app is where they can go to see pro progression as to you know how their skin quality is improving based on treatments that they’re getting with consultations with their providers. So this mobile app is where the AI is really being embedded. And so what I’m doing from a research point of view is to really solve three problems. So the first problem is to say, what are the various conditions that need the treatment? So there’s a diagnosis angle, right? So it’s like you might have acne, you might have dark spots, rosacea, and a couple of other conditions. And then the second aspect would be after knowing what these conditions are, we need to know how severe it is. So the severity is important. So in that way, we can decide which medications or which custom solutions from the Rx point of view do we want to prescribe from the dermatologist’s perspective, since we actually have a group of a hundred plus dermatologists on standby, right? So we want to make sure we make it easy for them to prescribe. And then the third aspect is the progression, right? To be able to say, is this solution leading to an improvement or is it making things worse? Right. So based on these things. We would have this shared in agent intelligence, which looks at all these aspects to say, okay, now we need to change strategies to make sure that we can improve our potential customer lifetime value with this patient or customer. So that would be one of the big things. But then I’m also looking at privacy preservation. Cause as you can imagine, when you’re trying to put these types of solutions into places like Washington or California. We have to follow certain state regulations, right? Like this in California, I think the biggest one would be CCPA. So in order to be like CCPA compliant, it’s really important that we don’t store the personal information of any of these patients. And so that’s why even when it comes to patient selfies, which as we know that if we look at global regulations, it wouldn’t pass the biometric bar if we did that. So one of the things that I’m doing with the agents is to say, can we get the same correct predictions at lower resolution where it’s like almost impossible to identify the patient? So lower res so low resolution is one angle. And then the other angle is the transformation that was done to make it lower resolution. Can we make it difficult for that to be reversed? Because then you can have a hijacker come in and actually get the original resolution, and that would be a problem. So We want to make sure that there’s a way to reverse that. And so that’s something that I was also talking about in the conference recently. That there actually are some methods that we can use to make it difficult to reverse the resolution. And also then the next thing I would say is segmentation, right? So if you actually don’t need to see the entire selfie, but you just have to focus on maybe the forehead or the mouth, right, or just anything else, then that’s really all we would need, right? So that way we’re already making it really difficult, right, to know who the patient is because you’re just looking at a feature. Maybe there could be some people that have very good intuition to say that, hey, I know whose mouth that is, but I don’t think that can be done at scale. So I think we’re already safer there.

Chris Sowa (39:27): I don’t think that would be a very well known skill, right? If people can just identify for their mouth. All right, well, thank you very much, Rishov. That was really a great discussion. That’s the show. So if you sign off for AI decisions in your organization and you have an answer to who signed off on this, and or you’re still working one out, we’d love to hear it. We’ll have a link in the notes. Thank you very much. Until next time. This is Chris Sowa with RedactSure.

About the show

HAL: Who Signed Off On This? is a show of short conversations with the people who have to sign for AI: how they decide what to approve, where they insist a human still looks, and what they would need to see before they trusted a machine with more. HAL is the computer from 2001: A Space Odyssey. The show is about the people who make sure that never happens on their watch.

About the host

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.