Press Release

Surgery is the most complex test for AI. That’s why the biggest gains are here.

AI earns its place in the operating room by removing the grind of logistics and making people better at their jobs. Here’s what that looks like in an Proximie OR built on NVIDIA Open Models.  

By Richard Carter, Proximie CTO  

One of the hardest things about MedTech is deploying it to solve a real problem, not just support human judgment with a digital proxy in the name of efficiency. This is especially true in healthcare, where patients are humans at their most vulnerable place. This is particularly true for surgery, where patients are often unconscious and unable to consent to the choices made on their behalf.  

It is the most high-stakes, highly regulated environment in healthcare, making it one of the most AI-resistant domains conceivable. And yet surgery, and procedural medicine more generally, is one of the most exciting frontiers in the thoughtful application of AI.  

Such an unforgiving environment demands that AI adds value from the outset, which makes surgery the exemplar of how AI can make people better at their jobs. Not by substituting human judgment and skill, but by removing the manual grinding of logistics and communication gears so that human judgment and skill is all that remains.  

Most of what makes a day in the OR unpredictable happens around the procedure, not in it.  

Procedural medicine is, as its name would suggest, highly process oriented. This is true both within the procedure, where scheduled procedures follow a set sequence of steps, and around the procedure, as scheduling, clean up and next-case preparation tasks take place. Only around 10 to 15% of the variation in OR time is clinical; the rest is the workflow around it. And up to now, that gear grinding has been entirely human-driven because only humans could fully understand the context and apply the judgment to work out what needs to happen next: whether that’s bringing people or equipment into a room to perform a task, finding the information for a surgeon mid-procedure, or capturing and storing information for medical records.  

And a record typed up after the fact is only as good as someone’s memory of a busy day. Any AI trained on that record inherits every gap, and every error.  

Today, Proximie already puts more patients through the same ORs.  

Using NVIDIA Cosmos 3, a world foundation model, alongside Proximie’s real-time audio, video, and device signals platform is transforming this space. Agentic notifications and ambient voice technology already automate core workflow functions: the next patient is called when the room is ready, live alerts flag when a list is running late or capacity opens up and operative notes are captured by voice.  

Across health systems we have seen case volumes increase by 8.2%, turnover speed up by 45%, and 26% more first-case-on-time starts. Behind those numbers are more patients treated in the same rooms with the same team, who spend less time typing, chasing and checking.  

But Proximie’s ambition is much greater.  

Proximie sees and understands the procedure, the room and every connected device at once.  

Proximie is used in more than 700 operating and procedural rooms, and its platform is unique in capturing both the procedure itself through intraoperative views, as well as the whole room, through wide-field views that give operational context.  

By combining what the surgeon sees with a view of the room and signals from the devices themselves, Proximie’s models, using Cosmos 3, have a far richer set of data sources to learn from. That richness cannot be downloaded with a model. It has to be gathered, case by case and room by room, over years of real procedures. And this is exactly what Proximie has spent a decade doing.  

The quality – and more importantly, the diversity – of these data sources means the model can “see and understand” far more than any one person in the room, no matter where they’re positioned or what their role.  

The surgeon is focused on the procedure, the scrub nurse on the instruments, the coordinator on the schedule; each sees part of the picture, and the gaps get filled with calls, messages and logging information by hand. That is time pulled away from the patient. Proximie brings the clinical and the operational together in one place, so the team can spend that time with the patient instead.  

NVIDIA’s open models get better and smaller when they learn from surgical reality.

NVIDIA’s open models are built to be adapted. What they become depends on the data they learn from, and that is where a decade of recorded surgical reality counts.  

With UC San Diego Health, Proximie is co-developing a multimodal foundation model (one large model that learns from several kinds of data at once) for perioperative intelligence using NVIDIA open models, and it is one of the first partners in NVIDIA’s Open-H-Hospital, an open initiative to bring real hospital operations together with staged environments and simulation to train the next generation of hospital AI and robotics.  

The results are already strong.  

Zero-shot, meaning before it had seen any of Proximie’s data, NVIDIA’s model identified  key equipment in the OR with an F1 score well over 80%. Turning it on Proximie’s own data with LoRA fine-tuning framework improved the performance further. The improvement wasn’t just an increase in accuracy, but also a substantial reduction in model size.   A smaller model means several models, each handling a different camera or data feed, can run at the same time on the one computer in the room. That’s important because that same computer  has other work to do too, not least protecting privacy.  

Privacy is protected before any data leaves the room.  

Whenever video leaves the room, privacy must be protected, usually by obscuring identifying areas such as faces, exposed skin (which may have tattoos or birthmarks) and text on screens or whiteboards can all identify someone. Proximie’s models do this in the room, blurring the entire image at source before the video goes anywhere, an approach it has patented.  

More than this, the blurring must remain in the surgeon’s control so that, for example, a virtual surgical training session on facial or dental surgery still shows trainees what they need to see. This demands processing power not just to apply the blurring, but to keep the surgeon in control through an intuitive user interface and real-time feedback.  

Real surgery is the foundation. Synthetic data fills the gaps.  

The foundation is in place, and the work ahead now is to get a fleet of models, each suited to a different view or function, working together. While some views are fixed, and always have the same perspective, others can vary hugely when, for example, cameras in the OR lights or Proximie’s repositionable wireless cameras are used to capture specific scenes. More challenging still is when video is across different rooms, as can be the case in cath labs and interventional radiology.  

Proximie is working to combine fleets of models running both locally in the OR (so the data never leaves the suite of rooms delivering treatment) and in the cloud, where less latency-sensitive models can run more cheaply on de-identified data after the fact.  

Real footage has one limit: the moments that matter most are, thankfully, the ones that happen least.  

No hospital can wait for enough rare events to train a model on them. That is where the biggest opportunity lies for open models. We now have the ability to generate realistic synthetic scenes of the operating room, so models can learn from events too rare to capture. Grounded in a decade of real surgery, that synthetic data stays true to not just how operating rooms actually work, but how each one works as an individual room.

The next step is an OR co-pilot that gives you the green light.  

This combines to offer a completely different way of gathering and recording information. From within the case, focused on the surgical site, the models will estimate the time left to run. Important moments can be recorded not just in words, but seamlessly as a video of what happened. Seldom-used equipment can be called for ahead of time within the procedure, and in time brought in by a robotic assistant, alleviating today’s burden of moving everything into the room “just in case”. Each surgeon’s preferences are ambiently noted, and the preparation for the next case adjusted. Loved ones can be kept up to date with progress without the surgical team stopping to send a message.  

The same intelligence can support the checks that keep patients safe. Every time an implant is used, the team runs a safety check. Picture that check happening exactly as it does today, with intelligence in the background following the conversation, the procedure and the devices, and giving the team a simple traffic light: green when everything matches and it is safe to implant and move to the next step, red when the AI wants someone to check again. People still run the check and make every decision. What changes is that the steps that matter most get a second check that never tires.  

Surgical intelligence is a room that runs itself, so the people in it can focus on the patient.

All of this is centered not just on the patient, and achieving the best outcomes, but on creating a better working environment for the team delivering care. With less logistics and communications grind, the surgeons, nurses, and technicians are freed to focus on the human aspects of their roles.  

That means more time with patients, less time documenting, and fewer unexpected moments because preparation was incomplete. The surgical team works truly as a team instead of an atomized group of individuals operating processes at the edge of care.  

The team’s full focus becomes caring for those who most need their skills; the difference is they now work in a truly intelligent, connected OR.

‍