Sony Patent Would Let an AI Model Train NPCs Based on Your Player Data

If one thing's been abundantly clear over the last year, it's that Sony and Microsoft are both champing at the bit to work AI features into the next generation of consoles. And now, Sony has filed another patent that would use AI to train NPCs based on data generated by you, the player.

This new patent, called "AI-adapted non-player character for video game system," describes a process of taking data generated from playing a game and then using it to modify NPCs in order to make them more lifelike.

But unlike Nvidia's ACE program, which uses a large language model to essentially script NPC dialogue, Sony's proposed system could change everything about an NPC, from its design to its behavior in, say, a boss fight. Sony asserts that this is necessary to create more lifelike games, without bloating development time and budgets even more than they already are.

The patent says that "developing such believable NPCs is challenging due to limitations of modern game development and often relies extensive scripting and programming. Additionally, NPC development often relies on sophisticated systems and extensive resources to track, animate, and evolve NPC behaviors. In a typical development environment, NPC management often utilizes costly and efficient rendering engines, simulation engines, and scalable systems capable of supporting increasingly detailed virtual environments and lifelike NPC interactions."

The solution, according to this patent, is to use AI to do most of this work. It continues, "The techniques described herein overcome the foregoing challenges and others by providing an artificial intelligence (AI)-adapted NPC. By using the techniques described herein, video game systems can be improved." Using AI to shortcut development time is nothing new, but that's not the whole reason that Sony patented this technology. It seems to think the player data will make NPCs even better than before."

Sony continues: "Additionally, by using the techniques described herein, lifelike NPCs that behave based on behaviors exhibited by video game players and/or responsive to behaviors exhibited by video game players can be seamlessly integrated into video game systems and video game content thereby improving video game systems without the extensive scripting and hardware resources."

Just like any of the other patents that Sony has been spitting out at a rapid clip, there's no guarantee this system would actually make its way to a PlayStation 5 (or more likely PlayStation 6) near you. However, the game industry has been experimenting with ways to use AI to train NPCs for years now. Just last year, Inzoi used LLMs, along with player input to shape how NPCs would interact with the player and each other.

It's only a matter of time, then, before AI is used to train different parts of NPC behavior with your data. I could see a world where the bosses in a game like Elden Ring evolve to counter your playstyle, or an enemy in a horror game stalking you more effectively because it 'knows' how you react to different stimuli.

However, there would be questions about security. As cool as having smarter NPCs would be, if these models are processed in the cloud, there would be a risk of your data leaking out onto the internet. And while it doesn't seem like advertisers would want know how you react to a giant monster swinging an axe at you, it probably will end up being used for 'enhanced marketing,' too.

This is hardly the first of Sony's strange new patents. Just a couple of weeks ago, the company patented a system where AI would detect problems with your console and deliver vibe-coded patches. It also patented a controller with an NFC card reader that would bring tap-to-pay to your PlayStation. The next generation of consoles is shaping up to be a strange one, to say the least.