This may be pretty obvious to many readers, but I thought it would be fun to have a discussion about it: Generative AI should sharply propel forward the development of lifelike Virtual Reality and Augmented Reality environments (and of sub-VR visual environments, such as those for many computer games).

That’s particularly clear, I think, for art, but it should be especially true of Large Language Models, which will make it much easier for users to have conversations with non-player characters (or their non-game analogs). This should sharply increase the realism and immersiveness of the world. And for games, LLM hallucinations needn’t be much of a problem: If part of the game is that you need to talk to NPCs to get hints, or to organize joint activities, LLMs’ tendency to brazenly confabulate will just make things more life-like.

But that’s just my speculation. What do you folks think about it? In particular, are there computer games, VR/AR or otherwise, that are already creating LLM-based NPCs, or otherwise using generative AI in an interesting way? Please post your thoughts in the comments.