Remove ads

Ted Chiang: ChatGPT Is a Blurry JPEG of the Web

liberty90 said:
Something like Dwarf Fortress Adventurer Mode with deeper and less mechanistic feeling to the characters?

Adventurer mode - Dwarf Fortress Wiki

dwarffortresswiki.org dwarffortresswiki.org

And yeah, yeah, I wouldn't want to claim that this is computationally cheap. But costs of computation are historically falling down long-term, pretty significantly.
Click to expand...
Click to shrink...
Right but I really doubt you're playing dwarf fortress adventure mode for the conversation. Which is my point. Your hypothetical dev has to license an LLM, train it, build some sort of database and API that is compatible with the hooks for the LLM, raise the price of their game to offset the cost of the LLM license, and… for what?

At the end of the day they MIGHT get a chatbot that can go one more line of dialogue before sounding insane? That the players aren't even interested in talking to?

They're not in there to talk to the NPCs.
 
Aineko said:
Being there to talk to the NPCs has, in a very real way, never been an option in video games.

If LLMs can get affordable enough, it might become so for the first time.
Click to expand...
Click to shrink...
I did reference Baldur's Gate 3.

They literally put in a spontaneous musical number that you can get into with one of the NPCs if you play your cards right.

That's always going to be more meaningful than an LLM gargling back some sort of grammar-checked garbage.
 
Aineko said:
'Dialogue trees can be executed well' is not the same thing as 'a modest set of pre-written strings is all you need for conversations', and pretending otherwise is embarrassing.
Click to expand...
Click to shrink...

That's not what you said. You said:

Aineko said:
Being there to talk to the NPCs has, in a very real way, never been an option in video games.
Click to expand...
Click to shrink...

Disco Elysium has no mechanics other than its dialogue system! It is a game purely about talking to NPCs and it's one of the best games of all time. I don't even need to start naming visual novels or whatever even though each and every single game where the conversation with NPCs is the point puts the lie to your preposterous claim. In order to talk up LLMs you have to push a significant fraction of video game history into the garbage bin. Stop it.
 
Yaminator said:
That's not what you said. You said:



Disco Elysium has no mechanics other than its dialogue system! It is a game purely about talking to NPCs and it's one of the best games of all time. I don't even need to start naming visual novels or whatever even though each and every single game where the conversation with NPCs is the point puts the lie to your preposterous claim. In order to talk up LLMs you have to push a significant fraction of video game history into the garbage bin. Stop it.
Click to expand...
Click to shrink...
No.

There are mechanics for "talk to the NPC" in a lot of games. Only a very obscure subset even try to look like it's anything but "trigger branch i of N".

In a limited way those NPCs can talk to you. But you can't substantively talk to them, because there's nothing there that could handle anything but multiple choice selections. It's a solid mechanic, but it isn't the player talking.

And back on the thing I actually said, being there to talk to a small finite state machine rapidly falls apart as either the sequence proceeds to its conclusion and stops or it keeps revisiting the exact same nodes. If not both of those in the same game.
 
Aineko said:
No.

There are mechanics for "talk to the NPC" in a lot of games. Only a very obscure subset even try to look like it's anything but "trigger branch i of N".

In a limited way those NPCs can talk to you. But you can't substantively talk to them, because there's nothing there that could handle anything but multiple choice selections. It's a solid mechanic, but it isn't the player talking.

And back on the thing I actually said, being there to talk to a small finite state machine rapidly falls apart as either the sequence proceeds to its conclusion and stops or it keeps revisiting the exact same nodes. If not both of those in the same game.
Click to expand...
Click to shrink...
To have it so those conversations are meaningful and relevant to the game itself you either have that finite state machine to keep the thing on task, or the entire game is just talking to an LLM.

We already have those, it is what all of those character LLM things that have been around for a while now do. That is the end point if you want truly open ended stuff that isn't being guided back to what is actually in the game by a finite state machine telling it what it can and cannot be saying.

That is the core problem with the concept. You still need all of the same background stuff programmed behind the scenes to keep things in line with the gameplay and story that was actually developed for the game. It is still a problem if your NPC companion goes off on a tangent about a magic weapon that isn't programed into the game with just enough details on how to get it that appear right to make it to the bug report stage.

It sounds nice to have NPCs who will describe to you their daily life if you ask, but if those questions don't match what other NPCs say, or worse don't match what the game's story intends, then it isn't actually adding anything real. Just empty words that don't match the virtual reality.
 
liberty90 said:
So chain of thought like:
"I'm now persuaded that I should give this hero the Fantasy Sword"
"Command: Give Item XXXXXX"
-[From the game engine]Error: No item-

Do you think that AI Agent is likely to ignore error message (from traditional game engine) entering it's LLM Chain of Thought?

Or do you think that it will sometimes utterly ignore issuing command to the engine, yet insist to the Hero that the sword was in fact given?

While AI Agents have issues with long-term consistency etc, they are able to use Tools and take Error messages (injected into their llm context) into some account.

And if the issue is rare enough, wouldn't in-character answer (from the player) like "Are you crazy? Look at your shop inventory again, idiot" work to take Agent back on track?
Click to expand...
Click to shrink...
I mean there is literally no way to know due to how LLMs work. Sometimes they just ignore some of the context for completely unclear reasons, or do things that have nothing to do with the provided query because of an incorrect probabilistic association.

The risk with an LLM agent is several fold: The agent delivers the wrong command to the engine because it parses the input incorrectly. The agent passes along the correct command but to the incorrect node. The agent parses the engine response incorrectly. The agent completely ignored the engine response and instead generates what it believes is the most likely response and delivers that instead. etc

It comes down to the fact that LLMs are just not precise, and they 'know' nothing, they just spit out most likely responses. This is damn limiting when trying to design a game to actually use them for something beyond a toy test scenario, because the longer they run the more they start to break, as they get further and further from the key context.
 
Last edited:
I don't think you can make it work with current LLM tech, but if you asked me a decade ago whether the stuff today's LLMs do was possible I would definitely have said no. Maybe some researchers *have* found a satisfying way to integrate them with a fixed rule system and it's in production right now.

It would certainly be nice. Chatting with LLMs can be fun, but trying to play a game feels like trying to grab a fistful of water.

I end up making my own rules, like getting an NPC to take a specific action without directly referencing it in any way, ot making ChatGPT guess the characteristics of a civilization based on archaeological evidence I describe.
 
E.I.G. said:
To have it so those conversations are meaningful and relevant to the game itself you either have that finite state machine to keep the thing on task, or the entire game is just talking to an LLM.

We already have those, it is what all of those character LLM things that have been around for a while now do. That is the end point if you want truly open ended stuff that isn't being guided back to what is actually in the game by a finite state machine telling it what it can and cannot be saying.

That is the core problem with the concept. You still need all of the same background stuff programmed behind the scenes to keep things in line with the gameplay and story that was actually developed for the game. It is still a problem if your NPC companion goes off on a tangent about a magic weapon that isn't programed into the game with just enough details on how to get it that appear right to make it to the bug report stage.

It sounds nice to have NPCs who will describe to you their daily life if you ask, but if those questions don't match what other NPCs say, or worse don't match what the game's story intends, then it isn't actually adding anything real. Just empty words that don't match the virtual reality.
Click to expand...
Click to shrink...
Most video games are large finite state machines (computers are finite state machines so you can't escape that part of it, just make it too big to see the box) while character 'actors' are very small ones because they're small parts of the game with high cost per significant state and transition, because those have to be explicitly scripted rather than emerging procedurally.

If you can use an LLM to make those actors able to react to more of the game's state, and also to respond to non-scripted player input? It could be different. And might make more-detailed NPC personal state more interesting, since the language model can actually present it in ways that would be infeasible for a writer. (Rewrite every line for each of a few dozen mood matrix possibilities...)


And then a bunch of this is "but it won't actually execute" to which I say all technologies are bad if you assume malfunction, and I am not going to play on that conversation node.
 
Aineko said:
Being there to talk to the NPCs has, in a very real way, never been an option in video games.

If LLMs can get affordable enough, it might become so for the first time.
Click to expand...
Click to shrink...
Seems like it would just wind up being the dialog equivalent of radiant quests, technically infinite but hollow and uninteresting
 
Aineko said:
Most video games are large finite state machines (computers are finite state machines so you can't escape that part of it, just make it too big to see the box) while character 'actors' are very small ones because they're small parts of the game with high cost per significant state and transition, because those have to be explicitly scripted rather than emerging procedurally.

If you can use an LLM to make those actors able to react to more of the game's state, and also to respond to non-scripted player input? It could be different. And might make more-detailed NPC personal state more interesting, since the language model can actually present it in ways that would be infeasible for a writer. (Rewrite every line for each of a few dozen mood matrix possibilities...)


And then a bunch of this is "but it won't actually execute" to which I say all technologies are bad if you assume malfunction, and I am not going to play on that conversation node.
Click to expand...
Click to shrink...
... so, the base problem remains, as it has always remained in this discussion, that somebody still need to program the NPC dialog stuff to interact with the rest of the game state.
It doesn't matter if it is a complicated finite state machine or an LLM, you still need to have the hooks made by the programmers for the details to be available.


Let me give an example: How do you make the NPC know how many times the player has broken a signpost?
Well, first you need a variable to track how many times that has happened. The game state itself doesn't know that detail unless it was programed to keep track of it.
Then you need to have that variable be explained to the NPC dialog system as the number of times a sign has been broken. This is either just a raw variable that is displayed in a text box unintelligently, a tree/table of variable responses based on number of signs broken, or a phrase that is provided to the LLM to prompt a response related to sign destruction.
At that point you can make accurate responses to how many signs have been broken by the player.

The LLM is more immersive than the first option, but harder to setup, and it is less controlled than the second option, while being a bit easier to setup. However, that is for a single check. There needs to be a setup LLM prompt detail for every part of the game state in order for your proposal to happen. A whole bunch of added complexity and context that is created by the developers to support the LLM fed NPC dialog.


I understand that this sounds worth it to many, but I also suspect they have no idea how much work and effort that is for something that might not pay off at all.
 
Aineko said:
No.

There are mechanics for "talk to the NPC" in a lot of games. Only a very obscure subset even try to look like it's anything but "trigger branch i of N".

In a limited way those NPCs can talk to you. But you can't substantively talk to them, because there's nothing there that could handle anything but multiple choice selections. It's a solid mechanic, but it isn't the player talking.

And back on the thing I actually said, being there to talk to a small finite state machine rapidly falls apart as either the sequence proceeds to its conclusion and stops or it keeps revisiting the exact same nodes. If not both of those in the same game.
Click to expand...
Click to shrink...

Only a obscure subset of games doing anything but branching dialog for the same reason only an obscure subset have completely physics based animation or bottom up AI controlling character behavior, in 99% of cases it simply makes things more complicated and harder to debug in ways that make a game worse unless that is the entire point of the game. Jurassic Park Trespasser having completely physics based animation didn't improve the gameplay. People don't add accurate life simulation to open world games even though we have had that technology since the 90s because it doesn't actually improve the gameplay of anything but life sim games. Neural network controlled behaviors like with Creatures don't exist outside of they software toy genre because it doesn't add anything to 99% of games.

Adding more simulations to a game don't make it better. Many times it makes games worse.
 
Or to put it in a way that some people will understand. An LLM based NPC becomes a parser game that doesn't even know the right answer to give.
 
Tasrill said:
People don't add accurate life simulation to open world games even though we have had that technology since the 90s because it doesn't actually improve the gameplay of anything but life sim games. Neural network controlled behaviors like with Creatures don't exist outside of they software toy genre because it doesn't add anything to 99% of games.

Adding more simulations to a game don't make it better. Many times it makes games worse.
Click to expand...
Click to shrink...

Heavy disagree. Open world games feeling 'dead is a HUGE problem I have with them.

If AI could somehow be used to generate a proper simulation so that things would continue happening without rigid scripting, and could go on forever, it would be a MASSIVE improvement.
 
How are we constantly talking as though the current level of AI is all we'll ever have..?

How many of you had a chance to try "gpt2-chatbot" while it existed? It's not a huge improvement on GPT-4, but the quality of its fiction writing at least is dramatically improved, and definitely made me laugh. It also seems a good bit smarter overall. Things really, honestly are still moving along at a decent clip.
 
Baughn said:
How are we constantly talking as though the current level of AI is all we'll ever have..?

How many of you had a chance to try "gpt2-chatbot" while it existed? It's not a huge improvement on GPT-4, but the quality of its fiction writing at least is dramatically improved, and definitely made me laugh. It also seems a good bit smarter overall. Things really, honestly are still moving along at a decent clip.
Click to expand...
Click to shrink...
Sure, but a game dev isn't going to be using a big commercial model like GPT 4.5 (or whatever they end up calling it) to power Random Farmer #17. You're looking at either a cheap local model, like quantized Phi-3 Mini or even smaller, or maybe a server hosting something like Llama3 8B if you're lucky. Those are better than the equivalents from a year ago, sure, but there's still a pretty significant gap.

(I'm also still in the "LLM's are plateauing at the higher end" camp, but we'll find out the truth of that eventually.)

More advanced models don't necessarily fix the "thousand faces with the same voice" problem either. A lot of the time it's just down to quirks in their instruct training. Not always as blatant as "As an AI language model" or the chain-of-thought pattern, but you can't really stop them from picking up specific turns of phrase, sentence structure, etc.
 
Last edited:
News from OpenAI.

They are releasing a combined text-audio-vision model that processes all three modalities in one single neural network, which can then do real-time voice translation as a special case afterthought, if you ask it to.
Click to expand...
Click to shrink...


Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: openai.com/index/hello-gp…

Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.
Click to expand...
Click to shrink...

 
I feel like people are comparing two massively different things with regards to LLM NPCs vs human crafted NPCs.

When thinking about LLM NPCs, I don't expect nor want something like Disco Elysium. What I want is something like Dwarf Fortress or Crusader Kings; emergent gameplay. I want NPCs that have meaningful interactions with each other and me, and by meaningful, I don't mean literary, I mean having actual impact. I want to be able to go into a town, have a minor interaction with an NPC that results in knock-off effects that drastically reshape the entire community...and then reload a quick save, do the exact same thing and have a completely different, yet plausible and engaging set of events occur.

I don't expect nor am I interested in LLMs that attempt to emulate great human writers.
 
Zhade said:
I feel like people are comparing two massively different things with regards to LLM NPCs vs human crafted NPCs.

When thinking about LLM NPCs, I don't expect nor want something like Disco Elysium. What I want is something like Dwarf Fortress or Crusader Kings; emergent gameplay. I want NPCs that have meaningful interactions with each other and me, and by meaningful, I don't mean literary, I mean having actual impact. I want to be able to go into a town, have a minor interaction with an NPC that results in knock-off effects that drastically reshape the entire community...and then reload a quick save, do the exact same thing and have a completely different, yet plausible and engaging set of events occur.

I don't expect nor am I interested in LLMs that attempt to emulate great human writers.
Click to expand...
Click to shrink...
That honestly sounds like something that would be done easier and cheaper with existing technologies. Like by creating a large number of lines, flags, and good RNG algorithm you could do something similar that the average person will never be able to tell any different.
 
You must login or register to reply here.
Remove ads
Back
Top