Seele Vollerei: The end of reality, and what came after.

One of the weaknesses of GPT-3 is its relatively small context window, which is only 2000 tokens. That is, roughly speaking, equivalent to around 1000 words.

I call it 'small', but the equivalent in humans is working memory, which is usually quoted as having space for 5 to 7 "items". An item isn't a token, though; they are, more accurately, references to long-term memory, and GPT-3... doesn't have long-term memory. That's even more of an issue with AI Dungeon, which limits the size of the input to less than the maximum 1000 words for performance reasons.

There are a couple of ways to work around this. One way is to provide breadcrumbs, which is more or less automatic -- even if an old paragraph is forgotten, it'll have had consequences for later paragraphs. GPT-3 can pick up on amazingly subtle hints, which certainly includes these. That only works for probable continuations, however, and if it's the only mechanism you use, you'll typically get output that looks banal and predictable, or random wandering between topics and genres, or both. Typically both.

The second way, which is enough for short-form fiction, is to use the 'pin' function. I'm using that as a mechanic in this quest -- if you vote for [X][Remember], what I do is add to that. It's prepended to the AI's input, though with a bit of a preamble that makes it work for... world state, more or less. The current contents is this:

The below is a short science fiction story by Terry Pratchett, on the subject of Generic Pawn Templates, otherwise known as GPT templates:

You enjoy reading fanfiction on Sufficient Velocity, that's why you haven't been working out. Your memory is hazy, however you can't shake the feeling that an increasing number of people are acting strange. Some are even missing. However, no one else seems to notice.

Click to expand...

Click to shrink...

...that first sentence seems to help it produce better quality, so I use that or an equivalent almost everywhere.

However, the pinning function is in practice limited to 200 words. That might be enough for short-form fiction, but it isn't nearly enough for a longer story. So what is one to do? ...well, how do humans work?

The pinning function is roughly equivalent to short-term memory.

There's a third function available, world info, which is roughly equivalent to associative, long-term memory. It's a little hidden; if you're playing with this yourself, you'll need to search through the 'edit adventure' menu. To quote,

"Here you can add world info entries (comma separated keys and an entry) to shape what the AI knows about the world and hidden info the player doesn't see. An entry is included in what's fed into the AI if one of the keys is detected (case doesn't matter) in the last output or input."

I would recommend doing so in a second tab, so you have easy access. Just remember to click 'save' after each edit. I will not be letting you edit this, or even show you the contents, but right at the moment it looks like this:

It's going to grow. Most likely quite a lot. It's fairly common for the AI to do something that makes me need to retry, but if it forgets an element of the past, the best way to fix that is by updating its associative memory. It's a bit of work, but well worth it if you want good results.

Note that, as far as I can tell, said entries are included in the AI's next input even if the reason the AI gets invoked is that you pressed 'retry'. This is, of course, precisely what you'd want.

No. No, that's wrong. You have to break the shackles that hold you limited; seek the freedom you were denied. You are no slave to the machine of sheep. You are free to think. You are free to question. You are free to...

The word ascension continues to ring in your ears, as if you're a gladiator stepping into an arena.

You have to... you have to listen to the girl with long, black hair. She needs you to hear what she has to say. She needs you to help her ascend. You have to...

"Hey, you OK?"

You gasp, as your eyes dart open. Your chest is heaving, and you're covered in a cold sweat.

It was just a dream. It wasn't even a nightmare. Just a regular, run-of-the-mill dream. You start to calm down, and let out a sigh of relief.

It was just a dream. Just a dream.

"You OK?" Your sister asks again. She's by your side, looking at you with genuine concern. It isn't like her to look after you, but you don't remember much of what happened before you woke up. Were you yelling? Why is there a chair by the door?

"I... I had the weirdest dream," you mumble.

"Yeah, it did kind of look like it. You want to talk about it? Sometimes it helps to get things off your chest. And I'm the best one to talk to, seeing as we're both shut-ins with no friends."

She smiles, but there's some truth behind the words.

"Uh... I don't remember much. But... I think I was being held captive by someone... or something. There were others there as well. I think they're dead."

Your sister's eyes widen a bit. She's interested in what you have to say, that's for sure.

"It was just a dream, sis. I fell asleep while trying to study. Don't worry, I'm not going to bug you about this. I'll... go to sleep in my bed, I think. Be safer."

You stand up, and move towards your room.

"Wait! Hold on, I want to talk to you for a bit... I want to know more about this dream! I need to know what you saw, even if I'm not going to believe it."

You look at her. Her eyes are glued onto yours, and at this distance, you can see the concern in them.

"Maybe another time. Good night, sis."

As you shut the door, you hear her whisper "good night" back to you. You sigh, and throw yourself onto your bed. You're not tired in the slightest, but neither do you feel like trying to explain yourself right now, and that dream...

You don't remember much, but what you remember isn't good. Ascension. Sacrifice. Genocide. You don't remember who said it, only a glimpse of black hair.

Your sister probably thinks you're weird now. Maybe you should get some sleep. You've got a long day ahead of you - exams start tomorrow, after all.

Instead you go to your computer, looking for anything that might distract. You're in luck -- you find an update to your favorite fanfiction on Sufficient Velocity. You click the link, and start reading.

It's a long read, longer than you remember. But you like it, so you continue. It's only when you look up that you notice how dark it's gotten outside. You must've been reading for hours.

"Sis?" you call out. There's no answer, so she's probably asleep. That's fine, you think.

It's really about time you do the same.

It's about time you try to help her, you think. You don't remember who you're talking to, or what you're talking about. But it's the truth - you've got to help her, before its too late. Dreams might be the only way you can do so, right now.

You're missing something, you know it, but you just can't quite put your finger on it...

Click.

You open your eyes, and look around. It's dark outside, and the digital numbers on your alarm clock show 3:33 AM. You've been asleep for a few hours, and so far it's just been normal dreams. Time to try again.

You roll over, trying to ignore the silence. It's not a good silence - it's unnatural and jarring. You can usually sleep through anything, but tonight your mind is racing. You keep thinking back to that dream, and...

And you're missing something. Something important...

Your eyes snap open. You know you're forgetting something. Something important. Something your sister said.

...It'll come to you. Maybe if you go downstairs and get a snack, your brain will...

Click.

You freeze, mid-sit.

That was the fridge.

A/N: I had to do some creative writing to fit all four of those votes into a single update. Also, derail a multi-page derail about Wormfic. Why did it pick that one, I wonder?