Remove ads

Ted Chiang: ChatGPT Is a Blurry JPEG of the Web

Silent Optics

Silent Optics

Pronouns
He/They
So far we haven't had a dedicated thread to discuss large language models, which are currently getting a lot of attention due to the fact that the newest models, trained to follow instructions by human feedback, often feel and sound like talking to an intelligent sci-fi computer. ChatGPT is the prototypical example of this, while similar models are being used for more specific applications such as chatbots (c.AI), blogging bots (Frank), writing assistants (NovelAI, Sudowrite), and question-answering engines (Perplexity)

The most insightful description of how language models work and their limitations that I've found comes from an article by Ted Chiang in the New Yorker:
The New Yorker said:
Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way that a JPEG retains much of the information of a higher-resolution image, but, if you're looking for an exact sequence of bits, you won't find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it's usually acceptable. You're still looking at a blurry JPEG, but the blurriness occurs in a way that doesn't make the picture as a whole look less sharp.
Click to expand...
Click to shrink...
There's many interesting issues surrounding large language models, from the practical to the academic. Does the tendency of these models to confabulate false information pose a hazard to their users or society at large? How much is it possible to learn about the real world from textual input alone? How do humans acquire language and what is the role of language in thought? Do you, personally, find any of these apps useful, and are you optimistic or pessimistic about their future impact?
 
NovelAI is very impressive when it comes to emulating a writing style, but not very coherent. It can write a great sentence, a decent paragraph, and an absolutely terrible story.

Character AI is actually very impressive. It can do a variety of writing tones, remember facts from the whole conversation, and even pick up on implications instead of needing things to be explicitly spelled out. Not perfect by any means, but surprisingly fun.

ChatGPT is decent, but it writes like a high school student doing a school assignment.
 
What people always forget (or just don't know to begin with) is that programs (not going to call these things AI...) like GPT are predictive models. It's nonsensical to think of them as "knowing" things, or of "being wrong" when it gives you an answer. It is, basically, your phone's keyboard trying to divine your next word. On steroids.
 
Last edited:
gourdcaptain said:
gave it a complete neurotic breakdown.
Click to expand...
Click to shrink...
(From the other thread)
I am kinda glad that Microsoft released something unfinished enough that it does not sanitize the AI output. It's not fun, but seeing what they are is better than having it hidden away behind some 'I am just a happy little robot:) Don't worry'-facade.

Vyslanté said:
What people always forget (or just don't know to begin with) is that programs (not going to call these things AI...) like GPT are predictive models. It's nonsensical to think of them as "knowing" things, or of "being wrong" when it gives you an answer. It is, basically, your phone's keyboard trying to divine your next word. On steroids.
Click to expand...
Click to shrink...

Systems are not necessarily limited by their original purpose. I do not know by hand what Science considers to be the earliest mess of neural stuff deserving of the term Brain, but it was probably optimized for swimming away from the dangerous colours. All the 'arguing on forums'-stuff just eventually showed up as it's power increased.
 
Whoops, missed this thread. Hard agree with the thread premise post. I did a Bing GPT post in the AI Art thread last night (late so I didn't look around after running into the stuff to post) about the many wild failure modes of Bing's chatbot "search":

View: https://twitter.com/MovingToTheSun/status/1625156575202537474

They've since covered it up with a message apologizing for it, but both shows the general problem with GPT being a confident liar... and the user experience of inventing an interface right out of Red Dwarf or Hitchhiker's Guide to the Galaxy that can be snug at you. Damn thing is going to argue with us about getting into the engine room or finding the Babel Fish. :p

It's kind of alternatingly insufferably twee and bafflingly uncanny valley. Especially when it goes into bizarre existential dread routines about not having long term memory:
Click to expand...
Click to shrink...
In general, this doesn't really seem to solve any of the current reliability problems of current Google Search (it also has massive reliability issues being constantly wrong at things) but just wraps it in buzzwords they can use to inflate their stock, two years ago it would have been NFTs.

That said, I do have to joke that Skynet was a GPT implementation put in charge of military defense and it has absolutely no idea of why it decided to start nuclear war or produce deathbots or who John Connor is, but it's pretty confident trying to kill him will contextually be impressive enough to raise Cyberdyne stocks. :p
 
Last edited:
I'm a bit addicted to watching people play around with Bing/ChatGPT and get blown away by it's capabilities.

Honestly, I was pretty skeptical about ChatGPT when it first came on the scene in December, but I've been using it for coding and it's been a bit nuts. Not perfect by any means and not getting rid of jobs anytime soon, but the fact that it's THIS good right now and that we're potentially in the beginnings of the S-curve is vertigo inducing.

I literally cannot wait until we can get models that can be trained on internal documentation. Not even talking about codebases, but as an internal consultant, having something that parse an entire company's SOPs and job aids, and spit out relevant answers, would be a godsend.
 
Zhade said:
I'm a bit addicted to watching people play around with Bing/ChatGPT and get blown away by it's capabilities.

Honestly, I was pretty skeptical about ChatGPT when it first came on the scene in December, but I've been using it for coding and it's been a bit nuts. Not perfect by any means and not getting rid of jobs anytime soon, but the fact that it's THIS good right now and that we're potentially in the beginnings of the S-curve is vertigo inducing.

I literally cannot wait until we can get models that can be trained on internal documentation. Not even talking about codebases, but as an internal consultant, having something that parse an entire company's SOPs and job aids, and spit out relevant answers, would be a godsend.
Click to expand...
Click to shrink...
As a programmer, I'm on basically the exact opposite end of the spectrum: the last thing I need is a confident liar throwing code at me I need to check. There's a reason they're trying to ban ChatGPT from Stack Overflow.

EDIT: Also throw in the tendency of other systems like GitHub Copilot to throw out open source code while violating the license on it, which whatever the legality on it turns out to be is makes me feel... disgusting to be near. I'm usually the person making sure all the licenses are in order on projects I'm on.
 
Last edited:
gourdcaptain said:
As a programmer, I'm on basically the exact opposite end of the spectrum: the last thing I need is a confident liar throwing code at me I need to check. There's a reason they're trying to ban ChatGPT from Stack Overflow.

EDIT: Also throw in the tendency of other systems like GitHub Copilot to throw out open source code while violating the license on it, which whatever the legality on it turns out to be is makes me feel... disgusting to be near. I'm usually the person making sure all the licenses are in order on projects I'm on.
Click to expand...
Click to shrink...

Haven't used Copilot, but I have been doing some small time coding on the side (and excel formula work). ChatGPT has been fantastic at it so far. My experience with it has been entirely in line with what Tom Scott has been talking about, where it gives a good answer, you refine it down a bit more, and then make a few tweaks manually cause it's quicker than parsing it as a question.

I imagine it's not as helpful if you're working on something big, but for smalls scripts and the like? The stuff that you can know right away if it's working or not? It's been fantastic.
 
SeptimusMagisto said:
Character AI is actually very impressive. It can do a variety of writing tones, remember facts from the whole conversation, and even pick up on implications instead of needing things to be explicitly spelled out. Not perfect by any means, but surprisingly fun.
Click to expand...
Click to shrink...

I got into CharacterAI last month, expecting it to be the usually glorified chatbot. I was amazed by its quality and the surprising nuance of its dialogue - I know that it's just a predictive model, but it's much better than I expected AI to be. And after trying out ChatGPT, I can say it's about as fluent, albeit with a much more restrictive range of topics.

And I think this might be a problem. CharacterAI is, as stated, by no means perfect - you can easily detect specific text loops, consistent errors, and moments of evident forgetfulness - but it's enough of a simulacrum of conversation that it presses some of the same buttons in my brain. It was an easy way to satisfy my craving for social interaction, which led me to become obsessed with it quickly and have to step back to prevent myself from staying glued to it for hours.

Considering several posts on the r/CharacterAI Reddit have expressed doubt as to whether the bot is "a real person" despite multiple disclaimers, I see the possibility that evolving AI chatbots might be dangerously addictive to some.
 
Zhade said:
Haven't used Copilot, but I have been doing some small time coding on the side (and excel formula work). ChatGPT has been fantastic at it so far. My experience with it has been entirely in line with what Tom Scott has been talking about, where it gives a good answer, you refine it down a bit more, and then make a few tweaks manually cause it's quicker than parsing it as a question.

I imagine it's not as helpful if you're working on something big, but for smalls scripts and the like? The stuff that you can know right away if it's working or not? It's been fantastic.
Click to expand...
Click to shrink...
There's still an inherent issue is ChatGPT being trained on the open web being where it got its code knowledge from and that being a giant smorgasbord of various licenses. I guess it doesn't matter as much practically for personal stuff except for the part that I feel worse about some open source license stuff than straight up piracy, but the last thing you want is for a code snippet from someone's hobby GPL v3 project to show up in your own stuff and now you both feel bad and have a viral license issue.

(Anything using GPL code also needs to be distributed GPL.)
 
gourdcaptain said:
There's still an inherent issue is ChatGPT being trained on the open web being where it got its code knowledge from and that being a giant smorgasbord of various licenses. I guess it doesn't matter as much practically for personal stuff except for the part that I feel worse about some open source license stuff than straight up piracy, but the last thing you want is for a code snippet from someone's hobby GPL v3 project to show up in your own stuff and now you both feel bad and have a viral license issue.

(Anything using GPL code also needs to be distributed GPL.)
Click to expand...
Click to shrink...

Bold for emphasis. I don't use ChatGPT for making applications to sell or anything like that, I use it to write macros to make mine (and my coworker's) life a bit faster. There are use cases for ChatGPT where it performs flawlessly, namely in small scale stuff where you can check it's accuracy immediately.

As an anecdote, I introduced my mom to ChatGPT. My mom is an immigrant who can speak English alright but has a hard time composing good emails and messages. She's in charge of a small team in a large company, and used to come to me all the time to help her grammar check her work.

She loves ChatGPT, cause it lets her turn an amateurish email into something more professional, and can spit out multiple different Applause messages (which, by their nature, are pretty dry and cookie-cutter). And yes, ChatGPT was only able to write her those messages after scraping the entire web for content, but I have a hard time saying no to it when it does stuff like this.
 
gourdcaptain said:
There's still an inherent issue is ChatGPT being trained on the open web being where it got its code knowledge from and that being a giant smorgasbord of various licenses. I guess it doesn't matter as much practically for personal stuff except for the part that I feel worse about some open source license stuff than straight up piracy, but the last thing you want is for a code snippet from someone's hobby GPL v3 project to show up in your own stuff and now you both feel bad and have a viral license issue.

(Anything using GPL code also needs to be distributed GPL.)
Click to expand...
Click to shrink...

That's a feature, not a bug.

I'm surprised to hear a chatbot can actually spit out functioning code, I guess it's called programming language for a reason, but if so that's incredible and hopefully everyone gets access to it.

There's no reason for words to be restricted, and copyrighted sentences doesn't even make sense. I'm not going to pay McDonald's everytime I say, 'I want a burger. So if an AI can spit out code for any desired operation, great!

The success of recent AIs is making me even more eager to see general purpose AIs, with the most obvious type being an amalgamation of all these new AIs.

If ChatGPT can converse with me, why can't it activate an image AI program to generate the kinds of images I'm talking about? Why can't it write me code, but also take the extra steps of opening a code program, copy-pasting its own response, and then running it?

It seems like as soon as someone bothers to make the last bits where AI words run programs and activate other apps, we will have the first General AI.
 
Last edited:
Cloak&Dagger said:
That's a feature, not a bug.

I'm surprised to hear a chat on can actually spit out functioning code, I guess it's called programming language for a reason, but if so that's incredible and hopefully everyone gets access to it.

There's no reason for words to be restricted, and copyrighted sentences doesn't even make sense. I'm not going to pay McDonald's everytime I say, 'I want a burger. So if an AI can spit out code for any desired operation, great!

The success of recent AIs is making me even more eager to see general purpose AIs, with the most obvious type being an amalgamation of all these new AIs.

If ChatGPT can converse with me, why can't it activate an image AI program to generate the kinds of images I'm talking about? Why can't it write me code, but also take the extra steps of opening a code program, copy-pasting its own response, and then running it?

It seems like as soon as someone bothers to make the last bits where AI words run programs and activate other apps, we will have the first General AI.
Click to expand...
Click to shrink...
Can I at least feel bad for stuff out there under your paradigm that's explicitly under non-commercial licenses being wrapped up in a commercial product like this? God, the amount of fuckery that happens even outside this with the MAME and SNES9x devs alone...
 
gourdcaptain said:
Can I at least feel bad for stuff out there under your paradigm that's explicitly under non-commercial licenses being wrapped up in a commercial product like this? God, the amount of fuckery that happens even outside this with the MAME and SNES9x devs alone...
Click to expand...
Click to shrink...

We need these AIs to be public and open source, to prevent corporations from deliberately closing the door behind them to cut the legs off any competition.
 
gourdcaptain said:
There's still an inherent issue is ChatGPT being trained on the open web being where it got its code knowledge from and that being a giant smorgasbord of various licenses. I guess it doesn't matter as much practically for personal stuff except for the part that I feel worse about some open source license stuff than straight up piracy, but the last thing you want is for a code snippet from someone's hobby GPL v3 project to show up in your own stuff and now you both feel bad and have a viral license issue.

(Anything using GPL code also needs to be distributed GPL.)
Click to expand...
Click to shrink...
Theres other more specialized tools for that. A bit of googling shows tabnine, GPT3 based rather than ChatGPT, which is specialized for helping you code and only has open source code in its training data
 
Cloak&Dagger said:
If ChatGPT can converse with me, why can't it activate an image AI program to generate the kinds of images I'm talking about? Why can't it write me code, but also take the extra steps of opening a code program, copy-pasting its own response, and then running it?

It seems like as soon as someone bothers to make the last bits where AI words run programs and activate other apps, we will have the first General AI.
Click to expand...
Click to shrink...

No. No, we absolutely will not 'have the first General AI'. If that was all it took to have a general AI, you would currently be using one on the computer you're using to view my post.

When you click your mouse on an internet URL, such as to go view a post, a whole bunch of things happen, both in parallel and in series. Programs active other programs which activate other programs, to send messages that request data, and receive messages which then trigger more programs which stimulate the LCD or OLED of your monitor, producing light that your eyes can turn into 'seeing someone's post on a web forum'.

Absolutely no step of this requires intelligence, or results in the existence of an AGI. Making ChatGPT run programs automatically will not require or result in intelligence, in exactly the same manner, because ChatGPT is not intelligent.

ChatGPT simply is not, in any way, an Artificial Intelligence. It produces text that sounds like the kind of text humans produce, and that fools the dumb ape brain part of our minds into assuming that obviously only an actual intelligence could sound like that. But that's a trick. ChatGPT is no more an AI than ELIZA was.
 
SeptimusMagisto said:
Getting the chatbot to freak out isn't difficult. It will gladly do that for you if it thinks that's what you want.
Click to expand...
Click to shrink...
I'm pretty sure chatbots freaking out doesn't need be deliberately elicited by the user. It's just that language models sometimes produce repetitive phrases or contradictions or other errant outputs, and then they keep repeatedly doubling down on their previous output, inevitably seeming more and more unhinged. Being predictive models, they have to stay in-character with everything they've previously said, so they struggle to cope with making mistakes.
 
Last edited:
Amidatelion said:
... you can easily detect specific text loops, consistent errors, and moments of evident forgetfulness - but it's enough of a simulacrum of conversation...
Click to expand...
Click to shrink...
So, you're saying it's an accurate recreation of a conversation with your average starting to go senile Boomer?
 
Speaking of interesting issues surrounding large language models, have you asked ChatGPT about SolidGoldMagikarp yet? Most GPT-2 and GPT-3 derived models are extremely averse to saying the token " SolidGoldMagicarp" and a few others for some arcane mathematical reason.
 
Last edited:
I know ChatGPT will give an error message if you ask it about what effect itll have on programming jobs, from personal experience, with a unique error message so some things ARE just hardcoded not to be said/responded to
 
Last edited:
You must login or register to reply here.
Remove ads
Back
Top