Remove ads

Ted Chiang: ChatGPT Is a Blurry JPEG of the Web

It honestly seems like 'occasionally our LLM botches things' should pose the same business problem with the same level of necessary tolerance and similar mitigation as 'occasionally our human employees botch things'.

I am curious whether there's actually some legal reason that it wouldn't.

(Obviously the 'we assume no responsibility for the chatbot we told you to listen to' was a villainous hail mary that got exactly the respect it deserved...)
 
Simon_Jester said:
I suspect that LLMs are going to hit practical limits whenever and wherever the material they create bumps into the walls of, for lack of a better term, accountability.

Because it looks like the courts aren't accepting the notion of "the LLMs we use for integral parts of our business model are not actually part of the company and we are not accountable for what they do, so sorry, there's no one for you to sue over this mistake." If they were, everyone would be all over LLMs even more so than they already are, because there's almost no downside to using them and considerable upside. But if you can plausibly lose real money over something dumb your LLM says, it's going to be hard to really get the maximum benefit from what the LLM does. You still need people poring over every line of text for most applications, and that's often only slightly easier than having to write it from scratch.

Even if the LLM is approximately as reliable as a human being in creating content, when a human being creates content there is someone to shout at if it fails. And the decision-making process the content-creator uses isn't opaque, so you can realistically figure out WHY the product is messed up.
Click to expand...
Click to shrink...

The counter-intuitive thing though is that for a lot of the moneyed interest's who want to cut out the middleman creative and pass the savings onto themselves by utilizing LLM's is that they do think an LLM would be more responsive and capable of adjusting faster than a human, because it would ostensibly be a matter of telling the LLM "Write Captain Crunch 2: The Search For Trix, but this time make it 20% funnier, 10% less boring, and maybe 5% less queer subtext so we don't piss off China."
 
Aineko said:
It honestly seems like 'occasionally our LLM botches things' should pose the same business problem with the same level of necessary tolerance and similar mitigation as 'occasionally our human employees botch things'.
Click to expand...
Click to shrink...
I may not have explained myself well. What I'm predicting is that the owner won't like it when the text is generated by a machine that they can't shout at for doing it wrong or "doing it wrong." While in practice navigating the inevitable screwups is not necessarily impossible, the fact that there's no longer anyone you can blame other than "I guess our machine stinks, lol."

Courts won't let corporations off the hook just because they use chatbots instead of real people to write documentation, answer questions, and provide services. Therefore, corporations will want to make sure they have real people available that they can hold accountable for failures, rather than a robot that is literally incapable of even understanding negative feedback.

It probably won't help that the robots will still need robot whisperers prompt writers whose skill set is basically black magic and whose output quality is not predictable from any naive oversight of the prompts they input.

Arthur Frayn said:
The counter-intuitive thing though is that for a lot of the moneyed interest's who want to cut out the middleman creative and pass the savings onto themselves by utilizing LLM's is that they do think an LLM would be more responsive and capable of adjusting faster than a human, because it would ostensibly be a matter of telling the LLM "Write Captain Crunch 2: The Search For Trix, but this time make it 20% funnier, 10% less boring, and maybe 5% less queer subtext so we don't piss off China."
Click to expand...
Click to shrink...
They think that, maybe, but I suspect that it's only going to take one or two rounds of disappointing productions to prove it doesn't work that way (especially the "make it funnier and less boring" part) before they decide that they'd rather hire a scriptwriter they can micromanage.
 
Simon_Jester said:
They think that, maybe, but I suspect that it's only going to take one or two rounds of disappointing productions to prove it doesn't work that way (especially the "make it funnier and less boring" part) before they decide that they'd rather hire a scriptwriter they can micromanage.
Click to expand...
Click to shrink...
See also: The Willy Wonka Experience
 
Aineko said:
It honestly seems like 'occasionally our LLM botches things' should pose the same business problem with the same level of necessary tolerance and similar mitigation as 'occasionally our human employees botch things'.

I am curious whether there's actually some legal reason that it wouldn't.

(Obviously the 'we assume no responsibility for the chatbot we told you to listen to' was a villainous hail mary that got exactly the respect it deserved...)
Click to expand...
Click to shrink...

The first thing is that if a human employee botches things, you can try to make them responsible for the cost of the botch, like if someone manages to convince a CS agent to sell them a car for a dollar, you can tell the CS agent to pay for the goddamn car.

The other thing is that LLMs are kind of scope-insensitive so while they may or may not be more likely to botch things than humans (and arguably in some contexts they are indeed less likely, because humans fuck up when asked to do rote tasks that they are nominally good at ten thousand times), they are way more likely to make catastrophic botches of the sort that a human would go "uh, wait, this is insane, lemme ask someone else" before committing.

And more generally, the kind of botches humans make are generally predictable and have been engineered against for a long time, while the kind of botches LLMs make are novel and the people who think they're a good idea to use in unsupervised mode don't even believe they exist . . .
 
I mean, yeah it'll be a while before AI is good enough that it consistently doesn't need supervision, and the person doing the supervising sometimes/often needs enough skill/attention that the cost of supervision is higher than businesses trying to cut costs were hoping for.
 
aattss said:
I mean, yeah it'll be a while before AI is good enough that it consistently doesn't need supervision, and the person doing the supervising sometimes/often needs enough skill/attention that the cost of supervision is higher than businesses trying to cut costs were hoping for.
Click to expand...
Click to shrink...
Which leaves it where it has always been- a "solution" searching for a problem while every tech bro out there tries to convince the gullible people with money that AI will solve all their problems.

Also from a business standpoint these things are just hilariously unsafe. In order to do anything related to your business you have to feed the model all your proprietary data. Otherwise you'll just be stuck with a model that spits out whatever generic catchphrase it can make from the open source Gutenberg project or whatever it was fed when you bought it.

But the AI as no notion of least privilege. Bob on the factory floor wants to know how many vacation days his team lead has? He can go ahead and ask! Or maybe someone with a grudge decides he'd really like to know the home address of that secretary who turned him down. Don't worry, the company AI knows that too! Or maybe a sales guy figures he'll just make a better deal by selling engineering documents on the side- no problem, chat GPT is happy to provide.

There's just no safe use of these things in a business environment at this stage. MAYBE copilot. But I REALLY doubt it. It's supposed to only have access to what its users do… but. A) are you really, REALLY sure? And B) do you actually know what your users have access to? Before if it was obscure the users probably didn't know they had access to so much stuff. But now here's this friendly prompt they can ask…
 
armorfiend said:
But the AI as no notion of least privilege. Bob on the factory floor wants to know how many vacation days his team lead has? He can go ahead and ask! Or maybe someone with a grudge decides he'd really like to know the home address of that secretary who turned him down. Don't worry, the company AI knows that too! Or maybe a sales guy figures he'll just make a better deal by selling engineering documents on the side- no problem, chat GPT is happy to provide.
Click to expand...
Click to shrink...
That's pretty easy to fix from a technical perspective. This is another case where the actual LLM is only part of your system. You'd use RAG and possibly function calling, and restrict which parts of the database it has access to based on the current user. RAG also reduces hallucinations / false outputs, though nothing can fully eliminate them.

Is everyone going to do it correctly, though? Absolutely not. Is it more useful than current systems once the hype wears off? No idea. Would I ever trust it? Not anytime soon.
 
Darth Crater said:
That's pretty easy to fix from a technical perspective. This is another case where the actual LLM is only part of your system. You'd use RAG and possibly function calling, and restrict which parts of the database it has access to based on the current user. RAG also reduces hallucinations / false outputs, though nothing can fully eliminate them.

Is everyone going to do it correctly, though? Absolutely not. Is it more useful than current systems once the hype wears off? No idea. Would I ever trust it? Not anytime soon.
Click to expand...
Click to shrink...
I'd qualify it by saying it's easy from a conceptual perspective and damn near impossible from an actual technical execution perspective.

I feel like what you're describing is kind of how copilot is claiming to do it… but can you trust it? I sure wouldn't.

Locking down access to least privileges and need to know has been the holy grail forever. The only solution is automation, which can fix almost as many problems as it causes… and somehow you have to teach your LLM and your automation how to tell sensitive data apart from other stuff.
 
armorfiend said:
Locking down access to least privileges and need to know has been the holy grail forever. The only solution is automation, which can fix almost as many problems as it causes… and somehow you have to teach your LLM and your automation how to tell sensitive data apart from other stuff.
Click to expand...
Click to shrink...
You never actually train the LLM on your sensitive data. That gets put into the LLM's context by RAG based on the user's query, or by a function call triggered by the LLM output. That, in turn, is restricted by user roles. There is some complexity in getting a system like that set up, but it doesn't need to rely on the LLM to keep secrets at any point.

I do agree that a ton of companies have bad security now, that many attempts at LLM implementation will make it worse, and that the people pushing for it are probably not very security-minded.
 
aattss said:
Seems backwards to me. They know the problems they're trying to solve. They just haven't actually solved them.
Click to expand...
Click to shrink...

The only problem the money people actually know that they want to solve with AI is that they have to give away a tiny portion of their money pile for the rest of it to keep growing.

The AI vendors, in turn, have a solution that's looking for problems.
 
That's pretty scummy, yeah. I'm not familiar with the actor so didn't recognise the voice, but you really shouldn't have made a voice matching someone IRL without their permission. Dunno why they did at all, honestly - the human intonation and speech patterns alone GPT4o have demonstrated would be enough to impress folk imo.
 
I imagine OpenAI settles before discovery because that would almost certainly involve revealing their training data, which in turn would open them up to even more lawsuits.
 
DracoDracul said:
I imagine OpenAI settles before discovery because that would almost certainly involve revealing their training data, which in turn would open them up to even more lawsuits.
Click to expand...
Click to shrink...
Like a lot of rich people, Altman probably thinks a fine is just a price you pay to do something, and a settlement is just a fine with a fancy name. How much could a mere multimillionaire like Scarlett Johannson possibly want in order to walk away? Thirty, fifty million? What's that compared to the thrill of cosplaying a niche movie?
 
Microsoft is introducing Copilot+, which requires PCs that can run AI locally.

Microsoft unveils Copilot+ PCs with generative AI capabilities baked in

https://www.engadget.com/microsoft-unveils-copilot-pcs-with-generative-ai-capabilities-baked-in-170445370.html said:
To be dubbed a Copilot+ PC, a system will need to deliver at least 40 TOPs of NPU performance and have at least 16GB of RAM and 256GB of storage. Qualcomm claims the Snapdragon X Elite delivers up to 75 TOPs overall. But the pure specs matter less than what Microsoft is able to actually do with the hardware.
Click to expand...
Click to shrink...

One of the Copilot+ PCs is the new Microsoft Surface Pro, which has the NPU to handle Copilot+ built into the CPU, and does not have a discrete GPU. Perhaps this is the way to go about it - enable consumers to utilise AI without worrying about things like Internet connections, subscription fees and whether there is a discrete GPU onboard.

(Memory bandwidth is not too big an issue for locally-run models if you have DDR5 or LPDDR5X memory, especially since most of these will be fairly small if Llama 3 8B and Phi-3 Mini are any indication. Memory capacity will probably be the bigger issue here, as it directly determines whether you can even load the model to begin with.)
 
For comparison purposes, the nvidia 4090 does about 100 TOPS.

Mind you, that's at fp16; LLMs and similar things are typically heavily quantized, so those 75 TOPS are probably at something inane like 4-bit quantization. Which is probably fine—a 70B parameter model quantized to 4 bits does much better than a 35B model quantized to 8 bits, even though they take up the same amount of memory and inference time—but it's worth noting that Nvidia is also working on quantization, and the 50xx series is virtually guaranteed to support that.

As it stands, "fp16" is already quantized down from fp32. It's one of the big advantages the 30/40 series has over the 20 series—halved memory usage and doubled inference speed, on a GPU that's already faster than the 20xx...
 
Maybe I'm biased towards the biomimetic style of engineering (being a biological entity myself) but I think they'd be better off getting neuromorphic chips going given that spiking neural networks are orders of magnitude more power-efficient. It would also be a step towards a vision of, like, Star Wars-style droids instead of these vast corporate data centres.
 
What they desire is President John Henry Eden, not C-3PO, so they're going for the ZAX form factor, not the droid one.
 
Why would end-users want a ZAX-sized AI? It's too cumbersome for many of the desired applications. Like anything to do with robot dogs, robot companions, remote-controlled vehicles (think anything to do with robot waiters and food delivery) and so on.

Even Microsoft seems to be branching out in this direction. After announcing the Copilot+ PCs, they have announced a Phi-3 variant that can be run on the NPU:

Microsoft introduces Phi-Silica, a 3.3B parameter model made for Copilot+ PC NPUs

https://venturebeat.com/ai/microsoft-introduces-phi-silica-a-3-3b-parameter-model-made-for-copilot-pc-npus/ said:
A Microsoft spokesperson tells VentureBeat that what differentiates Phi-Silica is "its distinction as Windows' inaugural locally deployed language model. It is optimized to run on Copilot + PCs NPU, bringing lightning-fast local inferencing to your device. This milestone marks a pivotal moment in bringing advanced AI directly to 3P developers optimized for Windows to begin building incredible 1P & 3P experiences that will, this fall, come to end users, elevating productivity and accessibility within the Windows ecosystem."
Click to expand...
Click to shrink...

Microsoft must be taking the idea of locally-run AIs very seriously to make these announcements.
 
What's Microsoft doing with this push for local LLM execution hardware?
Article:
At a Build conference event on Monday, Microsoft revealed a new AI-powered feature called "Recall" for Copilot+ PCs that will allow Windows 11 users to search and retrieve their past activities on their PC. To make it work, Recall records everything users do on their PC, including activities in apps, communications in live meetings, and websites visited for research. Despite encryption and local storage, the new feature raises privacy concerns for certain Windows users.

"Recall uses Copilot+ PC advanced processing capabilities to take images of your active screen every few seconds," Microsoft says on its website. "The snapshots are encrypted and saved on your PC's hard drive. You can use Recall to locate the content you have viewed on your PC using search or on a timeline bar that allows you to scroll through your snapshots."

At first glance, the Recall feature seems like it may set the stage for potential gross violations of user privacy. Despite reassurances from Microsoft, that impression persists for second and third glances as well.


Okay, even assuming they don't fuck it up and start sending "anonymized" data back to the mothership: having a local database of everything you've done on your computer is a high-priority target for scammers, abusive spouses, controlling parents... Or all the issues of if it creates legally awkward databases of HIPPA information et. all!

Don't worry, it doesn't record passwords. In Microsoft Edge. Or DRMed content, because the computer doesn't exist to serve you. I'm sure it'll be as easy to keep disabled as Microsoft Edge. :p

Or to quote Molly White on Mastodon "back in my day we called this spyware".
 
gourdcaptain said:
Okay, even assuming they don't fuck it up and start sending "anonymized" data back to the mothership: having a local database of everything you've done on your computer is a high-priority target for scammers, abusive spouses, controlling parents... Or all the issues of if it creates legally awkward databases of HIPPA information et. all!

Don't worry, it doesn't record passwords. In Microsoft Edge. Or DRMed content, because the computer doesn't exist to serve you. I'm sure it'll be as easy to keep disabled as Microsoft Edge. :p

Or to quote Molly White on Mastodon "back in my day we called this spyware".
Click to expand...
Click to shrink...

I hope the open-source nature of the models and UIs means people can quickly port all of these to a more... local operating system to run LLMs.

(Remember how Microsoft is making hard to install Windows 11 locally without a Microsoft account? I don't think Copilot+ PCs will be any nicer here, unless the initiative literally gets back-ported onto Windows 10.)
 
gourdcaptain said:
having a local database of everything you've done on your computer is a high-priority target for scammers, abusive spouses, controlling parents
Click to expand...
Click to shrink...
I did see that and think it'd be a tad problematic if the bot constantly recorded your computer activity. Lots of these new AI features don't seem to be things the user triggers on command, which doesn't thrill me - what if I don't wanna use X right then? Or have secure info or my web browsing history easily accessed?

I'm fine with LLM models doing stuff, I just want to be the one to have it activate and do X.
 
You must login or register to reply here.
Remove ads
Back
Top