Remove ads

Scientia Weaponizes The Future

Reader mode
Remove ads
Christmas Update
  • Informational
    • Merry Christmas, everyone!

    I would like to offer you all an update. I'd hoped to have the next chapter ready for today, but it's not quite fully baked and I don't want to put out something mediocre after a long wait. It's about 4,500 words right now, with the remainder fully outlined, and covers Scientia's confrontation with the Simurgh after the events of the last chapter. I hope to finish it in the coming months, at worst. By the end of my winter break next month, optimistically.

    I apologize to all of you for the long hiatus Scientia has been going through. I've discovered that I'm one of those strange people that actually enjoys law school despite all the stress involved, but it's definitely been all-consuming of my energy this past year and a half.

    My life has been going shockingly well, and I hope you'll forgive me for sharing my joy with you all. My class rank is comfortably in the top ten, and I pulled all As this past semester despite law review eating my life, which is not something I ever expected to pull off in law school under any circumstances. With the law school curve, Bs aren't too difficult, but there are very few As to go around. A full slate of them is utterly incredible, and I'm still in shock. Some of them were difficult classes, too; my patent law and policy exam in particular was an eight hour long ordeal and one of the most difficult things I've ever attempted. I suppose it must have gone better than I thought it did at the time.

    This past summer I externed with a federal judge and got to write some of the court's decisions, which was a pretty incredible experience. For next summer I landed a dream job doing patent litigation at a biglaw firm in San Francisco with what seems like a nice work environment and reasonable hours (for biglaw). If there are any readers out there in the Bay area who would like to meet for coffee and talk, or offer advice, let me know in a message. I could use some local friends, and maybe summer housing tips.

    My law review note has sadly been eating most of the writing time I would have liked to throw towards Scientia. Funnily enough, it's about artificial intelligence. (If only I had charges of origin civilization knowledge or Prometheus to help me write it...) It turns out the science fiction author's skill of explaining how fantastical technologies work in an easily understandable way is really helpful for explaining how AI actually, really works in a way that is understandable to lawyers without resorting to vague metaphor. I'm pretty happy with how it's come out so far, and the note's central thesis about intellectual property protection as applied to AI comes to an interesting conclusion. It's a struggle, but I hope it'll be something to be proud of when I'm done.

    But that's more than enough about me; I'm happy and doing well, and I just wanted you all to know that, and that I haven't forgotten about this story and will finish it, no matter how long it takes.

    My absolute best wishes go out to all of you. I love you all, and wish you warmth, good food, friends, and lots of big joy-filled belly laughs. May your day be bright and full.
     
  • Informational
    • Happy Holidays! I'm glad you're doing well, and I am cheering for you from across the computer screen! ^.^
     
    TaliesinSkye said:
    explaining how AI actually, really works in a way that is understandable to lawyers without resorting to vague metaphor. I'm pretty happy with how it's come out so far, and the note's central thesis about intellectual property protection as applied to AI comes to an interesting conclusion
    Click to expand...
    Click to shrink...
    Sounds like a Intresting read, is it published anywhere?

    I'm building on ChatDev a virtual software development department in my thesis work. Aiming to get it to make programs larger than 400 lines and characterize hyperperamiters like LLMs impact on maximum output size.
     
    Yahoo! A hiatus takes as long as it needs to take, but it's always pleasant to see signs of life. Glad to hear things are going so well, happy holidays!
     
    SufficientEnergy said:
    Sounds like a Intresting read, is it published anywhere?

    I'm building on ChatDev a virtual software development department in my thesis work. Aiming to get it to make programs larger than 400 lines and characterize hyperperamiters like LLMs impact on maximum output size.
    Click to expand...
    Click to shrink...
    Not published, I'm still writing it. The general idea is that no form of IP protection, as it currently exists, will actually effectively protect AI models. Patents run into subject matter eligibility issues under § 101, and AI models are way too reverse engineerable for trade secrecy to hold. (It's possible to treat the model like a system of nonlinear equations and then solve for the nodes and parameters by querying the model for data. You can pull a substantial portion of training data out at the same time.) Copyright probably doesn't work because there's insufficient human authorship of the model itself, which is just the output of an algorithm, and the last resort option of contract law has too many limitations to be effective. For good or ill, the billions people are spending on AI models will probably produce things that they don't really own. Things should be interesting in the IP world in the coming years.

    The portion of the paper that's a description of how AI works is mostly done in first draft, I could share that if you're interested.
     
    TaliesinSkye said:
    Not published, I'm still writing it. The general idea is that no form of IP protection, as it currently exists, will actually effectively protect AI models. Patents run into subject matter eligibility issues under § 101, and AI models are way too reverse engineerable for trade secrecy to hold. (It's possible to treat the model like a system of nonlinear equations and then solve for the nodes and parameters by querying the model for data. You can pull a substantial portion of training data out at the same time.) Copyright probably doesn't work because there's insufficient human authorship of the model itself, which is just the output of an algorithm, and the last resort option of contract law has too many limitations to be effective. For good or ill, the billions people are spending on AI models will probably produce things that they don't really own. Things should be interesting in the IP world in the coming years.

    The portion of the paper that's a description of how AI works is mostly done in first draft, I could share that if you're interested.
    Click to expand...
    Click to shrink...

    As someone who is occasionally writing about AI for money (mostly hurling ordure at the VCs responsible - [1] [2]) this is the funniest possible outcome for AI as a VC toy, and I extremely much look forward to your paper when it comes out.
     
    divabot said:
    As someone who is occasionally writing about AI for money (mostly hurling ordure at the VCs responsible - [1] [2]) this is the funniest possible outcome for AI as a VC toy, and I extremely much look forward to your paper when it comes out.
    Click to expand...
    Click to shrink...
    It is funny, yes. I'll be happy to share the paper when it's ready, although it will probably be late spring before I know if it's publishing. If not, I'll throw it up on arXiv myself and share. It's rare for student law review notes to get published — roughly 5% odds on the whole — but this is a novel treatment of a topic that's very in right now, so I think my odds are good. The number of law students with computer science degrees writing law review articles is vanishingly small, so I also don't have a lot of competition for AI authorship by people who have some limited idea of what they're talking about. I have a four year CS degree and I feel like I'm operating at the edge of my competence or beyond, wading through some of these papers.

    Graviator said:
    Is this still possible if the model's internal shape isn't known?
    Click to expand...
    Click to shrink...
    Yes. Completely black box attacks are entirely possible. Based on the papers I've read I think you just need an idea of what category of AI you're dealing with, like whether it's a pure transformer or a generative adversarial network or whatever. The actual architecture, like the number of parameters or nodes or number of hidden layers, is unnecessary. You can extract all of that as you build out your copy.

    This is a good paper. It cites some other good papers in the evolution of model extraction attacks, too. Tramèr came up with the fundamental approach all the way back in 2016, and it's just been refined since then. There are proof of concept attacks even on big models like ChatGPT. There's also a whole niche hobby where people build their own small AIs and then improve them a bit by using large AI output to hone the smaller model, which is a type of extraction that doesn't aim for a complete copy. So far those efforts are better at capturing the style of large models than the accuracy of their results.

    Baughn said:
    Or if you can't set the sampling temperature to zero?
    Click to expand...
    Click to shrink...
    Wat. I think I missed a joke?

    Omegaprime02 said:
    First off, Happy Holidays!
    Second, glad you're doing so well!
    Third, from a family of lawyers, keep it up, we need more good people in the profession!
    Click to expand...
    Click to shrink...
    Thank you so much for this. I'm trying my best. I sometimes wonder how many lawyer readers I might have, it's good to know there are some out there.
     
    Last edited:
    Training a model to imitate the output of another model isn't new, I came across a use of the technique back in 2007 and it's undoubtedly older. However, I would draw a distinction between that and reverse-engineering the model's internal structure.
     
    Graviator said:
    Training a model to imitate the output of another model isn't new, I came across a use of the technique back in 2007 and it's undoubtedly older. However, I would draw a distinction between that and reverse-engineering the model's internal structure.
    Click to expand...
    Click to shrink...
    That's interesting, I hadn't seen any sources going back that far. That's before the current crop of AI techniques. You're right that there's a big difference in fidelity, a true copy vs. a poor imitation. They're both forms of reverse engineering in some sense, though.
     
    Graviator said:
    Training a model to imitate the output of another model isn't new, I came across a use of the technique back in 2007 and it's undoubtedly older. However, I would draw a distinction between that and reverse-engineering the model's internal structure.
    Click to expand...
    Click to shrink...
    I have a *fun* story about training a model on the output of another model.

    At a company that will be left unnamed, a manager decided that it would look really impressive for his unit of the company to roll out a shiny new AI tool to solve an already solved problem. Unfortunately, he didn't have the expertise on his team, didn't want to pay to put together training data, and wasn't smart enough to be able to consistently approach the problem in a reality-based way.

    So he hired a couple people who claimed to know a little bit about AI and proceeded to ask a lot of them and not give them the resources they needed to solve the problem correctly. And of course they needed to solve the problem anyway, and at some point one of them made the mistake of mentioning that unsupervised techniques exist.

    I don't know how exactly it happened, but they ended up using an unsupervised model to generate labels that the horrible ensamble of supervised models they had wanted to build in the first place could then be trained and evaluated on. All without ever doing anything to confirm it actually worked.

    They produced an over-engineered mess, and had the numbers to say it had almost perfect recall and precision... with respect to labels geneated by some shitty unsupervised model which had never been evaluated against actual ground truth labels. The manager was happy, everyone got paid, and the team got out of dodge before it could catch up with them.

    I was part of one of the teams brought in to fix it once an internal AI oversight body looked at it, screamed in horror and got it pulled from production. Turned out, once we finally got the resources to generate some labels to compare against, the whole thing was worse than useless and could be significantly improved on with a couple hard cutoffs.
     
    TaliesinSkye said:
    Wat. I think I missed a joke?
    Click to expand...
    Click to shrink...
    Well, I'm only guessing about how techniques like that work. But the output from the model is a probability distribution over tokens, not a single token. If you can see the distribution then I suppose it makes some sense that you can back-calculate the weights of the model, given sufficient samples and assuming you know the full set of hyperparameters.

    But if what you get from GPT-4 is a randomized output. Just one of the many possible tokens. Though, I don't know that predictably getting the rank-1 token would actually be useful... it doesn't seem like the random output would be either? Unless you're going to redo the same call thousands of time to get a better approximation of the total distribution, which sounds unfeasible to me?
     
    Baughn said:
    Well, I'm only guessing about how techniques like that work. But the output from the model is a probability distribution over tokens, not a single token. If you can see the distribution then I suppose it makes some sense that you can back-calculate the weights of the model, given sufficient samples and assuming you know the full set of hyperparameters.

    But if what you get from GPT-4 is a randomized output. Just one of the many possible tokens. Though, I don't know that predictably getting the rank-1 token would actually be useful... it doesn't seem like the random output would be either? Unless you're going to redo the same call thousands of time to get a better approximation of the total distribution, which sounds unfeasible to me?
    Click to expand...
    Click to shrink...
    It works with random queries and output. I'm fuzzy on how the math works, I'm afraid. You can read the original paper here. Let me know if you figure it out well enough to explain it.
     
    TaliesinSkye said:
    For good or ill, the billions people are spending on AI models will probably produce things that they don't really own.
    Click to expand...
    Click to shrink...
    Not owning some piece of natural scenery in the physical world doesn't stop photographers from owning a photograph. Selecting which of the AI's outputs are worth posting in public is arguably at least some amount of the same *kind* of authorship as picking where and when to take a photo, even if it's not the quite same degree.
     
    Random832 said:
    Not owning some piece of natural scenery in the physical world doesn't stop photographers from owning a photograph. Selecting which of the AI's outputs are worth posting in public is arguably at least some amount of the same *kind* of authorship as picking where and when to take a photo, even if it's not the quite same degree.
    Click to expand...
    Click to shrink...
    Sure, but legally this is somewhat settled with the case of monkeys taking photographs, presumably.
     
    Random832 said:
    Not owning some piece of natural scenery in the physical world doesn't stop photographers from owning a photograph. Selecting which of the AI's outputs are worth posting in public is arguably at least some amount of the same *kind* of authorship as picking where and when to take a photo, even if it's not the quite same degree.
    Click to expand...
    Click to shrink...
    I think the situation with AI is more like making a photocopy of a document and then trying to claim you wrote the document and it is your original work, and less like claiming ownership of a photograph of a tree.
     
    TheatrclBarristr said:
    I think the situation with AI is more like making a photocopy of a document and then trying to claim you wrote the document and it is your original work, and less like claiming ownership of a photograph of a tree.
    Click to expand...
    Click to shrink...

    Whether or not a work contains originality of its own is independent of the extent to which it infringes on the copyright of some other work. We are, let's not forget, in a thread about a fanfic.

    BurnBright said:
    Sure, but legally this is somewhat settled with the case of monkeys taking photographs, presumably.
    Click to expand...
    Click to shrink...

    The photographer in the analogy is the person deciding which outputs are successful enough to be worth posting in public and which ones are failures, not the AI. Or even making further modifications with the raw AI output as just one component, drawing stuff by hand as the base for img2img, etc.
     
    Last edited:
    Reader mode
    You must login or register to reply here.
    Remove ads
    Back
    Top