this post was submitted on 13 Jul 2023
9 points (100.0% liked)
Technology
378 readers
1 users here now
This magazine is dedicated to discussions on the latest developments, trends, and innovations in the world of technology. Whether you are a tech enthusiast, a developer, or simply curious about the latest gadgets and software, this is the place for you. Here you can share your knowledge, ask questions, and engage in discussions on topics such as artificial intelligence, robotics, cloud computing, cybersecurity, and more. From the impact of technology on society to the ethical considerations of new technologies, this category covers a wide range of topics related to technology. Join the conversation and let's explore the ever-evolving world of technology together!
founded 1 year ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I disagree. I can barely run a 13B parameter model locally. Much less a 175B parameter model like GPT3. Or GPT4, whatever that model truly is. Or whatever behemoth of a model the NSA almost certainly has and just hasn't told anyone about. I'll eat my sock if the NSA doesn't have a monster LLM along with a myriad of other special purpose models by now.
And even though the research has (mostly) been public so far, the resources needed to train these massive models is out of reach for all but the most privileged. We can train a GPT2 or GPT-Neo if we're dedicated, but you and I aren't training an open version of GPT4.
AI is more than just ChatGPT.
When we talk about reinterpreting copyright law in a way that makes AI training essentially illegal for anything useful, it also restricts smaller and potentially more focused networks. They're discovering that smaller networks can perform very well (not at the level of GPT-4, but well enough to be useful) if they're trained in a specific way where reasoning steps are spelled out in the training.
Also, there are used nvidia cards currently selling on Amazon for under $300 with 24 gigs of ram and AI performance almost equal to a 3090, which puts group-of-experts models like a smaller version of GPT-4 within reach of people who aren't ultra-wealthy.
There's also the fact that there are plenty of companies currently working on hardware that will make AI significantly cheaper and more accessible to home users. Systems like ChatGPT aren't always going to be restricted to giant data centers, unless (as some people really want) laws are passed to prevent that hardware from being sold to regular people.
But you can run it.
I've got a commodity GPU and I've been doing plenty of work with local image generation. I've also run and fine-tuned LLMs, though more out of idle interest than for serious usage yet. If I needed to do more serious work, renting time on cloud computing for this sort of thing actually isn't all that expensive.
The fact that the very most powerful AIs aren't "accessible" doesn't mean that AI in general isn't accessible. I don't have a Formula 1 racing car but automobiles are still accessible to me.
If we're just talking about what you can do, then these laws aren't going to matter because you can just pirate whatever training material you want.
But that is beside my actual point, which is that there is a practical real-world limit to what you, the little guy, and they, the big guys, can do. That disparity is the privilege that OP way back up at the top mentioned.
I have no idea what that original commenter's opinion on copyright vs training is. Personally I agree with the OP-OP of the whole thread. Training isn't copying, and even if it were the public interest outweighs the interests of the copyright holders in this regard. I'm just saying that in the real world there is a privilege that that the elites and ultra-corps have over us, regardless of what systems we set up unless capitalism and society as a whole is upended.
At this point we're just bickering over semantics.