this post was submitted on 20 Jul 2023
71 points (100.0% liked)
Technology
378 readers
1 users here now
This magazine is dedicated to discussions on the latest developments, trends, and innovations in the world of technology. Whether you are a tech enthusiast, a developer, or simply curious about the latest gadgets and software, this is the place for you. Here you can share your knowledge, ask questions, and engage in discussions on topics such as artificial intelligence, robotics, cloud computing, cybersecurity, and more. From the impact of technology on society to the ethical considerations of new technologies, this category covers a wide range of topics related to technology. Join the conversation and let's explore the ever-evolving world of technology together!
founded 2 years ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's almost certainly because OpenAI is throwing less computing power at it in order to decrease the cost.
It's enshittification, then.
I mean, they've gotta to be blowing absurd amounts of money at it. It's not remotely cheap to build a massively complicated web service at that scale, and eventually the numbers need to start adding up. I'm sure they have several good monetization plans, but not every instance of a business attempting to stop hemorraghing money is a conspiracy. You'd be doing the exact same thing in their shoes.
Enshittification is not a conspiracy because a conspiracy requires communication and planning. Enshittification is just how idiots act when trying to make money.
i wouldn't even say that, it's just the logical end point of capitalism
Yes.
And extra infuriating they want to roll this stuff out after making it LESS reliable.
And there are more and more offline GPT AIs available for free. Now everyone with an above average computer can have their own chatGPT.
I mean an "average" computer would require a pretty beefy set of hardware. I think most of the average local llama's would run fairly decently on a MacBook without issue nowadays (that m3 is going to be a pretty awesome beast). But the quality is pretty reduced even compared to something like 3.5 which most people thought wasn't all that great.
But really, I'm excited about researchers have access to more computer for smaller amounts (see this https://www.chatgptguide.ai/2023/07/20/worlds-largest-supercomputer-for-ai-training-is-out/) currently we have 1T models that are good, but we could pretty soon have 100T models from the open source community. Let's see whether we can scale the hardware needs with the parameter growth so we don't need A100s to run a decent model.
It's still pretty rough to selfhost an LLM. You can get one that's kind of okay on an average computer, but to get a really competitive one running locally at a good speed, you need a huge amount of RAM that is still beyond most average users (VRAM for GPU based projects).
I've been trying to get Vicuna going and the RAM usage is rough, 60gb is suggested, and I've got 64 and I think I need a lot more honestly.