this post was submitted on 24 Jun 2023
50 points (100.0% liked)
Reddit Migration
458 readers
1 users here now
### About Community Tracking and helping #redditmigration to Kbin and the Fediverse. Say hello to the decentralized and open future. To see latest reeddit blackout info, see here: https://reddark.untone.uk/
founded 1 year ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
One of the reasons they are doing this is because of the large language models being implemented. These companies are using Reddit to train the models. The reason is because of the voting on replies. Where else can you get millions of questions being answered with actual humans saying how good a response is?
The big boys in the current AI space will definitely pay for the API. They'll likely pay a lot for it as well.
Why pay the bloated and gouging costs for API access when you can just write a web parser and scrape the site the old fashioned way?
Scrapers can easily be disabled. Reddit won't look the same obviously. But this isn't a real obstacle.
then the scrapers start using residential proxy botnets
Then you just force them to change the syntax repeatedly and scraping will break with regular occurrence. Scraping is extremely fragile and not easily adaptable without human effort which costs money.
They may not need to. Already trained, already got the data that they need. Going forward they can just continue training with the input from
The users (all the folks talking to ChatGPT directly for example).
There is no reason other apps need to be swept up in the same cost structure as LLM enterprises.
Exactly, the LLM excuse is just that, an excuse to purge 3rd party apps and push ads/get user data that is otherwise unavailable to them.