this post was submitted on 08 Jun 2023
26 points (100.0% liked)

General Programming Discussion

185 readers
1 users here now

A general programming discussion community.

Rules:

  1. Be civil.
  2. Please start discussions that spark conversation

Other communities

Systems

Functional Programming

Also related

founded 5 years ago
MODERATORS
 

Idea: Scrape all the posts from a subreddit as they're being made, and "archive" them on a lemmy instance, making it very clear it's being rehosted, and linking back to the original. It would probably have to be a "closed" lemmy instance specifically for this purpose. The tool would run for multiple subreddits, allowing Lemmy users to still be updated about and discuss any potential content that gets left behind.

Thoughts? It's probably iffy copyright-wise, but I think I can square my conscience with it.

you are viewing a single comment's thread
view the rest of the comments
[–] fruitywelsh@lemmy.ml 4 points 1 year ago* (last edited 1 year ago)

Well there is a tool that uses the API for this purpose https://github.com/rileynull/RedditLemmyImporter Maybe one of the little niches carved out for free API use, like for bots or something might work for that, but definitely should work now.

Another interesting idea is for getting historical data, pulling it from the archive.org project from the project r/datahoarders are doing now.