this post was submitted on 12 Jan 2024
39 points (100.0% liked)

Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ

1443 readers
15 users here now

⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.

Rules • Full Version

1. Posts must be related to the discussion of digital piracy

2. Don't request invites, trade, sell, or self-promote

3. Don't request or link to specific pirated titles, including DMs

4. Don't submit low-quality posts, be entitled, or harass others



Loot, Pillage, & Plunder

📜 c/Piracy Wiki (Community Edition):


💰 Please help cover server costs.

Ko-Fi Liberapay
Ko-fi Liberapay

founded 1 year ago
MODERATORS
 

I’ve been looking online for ways to download websites (game wikis mostly), in order to have them all in my collection and ensure that they dont get taken down or changed.

After trying linkwarden, which is fine for singular web pages, one has to manually link each individual page of the wiki in order to make a pdf.

With this in mind, the only other option that I’ve discovered is using wget recursively. Do any you of you have experience with this or reccomend alternative ideas? Any and all help is appreciated.

PS: I will most likely download official game guides which will cover most of the games, but looking for something to cover all my games library.

top 5 comments
sorted by: hot top controversial new old
[–] otter@lemmy.ca 13 points 9 months ago (1 children)

I've used HTTrack in the past

It worked well for the websites that I tried it on and it's pretty easy to run. You could give it a go?

https://en.wikipedia.org/wiki/HTTrack

[–] ShellMonkey@lemmy.socdojo.com 3 points 9 months ago (1 children)

I've got a container set up of this. Drops the output on the NAS and can be accessed from any box on the local net. The only issue is it has a tendency to need the container recycled every so often, like it just gets board sitting there and quits. 🤔

load more comments (1 replies)
[–] redcalcium@lemmy.institute 8 points 9 months ago* (last edited 9 months ago)

SingleFile extension can save a web page into a single html file where all media are neatly inlined inside the file. You'll have to do this manually on each page though, so it's not ideal for saving the whole website.

If you're comfortable running commands in terminal, you can use SingleFile CLI to crawl the whole website, e.g.: single-file https://www.wikipedia.org --crawl-links=true --crawl-inner-links-only=true --crawl-max-depth=1 --crawl-rewrite-rule="^(.*)\\?.*$ $1"

[–] vildis@lemmy.dbzer0.com 3 points 9 months ago

I use grab-site (unmaintained) for full site archival and wget -p -k for simple non-javascript single pages

I've heard good things about HTTrack, SingleFile and Archivebox but don't have any experience with them.

Archivebox looks the most modern and intuitive but is hosted on docker

[–] laserjet@lemmy.dbzer0.com 3 points 9 months ago