datahoarder

181 readers

1 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago

MODERATORS

archivist@lemmy.ml

A Data Hoarder, I Am Not (feddit.de)

submitted 8 months ago by otacon239@feddit.de to c/datahoarder@lemmy.ml

5 comments fedilink hide all child comments

So I’ve been consolidating all of my storage and removing all the duplicates and junk files.

In actual physical storage, this was spread across 12TB worth of hard drives, all partially full.

After everything was said and done, I’m using 1.3TB of space if you don’t include games. ¯\_(ツ)_/¯

This is stuff dating back to 2015. Sometimes it’s actually worth it to just clean up your junk files.

you are viewing a single comment's thread
view the rest of the comments

[–] ivanovsky@lemm.ee 2 points 8 months ago (3 children)

Any good tools to deduplicate files? Got a bunch of images with different names, on different folders, sometimes with different resolutions, spread out across 10 external hard drives that I need to go through...

[–] Lypropos 3 points 8 months ago* (last edited 8 months ago)

Dupe Guru has been a useful GUI on windows. It is also cross platform.

https://dupeguru.voltaicideas.net/

[–] Nomecks@lemmy.ca 3 points 8 months ago

md5sum to get the low hanging fruit

[–] otacon239@feddit.de 2 points 8 months ago

This was done by hand on my end. If I had needed to do it procedurally, I’m not sure what my approach would have been, but the direction I’d probably head in would be to look into finding duplicate MD5 hashes.