this post was submitted on 12 Nov 2023
0 points (NaN% liked)

Homelab

22 readers
1 users here now

Rules

founded 1 year ago
MODERATORS
communick@selfhosted.forum
rglullis@communick.news
0
Best setup to replace large cloud storage providers (alien.top)
submitted 1 year ago by Knufle@alien.top to c/homelab@selfhosted.forum
2 comments fedilink hide all child comments
 

Hi everybody, I hope you're all doing fine.

First of all, if this post is not a fit for the subreddit, just let me know and I'll delete it, but I figured it's okay since I saw some similar ones.

I have a goal which is:

I wanna setup a reliable, large (min 4TB), fail-safe, redundant, protected, capable, and backed-up storage to mostly replace Google Drive/Photos but also to use as a means to store some stuff to watch via Plex/Jellyfin and help to archive some stuff that gets lost in history as other people have done. That is if mixing these types of data is okay for my setup ofc.

I'll be using Google Drive/Photos as my main example/concern moving forward, but anything I mention about them is also true for all the other data like archives or everything else.

The thing now is I'm trying to research/study the topic in order to set it up the proper way, what I mean by that is, I don't wanna dump Google Drive/Photos just to have my data corrupted 2-5 years down the road, I wanna trust my personal server as much as I trust these services, I should add that I'm pretty new to the whole "storage server" topic, if that's even a thing.

So, with all of that out of the way, here's what I already know, and please feel free to point out things that are wrong or that I just missed.

  1. First of all, the drives themselves, apparently the two most trusted brands are Western Digital and Seagate, I'm not exactly sure if there's one that's better than the other, even though I read some studies, I still couldn't point my finger on it, but I'm leaning towards WD more for some reason.
  2. I need at least RAID 1 for data redundancy, essentially a real-time backup of my data, if one drive fails, the other still has all my data.
  3. I'll multiply whatever my target storage capacity is by 2 since I'll be using RAID 1, that's also clear to me.
  4. RAID is not a substitute for proper backups. It's essential to have a backup strategy in place that includes periodic backups to external drives, cloud storage, or a separate NAS. This safeguards against scenarios like data corruption, accidental deletions, or catastrophic server failure.
  5. I should use high-quality, reliable HDDs designed for 24/7 operation (NAS or enterprise-grade).
  6. The probability of data loss with RAID 1 is quite low for typical drive failures. However, it doesn't protect against other potential data loss factors, such as accidental deletion, data corruption, or multiple drive failures in a short time frame.

Alright, so these are the things I know/have learned in the last couple of days, now for the ones I don't know/am not sure about I'd deeply appreciate your insights/suggestions/comments.

  1. Should I even use RAID 1, or should I use something else? I'm still very new to the topic so I'm wondering about it.
  2. In case RAID 1 is enough for my use case and I only need 2 HDDs, would this enclosure be good enough? https://aliexpress.com/item/1005005685846728.html
  3. What even is ZFS? I know it's a file system, but should I be using it on my drives?
  4. What category/types of drives should I be looking for specifically? Not talking about brands here, I know there are some acronyms for drives as well but I don't really remember them and don't know which ones would fit my use case.
  5. I know I mentioned NAS but I'm pretty new to it, should I use some software to set it up for the drives as well, does that make sense?
  6. Is there a table listing the most reliable drives where you can mix and match them, filter by the best value for a certain type of application, or something like that?
  7. Same as 6, but for RAID in this case? Is there a place that lists the safest configuration or something like that?
  8. What exactly would be an ideal backup of my storage? Just the same physical drives but in a separate enclosure that copies the original data from time to time? Any insights on a proper way to set up that as well?
  9. Regarding the other types of potential data loss, is there a good way to protect my storage from them as well? Should I be following some standard guidelines for that? Is there a list?

I should add I already have a server set up and running a couple of docker containers like Nextcloud among them and I can access it through a VPN, I've mostly figured that part out, it runs Debian 12 too, I don't know if that matters but I just thought I'd drop it.

Alright, as I said before, my main concern is I wanna have something super reliable so I can safely dump Google Drive/Photos completely, I'll be so glad if I'm able to do that, thank you so much in advance for your help.

TL;DR I wanna set up the storage part of my server in a proper and reliable way, could you help me?

top 2 comments
sorted by: hot top controversial new old
[โ€“] aetherspoon@alien.top 0 points 1 year ago (1 children)
  1. RAID-Z2 or RAID-6 is probably more than sufficient, as that has two parity drives for a pool of drives. RAID-1 does make more sense if you are only using two drives though.
  2. It can work, but generally USB enclosures aren't the greatest approach for what you're trying to do. Do you have space for your two internal drives instead?
  3. I mean, it is exactly that - a filesystem. It is a rather advanced one, and it does have some features that would be useful for you, like snapshots so you can go back after messing up a file. It also tends to run fast at the expense of it being a bit hard to expand and eating more RAM than you'd expect.
  4. Generally, anything but an SMR drive.
  5. I'm not quite sure what you mean?
  6. PCPartPicker, sort by price-per-TB?
  7. There are only a limited number of RAID configurations, if that's what you mean?
  8. It depends; are you disciplined enough to actually run backups, or are you a normal person that needs automatic cloud backups? :P But seriously, do you think you'll run manual backups often enough?
  9. ZFS and btrfs both do a decent job at handling such things.
[โ€“] Knufle@alien.top 1 points 11 months ago

Hey, thank you so much for getting back to me, I've been researching and learning more about some of the stuff you mentioned, so let me follow-up to the ones I'm still understanding.

  1. I've been taking a look and if RAID-Z2 and RAID-6 do make more sense because of even more safety, I'd be definitely willing to go that route. With that said, considering I'd buy the drive enclosure I mentioned, for RAID-Z2 or RAID-6 I believe I'd need 2 enclosures setup as JBOD since it wouldn't support RAID-Z2 or RAID-6 at the hardware level, so I'd just set it up at the software level, does that make sense? I'd also need one extra for the backup as far as I'm aware, I believe that would make sense?
  2. I don't have the space for two internal drives, in fact, it's a laptop I'm setting up to be a server, so the space is very limited, the drive bay itself only supports 2.5 inches drives. Would it be THAT BAD to use a USB enclosure? Are there some evidence regarding that that you know of?
  3. Is any other type that is not SME recommended though? Or just any?
  4. I'm actually very disciplined and I'd be able to run my backups following a set repeating schedule no problem, but like, aren't there softwares for automating that? I was hoping so.