UDM-PRO, USW-Aggregation, USW-Enterprise-24-POE, U6-LR… build a server with i5/32GB NVMe boot drive, then some RAID drives… I took out a loan in this scenario as $1,000 wouldn’t cover my entire rack getting blown up.
this post was submitted on 26 Nov 2023
1 points (100.0% liked)
Homelab
22 readers
1 users here now
Rules
- Be Civil.
- Post about your homelab, discussion of your homelab, questions you may have, or general discussion about transition your skill from the homelab to the workplace.
- No memes or potato images.
- We love detailed homelab builds, especially network diagrams!
- Report any posts that you feel should be brought to our attention.
- Please no shitposting or blogspam.
- No Referral Linking.
- Keep piracy discussion off of this community
founded 1 year ago
MODERATORS
What is your job? Do you have exposure to life cycled hardware?
Bought a Dell R630 from ebay for a decent price, but I wish I've had spend more on larger capacity hard drives. I bought a bunch of old 600GB HDD running RAID 10 that right now im afraid to replace them.
3 optiplex 7040s micros - put 32gb ram and a 2tb ssd in each and call it a day
I figure you need to buy networking equipment too?
I would only add a single SFF so that I can fit a couple of big 3.5 disks for my backup and data hoarding needs.
Other than that, yeah.... Micro/tiny/micro is the way
This is what I did with 3060s. Eventually added a 4th, because... well... 3 is less than 4 and I had an empty slot in my rack.
For 1k i would start with a Unifi UDM-Pro, a Intel NUC and a Synology NAS.
Honest question... Why people with knoedge on how to do one, buy a Nas like synology? Are you not just paying double or triple for the same result you could have if making the NAS from scratch?
We use Synology at work to avoid paying CALs on a WS VM
Reliability and lower power consumtion than most Frankenstein-DIY cheap stuff recommended here ;)
No, you are not paying anywhere near double or triple.
My Synology came in at ~$750 for the chassis and 2 8TB IronWolf drives.
A custom build with TrueNas was coming in at over $1k.
Hm, yeah maybe I just don't know the pricing/cost of a Synology then.
In my country just the price of a 8Tb IronWolf drive costs almost 1 entire month of the minimum wage here.
The cheapest Synology NAS available here is the DS223J, and it comes with no drives included and costs 80% of two months of minimum wage.
It's way cheaper to repurpose old hardware or buy from AliExpress and make a DIY build, there is no comparison and also I have no idea of what "custom build" are you mentioning, as most NAS builds I've seen are pretty cheap as you don't need much horsepower and DDR4 memory has low prices nowadays.
I regret getting a UDM-Pro and recently swapped it for an n5105 OPNsense box. Luckily they keep their value, so I didn't lose any money on the UDMP.
Why do you regret that choice?
I have a UniFi system: APs, switches, CKG2, Gateway. I’m looking to add CKG2+ and some POE cameras
After my past Ubiquiti experiences I can't agree on the UDM...
2-3 second hand small form factor PC's running Proxmox, cheap 2 bay Synology NAS for backups.
Where would your primary storage live?
2 bay?
N5105 nas board, 32-64gb of ram, 1x 500gb nvme SSD, some sort of case, and a bunch of HDDs, I like the 8tb ironwolfs, they are cheap enough, but large enough.
Maybe the n6005 if you can find it. But it's a great server, handles most selfhost stuff. I run Ubuntu server on it, it's just the cleanest and easiest to use, no GUI needed.
What's nice is it's super low power, and cheap. So you can eventually migrate to a more powerful Proxmox server, on minipcs, like NAB6, than just turn the n5105 into a TrueNAS server, and even duplicate it for backups, and triplicate (if you are really feeling it), for redundancy. Getting a 2nd and 3rd Proxmox minipcs enables HA on VMs. So yea. That's my goal. ATM I gotta migrate to the Proxmox.
I loved migrating to 3 nucs from a 2015 synology, so think you are 100% correct. (It allowed me to use TB networking for a 26Gbe ceph network)
TB = Thunderbolt?
Same but with a N100 motherboard. Asus and Asrock have some ITX boards with this chip.
At least 2 mini desktops with as much RAM and ssd that I can get I'm it. Running proxmox and truenas and then setting up my jellyfin, homeassistant, and the rest will be a playground. I am a simple man
I would buy a single $1000 42u rack…
And do what with it? if your budget was $1000 and you only bought a rack, it'd be empty.
That's it. A $1000 rack and call it a day! Done.
All used: 2019ish Intel NUC i7, 32-64GB RAM, run ESXi 7, 4 Bay QNAP or Synology with a Celeron, 8TB spinners, TP-Link ER605, an Omada POE switch, and an Omada AP.
You end up with a great setup for VMs, a reliable Plex server using the NAS CPU, multi-WAN, rock solid VPN, and a UniFi/Meraki like experience, and you don’t notice it on the electric bill, your ears, the shelf, or the room temperature.
This doesn’t differ at all from my existing setup. My only regret was not starting with 64GB of RAM on the NUC instead of the 32GB I started with.
I will just get a nice amd board with ipmi and dump a good Ryzen cpu, Any linux, be debian or any Rhel based distro or even Proxmox and tons of drives plus a few nvme raids. Pretty much about that
- Supermicro H11SSL-N6 with an Epyc 7551P with 128G memory - €600
- PSU - €60ish
- Pile of refurb 4TiB disks - €100
- Mikrotik hAP ax² - €80
- HP Procurve 2848 - €40
- Misc gubbins - €180
There's a server, networking gear, and storage. I can sort the rest out later.
$1k wouldn't get me started for the electrical runs and cabinets for the hardware.
Just chiming in that the consensus on Mini PC clusters is pretty cool.
Completely agree. That's where it's at!
I have a rack full of R710s that barely get used anymore because energy is so freaking expensive. I’d either do everything in the cloud or use lots of low powered machines at home.
Will do almost what i have now: compact (ITX/mATX) board with C612/2600v3/v4, maxxed with memory. SAS board/NVME/10G if you want/need. Silent and efficient for 24/7
I would do pretty much what I do now with two mini PCs and my desktop PC running background services in a three node cluster. I change my mind too often though and just did a bit of a rebuild over the holiday, so by next weekend I may have a completely different goal.
I having considered replacing the desktop with a laptop for more portability.
I would also not mind getting a 2.5 Gbps switch. I have all 2.5 Gbps devices on the network except the switch which is a little silly.
A bigger NAS with more drive bays
I'd separate my storage and put that in its own server.
Then, I'd probably go for multiple low energy sff "servers" instead of one powerfull one.
Id go to https://labgopher.com/ and find some cheap, newish hardware. Would not buy anything brand new.
I would buy a single n305 mini pc with at least 2 2.5gb nics, and maybe a godlike pc for vm's to play around with
Buy a new N100 microPC with 32GB RAM and m.2 drive, Sabrent 5-bay USB_3 DAS, a couple 10TB drives. Easy low-power single box home server with room to expand. You also could add a good switch and box of Cat8 (cable always > WiFi)
Spend the rest on another hobby!
A couple of gen9 Proliant servers. They're cheap, easy to source, plenty powerful for a homelab, have surprisingly good power management, and they're much quieter than previous generations (because of the power management). If you go with LFF drives, you can find surplus ones which have plenty of room for homelab stuff. SAS drives are so cheap, I've bought enough extra drives to replace any which fail.
For instance: https://www.ebay.com/itm/284061636798 is less than $200 with dual CPUs, a RAID contoller, and iLO for out of band management. You can source memory on eBay for cheap (for instance https://www.ebay.com/itm/266287238575), and as I mentioned, SAS drives are so cheap they're almost disposable (https://www.ebay.com/itm/225874909271).
So total cost for one of these servers with 128GB memory and four 8TB (24TB usable with RAID 5) drives would be $463.48. You could spin up two of them for less than your $1,000 budget and be able to do a BUNCH of cool stuff with them. Or you could just pack one with like 512GB memory and do everything on one server with virtual machines.
On my gen 9 DL380s with 12 4TB drives, I'm getting ridiculous disk speeds:
[root@neuromancer vms]# dd if=/dev/zero of=bigfile bs=16M count=1024 oflag=direct status=progress
16475226112 bytes (16 GB, 15 GiB) copied, 10 s, 1.6 GB/s
1024+0 records in
1024+0 records out
17179869184 bytes (17 GB, 16 GiB) copied, 10.3636 s, 1.7 GB/s
So over a gig and a half per second direct I/O writes. I spin up VMs on these servers in literally minutes, and I've got enough memory to have dozens of virtual machines. I have RHEL, Fedora, and Windows machines (my wife is a Microsoft sysadmin, she tests stuff on those).
The downside is that even with good power management, they do draw a fair amount of power and generate a fair amount of heat. I have three of these in my home office, and during the summer, it kept my office slightly warmer than I like.
For the OS, I use the free developer edition of RHEL - those skills are very marketable. https://developers.redhat.com/. I use RHEL for my VMs so I can play with stuff like NFS services, the automounter, user management, even stuff like OpenShift cluster members as VMs. I've learned a lot using my homelab, and it's helped my career a lot.
That's about $1000 more than most people have.
My suggestion is invest in networking equipment but it will not cost you $1000. Maybe a switch and a couple mini PC's and if you have to buy used retail it's maybe $200. If you want to get into NAS and streaming than you're looking at spending some money because reliable, preferably fast storage is a must and expensive
What do you guys do with these home labs?
get a synology nas instead of a giant enterprise server. I only boot it when i need to use it as the power consumption is so high.
, but $1,000 in cash
not sure how this would help me, I've spend 10k or more, but I could get a t-shirt I guess?