Self-Hosted Main

Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.

You don't have to do it, 99.9999%^([citation needed]) of the population don't. If it doesn't interest or excite you then it's not worth it.

Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you'll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That's a more realistic set of expectations.

I've done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn't exist yet. VPNs weren't a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn't you being inadequate, it's you trying to do multiple jobs that each require constant professional development.

Back in the day you had to find a book if you didn't know people

Most blogs and articles I find are about what they are which I already know.

Exactly.. I neither want "what z is" nor all the history behind z, the Z inventors life, his grand parents lives etc. I just want a solution to the problem im having..

Skill issue.

It was pretty easy for me.

Invest some time in understanding Docker containers and then install something like Portainer (for example on top of Proxmox + VM, or ProxMOX + LXC, or some Linux or bare metal) and then installing new apps will become very trivial once you understand how to "create" the containers and how to map the volumes and ports.

As a software engineer of 17 years, you’re only 2 years in, you ain’t hot shit yet and have lots to learn. It can be difficult. Also, you’re probably just not great at searching/finding answers yet. Sometimes you need to look further down the Google results, or comb through closed issues/PRs in GitHub, or read a weird issue of StackOverflow that has a comment that links to another issue with the exact problem.

Researching / finding answers / problem solving / learning new things will be the #1 skill that will get you somewhere.

I've studied automotive and never really graduate, but I love reading manuals since I was a kid and always believe that even I don't know what am I doing as long I can read It's manual, guide, documentation and read support I could get a grasp of it even if it takes more time.

A part that made it easier for me to understand self-hosting is the raspberry pi. It let me thinker self-hosting without worrying about power bill, how can I easily reinstall its OS even if I make bunch of mistake, make different copies of my setup on different sd card without worries of hdd or ssd.

Then from that just upgraded stuff from what I've learned.

I would say the opposite: self hosting nowadays is very easy. Is the multitude of options and configuration possibilities that is daunting. We want too much and this increases complexity.

The responses you get here are from people that have already perfected their setup so take them with a grain of salt.

I mean I have my setup running with Traefik, Crowdsec, Authentik and that's before the request even reaches the application.

2 months ago that was only Traefik.

A year before that I was using Nginx Proxy Manager instead of Traefik because it was easier to manage and understand.

Half the fun is evolving your homelab. Trying to start out with the full stack of things someone suggests is daunting and nearly impossible.

Take things one step at a time. And honestly if you don't understand what the documentation is talking about, YouTube videos are great. I've had to use it lots to understand how Authentik works but now I understand the docs

As with everything in life you need to crawl before you run. And as someone mentioned before skills issue. You can assume most people who self host are Linux engineers / Devops engineers. So they do these kind of setups daily. This is why these channels exist. So we can assist or guide. So you can learn as well.

Enjoy it man! All your problems can be solved with enough effort. THis is the best part, the part where you are forced to learn a bunch of new stuff to move up in your skills :)

Best of luck! I love this shit!

I am right there with you. I am a Mechanical Engineer and if we tried putting out a product with the limited documentation that these come out with, we would be taken off the project. ME projects have documentation meant to guide a 10 year old through the process and details. The documentation would fill a 3" 3ring binder.

Yet these network products are released with 1 page of upper level description of the product. Yeah I already know what the product already does, that's why I got it, how about the actual documentation?

Depends on what you're trying to do.. Plex/Jellyfin server on a Pi with Zerotier for remote access? Super easy and you can get running in 20 minutes with zero previous knowledge.

Self hosted email and reverse proxy with auto SSL and best security practices? Well that's a rabbit hole...

On the one hand, I feel you.
On the other, man, I am so happy you are suffering! lol If you, a Software Engineering student are struggling, then it does not make me feel quite as clueless anymore as a normie :)

Do you have a purpose to host whatever you happened to name here?

There is a reason to host reverse-proxies, however, one can do without them in a self-hosted environments. First, one needs to understand the point behind reverse-proxies (this is an example btw, you might very well know the how and why behind them), and only then would the instructions to set it up start to make sense.

Use a decent VPS instead of trying to host off a residential internet connection behind 47 layers of NAT and you don't have to worry about 90% of that stuff

i.e. use the right tool for the job

I'm confused why you say you can't find information on how to set up home lab projects. There are hours and hours of content on YouTube that also do sometimes point back to blogs and GitHub repos.

I will say that the problem I have is once they show you how to setup say a reverse proxy there's generally not follow through in either deep diving into all the additional settings or more advanced setups. If there is the other problem you'll run into is that it can become outdated quickly so it's a double edged sword. But setup and getting things running in a home lab there shouldn't be any shortage of info to consune.

Start small

Set up a nextcloud or something then use NGINX on another VM to connect to it as a reverse proxy

That'll help you understand cf a little more

You can't just drive in and be an expert, it'll take years to get good

TBH, I don't think Software Engineering, especially just 2 years of experience, is going to really help you understand network architectures/distributed systems. You can probably build some small apps, but designing larger systems is a skill that requires intentional practice.

The best thing you can do is figure out how to containerize the stuff you want to run and store the configurations in source control. Figure out how to isolate your data and back it up, and then experiment with configuration changes to see how they change system behavior.

A few specific things to learn/practice.

• Learn how to break down a larger problem into distinct components with specific responsibilities.
• Learn about docker, what problem it solves.
• Learn how to experiment with the tools to find out what they can do, and how you can configure their behavior (docker is very helpful here, because you can spin up a temporary sandbox to figure things out without risk of breaking a "live" system).

I think the reason is that there's a preferred method for doing things for each person.

Like take reverse proxies for instance.

You can use nginx proxy manager if you're comfortable with docker but uncomfortable with CLI or text file configs.

Or you can just use straight nginx.

Or if you want you can use Caddy, which is what I personally do, because I dig how well Caddy does automatic management of letsencrypt certs. But that means being comfortable grokking how to write a Caddyfile (the config), and how to launch a daemon on some sort of Linux.

So if you want to take my route you've got even more choices, you can run it on a Linux VM, a Linux container, or let docker manage your Linux container. Or you can use someone's Caddy docker container and config that for yourself.

Personally I hand configured a Linux LXC container on proxmox and am using that for all the reverse proxying on my proxmox NUC server.

It's just so many variables now.

I remember back in the day you just installed a LAMP server and ran everything with PHP.

I'm not sure which is better.

But I've got to say, the sheer amount of abstractions and ways to orchestrate a system are mind numbing.

Like you can do proxmox or truenas or truenas scale or Synology or rancher or bare metal. There are so many things to decide on now.

And I've been running Linux for the past 18 years or so.

The biggest thing for me is that you should decide on the outset if you want a simple setup, or if you want to play with bleeding edge enterprise stuff.

One is relatively plug and play and somewhat tends to be less secure.

The other can be fun for some or job training for others. I find the people who find it fun to be a bit weird, like they are basically LARPing being a system administrator. But it might be just dressing up for the job they want instead of the job they've got, or they just like it and that's fine for them.

It's not really for me. I look for simplicity, but also security. Luckily I'm a fan of minimal installs and am happy with operating on a CLI. OpenBSD is a pretty solid base to build on. It makes you think about everything you add and how it's a possible security hole.

Anyway, enjoy reading manuals. That's most of what we do here. Join a forum or two (a real one). Find a supportive community there, ask good questions, contribute good answers. It's computer karma and by helping others you're broadening and solidifying your own comfort levels.

Don't give opinions as fact. They are opinions.

This comment is clearly riddled with opinions. YMMV. Good luck with your goals.

just chip away at it, I'm slowly learning SWAG/reverse proxy. Its complicated for me but its rewarding and eventually becomes easy. Such is the way with linux from my experience

For me and I'm not sure if it's been mentioned here already or not, but I don't really have any programming skills or super geek out on this stuff. But I like the tools, right? So teaching myself some of this is hard, especially when there is an attitude with most of this stuff that you should know everything about docker containers and weird networking rules, etc. Sometimes people are helpful, more often than not, it's a super short answer you have to interpret and decipher to figure out.

I self host for the same reason I play Factorio, it's a constant stream of solving problems and adding new ways to do things. There is a lot to do, but you don't need to do everything right away. Like a lot of people said, just break it up. I use Vikunja to create a list of things I want to and then break those things down into hour or weekend projects.

There is a challenge in the "bootstrap" process. But once you get something going then its easier to keep adding. Stuff like "Reverse proxy, VPN, Cloudfare bullshit" is not super necessary at first. Start with just getting an app going at an ip address and port that's only accessible on you LAN, then add the other stuff.

so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things.

no offense but maybe you just aren't using the right search terms.

Because security isn't easy. Know you know why system admins have a problem with programmers that just expect things to work.

People are overcomplicating stuff because they fear for security.

Move fast and break stuff, and just start somewhere.

You will not die if you run your setup without reverse proxy for a week.

Just go step for step till everything is nice and secure.

Also the thing about understanding everything is a habit that you get in school. But in bigger software projects it's not feasible at least in my experience.

If you don't want to deal with these things and it's just you or maybe a few others, here's the easy way:

• Define the services for each host (i.e. physical + virtual machine) in a single Docker Compose file. You can even put all of these docker-compose files in a single Git repo

• Install Tailscale on all your machines - this is really easy and will let you securely access everything without having to forward ports or worry about the attack surface of the public web

• Set up something like Heimdall or https://github.com/tailscale/golink to have easy to use shortcuts for your services

This gets you out of SSO signin, certificates, reverse proxy setups, all the things that are (understandably!) complex and annoying to set up

If you've never gone and built your own system, or started a network from scratch without a container based system, and you've relied on stacking your own environment for a long time, you might necognize it as you would any skill.

I'm sure I could build a shed, for E.g. I know to get the wood, the panels, the cement, and all the things I'll have all documented in front of me.

Then I will start, realize I forgot I also needed a permit, which I then found out I might also need to run power to it, so now shed has become something that doesn't feel impossible, it feels daunting

Why? Well, I don't build sheds, I don't keep building things that would allow me to have all the framework, project of getting my shed built. So, as with a homelab, take it one step at a time, and if rhe shed isn't the house, you're allowed to skip a few things and stick with what works for you.

I use homelab and self hosting as a passion, but I also have 25 years of networking and systems administration experience so docker and containers just made my normal, tedious job of building up complex network infrastructure, feel like a job, and my homelab and self host happens to be a hobby

Treat your homelab as a passion or simple hobby, or use it as a skill building tool for work

Overall it's just time, experience and expectations that either drive you forward or set you back

Networking is extremely dense and extremely hard.

Its also a space filled with professionals who do it for a job. Many on this sub are current or ex networking or security professionals who understand this shit like the back of their hand.

Im no idiot with IT - I’ve had pc’s my whole life and work in close proxiimity to IT. But networking, security and the clusterfuck of everything associated with it is honestly about as hard to get into as actuarial science.

It requires such a heavy base of fundamental concepts that if you dont start at the bottom and work up, you have no clue wtf is going on.

Begging the question.

Also, "Reverse proxy, VPN, Cloudfare bullshit" - you don't need these things.

I can’t get anything actually running!

Most of my time in my sysadmin work is sitting there staring at the screen reading error logs about why some new product isn't working.

make sure you ACTUALLY have a solid grasp of networking. dig into the osi model or the 3 layer dod model. Go further than "yup. encapsulation. got it". Look at the actual headers in layer 2 and layer 3. Figure out how packets get routed and how frames get forwarded. Look at how NAT actually mangles packets.

Understanding hos the traffic flows will make troubleshooting much easier.

I got hooked by Youtube videos from TechnoTim Christian Lempa and others. Videos include screen capture what they are doing so it bridges some things that may be missing from text guide because it may be 'obvious' if you are doing it all the time.

I keep to principal that I have two 'sections' in my lab 'production' and 'testing' Testing I am mucking about testing stuff it works and production has some stuff that I rely on.

Because it's a complicated topic, you're just feeling the actual responsibilities of doing it right that many are ignoring now.

On a certain level, most of those things don't matter.

Yes, reverse proxying and being able to point a domain is nice

A VPN more secure

Cloudflare (either tunnels, DNS, or DDoS protection) is also good

You can always do better with more "best practices" but if you want to stream a movie to some friends? Throwing up a server, opening a port and providing an IP just works as well. The better you get the more fancy you will be.

Its very much a tinkering hobby. Find a reason you want to tinker, and if you enjoy the tinkering you will keep up with it. My current dashboard points to a direct IP. Most of my services are reverse proxied but for some reason I can't get CORS to work well with dashy. At some point I'll get around to fixing it, but it gets the job done now.