First: Good for you, enjoy the journey! Second: Just as others already pointed out, Mastodon is not really a beginner project. You want to understand what you are doing, not just make everything work no matter what. Some reasons why I'd not start with Mastodon:

• Complex deployment stack (for beginners)
• Needs regular maintenance
• Security considerations (if you haven't managed/hardened a server before)
• Long term project

So instead: Have a look at awesome-selfhosted for ideas. A personal dashboard, photo gallery or a PiHole/AdGuard is a good start.

About Docker; it's a bit more than just dependency separation. It's a kind of virtualization, but without each container running it's own kernel. Advantage is: Docker images run (with some configuration) relatively lightweight out of the box. So there's no need to install the applications natively. While I'm a great fan of Docker, you'd probably learn more installing things natively in the beginning. Or maybe do both, it's up to you. However, if you decide to use Docker, be sure to understand what's going on under the hood. That's where the fun begins. Everyone can pull and start images, but not everyone knows how to customize or build them themselves.

No matter what you decide to do, have fun. And if you've any questions, there's plenty of documentation online or just ask. The selfhosting community is very welcoming towards new members ;)

I'd start with internal services first before opening yourself up to the outside, starting with Jellyfin possibly. Docker is worth learning as it's agnostic to the system you are using and helps with prototyping/testing quickly - something invaluable as you learn and make mistakes in your journey. From maintaining the fun stuff, such as your media service, you'll learn the other stuff, like ssh keys, Reverse proxy &c: Iterative improvement is the game when it comes to your homelab and yourself.

You don't require docker to self host. You have a lot to learn, so trimming down the amount of things your doing is the right idea. Ignore it for now.

You don't have to buy a domain, you can use your IP directly, or use a free dynamic dns service.

Mastadon may be too complicated for a first host for a beginner. I would recommend trying something simpler first, but it's your hobby so do it how you want to.

As others have suggested I would stick to internal only until you're comfortable. Dealing with automated port scanning from all over the globe is a nightmare if you aren't running a tight ship, and a collasal headache to deal with if they get in. I started with pihole, jellyfin, and some other simple docker services. Check out linuxserver.io if you go the docker route, they have good prebuilt compose files for you to use and the community has good info on their forums. You don't need a domain but it's preferred if you want to host something public-facing. Make sure you have fail2ban configured if you open stuff up to the web, it makes dealing with the automated attacks easier.