Too complicated and poor beginner documentation
this post was submitted on 18 Oct 2023
6 points (100.0% liked)
nixos
46 readers
1 users here now
All about NixOS - https://nixos.org/
founded 4 years ago
after spending +3 years getting experienced and reading a lot of the source code (getting past the documentation/learning-curve problem), I can say there are at least two fundamental flaws, with nixpkgs moreso than nixos:
(Edit: Added 0)
-
The culture issue. See thread below
-
The messy mono-repo was never desgined to be searchable. It might feel searchable to a newby, but many teams have dumped tons of effort and slapped together lots of hacks to make niche-package-versions even halfway searchable. Devbox is doing a good job of fixing this, but its not there yet.
-
The monorepo design is unmaintainable/unscalable from a package maintainer standpoint. There's a ton of contributor burnout, there's no automated quality control on packages, to outsiders its not obvious how to report package-specific issues or how to edit/fix/contribute to a single package, and instead of 30min to publish a hello-world npm or cargo package, users need to make a PR on the core, and get it approved. Meaning publishing a hello-world package would get rejected anyways. The good news is flakes hub is fixing this, and I'm really excited for that.
The good is;
- The pros still outweigh the cons. My projects from 3 years ago that I haven't touched still work (0 bitrot) first try, no manual install/setup needed.
- people have put a ton of effort into nix. Its truely amazing how many things work in nix despite how absurdly difficult it is to get things working with nix
- it is pretty much as reproducable as it can be and nothing else is even remotely close.
The messy mono-repo was never desgined to be searchable
I'm not sure what you mean by "searchable"?
https://search.nixos.org/packages exists and it's generally pretty damn good.
The monorepo design is unmaintainable/unscalable from a package maintainer standpoint. There’s a ton of contributor burnout, there’s no real quality control on packages
There is contributor burnout and perhaps some quality issues here and there but the monorepo isn't the reason for that. In fact, it'd be a lot worse if we had hundreds of smaller repos instead as wide-reaching changes would become basically impossible with our current manpower.
There have been calls to fragment Nixpkgs for years but they've almost always been struck down because none of what was suggested would improve anything about our current maintenance issues.
to outsiders its not obvious how to report package-specific issues
You simply open an issue on GitHub. I don't know how it could be any more clear.
I just checked and we don't explicitly document this in CONTRIBUTING.md but I don't think we should need to. It's just too obvious IMO.
how to edit/fix/contribute to a single package
https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#how-to-create-pull-requests
flakes hub is fixing this, and I’m really excited for that.
FlakeHub fixes none of the maintenance issues of Nixpkgs.
You actually bring up a very good point that I missed; another problem, maybe the largest problem with nix, is the culture issue.
A comment like "what do you mean nixOS doesn't have good documentation??? we have tons of good documentation!" feels very similar to reading what you just said about "You could simply open an issue on Github, I dont know how it could be clearer". Neither of those are a question. They're not asking "how can we do better? (I'm sorry you, as a new user, had a hard time)". Those comments are statements, and they say "you [person struggling to report a package-specific issue] are the problem; our system doesn't need to change. YOU need to change".
This culture issue has real consequences. It's the reason Determinate Systems has a separately maintained nixpkgs-installer script, and separately maintained documentation. Its the reason I also have my own independently maintained nixpkg-installer and nixpkg-uninstaller script. _
Maybe you haven't had our problem because you're really skilled and familiar with nix/github/whatever. Maybe you haven't run into our problem because your use-case doesn't overlap with ours. I am glad you don't run into the problem. But that doesn't somehow make the problem not-a-problem.
Adopt a legacy codebase that is massive and requires EXACTLY ruby 2.6.0-rc1. On ubuntu, using rbenv, it takes a newby 30 sec to list all the available versions, and 30 sec to install ruby 2.6.0-rc1.
Try finding that version on search.nixos.org.
Even excluding versions, try searching for the extremely-commonly-needed "Core Foundation" package on search.nixos.org. I assure you the package does exist on nixpkgs, I've been using it for years. As a newby had to spend WEEKS looking through the source code and learning nix-lang quirks just to find it.
_
But more importantly please ignore those details and look at the bigger picture; we are on the same team. I'm not insulting or ignoring the massive accomplishments nix team has made. They (maybe you as well) are giants that have moved moutains and accomplished things I wouldve considered basically impossible. I want to help the core devs have LESS work. I want to have productive discussions about the trade-offs of federated vs monorepo, searchability, documentation improvements, installer scripts, etc.
But we can't.
Not until the discussion starts with "I agree there's a problem" instead of "there is no problem other than YOUR lack of skill"
Yup, figuring out how to install gcc 10 was a pain. Turns out I needed stdenvgcc10, but how would I have known that without reading like the entirety of nix pills? I don't want to read a manual for 2 hours to figure out how to do something basic. Not to mention nix pills is SO dense that as someone who can't learn from textbooks, its impossible.
they say "you [person struggling to report a package-specific issue] are the problem; our system doesn't need to change. YOU need to change".
I have not met anyone in the Nix community who's opposed to fixing actual systematic issues. I just highly doubt that the discoverability of github issue tracking in particular is a problem that Nixpkgs is in any way responsible for.
I too wouldn't be entirely opposed to adding i.e. a little section on how to report issues but when you've read half a README, then opened CONTRIBUTING.md and read through that, you really should have discovered the "issues" tab to report your, well, issue by then.
It's the reason Determinate Systems has a separately maintained nixpkgs-installer script, and separately maintained documentation.
No. The reason they have separate instances of those is that they allow a green-field approach to things. "Move fast and break things" is great for development but you can't do that when the entire ecosystem relies on the things you might be breaking.
The installer is a great counter example actually. If someone wanted to replace the regular installer with the detsys installer right now, the greatest opposition they'd likely face is "hey, let's be careful to not break users' setups, does (niche feature) still work?".
I wouldn't be surprised to see it replace the current official installer within the next year.
Adopt a legacy codebase that is massive and requires EXACTLY ruby
2.6.0-rc1. On ubuntu, using rbenv, it takes a newby 30 sec to list all the available versions, and 30 sec to install ruby2.6.0-rc1.Try finding that version on search.nixos.org.
Nixpkgs has never has supported that version and does not support using multiple versions of Nixpkgs either (not even the currently maintained branches). You can try to and it'll probably (perhaps even likely) work but it's not "intended" to and nobody will want to deal with issues you might encounter with that.
We regularly kick out packages that have stupid version requirements like that for a reason. Eventhough we could technically have an infinite amount of versions of any package we choose not to because it's a maintenance burden we cannot support.
The "proper" way of handling an issue like that (I'm sorry to say but depending on some old specific version is actually an issue of the dependant) is to "vendor" the dependency; copying its expression out of the Nixpkgs tree and maintaining it yourself.
try searching for the extremely-commonly-needed "Core Framework" package on search.nixos.org
According to repology, a package under than name exists in no repository and it knows about a damn lot of repositories:
https://repology.org/projects/?search=Core+Framework&maintainer=&category=&inrepo=¬inrepo=&repos=&families=&repos_newest=&families_newest=
I don't know which "Core Framework" you are referring to either.
But more importantly please ignore those details and look at the bigger picture; we are on the same team. I'm not insulting or ignoring the massive accomplishments nix team has made. They (maybe you as well) are giants that have moved moutains and accomplished things I wouldve considered basically impossible. I want to help the core devs have LESS work. I want to have productive discussions about the trade-offs of federated vs monorepo, searchability, documentation improvements, installer scripts, etc.
But we can't.
Not until the discussion starts with "I agree there's a problem" instead of "there is no problem other than YOUR lack of skill" I've yet to see a "bigger picture"-issue described in what you wrote.
At this point I'm not sure whether we're talking about the same Nix community anymore. We have a lot of those "big picture" issues in the Nix community and we're aware of them. What we need the most help with is fixing them, not finding them.
If many people report a problem, the correct response is never "no, you're wrong, that's not a problem". There's no technical aspect to it, that is just common dececency when talking to another human.
That's the culture issue with Nix.
It doesn't mean an issue will never be "wont-fix" or that a issue won't genuinely be a skill-issue. It means it's not hard to say "I'm sorry, I disagree with adding support for that feature" or "here's a tutorial for [skill issue]" instead of saying "no, you're wrong, what you reported is not a problem". When you mention editing the CONTRIBUTING.md you're not offering help, you're just setting up a punchline for an "[but even that wouldn't help this IDIOT]". There's no sincerity, no sympathy or attempt to understand.
I don't know how to explain it any more clearly... you're not convincing anyone to help you on those "big issues" with words like "you really should have discovered the issues tab".
I have not met anyone in the Nix community who's opposed to fixing actual systematic issues
True, there is no one actually broadly and openly opposing change willingly or based on conviction. I think the main inhibitor to change in the nix (-pkgs / -OS) communities stems from fear. The situation is already pretty unpleasant, I think we can all agree. Contrary to that, Nix* has gained a lot of momentum in the last couple of years. Fundamental change would affect a lot of people and can easily turn the situation from unpleasant to unbearable. So people are not easily convinced to take a chance on ideas and just "try it out for a release or two" but rather die on the hills of their own personal arguments pro or against something. This ultimately leads to people blocking each other in and even though there is a will and a need to change, there is no change or it is coming very, very slowly. My personal fear is that the shift from unpleasant to unbearable will happen not because of wrong action but because of fearful inaction.
https://search.nixos.org/packages exists and it's generally pretty damn good
It's actually not. If you search for go you get the go package, but if you search for go 1.21, no go package will even appear on the first page. The package you are looking for is somewhere on page 7 or 8.
Indeed, that looks like a bug. The version field is there but it doesn't appear to be searched.
Could you open an issue on https://github.com/NixOS/nixos-search?
I have to agree on those two cons. Contributing to nixpkgs is a major pain point for me. I don't mind having to get PRs reviewed, but the speed and the lack of consistency have really put me of from doing that again.
Quality control is also a major pain point. Nixpks supports a bunch of platforms and architectures, but anything other than tier 1 (x86_64-linux, gcc + glibc ) can and will just randomly break because maintainers and contributors don't usually care for anything that's not right in front of their eyeballs.
Personally, I avoid touching the monorepo like the plague and stick to putting everything into flakes if possible.
Agreed, flakes are the way. Its just making them searchable hasn't been easy/realistic until recently.
Also I feel like flakes are kind of tainted by always pulling in nixpkgs as a massive dependency chain. I think there should be a standard library separate from the packages, and nixpkgs.lib is 80% pure functions. So I've been working on making a "lib" flake that
- Is 100% pure (no stdenv)
- is versioned
- doesnt link/depend on all of nixpkgs
I know it still won't be practical to avoid depending on nixpkgs, but I think its a step towards breaking up nixpkg into organized/managable chunks.
I actually think I have seen this being discussed, but does not seem to be at the RFC level yet. Probably drowned in all of the flakes vs monorepo discussions.
One prerequisite for this that's in the making are flake versions: https://github.com/NixOS/rfcs/pull/158
Oh cool! I didn't know about that. Maybe I can work eith the other person who is doing that.
I just remembered the thing I really dislike about NixOS is that I cannot declare flatpaks, appimages, and the like. It would be really useful if this feature was added somehow seeing that there always seems to be an odd package that doesn't function fully from nixpkgs and the workaround is mainly flatpak.
If anyone is interested, it looks like there is https://github.com/gmodena/nix-flatpak. Will try it out soon.
for appimage, see e.g. https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/networking/mockoon/default.nix
Thanks, I'd like to understand appimage tools, and nixified Docker / Podman to start up a headless server, but they all seem pretty difficult for me right now.
I dislike that it is hard to find relevant instructions on what you are looking for. That being said, there are many channels- Matrix, Discourse, Discord, Lemmy, that have a lot of smart people that are willing to help.
I'm not a programmer and I run NixOS on my personal, family, and friends' PCs. After about three months of struggling severely, I was able to get my one flake git repo to the point where I have installed it and replaced other linux distros with NixOS on these machines. I also don't often understand the warnings that are displayed when rebuilding a configuration, but have been able to solve them so far with a little guess work sometimes.
I'm setting them up with syncthing for their home directories and am looking into some easier than rsync based programs to keep them backed up. Then I can easily reinstall NixOS, git clone the flake, and have them back up and running in the event that they nuke their PC. This hasn't happened yet, unlike on other distros. Plus, I have only received complements about the systems. Someday, if Wayland gets there, I will be able to remotely access them. There probably is a way, but I haven't figured out an easy way to do so yet.
So overall more happy and satisfied, because all things tech take time and learning.
Next step is to port my Ubuntu server over to NixOS and probably use Docker to spin up webapps. Maybe even declare them the nix way.
Yesterday, I ran into an issue where two versions of Qt couldn't run at the same time. This problem has been mentioned by a few people before, with no solutions offered for users. In my case, I was trying to start a nix shell with the rpi-imager. Thankfully, I was able to use the cli, but this was still unexpected and I was under the impression that nix was supposed to fix problems like this.
This is something that Nix cannot fix. Essentially the symbol namespaces in Linux are flat, so if you load two versions of the same library (or two different libraries exposing the symbols even) the first one wins. This is in contrast to eg. Windows where symbols are organized in a tree structure, which has advantages like fewer conflicts, but also comes with disadvantages of its own (eg. it would not be easy to implements something like LD_PRELOAD on such a structure). So on Linux, this is by design / works as intended.
For a bit of background information, you can have a read of https://github.com/NixOS/nixpkgs/issues/31189
Why is this only an issue on NixOS, then? I can't find a single instance of this issue on another distro.
Here is the relevant github issue: https://github.com/NixOS/nixpkgs/issues/37864
Because other distros usually can't have multiple versions of the same library to begin with.
If these these applications can coexist in other distros, and they can only have one version of Qt, then that means packages in NixOS are overly strict about library versions. I don't really care what the underlying reason is for this issue, I just know it's an issue exclusive to NixOS that frustrates me.
No, the actual issue was an impurity (not working in Nix' pure model). Impurity is a bug; it was fixed years ago.
They could, but they usually don't, because of reasons like this. On "classical" distros, library updates are done much more reluctantly because an update to a share library does not trigger a complete rebuild of the system. So the solution is not technical, it's organizational more often than not.
That being said, the statement that symbol conflicts do not exist on other distros is plainly not true.
That being said, the statement that symbol conflicts do not exist on other distros is plainly not true.
I have never claimed such a thing.
Classical distros have exactly one instance of a library ABI's .so in most cases which is usually the only place where any given symbol is defined.
You could technically provoke a symbol conflict using LD_PRELOAD and the like but it's not something you commonly run into because package upgrades always replace the previous version entirely.
You could technically have multiple conflicting sos on classical distros too by prefixing a more detailed version but you don't; doing such things kinda what differentiates Nix from classical package management.
This QT issue in particular was an impurity (working outside of Nix' pure model; not as intended) caused by "installing" qt libraries into your environment imperatively (which isn't something you should do anyways) that was solved a couple years ago.
I have never claimed such a thing.
I switched context there quite a but. That part was directed at the underlying tone of the thread that implies that "other distros do not have these kind of conflict". Should have put that in a separate reply to make it clearer. Please excuse the confusion 🙂
It's accumulated a lot of cruft over its >20 years of existence. Especially the language, which can't be changed due to otherwise breaking backwards compatibility.
There's Nickel which is sort of Nix expression language 2.0 but, honestly, the Nix expression language is perfectly adequate for what it needs to do.
Not to say it couldn't be better I wouldn't put it inn any top 10 of what is holding Nix back.
It has one major advantage over Nix though: it comes with a built in code formatter!