this post was submitted on 09 Jan 2025
19 points (100.0% liked)

Programming

424 readers
11 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 2 years ago
MODERATORS
 

yes, not a unix os but rather unix-like, and i want to program all of it on python, is that possible?? even the kernel, i want it all python. i know most kernels use c++ or c* but maybe python has a library to turn c* into python?? i'm still sort of a beginner but thanks and i would appreciate the answers

top 13 comments
sorted by: hot top controversial new old
[–] ursakhiin 6 points 6 days ago

I'm not going to tell you you shouldn't do that, I think everybody else has done enough telling others what to do. I'll try to focus more on what you'd need to accomplish and why what you're asking hasn't been done.

Building an OS involves a lot of complex work using very low level calls. The easiest way to think about it, IMO, is that whatever language you use needs to be able to communicate directly with the hardware without any abstraction between the code and the hardware after it's compiled.

Basic Python, out of the box, requires multiple levels of abstraction to run.

(I'm simplifying here) You write code which is run through an interpreter. The interpreter is a compiled application that translates Python into code the operating system can understand. Then the operating system translates that to calls the hardware can understand.

In that process, the python code is translated to byte code, assembly, and machine code. The Python virtual machine handles memory management for you. It also handles some processing concepts for you.

You'd need to start by finding (or inventing) a solution that compiles Python to assembly without the need of an interpreter or OS in between you and the hardware. It's worth noting here that Python itself isn't even fully written in Python and is instead written largely in C because Python isn't a compiled language. You'd then need to extend Python with the ability to completely manage memory and processor threads without the VM. You'd need to do that because that's really the main purpose of an operating system.

Something we learn in programming is choosing the right tool for the job. Python isn't a great option for this type of project because the requirements just to get to where you can start are so high that it's not really considered worth while. Is it possible, yes, in theory. But without the python interpreter and VM, you'd have to ask if you're really developing Python or something else that just uses pythons syntax.

[–] deegeese@sopuli.xyz 12 points 1 week ago (1 children)

What OS will run the Python interpreter?

[–] FrameXX@discuss.tchncs.de 2 points 1 week ago* (last edited 1 week ago) (2 children)

Can't Python be translated into machine code and packaged into a binary? I swear I have no experience in OS development, just curious.

[–] deegeese@sopuli.xyz 5 points 1 week ago

Like Java, you can distribute a binary which bundles an interpreter/VM, but your code is still running inside a host OS.

[–] zagaberoo 2 points 6 days ago

You're absolutely right, you could take any binary that runs under an OS and set up a bootloader to execute it directly without an OS.

The problem is that all programs, even ones in C, rely invisibly and enormously on the OS abstracting away hardware for them. The python interpreter doesn't know the first thing about how to parse the raw bytes on a hard drive to find the location of the bytes that belong to a given file path. Files and filesystems are 'fake' when you get down to it, and the OS creates that fiction so each program doesn't have to be customized per PC setup.

So, ironically, to be able to truly kernel hack in python like you want would require writing tons of C to replace all OS hooks (like fopen to interact with a file, e.g.) with code that knows how to directly manipulate your hardware (speaking PCIe/NVMe to get to the disk, speaking GPT to find the partition on the disk, speaking ext4 to find the file in the partition, e.g.).

OSes are complex as hell for a reason, and by retrofitting python to run on bare metal like that would require recreating that complexity in the interpreter.

[–] brian@programming.dev 6 points 1 week ago (1 children)

you could in micropython at least. it's not unixy but for example see https://github.com/Rybec/pyRTOS

[–] mox@lemmy.sdf.org 6 points 1 week ago* (last edited 1 week ago)

Micropython is an interpreter, implemented in C. Anything running in it wouldn't be an operating system in the sense that we usually mean. Anything incorporating it wouldn't satisfy OP's goal of being "only Python".

If a CPU were developed that used Python bytecode as its official instruction set, perhaps using micropython implemented as microcode, then it might work.

[–] thingsiplay 6 points 1 week ago

To program an operating system, you need deep knowledge how internals work. The language you are using needs low level access. And I don't think a garbage collected language is a good fit for an operating system either. Especially an interpreted language like Python requires an interpreter to run under. And Python is not the fastest language either, which is fatal for a low level os functionality.

What do you expect from turning C code into Python code? Python does not have low level access, it requires C (even Rust requires some C functionality). I don't think it is even possible to write an OS purely in Python.

[–] MajorHavoc@programming.dev 6 points 1 week ago* (last edited 6 days ago)

The essence of your answers is "yes, but...". And the "but" is mostly about how slow Python is in contexts that need to be astonishingly fast.

It depends how complex the hardware is and how much time we're willing to waste.

Technically, when I deploy a Python program to a BBC Microbit, that's (more or less) what is happening. Pure Python code is making every decision, and is interacting directly with all available hardware.

We could still argue semantics - virtually no (modern) computer exists that isn't running at least one tiny binary compatibility driver written in C.

I believe the compiled C binary on a BBC Microbit to bootstrap a pure Python OS is incredibly small, but my best guess is that it's still present. The C library for Microbit needed to exist for other languages to use, and Python likes calling C binaries. So I don't imagine anyone has recreated it in pure Python for fun (and slower results).

(Edit: As others have pointed out, I'm talking about MicroPython, which is, itself written in C. The Microbit is so simple it might not use MicroPython, but I can't imagine the BBC Microbit team bothered to reinvent the wheel for this.)

Of course, if you don't mind that the lowest level code has got to be binary, and very few people are crazy enough to create that code with Python, then...

It begs another interesting question: Just how much of an OS can we get away with writing in Python.

And that question is answered both by RedHat Linux and Debian Linux - and the answer is that both are built with an awful lot of Python.

In contrast, Android is mostly Java with ~~lots of C~~ a C Linux kernel. Windows is mostly C# and lots of C. iOS is mostly Objective C and lots of C.

You can have an OS built with almost any language you want, as long as you also want parts of it built in C. (Edit: This is meant to amuse you, not be guidance for what is possible. Today, we love our C code. C didn't always exist, and might someday no longer be our favorite hardware driving language.)

An interesting current development is discussion around rebuilding parts of the Linux Kernel with Rust, which can run just as fast as C. This would effectively cause RedHat, Debian and Android to replace some of their C code with Rust. To date, there's been a lot of interest and discussion and not a lot of (any?) actual funding or work completed.

[–] bitcrafter@programming.dev 4 points 1 week ago (1 children)

I would not recommend this as an exercise for a beginner, but RPython is a subset of Python with a C backend; it is used as the basis of PyPy (an implementation of Python), so it may be possible to use it to implement the low-level parts which then can be used to bootstrap a full Python virtual machine.

[–] Corbin@programming.dev 3 points 1 week ago

In short: If you'd like to learn more, come visit #pypy on Libera IRC. It's an interesting discussion topic, particularly if we want standard-library imports like math, sys, or json to work.

RPython is not capable of translating to bare metal today; it depends on libc and libffi for many features even when not producing JIT compilers. It's also intended to operate on a layer of syscalls: rather than directly instructing hardware, it wants to make fairly plain calls, perhaps via FFI, passing ordinary low-level values. So, any OS developer would first have to figure out how to get RPython to emit code that doesn't require runtime support, and also write out the low-level architecture-specific hardware-access routines.

That said, RPython is designed to translate interpreters, and fundamentally it thinks an interpreter is any function with a while-loop, so a typical OS would be a fairly good fit in terms of architecture. RPython knows the difference between high-level garbage-collected objects and low-level machine-compatible values; GC would be available and most code would be written in a statically-typable dialect of Python 2.7 that tastes like Java or OCaml.

The OS would be the hard part. RPython admits the same compositional flexibility as standard Python, so it should be possible to hack PyPy into something that can be composed with other RPython codebases. This wouldn't be trivial, though; PyPy in particular is tightly glued to RPython since they are developed together in a single repository, and it wasn't intended for reuse from the RPython side.

If all of that sounds daunting, and what you would like to do instead is take an existing kernel or shell with C linkage and ELF support, and extend it arbitrarily using Python code, then PyPy can help you in that direction too. Compile a libpypy and embed PyPy against your kernel, and you can then run arbitrary Python code in a fairly nice environment which supports Python-first development. Warning: while the high-level parts of this might be nice, like Python's built-in REPL tools, the low-level parts could be very nasty since this embedding interface is old and rotting, to say nothing of actually getting bare-metal code that doesn't make syscalls.

[–] senkora@lemmy.zip 3 points 1 week ago

What you are looking for is some way to shortcut the process of learning to write an operating system by re-using your existing knowledge of Python.

(I'm not judging that; I understand why you want to do it)

The simple truth is that there is no way to do that. Any solution that involves using Python in a kernel would cost you more in terms of complexity and time than learning C would.

It is rarely worth it to use a language outside of the domains that it is normally used for.

[–] FizzyOrange@programming.dev 1 points 6 days ago

You could do some of it in Python, but some stuff needs low level access to registers, e.g. trap handlers and context switching.

Should you do that? Absolutely fucking not. It would be hilariously slow and inefficient. Hundreds of times, maybe thousands of times slower than C/C++ kernels.