From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
I tried it briefly, but its hot garbage if you dont have potent hardware.
The amount of iterations you have to do, to get proper answers and the time it takes to produce them is a waste of time.
Personally I'd just recommend either Alpaca or GPT4All, both of which are on Flathub and much easier to set up (or at least GPT4All is; I haven't tested Alpaca yet).
Alpaca is great, I can even run it on my oneplus 6t, albeit slowly and the max size I got running was llama 7b
Alternatively, you don't even need podman or any containers, as open-webui can be installed simply using python/conda/pip, if you only care about serving yourself:
https://docs.openwebui.com/getting-started/quick-start/
Much easier to run and maintain IMO. Works wonderfully.
And llamafile is a binary you can just download and run, no installation required. "Uninstallation" is deleting the file.
You might instead just install the Alpaca flatpak. I found it a very easy and quick way to get started.
I did try to use it on Fedora but i have a Radeon 6700 XT and it only worked in the CPU. I wait until ROCM official support reaches my older Model.
I have the same setup, you have to add the line Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0" for that specific GPU to the ollama.service file
ollam runs on the 6700 XT, but you need to add an environment variable for it to work... I just don't remember what it was and am away from my computer right now