Latest 10 recent news (see index)
November 06, 2026
Making musl and proprietary NVIDIA drivers work together
As you may be aware, one of the limitations of musl-based distributions is that they will not function to their proper extent when used in computers with various NVIDIA graphics cards.
This is due to the proprietary nature of the driver, being built for glibc. I have been experimenting with trying to make this work, and this post is the summary of that effort.
Context
The NVIDIA graphics cards are tricky on Linux. While most other GPU vendors by default use free software graphics drivers (with kernel drivers in-tree in Linux, and userspace supplied by the Mesa project), NVIDIA in general does not, relying on an out of tree kernel driver and proprietary userland libraries that are supplied by the vendor.
A free software driver does exist, called Nouveau (kernel and OpenGL Mesa driver) and NVK (Vulkan driver). This allows the cards to work, but usually with significantly reduced performance, due to various reasons.
Currently the state depends significantly on what generation of NVIDIA card you have.
In short:
- Anything older than Maxwell (GTX 700 series) is unsupported by the proprietary driver on modern systems, and supported to some extent by Nouveau, but may have issues due to poor maintenance nowadays; performance should be okay when it works.
- Maxwell (GTX 700 and 900 series) and Pascal (GTX 1000 series) is supported fully by the legacy 580 branch of the proprietary driver, which also supports newer hardware. It works with Nouveau, but with very poor performance, as the cards can never reclock from their very low boot clocks (which would require firwmare that NVIDIA never released).
- Turing (GTX 1600 series and RTX 2000 series) and anything newer is supported
by the current main proprietary driver branch (at the time of writing, 620)
as well as the legacy (580) branch, and by Nouveau, as these cards use the
GSP firmware that is included in
linux-firmware. However, using the FOSS driver means only getting 40-80% of the proprietary driver’s performance. Other than that, the experience is often okay, particularly with NVK.
In even shorter, if you have RTX 2000 and above you are mostly okay with FOSS but lose performance, if you have newer than GTX 600 but older than GTX 1600 you are screwed and need the proprietary driver, if your card is really old it may or may not work well and you may not be able to use it at all.
Proprietary drivers and musl
We now have a pull request
open for both the 580 and 620 branch. It is not yet entirely ready but works,
and is available for x86_64 and aarch64 architectures as NVIDIA supplies
it.
To get there, there were multiple things to take care of. Particularly, using stock musl is out of the question and will likely never work. I will elaborate on each thing.
GLVND
The GLVND library was created by NVIDIA to allow for vendor-neutral OpenGL dispatch. It provides the libraries that applications link and exposes the OpenGL (and other) API. It then dispatches the calls to the implementations, which may be e.g. Mesa or the proprietary driver.
Most distributions nowadays use GLVND. Mesa comes with its own builtin dispatch code and does not necessarily require GLVND if you don’t build it with it. Musl distributions in general use this. Chimera at this point does not, and Void Linux on musl does not as it shares packaging with the glibc branch.
Chimera switched to GLVND on November 5 2026.
Switching is rather straightforward and does not need much documentation.
One just has to package GLVND, rebuild Mesa with it (to avoid using its
builtin dispatch libraries) and take care of packages that have dependencies
on mesa-devel.
GLVND is not only useful for NVIDIA drivers, but in general it is the primary thing users will find useful. There are also some vendor drivers on ARM and elsewhere but those are niche. The proprietary drivers mandate GLVND since many years back (this is also very useful on laptops with hybrid graphics, because then you may still want to keep Mesa for the iGPU).
Gcompat
The gcompat project provides
a compatibility shim to allow for limited usage of programs linked with glibc.
It consists of two components:
- the loader, which acts as the dynamic linker (
ldso) for the programs. For example, onx86_64,glibcprograms expectld-linux-x86_64.so.2as their dynamic linker. The loader does not do much, as it only sets up a library preload for the below, and execs into the realmuslone. - the shim library,
libgcompat.so.0, which implements various symbols that are inglibcand used by programs, but not inmusl. It does not implement everything. It is what provides the bulk of the functionality, and is useed by the means ofLD_PRELOAD, automatically by the loader.
Unfortunately for us, the latest tagged release of gcompat does not provide
all the symbols the proprietary drivers require to resolve. Fortunately for us,
the latest Git branch does. The development nowadays is very slow so we just
bumped it up to the latest revision.
Patchelf and initial packaging
You can see how the packaging works in the pull request, and that part is
rather unimportant for now, and does not require elaboration. It works in
a similar way to other distros, downloading the .run archive from NVIDIA
and installing everything to the correct places.
Of course, in our case, we cannot install it directly, because it depends
on glibc and other stuff. We also can’t just pull in gcompat as is, because
most of the driver is libraries that have no interpreter.
Let’s see what happens if we run ldd on libnvidia-glcore.so.580.196.07,
which is the core OpenGL library.
$ ldd libnvidia-glcore.so.580.196.07
ldd (0x739c4a8b1000)
libm.so.6 => ldd (0x739c4a8b1000)
libc.so.6 => ldd (0x739c4a8b1000)
libdl.so.2 => ldd (0x739c4a8b1000)
libnvidia-tls.so.580.196.07 => /lib/libnvidia-tls.so.580.196.07 (0x739c4a8a3000)
libnvidia-gpucomp.so.580.196.07 => /lib/libnvidia-gpucomp.so.580.196.07 (0x739c3fa00000)
libpthread.so.0 => ldd (0x739c4a8b1000)
librt.so.1 => ldd (0x739c4a8b1000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by ...)
Error relocating /lib/libnvidia-gpucomp.so.580.196.07: mallinfo: symbol not found
Error relocating /lib/libnvidia-gpucomp.so.580.196.07: __strftime_l: symbol not found
Error relocating libnvidia-glcore.so.580.196.07: dladdr1: symbol not found
Error relocating libnvidia-glcore.so.580.196.07: gnu_get_libc_version: symbol not found
Error relocating libnvidia-glcore.so.580.196.07: dlvsym: symbol not found
Error relocating libnvidia-glcore.so.580.196.07: backtrace: symbol not found
Error relocating libnvidia-glcore.so.580.196.07: unsupported relocation type 37
There is a bunch of stuff here. So the libm.so.6, libc.so.6, libdl.so.2
etc resolve to ldd as they are reserved in the musl ldso:
static const char reserved[] = "c.pthread.rt.m.dl.util.xnet.";
The dynamic linker will ignore the suffixes.
For some reason, some of the libraries also have DT_NEEDED on ld-linux-x86-64.so.2
which is the glibc dynamic linker. There are also some relocation failures
that we’ll get to later, for now we take out patchelf.
So, patchelf is a tool that lets you dynamically patch ELF files for things
like library dependencies (DT_NEEDED), dynamic linker (DT_INTERP), RPATH,
and even some funky dynamic symbol remapping.
We use this to sanitize every library and executable being installed. Let’s begin with something like the following:
$ patchelf --remove-needed ld-linux-x86-64.so.2 \
--remove-needed libdl.so.2 --remove-needed libm.so.6 \
--remove-needed libpthread.so.0 --remove-needed librt.so.1 \
--replace-needed libc.so.6 libc.so \
--add-needed libgcompat.so.0 libnvidia-glcore.so.580.196.07
What this does is drop all the dependencies we don’t need (this is optional
as musl ignores them, but why not? And even on glibc since recent versions
they are just compatibility symlinks that point to the libc itself) as well
as add a new dependency on libgcompat.so.0 to supply the symbols.
This clears up the list a lot, and now only one error remains:
Error relocating libnvidia-glcore.so.580.196.07: unsupported relocation type 37
We will get to this below. You now have a rough idea about patchelf.
So, about that relocation 37
What is this? Let’s check elf.h.
$ grep R_X86_64 /usr/include/elf.h|grep 37
#define R_X86_64_IRELATIVE 37
Googling for this tells us that it’s a relocation type for GNU indirect
function (ifunc). Ifunc is a mechanism for dispatching to different versions
of a function based on a runtime condition. It is done by having a resolver
function which does the decision logic, and returns the implementation, which
is then called. A good writeup is here.
This is intentionally not implemented in musl. In glibc it is used to
pick different implementations of functions like memcpy based on CPU
microarchitecture levels for performance reasons. However, indirect functions
are also a major attack vector,
bypassing RELRO among other things, and in general there are other ways to
do runtime dispatch anyway.
I remembered that there was a project called Alpaquita Linux which is basically
Alpine with performance patches. It has the musl-perf package which provides
a patched musl with string functions (and other things) from glibc and for
this it also has patches for ifunc. We don’t need all of them, particularly
the one for static linking is useless for us, so we take the one we need
and its aarch64 counterpart
for a good measure.
This just adds handling of the relocation. We then package the patched libc
as musl-nvidia and poof, unsupported relocation is gone.
There are now no errors in ldd. That means we’re done, right? Nope :)
Thread-local storage
Oh boy.
$ readelf -a libnvidia-glcore.so.580.196.07|grep TLS
L (link order), O (extra OS processing required), G (group), T (TLS),
0x000000000000001e (FLAGS) SYMBOLIC STATIC_TLS
113: 0000000000000000 0 TLS GLOBAL DEFAULT UND _nv005tls
123: 0000000000000000 0 TLS GLOBAL DEFAULT UND _nv000tls
205: 0000000000000000 0 TLS GLOBAL DEFAULT UND _nv002tls
224: 0000000000000000 0 TLS GLOBAL DEFAULT UND _nv001tls
$ readelf -a libnvidia-tls.so.580.196.07|grep TLS
L (link order), O (extra OS processing required), G (group), T (TLS),
TLS 0x003d40 0x0000000000004d40 0x0000000000004d40 0x000000 0x000038 R 0x8
0x000000000000001e (FLAGS) SYMBOLIC STATIC_TLS
9: 0000000000000010 8 TLS GLOBAL DEFAULT 19 _nv008tls
11: 0000000000000008 8 TLS GLOBAL DEFAULT 19 _nv007tls
14: 0000000000000000 8 TLS GLOBAL DEFAULT 19 _nv006tls
17: 0000000000000020 8 TLS GLOBAL DEFAULT 19 _nv005tls
25: 0000000000000018 8 TLS GLOBAL DEFAULT 19 _nv002tls
28: 0000000000000030 8 TLS GLOBAL DEFAULT 19 _nv001tls
31: 0000000000000028 8 TLS GLOBAL DEFAULT 19 _nv000tls
$ readelf -a libnvidia-glcore.so.580.196.07|grep TPOFF
00000000022f5f98 0000007100000012 R_X86_64_TPOFF64 0000000000000000 _nv005tls + 0
00000000022f5fa8 0000007b00000012 R_X86_64_TPOFF64 0000000000000000 _nv000tls + 0
00000000022f5fd0 000000cd00000012 R_X86_64_TPOFF64 0000000000000000 _nv002tls + 0
00000000022f5fe8 000000e000000012 R_X86_64_TPOFF64 0000000000000000 _nv001tls + 0
$ readelf -a libnvidia-tls.so.580.196.07|grep TPOFF
0000000000004fc8 0000001100000012 R_X86_64_TPOFF64 0000000000000020 _nv005tls + 0
0000000000004fd0 0000000900000012 R_X86_64_TPOFF64 0000000000000010 _nv008tls + 0
0000000000004fd8 0000000b00000012 R_X86_64_TPOFF64 0000000000000008 _nv007tls + 0
0000000000004fe0 0000000e00000012 R_X86_64_TPOFF64 0000000000000000 _nv006tls + 0
So, about TLS. Normally, when you use OpenGL in an application, the implementation
is dynamically loaded at runtime, not linked to the application. The OpenGL
functions you use are then dynamically requested, usually through a portable
wrapper (e.g. SDL_GL_GetProcAddress).
There are various TLS models. You can read up on them here.
Look at the DT_FLAGS for the glcore and tls libraries. See the STATIC_TLS?
We have initial-exec, and our TLS needs to exist at a fixed offset from the
thread pointer. This in general precludes dynamic allocation of the TLS memory.
A quick test:
q66@asusbook: /home/q66$ cat dynlib.c
#include <stdio.h>
__thread void *init_ptr __attribute__((tls_model("initial-exec")));
void upd(void) {
init_ptr = (void *)0xDEADBEEF;
}
q66@asusbook: /home/q66$ cc dynlib.c -o dynlib.so -shared -fPIC
q66@asusbook: /home/q66$ cat test.c
#include <stdio.h>
#include <dlfcn.h>
typedef void (*func_t)(void);
int main(void) {
void *p = dlopen("/home/q66/dynlib.so", RTLD_LAZY);
printf("SO %p %s\n", p, dlerror() ?: "");
func_t upd = (func_t)dlsym(p, "upd");
upd();
}
q66@asusbook: /home/q66$ cc test.c -o test
q66@asusbook: /home/q66$ ./test
SO 0 Error relocating /home/q66/dynlib.so: init_ptr: initial-exec TLS resolves \
to dynamic definition in /home/q66/dynlib.so
zsh: segmentation fault ./test
q66@asusbook: /home/q66$ LD_PRELOAD=/home/q66/dynlib.so ./test
SO 0x700c30811080
When trying to use the driver, we would get the exact same error when trying
to open the GL implementation. On glibc it works because it reserves some
extra space in its static TLS at the beginning and gives them out for as long
as there is space. It is limited, but enough for the driver.
The correct thing here would be to use TLSDESC (-mtls-dialect=gnu2) which
is efficient enough and allows for dynamic TLS allocation, but NVIDIA apparently
uses some rather old toolchains.
Since we don’t want to globally preload libnvidia-glcore into each process,
I ended up patching the dynamic linker to handle this scenario with the same
approach. I will not elaborate on the details of the patch as you can see it
in the linked pull request and it is commented. It boils down to checking if
the library has DF_STATIC_TLS in DT_FLAGS during dlopen and if so,
checking if we have enough space for its whole TLS image in the spare memory,
and if yes, avoiding allocating dynamic TLS for it.
That allows the above example to work without LD_PRELOAD.
(Another approach could be placing the TLS symbol definitions in something
that is not runtime-loaded, e.g. placed into libGL or something else, and
interposing the definitions; this could theoretically avoid patching the
libc for this specific thing, but we have the ifunc thing too).
Procuring some hardware
I do not have any NVIDIA hardware. Before I could test, I would need to put a computer together.
Fortunately, my girlfriend still had her old computer’s GTX 980 Ti laying around. It is only supported by the legacy driver, but good enough for now. I then obtained some random old computer hardware (i3-7100, some RAM and a board, etc.)
It resulted in this monstrosity.
I could finally try if this thing works.
Initial testing and static mutex initializers
After this, the packaging was seemingly complete and I went to do some testing on the hardware above.
Well, it turns out trying to start anything graphical would just hang.
I ran ls with libnvidia-glcore preloaded and saw:
(lldb) bt
* thread #1, name = 'ls', stop reason = signal SIGSTOP
frame #0: 0x00007b0f72259376 ld-musl-x86_64.so.1`__syscall_cp_c(...) at pthread_cancel.c:38:1
frame #1: 0x00007b0f7225843f ld-musl-x86_64.so.1`__futex4_cp(...) at __timedwait.c:24:6 [i
nlined]
frame #2: 0x00007b0f7225841b ld-musl-x86_64.so.1`__timedwait_cp(...) at __timedwait
.c:52:7
frame #3: 0x00007b0f72258526 ld-musl-x86_64.so.1`__timedwait(...) at __timedwait.c:68:6
frame #4: 0x00007b0f7225b6d3 ld-musl-x86_64.so.1`__pthread_mutex_timedlock(...) at pthread_mutex_timedlock.c:85:7
* frame #5: 0x00007b0f67c033db libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_a033b0 + 43
frame #6: 0x00007b0f67bd8036 libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_9d8000 + 54
frame #7: 0x00007b0f67bd980d libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_9d9710 + 253
frame #8: 0x00007b0f67c0340a libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_a033b0 + 90
frame #9: 0x00007b0f67bd7fe2 libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_9d7f60 + 130
frame #10: 0x00007b0f67bdcd5e libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_9dcce0 + 126
frame #11: 0x00007b0f675df237 libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_3df160 + 215
frame #12: 0x00007b0f6a598132 libnvidia-gpucomp.so.580.196.07`___lldb_unnamed_symbol_3397eb0 + 642
Hm, pthread_mutex_timedlock in libnvidia-gpucomp?
Inspecting the assembly revealed that it’s re-entering a lock. That suggested
that this mutex should be recursive, and for some reason it is not. Then a
friend remembered that glibc has static initializers for these, particularly
PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP, while musl does not support static
initialization, only via pthread_mutex_init, for this case.
How to deal with this? On glibc, the mutex structure is publicly visible,
and has roughly this layout:
struct mutex {
int lock;
unsigned count;
int owner;
unsigned nusers;
int kind;
...
};
Then the initializer looks like this:
enum
{
PTHREAD_MUTEX_TIMED_NP,
PTHREAD_MUTEX_RECURSIVE_NP,
PTHREAD_MUTEX_ERRORCHECK_NP,
PTHREAD_MUTEX_ADAPTIVE_NP
}
# define __PTHREAD_MUTEX_INITIALIZER(__kind) \
0, 0, 0, 0, __kind, 0, 0, { 0, 0 }
# define PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP \
{ { __PTHREAD_MUTEX_INITIALIZER (PTHREAD_MUTEX_RECURSIVE_NP) } }
That means it’s initializing the structure with the kind field.
Meanwhile, musl mutex type is opaque:
typedef struct {
union {
int __i[sizeof(long)==8?10:6];
volatile int __vi[sizeof(long)==8?10:6];
volatile void *volatile __p[sizeof(long)==8?5:6];
} __u;
} pthread_mutex_t;
The mutex type is encoded in the first integer:
#define _m_type __u.__i[0]
In this layout, the kind would look like this, mirroring the glibc structure:
#define _m_kind __u.__i[4]
I found that musl does not use this in any way, however.
We can use this. The strategy I took was creating a new shim library containing the following:
int __nvcompat_mutex_lock(pthread_mutex_t *m) {
patch_mutex_kind(m);
return pthread_mutex_lock(m);
}
int __nvcompat_mutex_timedlock(
pthread_mutex_t *restrict m,
const struct timespec *restrict at
) {
patch_mutex_kind(m);
return pthread_mutex_timedlock(m, at);
}
int __nvcompat_mutex_trylock(pthread_mutex_t *m) {
patch_mutex_kind(m);
return pthread_mutex_trylock(m);
}
The patch_mutex_kind fetches the (on musl, unused) kind field and writes
it into the type field`, atomically, if it’s unset. On statically initialized
mutexes, it is always unset, as it’s only written once, and the static initializer
for the default mutex just zeroes everything.
The actual encoding of the type is the same between both libcs (with the caveat
that glibc also supports adaptive mutexes, which we map to regular ones), so
there is not much remapping to be done. You can see the code in the pull request.
How do we use these symbols? Via patchelf again. It supports --rename-dynamic-symbols,
which takes a mapping file where each line looks like oldsym newsym. We just
provide ours like this:
pthread_mutex_lock __nvcompat_mutex_lock
pthread_mutex_timedlock __nvcompat_mutex_timedlock
pthread_mutex_trylock __nvcompat_mutex_trylock
Then we patch our libnvcompat into NEEDED of all the libraries.
And that’s mostly all.
Open-source components
There are various bits and pieces NVIDIA has released over time as actual
FOSS code. This includes nvidia-xconfig, nvidia-settings, nvidia-modprobe,
nvidia-persistenced, and the egl-gbm and egl-wayland external platform
libraries.
These come bundled in the driver archive. Since they can be built from source, we package them separately, to minimize the amount of stuff that needs to be shimmed, and then depend on the from the driver package.
Testing for real
Yes, it works.
For the 620 driver, we also had to replace libstdc++.so.6 in a couple places,
where it was being used for the new operator. This is provided in an way that
is ABI-compatible by libc++abi.so.1 so we just replace it.
The packaging will need some more refinement, and more testing will be needed. It was reported to me that the 620 driver works with RTX 3070 Ti. It was also reported to me that it works at least partially on AArch64.
But anyway, that’s it for this writeup.
November 03, 2026
New images
The 20261103 set of images is now published.
This is an incremental refresh.
Changes
It wasn’t originally planned to take a year refreshing the images but here is finally a new version.
There isn’t anything particularly special about the images themselves but this comes with a full set of updates around the system that happened in this year.
The most notable change that affects live usage is that the base image now comes with NetworkManager, replacing iwd. That should make networks easier to configure even for base systems without GUI (wireless configuration and so on) while being ready for removal of AF_ALG.
This set is based on kernel 7.3, GNOME 51, and Plasma 6.8. Newly, none of them come with the X.org server.
Next time
The next set hopefully won’t take a year again…
October 07, 2026
Creating distro build tooling for a small community
A lot of people have been commenting on this over time so I figured I would put together a little post summarizing my thoughts on the matter and everything else.
The post is going to be quite large. Every section can be read more or less separately, but they are all connected to each other.
Catering to the community
Chimera is a small project. It exists primarily to serve its community and not any external entity.
Making sure the maintenance cost is as low as possible is crucial. Chimera and its tooling are designed to automate away most of the boring stuff and make sure the packager’s job is pleasant and not bothersome. In a way, we aim to empower every user to be a packager. The barrier of entry is meant to be very low.
Some things that implies (I will try really hard to make this reasonable to follow, unfortunately the years have left their mark and I might be taking some assumptions for granted, so please bear with me; this also applies in the later sections):
- Everyone uses the same tools. Inefficient UX patterns are identified,
and either fixed, or
cbuildis extended to mitigate them. Feedback is taken into account. - Everything is
cbuild. One tool does everything, in a streamlined way. Managing the repository, the build environment, the repo generation and signing, even parts of the VCS handling and common maintenance tasks. No external helper stuff. - The remote build infrastructure just runs
cbuildand not much else. - The tooling does much of the bulk of making sure your packaging is correct and clean, most issues are hard errors. Heavy sandboxing, build environment consistency, unit tests by default, etc.
- You can run it on any Linux. If you run it on Chimera, you can immediately test your work. You can take the repo and bring it somewhere else. That also means the builder machines in the remote infra can run anything.
- Chimera is a collective effort. You are expected to share the stuff you
make, and get it upstreamed to us. The tooling is not intended for local
things that won’t get shared, and it provides no guarantees or obligations
for such usage;
cbuildis a developer tool, not a user tool. But every user can be a developer. - The tooling should easy and straightforward to use, and fun. It should not make things hard for you. Every user can be a maintainer. It shouldn’t be unnceessarily intimidating. We don’t gatekeep here when possible. You should also be having fun using the system and being here.
- You can replicate the entire remote infrastructure of the project on your machine in an hour or something. You can replicate the heavy bulk of things in like 5 minutes.
- The way things work is supposed to steer you towards implicitly doing the right thing, and punish incorrect patterns by making them harder than the correct ones. E.g. it’s really difficult or impossible to manually patch files with regex, or to do internet-reaching stuff during the build, or to touch the build environment filesystem. It’s really easy to manage patches though, there are build styles for common build systems, there are fine grained utility modules for doing all sorts of common annoying things, etc.; the tooling also automatically checks your stuff for correct formatting and other lint issues, validates your dependencies, validates your metadata including minor things like whether your build dependencies are sorted correctly and whether the SPDX license expression is correct and lost of other nits, and so on.
- Extensive documentation for the build system and packaging.
A common workflow setting up everything from scratch would look like so:
$ # set up environment with your cports fork
$ git clone https://github.com/my-user/cports
$ cd cports
$ # prepare a signing key
$ ./cbuild keygen
$ # prepare a build environment
$ ./cbuild bootstrap
$ # write your template here; then build it
$ ./cbuild pkg user/my-cool-program
$ # make a branch for submission
$ git checkout -b my-cool-program
$ # automatically makes a commit with the correct message and all files included
$ ./cbuild commit user/my-cool-program
$ # push and publish PR with the link
$ git push origin my-cool-program
An update would be done like this:
$ ./cbuild bump-pkgver user/my-program 4.20.69
$ ./cbuild prepare-upgrade user/my-program
$ ./cbuild pkg user/my-program
$ ./cbuild commit user/my-program
$ git push
The tooling provides common utilities for keeping your local repository clean, e.g. pruning packages and so on; it provides maintenance tools for bumping versions and revisions, checking dependency graphs, checking for new versions via update-check (which for most things distributed e.g. via common git forges requires no extra effort or specialized code), checking if your local repository can be unstaged against the remote one (verifying if you really rebuilt everything that needs it), updating sha256 checksums in templates. It even supports custom template-specific actions, such as building bootstrap tarballs for compilers. It supports aliases for less typing.
Checking for if a package has an update:
$ ./cbuild update-check main/firefox
main/firefox: 157.0 -> 157.0.1
It supports fun things when it comes to packaging bulk batches. For instance, it integrates git:
$ # build everything changed in your local commits
$ ./cbuild bulk-pkg git:origin/master..HEAD
$ # build a specific commit
$ ./cbuild bulk-pkg git:commithash
$ # build a range of commits but skipping stuff that has "test" in commit message
$ ./cbuild bulk-pkg git:from..to+!test
It supports common stuff, like
$ # build everything you have in your local repo that has a newer cports version
$ ./cbuild bulk-pkg status:outdated
There is a lot more that I could mention. I also wanted to talk about how we got here, how we started, and how our infra works.
Personal history
I started using Linux in 2006/7 and soon ended up on Debian as my long-term operating system. Around the same time I started to seriously get into programming and the intersection of that ended up being packaging for Debian. Debian has a large bunch of tooling to deal with packaging and particularly the community-driven approach really appealed to me so I maintained a bunch of packages for a while. That lasted for some time until I drifted away towards other things and eventually settled on FreeBSD.
During that time I didn’t do any work for the FreeBSD project itself but I did experiment with various things that were adjacent, none of them going anywhere in the end, but I did gain a lot of helpful experience on the way.
In 2016 I started messing around with Void Linux (particularly as I had to use Linux on work computers) and around 2018 I ended up using the ppc64le (POWER) platform for my workstation, which FreeBSD did not support at all at the time. I was already familiar with Void and decided to start a ppc64le port, which I maintained downstream until 2023 and gradually it expanded to support the big endian ppc64 as well as the classic PowerPC. Doing the downstream work led to me becoming an upstream Void maintainer as well and I took care of the compiler toolchains among other things for several years and became interested in improving the build tooling.
In 2021 I started the Chimera Linux project, which gradually got better and more usable, and that led to me shifting entirely to that, deprecating the POWER port of Void and resigning from the Void team in 2023.
Void Linux
When I started using Void in 2016, it mainly caught my attention as a system with a fairly low barrier of entry that did not get in my way, while still being “normal” enough to act as a regular Linux system. Initially I found the way e.g. runit worked in there interesting and in many ways fresh over classic distros particularly in the pre-systemd era, while I found the post-systemd era rather unfun and daunting in general.
This was also why I ended up picking Void for the ppc64le port, as it felt small scope and approachable enough to give me a chance to reasonably maintain it by myself, while also having a compact and easily reachable community that wasn’t aligned with any particular commercial entity.
When I started the ppc64le port, I was already marginally familiar with the build tooling of Void but only really properly picked it up for the port.
Getting started with xbps-src
For someone coming from most other Unix-like systems, xbps-src is really nice.
In Void, all the packaging of the distro, along with the build tools, exists
in a singular Git repository (void-packages). In a way, this mirrors the
ports systems as they exist on the BSDs. However, the ports systems are based
around Makefiles and you interact with ports individually using make (with
various external ports management tools existing to simplify that, working
on top of that).
In Void, software is packaged using “templates”. Every piece of software
consists of a template containing metadata fields (you know, package name,
version, dependencies, source URLs, etc.) plus (optionally) functions that
define the logic of how a template is built, along with extra files that are
necessary for the build (not always) and patches (not always). The tooling
for building templates (xbps-src itself) also exists in this tree.
Unlike most other systems, when you build a package, the build process does
not run in the system you are running the build on. Instead, xbps-src
constructs a small container (using Linux namespaces) that represents a
minimized Void system, puts stuff in it according to the template metadata
(build dependencies and whatnot) and then runs the build. At the end, you
get a local repository of xbps packages that you can install from.
The general workflow looks like this:
$ git clone https://github.com/void-linux/void-packages
$ cd void-packages
$ # create the build container
$ ./xbps-src binary-bootstrap
...
$ # build the thing you want
$ ./xbps-src pkg some-program
...
The templates are similar to e.g. PKGBUILD format of Arch Linux or
APKBUILD format of Alpine, but those do not use the containerized
approach (at least not always).
The xbps-src system is a collection of Bash scripts, and technically
the templates are also just Bash scripts, but with a set of expectations
so they in general do not utilize the full extent of the syntax and features.
When working as a packager, you basically just create a new directory in the
srcpkgs, put your template in it along with other stuff, build it, and
then you have a repo you can install it from. If you wish to submit it
upstream, you create a branch, commit it, and create a pull request.
Someone will take a look at it, and if it gets merged, the central build
infrastructure picks it up and shortly it becomes available in the central
repository.
The system neatly decouples build dependencies and runtime dependencies, with runtime dependencies typically scanned automatically from ELF files and their metadata as well as other file types, so the template specifies whatever it needs in the build environment but the runtime dependency list is largely automatic.
The system supports “build styles” so you can e.g. declare that a template uses the GNU Autotools, or Meson, or Cargo, or whatever, and it takes care of most of the groundwork, allowing for templates that are often declarative without any build logic, lowering that barrier of entry.
It also supports cross-compiling really well, letting you cross-build most of the repository to any other architecture. That’s neat, but often sloppy, and cross-built packages end up with subtle brokenness that native packages would not have, due to build systems subtly not passing certain checks and so on. You also can’t run unit tests for projects when cross compiling them, in most cases.
The system also has a really handy update-check mechanism which will scrape
upstream URLs and find version infos in them, then present you with any updates
that may have happened since the last template version update. The project has
a nightly job which generates a summary once per day, letting packagers stay
on top of things.
Upstream build infrastructure of Void
The build infrastructure of Void consists of several computers (each handling
a different architecture port) that are plugged as workers into the central
orchestrator, using the Buildbot software. It will
receive updates from the Git repository (void-packages), collect a batch
from the changes, figure out the correct order, and use xbps-src to build
the packages.
It will also sign the packages with the right keys. This is a separate step
that xbps-src does not handle (the output is unsigned). Eventually, things
make it in the upstream repository and get mirrored.
The way the Buildbot is managed I can’t tell you much because I never saw
much into it. The admins have an infrastructure that is based on Ansible,
Terraform, and a bunch of other pieces that are put together in ways unfamiliar
to me. Additionally, xbps-src does not do the batch sorting for you, so
other pieces of the process have to do it. Things outside the void-packages
repository itself are relatively opaque.
When I maintained the POWER port, I had my own set of scripts to keep things going and did not reuse any of the Void infrastructure at all. This was likewise opaque and purpose-built for my port.
Starting Chimera
In early summer of 2021, I started Chimera. The initial push for me was that I wanted to experiment with a different userland setup, as well as have a more personal project to work on without having to deal with others’ efforts and pre-existing work, but also I wanted to try out my own take on the build system.
There were some initial points I was unhappy with in xbps-src while
maintaining the POWER port.
- The shell-based system was way too slow. Parsing the complete collection
of templates in the repository would take potentially as much as half an
hour, and there weren’t other ways to introspect the templates. Therefore,
my tooling would call into
xbps-srcand employ various caching tricks to make things manageable. - The shell-based system was very often fairly sloppy, letting various wrong behaviors through. For instance, network access is permitted through the whole build, there is nothing ensuring consistency of the build container after the build is done (the entire thing is read-write and the template can do whatever it wants), the correctness lints for the resulting packages are fairly slim, the ELF scan step and other things would take an eternity due to slow shell code, and limited opportunities for doing more due to shell being excessively slow.
- A lot of useful tooling is separate from the main system and maintained
in the
xtoolsrepository, including extra lints and so on. This is not mandatory or anyhow verified however. - Lots of sloppy templates resulting from the prior points. E.g. the templates
are often littered from manual in-place
sedcalls and similar rather than using proper patches (which leaves in calls that eventually no longer do anything due to upstreams changing), patches are applied very fuzzily which occasionally results in subtly mispatched things, and so on. - The Void project does not run per-template unit tests (typically the test
suite of the software being packaged) on builders, which would catch a lot
of errors, particularly on
musltargets and so on. It does run them in pull request CI, but I do not see this as enough. Often the check runs of templates are poorly maintained and broken. - The Void project cross-compiles all architectures other than
x86_64andi686, resulting in the other-arch ports being notably lower quality. - Void has a wonky staging system. When large batch changes are being done, the repos may end up in a strange state for a while and users are advised to avoid upgrading until everything is finished. It has a rudimentary staging system which prevents the repos from being changed while a big batch is being rebuilt for changes shared library SONAMEs, this works only for shared libraries however, which means packagers will often forcibly stage the repo for a batch rebuild, then do stuff in several commits, wait for everything to clear, then unstage the repos. Various parts of the related infrastructure are also lacking, such as clearing obsolete and removed packages from the repos, which is/was done manually.
There are others but these were my main gripes.
Chimera started with its build system. Many experiments were done. Initially,
cbuild was a ground-up rewrite of xbps-src, using Python. In cbuild,
the templates are also Python scripts, but through low-level bits that Python
allows, don’t always look as such. It came with a very minimal set of initial
packages that was basically a basic Void-like chroot, using xbps as its
package manager. Thus cports was born.
There were two initial goals:
- Make it fast. I should be able to parse several thousands of build templates and dump all their metadata in under a second, rather than many minutes. Chimera currently achieves this.
- Make it strict. No network starting with configure step. Read-write access
only in the build directory (and destination directory for install step)
and full consistency of the build container guaranteed at all times.
Heavy linting and straight up denying various misbehaviors at all times.
Sandboxing done with namespaces only, using Bubblewrap (
bwrap). Thecbuildsystem runs outside the sandbox (in your host environment) while any calls to the build system of the project being built are sandboxed. - Make it highly portable, with Python (no external modules), Bubblewrap,
and Git, along with the package manager binary, being the only dependencies.
You can use
cportson any Linux system with the basic dependencies. - Make it able to do everything
xbps-srccan do, but better.
Chimera started with the ppc64le target only, being developed on a POWER
workstation. Over time, as things cleared up more, more targets were added,
along with additional goals and other changes.
Over the next months, the core packaging started to become more defined,
settling on the userland tooling and other things, as well as transitioning
from xbps to apk. The original idea was to use pkg from FreeBSD, but
this proved to not be ready for our style of use at the time, and upstream
suggested that we’re best off using something else.
Fundamentally, using cbuild in the basic sense feels similar to xbps-src.
The big difference is that cbuild signs everything, not needing a separate
step, and no unsigned packages are allowed. Therefore, initial setup looks
more like
$ # generate your signing key
$ ./cbuild keygen
$ # bring up the container
$ ./cbuild bootstrap
$ # package whatever you want
$ ./cbuild pkg main/firefox
Settling on apk
We didn’t stay on xbps for very long. Soon, the switch happened to apk.
Using apk brought over various benefits.
- Unlike
xbpswhich is driven by ad-hoc logic,apkhas a real dependency solver. This means way fewer surprising behaviors and way fewer workarounds and less effort needed from packagers to make sure that users’ systems do not break in surprising ways. - Handling of stuff like shared libraries is way nicer in
apk. Inxbps, dependencies are driven purely by name, and shared library providers and requires have separate metadata fields. These will get checked andxbpswill not permit things to proceed if unmatched, but dependencies are done by name. When a package is being built, the system will consult this massive central file (common/shlibs) matching SONAMEs to package names to generate dependencies on top of the shlib metadata. This results in poor UX (you cannot match the shlib providers etc. in the same way as names when e.g. installing) and is inflexible as it’s limited to shared libraries. Inapk, virtual packages are used instead, so you e.g. provideso:libfoo.so.1=1as a virtual package name. You can then search by this name, install by this name, etc. and it can be used for other types as well, e.g.cmd:foo(apk add cmd:i-know-command-name-but-not-package-name) and others. That also means more things can participate in the staging system on the build system side. And no central mappings, as the build system can easily query things from the repository. - Unlike
xbps,apkhas real support for triggers. Triggers inxbps-srcare emulated with package scripts. As an example, consider e.g. updating fonts cache. Thefontconfigpackage has a trigger (which is a script) and metadata to run the cache update when/usr/share/fontsis modified in any way. Then when something modifies the directory, the trigger will run at the end of the transaction, after all filesystem changes are done. Inxbps, you can only have scripts that run before or after a package changes and are owned by the package;xbps-srcemulates triggers by checking if the package contains a directory that would drive the trigger and if it does, insert a call in its package script; that means if several packages change the fonts, each of them contains a cache update script, which will run several times (and not at the end). This means things become quite slow for large triggers and the transaction breaks atomicity (because the shell script run in the middle of the transaction, rather than at the end, needs all prior things committed to the final filesystem location).
There were others, but these are notable.
Building an infrastructure and a new core tenet
Building the infrastructure was interesting. We ended up using Buildbot again, after some deliberation and original intent to build a new orchestrator from scratch. There is an important distinction in our buildbot, however.
I wasn’t happy with the opaqueness of Void’s infrastructure, and with the way
it does so much work beyond xbps-src. I declared an entire new rule that goes:
What the packager runs exactly matches what the remote build machine runs.
The build steps of Chimera’s infrastructure go approximately like this:
- Central orchestrator is poked by a webhook that a change in
cportshas happened. It does not matter what change. - Central orchestrator tells worker machines in the fleet (one per arch)
that
cportshas changed. It does not tell them what. - Worker machines pull their copy of
cports. It does not matter what has changed. - Worker machines update the build container (
./cbuild bootstrap-update). This updates the core packages of the container to match the repository. - Worker machines run
./cbuild bulk-print-ver status:unbuilt. This generates a list of all packages along with their versions that are present incports, can be built, and are not already built in the repository. This step is separate only for the purpose of presenting it to packagers in the Buildbot UI, so that you can have a separate view for each package being built and so on. This list is correctly sorted. Each entry is saved in a Buildbot variable, again for the purpose of presentation mainly. - Each entry in the list from the prior step is built. The result is already signed and ready to be used, but for now remains in stage area.
- An unstage step is attempted. If any provider is removed in the new
packages (e.g. something rebuilds and stage provides
so:libfoo.so.2while the original provider wasso:libfoo.so.1) and there is still any package depending on the old name, the unstage will fail. That ensures that all reverse dependencies are always rebuilt as necessary before publishing. On successful unstage, the staging area is merged into the repository. - Repository is pruned for outdated stuff.
- Repository syncs to the final primary mirror. First, changed packages get uploaded in one step, then indexes are replaced, and only then old packages are deleted. This is to ensure that users don’t end up with any index that contains packages not present yet.
Steps 4, 5, 6 can be condensed together into a single cbuild command:
$ ./cbuild bulk-pkg status:unbuilt
The main tricky part with batch builds like that is correct sorting. Imagine having packages A and B in the bulk that you want to build. There is a package C, which A depends on, and which depends on B. Since that makes B in the dependency tree of A (through A->C->B), B needs to be built first. You do not know this by knowing only A and B, since C is not in the batch.
Since cbuild is very fast and can parse the entire collection very quickly
without caching, it can consider all the intermediates and do a proper sorted
graph trivially. It can also simply parse every template to consider whether
it’s buildable, and match it against the repo state. Therefore, on every build,
it can always do that from scratch and the orchestrator has to do nothing,
with each step being largely independent of the other.
The build fleet doesn’t do anything else. There is other auxiliary infra, such as our IRC commit bot (which announces changes driven by a webhook) and our nightly update-check equivalent to Void, letting packagers stay on top of things.
In general, everything that any piece of our infrastructure does can be easily done locally.
Our fleet is self-hosted. The orchestrator and most builders run on Chimera host systems, but they don’t have to. Currently these are:
- The
x86_64machine, which is a 16-core EPYC root server at Netcup, running Chimera. - The
aarch64machine, which is an 80-core Ampere Altra, running Chimera, owned by me. - The
ppc64lemachine, which is an 18-core/72-thread POWER9, running Chimera, owned by me. - The
loongarch64machine, which is a Loongson 3A6000 4-core/8-thread, likewise running Chimera, owned by me. - The
riscv64machine, which is a Milk-V Pioneer 64-core machine kindly provided to us by Zach van Rijn of Adélie Linux, running Fedora.
The builders do not publicly face the Internet, the orchestrator does, on its own machine. The primary repository is also on its own separate machine. Each architecture has its own signing key.
Final words
All of this may possibly even be a bit too long and too much to take in. Therefore, I will end it here and hope that I have not forgotten anything. I hope this was interesting to read and I didn’t bore you to death, and that it wasn’t unnecessarily difficult to follow.
As always, you can ask anything on our IRC.
March 13, 2026
Plans to possibly retire the big-endian PowerPC/POWER platforms
Unless circumstances change, we are planning to retire support for the big-endian 32-bit PowerPC and 64-bit POWER platforms. Read below for when and why and how you can potentially change this.
Note that this is not about ppc64le, which is not affected by this.
Reasons
We’ve been supporting these platforms in Chimera since a while back, with ppc64 being suppported in repos since the middle of 2023 and 32-bit ppc a while later.
We brought up the platform support to give people interested in the platform a base to bring the platform towards greater usability and fix the long-standing bugs that plague it. This has not happened, and so far things have been getting very slowly worse instead.
Occasionally users come in looking to test it and often expect a fully working platform but generally quickly lose interest when faced with various issues and in the end nothing gets fixed.
For instance, it has not been possible to run a working GPU either platform for several years, due to unfixed regressions in Mesa, requiring use of the old Amber branch to get these working (which most people opt for and which also ends up counter-productive for fixing things upstream).
Likewise, there has not been an up to date web browser, and things are looking to regress further e.g. with WebKit having adopted Skia for rendering (WebKit was the sole web engine left where rendering didn’t have broken colors).
We don’t have a dedicated maintainer for the platform and so far I (q66) have been working on keeping things at least building but I don’t have the resources or motivation or real interest in keeping this going.
Our build infrastructure for this is also lacking, with both platforms being built on a single virtual machine hosted at OSUOSL. The performance is for most part acceptable but e.g. we’re frequently running out of disk space as the repos grow as the machine is barely capable of hosting two sets of repos.
I have some POWER9 hardware that could be set up as a build machine to fix this but I’m not sure if this is worth running if I’m going to be the only person giving this attention, so for now this is not going to happen.
Timeframe
Therefore, in order to lighten my workload I plan to retire the repositories and builders by July 2026.
In the meantime, if someone steps up and is willing to dedicate time and resources to maintaining things well, actually using it, and fixing the issues that make it unusable for most people, I am willing to retract this decision and set up a better builder, but only if it actually seems like there is a future for the platform in the distro.
If not, the repositories will be removed after the middle of the year, along with packaging only relevant for these platforms (such as mesa-amber).
December 20, 2025
New images
The 20251220 set of images is now published.
This is an incremental refresh.
Changes
This set has been overdue for a while so it mainly comes as an update with new versions.
This set is based on kernel 6.18 and comes with latest versions of desktop environments and most other software.
Newly, the live images will not automatically import LVMs and ZFS pools, as that was found to be unintuitive and sometimes system-breaking (with ZFS-on-root setups).
Additionally, the images now come with an experimental TUI
installer. It can be invoked as chimera-installer from the
command line.
The installer supports the following:
- Local and remote installs
- APK mirror setup, with up to date mirror list fetched from the repo
- Basic setup (hostname, timezone, root password, user account)
- Installing extra packages
- Kernel version selection
- Bootloader setup (GRUB and systemd-boot, including BIOS/EFI/OF)
It does not support the following:
- Disk partitioning and formatting
- And probably a lot of other things
You are expected to partition, format, and mount your filesystem layout beforehand and then point the installer to the mounted root. This is by design and will remain that way, however a separate tool for simplified disk setup may be provided later on.
However, it does validate your mounted filesystem tree for baseline
correctness, performs detection of system-specific things (like ESP)
and automatically generates crypttab for encrypted setups (this is
rudimentary so if you use keyfiles or other advanced things you need
to tweak it afterwards).
Next images
The next set will come early next year with some more major changes.
April 20, 2025
New images
The 20250420 set of images is now published.
This is an incremental refresh with some major changes.
Changes
There are some major changes in how the images are set up.
One is that EFI/BIOS images now use the Limine bootloader for the
image itself, instead of GRUB. This does not affect what bootloader
is used for the installed system (that is the user’s choice) but
it’s the first step towards moving GRUB to the user repository
eventually.
The PowerPC/POWER images still use GRUB. For installed systems,
the recommended choice is systemd-boot on EFI and GRUB elsewhere,
Limine does not yet have distro integration.
The x86 images newly use MBR for best compatibility with all hardware.
Architecture support has been expanded. On top of the existing archs from before (AArch64, PowerPC, POWER LE/BE, RISC-V, x86_64) we now ship images for the LoongArch64 architecture.
The AArch64 image list has been expanded, with Rock64 and QuartzPro64 being added.
Various fixes have been made; notably, issues with cross-device links
and local installs with separate /boot should be gone, and apk
will no longer segfault when network connection is unavailable.
Everything has been updated and the images come with kernel 6.14, GNOME 48, Plasma 6.3.4, and a variety of updated software.
Planned changes
For the next image set we plan to provide images for mobile phones and generally more mobile packaging.
Additionally, the ARMv7 architecture will be introduced to the builders and then to release images.
Device SD images will automatically grow their root partition and filesystem on first boot from next time but this has not yet been deployed.
Additionally, Limine will get distro integration and will become a convenient option for installation.
March 20, 2025
Not dropping RISC-V support after all (maybe)
As circumstances have changed, we are not dropping RISC-V repos for the time being. Instead, newly rebuilt repositories are introduced, built on hardware, with tests.
This support is provisional for now, with the new builder still being evaluated to see how it holds up in the long term.
The situation now
Shortly after announcing the drop, we were offered remote access to a Milk-V Pioneer machine by Zach van Rijn of Adélie Linux. This machine was originally intended for another purpose which never ended up materializing.
I proceeded to do a full world rebuild on this machine, after some environment setup to allow our infra bits to run. This world rebuild is now finished, and makes up the new repository.
For most part, it was relatively stable during the build (we had to build our own kernel to prevent the draft RVV 0.7 in the CPU from interfering, and there were two crashes but it was also under total continuous load the whole time).
The performance is fairly acceptable, though nowhere near my original idea of being similar to Cortex-A72; the cores are more comparable to Cortex-A55 in practical performance, especially since we have to disable vectors. As there is still 64 of them, most of the large projects build fairly fast (anything written in Rust builds very slowly, however).
By now, the original repositories have been replaced, and the new
machine is plugged into the infrastructure. Do keep this in mind when
upgrading existing installations, and use the --available flag with
apk (every package in your system will be reinstalled).
Either way, we will continue to monitor the builds and see how the new machine holds up. If it works well, it will stay; if significant issues arise, we might end up dropping the architecture after all, at least until something significantly better is available.
The current repository is in the same tier as the LoongArch64 repo. The specifics are very similar - i.e. no LTO, tests on and enforced. The overall coverage is also fairly equivalent.
March 12, 2025
Dropping RISC-V support
UPDATE March 20 2025: The architecture is not being dropped for now after all. See the newer article for details.
The next set of images will drop RISC-V support. The builder is currently still going but within the next few days it will stop, and the repositories will stay in place but frozen.
Nothing will change in packaging (the build profile will remain, template support where present will remain, cross-toolchains will remain) but there will be no more updates to the repo for the foreseeable future.
The situation
The initial plumbing for RISC-V was added in the distro in July 2021
and repos later in the year, i.e. it has been there almost from the
start. During all this time, the builds have been supported by doing
so on an x86_64 machine with qemu-user binfmt emulation coupled with
transparent cbuild support for this.
The reason for doing it this way was that there wasn’t any hardware we could use for performance reasons; I had obtained a SiFive HiFive Unmatched board in October 2021 and this proved to be useless for builds as the performance of this board is similar to Raspberry Pi 3. Other boards came later, but none of them improved on that front significantly enough.
This was expected to be a temporary state that would resolve itself within 2-3 year time; it is Q1 2025, and the options are the following:
- HiFive P550 that was released recently has performance similar to Raspberry Pi 4 and is unsuitable for the task; this board was originally supposed to be released several years ago as part of the SiFive and Intel collaboration (Horse Creek) but now got released with a Chinese SoC instead
- Milk-V Pioneer is a board with 64 out-of-order cores; it is the only of its kind, with the cores being supposedly similar to something like ARM Cortex-A72. This would be enough in theory, however these boards are hard to get here (especially with Sophgon having some trouble, new US sanctions, and Mouser pulling all the Milk-V products) and from the information that is available to me, it is rather unstable, receives very little support, and is ridden with various hardware problems.
- Things based on Spacemit K1 (e.g. Milk-V Jupiter) have an 8-core SoC that is technically an out-of-order design, but in practice the per-core performance is reportedly even worse than the JH7110, so it is unsuitable.
- Boards based on JH7110 (e.g. VisionFive 2, the new Framework board etc.) utilitze 4 U74 cores (same configuration as my HiFive unmatched) that are simple in-order designs and therefore are unsuitable (similar to RPi3).
- My HiFive Unmatched, which is the same situation as above.
- Other available cores are usually much worse than any of the above.
The promising option (Milk-V Oasis with 16 SiFive P670 cores) that was first announced in 2023 ultimately ended up being canned due to issues the SoC vendor has, and nobody has ever seen a single production chip, let alone a board. As far as I can tell, no other options are coming up.
It is unsustainable to stick with the current situation with the emulator. Doing so has numerous problems:
- We could never actually run tests on the packages being built, because the emulator is unreliable and will result in false positive failures. Disabling stuff conditionally for RISC-V is not a viable option because they are not RISC-V issues and will always happen with emulation, so all the RISC-V packages were being built without tests.
- It is very slow, being by far the slowest builder in our fleet. It is still several times faster than e.g. the JH7110 would build things. The performance is actually rather variable; things that can parallelize really well run at a fairly reasonable speed due to being able to spawn many emulators, while things like configure scripts that are single thread and fork a lot run very slowly. Either way, overall, it is much slower than any of the other builders, despite RISC-V being until the introduction of LoongArch64 builds the only architecture with no LTO.
- Most importantly, it is unreliable. The
qemuemulator likes to hang during various workloads, with the emulator going into sleep state and remaining there forever. When that happens, the builds have to be manually canceled and restarted (it is not deterministic). This used to be worse before before some fixes, but even with latest version of the emulator it still happens, particularly during Go builds (since we rebuild every Go program upon toolchain updates for secfixes, any such rebuild can require many manual cancelations and restarts). - It burns a ton of power for how slow it is, because it fully loads a beefy x86 machine, and I’m not happy at all about that.
At this point, to have a relatively sustainable base, we’d need a board that is at least as powerful as Raspberry Pi 5. This would still make the slowest builder in the fleet, but it would likely be faster than the current emulation arrangement while also being more reliable.
However, the industry does not seem to be interested in producing such machines and for most part focuses on embedded (low-end) as well as things entirely irrelevant to a distro (AI/NPU etc.) that do not help at all; at this point I don’t think we can wait any longer, especially as no remedy has been announced.
We have no such problem with the other architectures; obviously x86 and ARM are at this point mainstream and this does not surprise anyone, but even the likes of LoongArch have perfectly acceptable hardware (not the fastest, but also not a bottleneck) that performs reliably.
Will RISC-V support be reintroduced?
If acceptable build hardware is released and is reasonably available to us, the architecture will be reintroduced.
If that happens, the repositories will be rebuilt from scratch, as if a new architecture, with a process similar to how it was recently done with LoongArch64. It will be a tier-2 architecture with enforced tests and without LTO just like LoongArch64.
However, whether or when that will happen is currently a big unknown due to such hardware not existing and nothing being even announced.
Nothing will change in the other architecture support. The new tier list will be:
- Tier 1 for
aarch64,ppc64le, andx86_64 - Tier 2 for
loongarch64 - Tier 3 for
ppc64andppc
There is also some chance of ARMv7 and ARMv6 32-bit repositories being introduced in the next few months’ timeframe, as we may be moving to an oversized Ampere Altra machine for all ARM builds (right now AArch64 is served by a Hetzner Cloud VM and can’t take any more load). This is yet not set in stone, however.
February 14, 2025
New images
The 20250214 set of images is now published.
This took longer than originally expected but there have been major changes that warranted waiting a bit longer for it.
Changes
The images come with a fresh version of apk-tools. This version
finally supports several features that we began using, particularly
variable expansion and being able to migrate most of its files into
a system-wide /usr location.
That means you finally have a way to properly change your mirror of choice without having to mess with the repository definitions. The process of doing that is in the relevant documentation section.
The repository definitions have been updated to use the new v3-style index naming, though backwards compatibility is also provided.
Kernel 6.13 is used in the new images. That means updated hardware support and other things.
Both the GNOME and Plasma images (the latter is still experimental) come with the latest versions of their respective desktop environments.
Various fixes have been made to allow the live system to work better and more seamlessly on more machines.
Additionally, 32-bit PowerPC images are now a standard release architecture and included in the batch. We have some plans to also introduce support for the LoongArch64 ISA, which may join them next time.
Due to all of these changes as well as updates in the infrastructure, this new set is the recommended baseline for installation. Older images have an out of date package manager and installation scripts, which may be problematic with the current layout.
December 27, 2024
Entering beta
Today we have updated apk-tools to an rc tag. With this,
the project is now entering beta phase, after around a year
and a half.
What actually changes?
In general, this does not actually mean much, as the project is rolling release and updates will simply keep coming. It is more of an acknowledgement of current status, though new images will be released in the coming days.
Changes since alpha
At the point of entering alpha, the cports tree had roughly
1000 templates, most of them in main. There was a single
large desktop (GNOME) and a single major web browser (Firefox)
and an assortment of other software.
At this point, the tree contains ~2800 templates, i.e. almost 3x more. We have all major desktop environments, all major browsers, and overall much larger collection of both small and large programs.
The repo was also at ~6000 commits at the time, by 11 authors; now it’s almost 20000 commits, by over 100 authors.
Significant under-the-hood improvements have been made in service
management, our build infrastructure, the cbuild build system
which is now significantly more powerful and has much better UX,
global switch to the mimalloc allocator, stateless /var and
progress towards stateless /etc, improvements in core userland,
introduction of libdinitctl, introduction of sd-tools,
and a lot more.
Infrastructure situation and sponsorship
Currently, we support 5 architectures (aarch64, ppc64le, ppc64,
riscv64, x86_64), 3 being tier-1 (aarch64, ppc64le, x86_64).
This list will likely remain stable in 2025. The infrastructure is self-funded and we control all of it. Besides the unsatisfactory RISC-V situation, all of the machines are sufficient.
It would be nice to introduce CI for more architectures during next year, particularly AArch64.
Chimera is a FOSS project and therefore does not and will not take donations, and is driven by its community. However, for the past half a year, I (q66) have been working on the project through my employment at Igalia, thanks to a contract with Rubicon Communications, LLC (aka Netgate). This collaboration will continue during 2025 and is a significant help and a boost for the project’s progress, as it lets me dedicate much more time.
Therefore, huge thanks to Netgate for giving me this opportunity.
Plans for 2025
During 2025, some notable things will be coming too:
- Complete system logging overhaul
- Support for mount units in service management
- Support for network mounts in service management
- Better cgroups support and progress towards removal of elogind
- Support for service-based timers
- Overhaul of service configuration files
- Switch to dbus-broker as the system and session bus provider
And likely much more than that. On the infrastructure side, we plan to automate more things, and introduce better build hardware for the RISC-V architecture if possible, as right now it is a major bottleneck.
Upcoming images
A new image set will be released before end of the year to match
this announcement. They will come with various fixes and a new
version of apk-tools.
