Arch Linux 2024

[Grogan]

There was a grub update today, for the first time in over a year. I keep that in my ignore list, so I don't miss it in a long list of updates and normally I wouldn't touch a working grub (sometimes for the life of the machine lol) but I've had occasional failures to load the kernel. The grub menu comes up, and I can choose but then it's just a blinking cursor where shows "booting Arch Linux" etc. (I can only see that if I let the timer run out, otherwise it's just a black screen). It hasn't happened in a month or so. I'm betting it's related to the FAT32 filesystem, can't find the whole kernel image. Just the way it happens makes me think that.

Anyway, I updated it, mounted the EFI partition at /boot (I manually mount and unmount that by script when I install a kernel) and ran

Code: Select all

grub-install --target=x86_64-efi --efi-directory=/boot --removable

To install it to the default location, I don't want it invoking efibootmgr to write to NVRAM on this board, it stuffs that up. It has no choice but to work, in this configuration.

I just use a hand edited grub.cfg, so that doesn't change (I made a new backup copy first just in case). I never like that grub-mkinstall garbage generator.

[Zema Bus]

I've been ignoring grub updates on Arch for a number of years, I got burnt by that before. One time I decided to let it update grub and it broke it again, so I've left it in my ignore list ever since.

[Grogan]

On the old computer I had an old Slackware install on /dev/sda1 and used that grub for many years. The partition was too small to matter (20G) so that was fine. I didn't boot the distro anymore, just mounted and edited grub.cfg. Then came a day when there were on-disk changes to ext4 and I found that newly formatted filesystems broke grub. I had formatted my arch filesystem and restored a current tarball snapshot to "defrag" (more like write all the files in directory order). I could still boot my LFS because I hadn't formatted that since the fs change. I was still able to boot into that old Slackware install, and I had to compile a newer version of grub and install it. I'm still booting that system from that.

So sometimes there are reasons that grub needs an update.

I'm always leery of grub updates. Sometimes they don't compile right too and are subtly broken (by the distributor I mean). We had one on this server last week. I was walking on my asscheeks waiting for the server to reboot after grub and kernel update

[Grogan]

That just happened today, after all that time, when I went to reboot to Gentoo to work on it. It then happened again when I booted back to Arch.

I'm going to try reformatting the FAT32 partition and blasting the data back with tar.

P.S.

I did that...

Code: Select all

mount -t vfat -o umask=0077 /dev/nvme1n1p1 /boot
cd /
tar czfp /storage/efipartitionboot-feb252025.tar.gz boot
umount /boot
mkdosfs -F 32 /dev/nvme1n1p1
mount -t vfat -o umask=0077 /dev/nvme1n1p1 /boot
cd /storage
tar xzfp efipartitionboot-feb252025.tar.gz -C /

vi /boot/grub/grub.cfg

(to change UUID... I thought the -i switch with mkdosfs would be able to specify one as the volume ID, but it's not a UUID, has to be a 32 bit hex string and won't take a hyphen, so I decided to play it safe and use what the kernel is getting from it, D238-4384)

e.g. boot stanza:

Code: Select all

menuentry 'Arch Linux' {
        insmod gzio
        insmod part_gpt
        insmod fat
        search --no-floppy --fs-uuid --set=root D238-4384
        linux  /vmlinuz-6.13.4 root=/dev/nvme1n1p2 ro mitigations=off split_lock_detect=off loglevel=3
}

That's all I had to do, as I'm just using the default path "EFI" rather than writing anything low level like you would with an MBR partition, or writing to any NVRAM. (I mean I don't need to run grub-install again)

It boots, but we'll have to see what happens in the coming days. There should be no fragmented files now at least. (FAT32 is terrible for that, there's no algorithms to avoid it)

[Grogan]

Grub malfunctioned again when I got up today and booted up. So it probably wasn't the filesystem, unless it's grub's vfat driver.

I tried compiling grub (intermittently crashing out?) and running grub-install again, and we'll see what happens. I removed ia32 support while I was at it (I won't ever be booting a 32 bit kernel on this 64 bit system). I probably could have removed "bios" (what they call it) support for booting from MBR too as unless I transplant the system to a disk with MBR partition tables, I'll never use that either, but I built that just in case.

[Zema Bus]

Odd that that keeps happening.

[Grogan]

It happened on the first boot with my new kernel. I powered off and it booted normally, and it happened again when I rebooted again to test it.

I have decided to get my kernel the fuck off that EFI partition. I want it on ext4 filesystems. I'm putting it in the root directory of each OS now. (I can easily do that because I manually edit grub.cfg and don't have a distro kernel installed)

All I have to do is put the kernel image in the root directory, and change the UUID in grub.cfg to make the kernel path relative to / instead of /boot in the line that actually loads the kernel and sets the rootfs to boot from (still /vmlinuz-6.13.5). That's the only place I use a UUID, for grub to find the kernel (not to set the rootfs or in fstab), and if I format the partitions, with ext4 I can use tune2fs to change them back to these after formatting so I don't have to edit anything if I restore a tarball snapshot.

Like this:

Code: Select all

menuentry 'Arch Linux' {
        insmod gzio
        insmod part_gpt
        insmod fat
        search --no-floppy --fs-uuid --set=root 73ce9b88-c44b-4454-a673-9e111ccc0dfe
        linux /vmlinuz-6.13.5 root=/dev/nvme1n1p2 ro mitigations=off split_lock_detect=off loglevel=3
}

menuentry 'Gentoo' {
        insmod gzio
        insmod part_gpt
        insmod fat
        search --no-floppy --fs-uuid --set=root fde6835f-ba29-407c-a345-a5ba24c9435d
        linux /vmlinuz-6.13.5 root=/dev/nvme1n1p4 ro mitigations=off split_lock_detect=off ipv6.disable=1
}

I don't need to load a grub filesystem driver for ext4.

So now it's only the boot loader on the EFI partition.

I can still build the kernel once for both OSes, I just need an extra line in my install script to also install the kernel image to the gentoo partition. This is how I install my kernels:

Code: Select all

#! /bin/sh

if [ -f "arch/x86/boot/bzImage" ]; then
  read -p "Enter OLDVER or leave blank to skip uninstall: " OLDVER
  read -p "Enter NEWVER: " NEWVER
else
  echo "You're in the wrong dir or your kernel isn't built"
  exit 1
fi

echo "Mounting filesystems CTRL+C to abort 5s"
sleep 5
mount -t vfat -o umask=077 /dev/nvme1n1p1 /boot
mount -t ext4 -o noatime /dev/nvme1n1p4 /mnt/gentoo

if [ -z "$OLDVER" ]; then
  echo "Not removing old kernel"
else
  echo "Removing old kernel"
  rm /vmlinuz-"$OLDVER"
  rm /mnt/gentoo/vmlinuz-"$OLDVER"
  rm -r /lib/modules/"$OLDVER"
  rm -r /mnt/gentoo/lib/modules/"$OLDVER"
fi

if [ -z "$NEWVER" ]; then
  echo "That was silly of you"
  umount /mnt/gentoo
  umount /boot
  exit 1
else
  echo "Installing Linux $NEWVER"
  install -m 600 -o root arch/x86/boot/bzImage /vmlinuz-"$NEWVER" || exit 1
  install -m 600 -o root arch/x86/boot/bzImage /mnt/gentoo/vmlinuz-"$NEWVER" || exit 1
  make modules_install
  make INSTALL_MOD_PATH=/mnt/gentoo modules_install
  echo "Opening editor for grub.cfg"
  vi /boot/grub/grub.cfg
  sleep 1
  sync
  umount /boot || exit 1
  umount /mnt/gentoo || exit 1
  echo "/boot and /mnt/gentoo unmounted"
  echo "Linux $NEWVER is installed"
fi

I've rebooted both OSes several times and it hasn't malfunctioned, but that doesn't mean I've solved it. I STILL don't want my kernel image on FAT32 even if it's not the problem. It just seems like it's something like that, the way it started happening more frequently than ever, after things were written back to it differently.

[Grogan]

It did happen once more last night even with the kernel image on the ext4 root filesystem. I was rebooting ridiculously to test it. It didn't happen anymore after that.

New tactic. I got brave enough to see if the old set root nomenclature still worked for EFI and NVME drives etc. and it does! No need for that search line to set root by UUID. Since I have NVME built right in the kernel, not relying on userspace udev, the drive ordering never changes, so it should be safe. hd1,2 and hd1,4 are /dev/nvme1n1p2 and /dev/nvme1n1p4 respectively. My life is now UUID free.

I also kind of knew those insmod commands for grub drivers were unnecessary (the grub-mkconfig garbage generator puts them there, at least the way Arch has it configured, so I used them), and I removed those too. The usual ones are already built in to the grub boot image (BOOTX64.EFI). Now this is the ultimate simplicity with less to second guess.

Code: Select all

menuentry 'Arch Linux' {
        set root='hd1,2'
        linux /vmlinuz-6.13.5 root=/dev/nvme1n1p2 ro mitigations=off split_lock_detect=off loglevel=3
}

menuentry 'Gentoo' {
        set root='hd1,4'
        linux /vmlinuz-6.13.5 root=/dev/nvme1n1p4 ro mitigations=off split_lock_detect=off ipv6.disable=1
}

[Zema Bus]

That's much cleaner! Think that will finally put an end to the issue?

[Grogan]

I won't really know for sure for a few months, if it seems to stop happening. If I say eureka, it will probably happen on the next boot

However, my philosophy is that if you want something to work, try to give it no choice but to.

[Grogan]

Another grub update in Arch today... "Fast forward to git master". Hmm... I hope somebody knows what they are doing.

I guess I'll build it. (once I take control of a package, I seldom give it back)

[Zema Bus]

I installed CatchyOS to see what that's like. I installed it with the MATE desktop and I also installed Cosmic. It looked like they had some nicely themed defaults for some of their DE's but not for MATE, it was the ugly default theme, so I spent a few minutes fixing that. It comes with the Catchy Browser, which is just a rebranded Firefox. That was the only browser in the live installer image, but during the install I opted to also install Firefox.

I also installed GhostBSD, also with MATE, it looks pretty good out of the box, I didn't feel the need to make any immediate changes. I'm not sure of a good replacement for Slackware so I'll probably just install Slackware again, probably with XFCE this time and configuring it with a dark black theme.

[Grogan]

Cool, I would like to try CachyOS some day. I think to get the x86_64-v3 packages you have to configure it to use the repositories

https://github.com/CachyOS/linux-cachyo ... nd-cachyos

(lol on the jump URL that github constructed)

I've thought about replacing the Gentoo with a Slackware Current install, but I've got a good setup here. Slackware would be a lot of work to recompile everything nowadays.

[Grogan]

lib32 support has been dying a death of a thousand cuts in Arch. I've been noticing a lot of lib32 packages that are gone, while messing with wine the last few days. I wanted to build a lib32-ffmpeg package but found so many dependencies to be non-existent that it wasn't practical.

Now this:
https://archlinux.org/news/transition-t ... e-staging/

We are transitioning the wine and wine-staging package to a pure wow64 build. This change removes the dependency on the multilib repository for wine and wine-staging.

The main reason for this is to align with upstream Wine development, which simplifies packaging and the dependency chain.

Potential Issues:

OpenGL Performance: A known limitation of the new WoW64 mode is reduced performance for 32-bit applications that use OpenGL directly
Breaking Changes: Existing 32-bit prefixes needs to be recreated

If you are facing issues with 32 bit prefixes, please recreate these and reinstall the application.

The problem with that is that many 32 bit games don't work with this. I've tried Wow64 wine builds. One of the reasons I stick with Arch is that it provides a good lib32 environment for this shit. With that changing, I might stop fighting with this distro.

[Zema Bus]

I wonder how that's going to play out.

[Grogan]

I also heard that when KDE is upgraded to Plasma 6.4, Arch has removed plasma-x11-session by default and you won't be able to start it, until you manually install the package. You wouldn't know that at this time, unless subscribed to their dev mailing lists. That's rather rude of them. I would have thought they'd be the last distro to try and forcibly cram Wayland down people's throats.

RFC - Manual changes for Plasma 6.4.0

Tomaz Canabrava Wed, 18 Jun 2025 13:24:14 -0700

Plasma 6.4.0 will need manual intervention if you are on X11

On Plasma 6.4 the wayland session will be the only one installed when the
users
does not manually specify kwin-x11.


With the recent split of kwin into kwin-wayland and kwin-x11, users running
the old X11 session needs
to manually install plasma-x11-session, or they will not be able to login.
Currently pacman is not able to
figure out your personal setup, and it wouldn't be ok to install
plasma-x11-session and kwin-x11 for every
one using Plasma.

tldr: Install plasma-x11-session if you are still using x11

That's bollocks, of course you can have both x11 and wayland support installed... and pacman knows what packages you have installed.

[Zema Bus]

So they'll just remove it, sounds like something Microsoft would do.

I heard that the next Ubuntu release (with the default Gnome desktop) will be dropping X11.

[Grogan]

They could have pacman prompt, y/n, when installing either the wayland or x11 package. "This is for Wayland support only, do you wish to install plasma-x11-session" and vice versa for the other. What they are doing isn't very nice, if nothing else they should install both and make someone go read that they can remove the one they don't want if they wish to instead of the other way around.

[Zema Bus]

They've been busy lately:

linux-firmware >= 20250613.12fe085f-5 upgrade requires manual intervention

2025-06-21 - Jan Alexander Steffens

With 20250613.12fe085f-5, we split our firmware into several vendor-focused packages. linux-firmware is now an empty package depending on our default set of firmware.

Unfortunately, this coincided with upstream reorganizing the symlink layout of the NVIDIA firmware, resulting in a situation that Pacman cannot handle. When attempting to upgrade from 20250508.788aadc8-2 or earlier, you will see the following errors:

linux-firmware-nvidia: /usr/lib/firmware/nvidia/ad103 exists in filesystem
linux-firmware-nvidia: /usr/lib/firmware/nvidia/ad104 exists in filesystem
linux-firmware-nvidia: /usr/lib/firmware/nvidia/ad106 exists in filesystem
linux-firmware-nvidia: /usr/lib/firmware/nvidia/ad107 exists in filesystem

To progress with the system upgrade, first remove linux-firmware, then reinstall it as part of the upgrade:

# pacman -Rdd linux-firmware
# pacman -Syu linux-firmware

[Grogan]

I'm sure glad I maintain my own firmware files. That's bollocks and it's just going to make it more difficult for people.

[Zema Bus]

Had to do this on the Raspberry Pi 5 too. After the last update including Plasma 6.4 the HDMI bug reared it's head, I don't know why it didn't occur before under Plasma 6.3. On a Raspberry Pi there's no option of switching to a DP monitor, but a mini HDMI to DP adapter would probably be a solution.

I wanted to see if CatchyOS does anything different than Arch when it comes to this sort of thing so I ran pacman -Syu before fixing it. I like the inclusion of the notice from Arch. I normally check the Arch website first but I don't always remember.

catchyos_linuxfirmwareNotice.jpg (114.73 KiB) Viewed 427 times

[Grogan]

All Arch based distros will be doing this. It's still going to wind up with most people having a shit tonne of firmware they don't need installed anyway, as they say, pacman has no way of knowing what firmware you're going to need. Even on ARM there's a bunch of unnecessary firmware for most devices. There are ARM SoCs that have Nvidia hardware, but not ours.

Code: Select all

core/linux-firmware           20250508.788aadc8-2  20250613.12fe085f-6  -727.25 MiB       0.00 MiB
core/linux-firmware-amdgpu                         20250613.12fe085f-6    83.86 MiB      13.63 MiB
core/linux-firmware-atheros                        20250613.12fe085f-6   113.76 MiB      32.49 MiB
core/linux-firmware-broadcom                       20250613.12fe085f-6    20.49 MiB       7.43 MiB
core/linux-firmware-intel                          20250613.12fe085f-6   321.78 MiB      42.15 MiB
core/linux-firmware-mediatek                       20250613.12fe085f-6    50.39 MiB      19.07 MiB
core/linux-firmware-nvidia                         20250613.12fe085f-6   152.25 MiB      99.50 MiB
core/linux-firmware-other                          20250613.12fe085f-6    56.99 MiB      13.80 MiB
core/linux-firmware-radeon                         20250613.12fe085f-6     4.96 MiB       0.80 MiB
core/linux-firmware-realtek                        20250613.12fe085f-6    20.67 MiB       2.75 MiB
core/linux-firmware-whence    20250508.788aadc8-2  20250613.12fe085f-6     0.00 MiB       0.04 MiB

(firmware-whence is just the license files for all the firmware... I don't care about that, I don't copy it)

Probably all we need on the rp5 is linux-firmware-broadcom (but don't believe me on that just yet). I'm going to try cleaning that up.

What HDMI bug are you referring to, the one that causes artifacts and bad performance on a secondary display?

[Grogan]

I had to use pacman -Rdd to remove linux-firmware because it's a dependency for the distro kernel package (linux-rpi-16k)

Then I simply installed linux-firmware-broadcom (it doesn't install the linux-firmware meta when you do that). Then pacman -Syu to get all the updates without complaint. The kernel package did print a note that "firmware is required for some devices" but that's got nothing to do with what's installed, it's just a message that scrolls by.

With the distro kernel, when you install firmware, pacman rebuilds the initramfs... that means I now have less crap in my image.

Everything is working, wifi, video, HDMI audio. Broadcom is all we need on the RP5

[Zema Bus]

That's good to know. The HDMI bug has been around for years, it happens in Linux with many HDMI displays that poll all the inputs before turning off. When they do that it wakes right back up and the timer is reset.

[Grogan]

I'm just using Xorg's facility for display power management because xfce4-power-manager bites the weenie (though it works, it doesn't seem to like to let you disable it). Since it seems to be KDE/Plasma, you could do the same, disable KDE's power management and just use xset to control yours.

[Zema Bus]

In this case it's Plasma 6.4 without X11 so xset wouldn't work, I'd have to install the Plasma X11 session. But I don't think that would solve this particular issue since the screen does actually turn off, it's just that it immediately turns back on again. What seems to be happening is when an HDMI monitor polls all the inputs when shutting down this gets interpreted as a hot plug event, so a signal is sent to the display which then wakes back up. Here's the bug report.

[Grogan]

Ahh Wayland. A small library and a bunch of protocol definitions, where any DE has to implement most everything themselves.

I like X11 because it provides APIs for people to write nice window managers that all work expectedly, with back end facilities provided by X. With Wayland, there will be a lot less diversity because writing that shit is hard.

While that was X11 6 years ago, it does sound like the same problem re-implemented by KDE power management.

[Grogan]

Here's another thing that Arch is taking away. I just went to build xorg-server (a -2 pkgrel) and I see they are disabling ACPI support, with -D linux_acpi=false. This disables the X server's use of acpid.

I guess they think everybody uses desktop environments with power managers that take over that stuff. I don't! Not on mine they're not.