Booting a Raspberry Pi 3B with UEFI and a Hybrid MBR

Previously in this miniseries about me installing NixOS on Raspberry Pi’s in different ways, I managed to install NixOS with a vanilla UEFI bootloader on an RPi 4 instead of via the SD card image with U-Boot. The approach isn’t NixOS-exclusive, by the way, it should allow you to install any vanilla Linux distribution with 64-bit ARM support on the RPi! What makes that possible is the RPi 4 UEFI firmware.

Earlier, I had tried to do the same on an RPi 3 – that didn’t work out though.

Azriel: Weird. What’s the difference between the Pi 3 and 4 that lead to it working on one of them, but not the other? The UEFI firmware seems to be mostly the same for both models.

The thing is: for the RPi 3, you have to use a legacy MBR partition table on your SD card. This leads to some issues during installation: my go-to bootloader systemd-boot doesn’t work on MBR-formatted disks, and GRUB is currently broken on NixOS on ARM.

The RPi 4 doesn’t have this limitation, and works fine with a modern GPT partition table.

Hailey: Why do you have to use MBR though? Shouldn’t the UEFI firmware support GPT-formatted SD cards? It’s an UEFI firmware after all, and IIRC GPT is the UEFI-native type of partition table.

That’s true, and the firmware itself does support GPT. The problem is the on-board bootloader of the RPi. That has to be able to load the UEFI firmware in the first place – and on the RPI 3 it only supports MBR. I explained that in more detail in that post.

Partition Tables

My friend David suggested trying using a hybrid MBR approach to circumvent the problem.

Ember: I think it’s worth explaining at this point what all this “MBR” and “GPT” stuff actually is.

Traditionally, on PCs, you’d have a so-called Master Boot Record in the first bytes of your storage device. This MBR contained the bootloader, so it could be found and loaded during boot.

The MBR also contained the partition table. That’s where information about all the partitions on the device would be stored, such as their positions and sizes.

Today, MBRs are mostly not used anymore. Instead, we use the more modern GUID partition tables (GPT). The bootloader is loaded from a dedicated EFI system partition (ESP) instead of from an MBR.

Interestingly, GPT-partitioned disks still do have an MBR: a so-called protective MBR. Normally, it contains data for a single partition spanning the entire disk. Of course, this partition doesn’t actually exist, its purpose is exclusively to keep MBR-only software from mucking around with the partition layout.

Ian: By the way, as GPT is short for GUID partition table, “GPT partition table” is a bad case of RAS syndrome.

Thank you, smartypants.

Hybrid MBR

To work with the hybrid MBR we’re about to create, we’ll use gdisk (also known as “GPT fdisk”). That’s a tool for working with GPT disks and their partitioning. Its main author is Rod Smith, who has a page with comprehensive information and instructions on hybrid MBRs, which helped me immensely and is the source of most of what I write here.

The general idea of a hybrid MBR is to take the protective MBR of a GPT-partitioned disk and put an actual partition table in there that mirrors the partition information the GPT contains.

This scheme allows for using the disk with both software that supports GPT and software that only supports MBR. For example, hybrid MBRs were used for dual-boot of older Windows versions on Macs. macOS already used a GPT, but those Windows versions didn’t support that yet. The hybrid MBR then allowed Windows to recognize the partition layout as it would with any other MBR.

Ian: Hybridizing the MBR is a pretty damn horrible and very fragile thing to do. The protective MBR wasn’t meant to do something like that, and a lot of tools will become confused and do weird things. Avoid it if you can.

Ember: So, when using a hybrid MBR it’s probably best to avoid having to mess with the disk layout again after setup. If you don’t break your partition table right away, you still would have to manually repeat any changes for both partition tables so that they’re in sync again.

Ian: If you think that sounds annoying, you’d be correct. So make sure to reserve enough spare space in the boot partition and consider using a file system with subvolumes or something similar, such as btrfs. Then you can mess around with the subvolumes if it becomes necessary in the future instead of having to re-partition.

For more information about what hybrid MBRs are and how to use them, take a look at Rod’s excellent article I linked above.

Preparing the SD Card

Enough chitchat, let’s get to the meat of the matter. These are the steps I took in order to prepare my SD card for installing NixOS on my RPi 3B.

Create Partitions

We start by partitioning the card normally with a GPT. I used parted for that, although you can of course use any other GPT-aware partitioning tool as well. For example, using gdisk would make a lot of sense as we’ll be using it for creating the hybrid MBR anyway, or gparted if you prefer a GUI. parted is just what I’m used to.

Either way, start by creating a GPT partition table if there isn’t one yet (mktable gpt in parted).

The first partition should be the EFI system partition. A size of a few hundred MB should be more than enough. Although I generally make it an even gigabyte, just to be safe. Mark it as ESP (toggle 1 esp in parted).

I generally create only one other partition spanning the rest of the disk. In principle, you can also create more partitions. Although, as Ian explained above, I’d recommend keeping it simple to make it less likely you’ll ever need to re-partition.

Formatting the Boot Partition

Find the path for the boot partition you just created, and run mkfs.fat -F 32 /dev/sdX1 on that to format it with FAT32.

Azriel: What’s the -F 32 option in the mkfs.fat command good for?

Ember: The reason it’s a good idea to use that is actually quite interesting and a little convoluted.

A FAT file system uses a file allocation table (or FAT in short, which is also what the file system is named after). The entries in that table are either 12, 16 or 32 bits big, depending on whether it’s a FAT12, FAT16 or FAT32 file system. That’s what the -F option for mkfs.fat specifies. In practice, these differ mostly in the maximum file system size.

By default, without -F, mkfs.fat will choose the type depending on the size of the partition. I’m not entirely sure what the exact limits used for that decision are, the man page keeps it pretty vague:

If nothing is specified, mkfs.fat will automatically select between 12, 16 and 32 bit, whatever fits better for the filesystem size.

I found an article that cites 260 MiB as the size under which a FAT16 is created, but I haven’t verified that. For our 1 GB partition a FAT32 is created by default, though.

Anyway, later on we’ll need to know hat kind of FAT file system we created, and to keep it simple, we can just force FAT32.

If you really want to, you could also leave that out and check what FAT type was created for your partition size, e.g. with sudo file -s /dev/sdX1, and use that information later on when it becomes relevant.

Ian: The current UEFI spec mandates support for FAT12, 16 and 32¹. Still, I’ve seen reports online that some implementations choke on ESPs with FAT16 or 12, so I guess it’s best practice to just stay with FAT32. These days, it’s the most common FAT type anyway.

Hybridizing

Now we can finally create the hybrid MBR itself. As mentioned, we’ll need the gdisk tool for that. For nixpkgs, it can be found in the gptfdisk package.

Ember: Just a reminder that gdisk’s author Rod Smith is also the author of the instructions we’re following here.

• Run gdisk on your card, e.g. gdisk /dev/sdX.

• Enter the recovery & transformation menu by entering r.

• I’d recommend printing your partition info with p so you have the partition numbers handy during the process, just in case you want to check something.

• Start the hybridization with h.

• You’ll be asked what partitions you want to add to the hybrid MBR. These will be the partitions visible to MBR-only software.

  We only need to hybridize the first partition. It’s the one we’ll put the UEFI firmware on, which we want the MBR-only RPi 3 bootloader to be able to see. Everything else, e.g. Linux and the UEFI firmware itself, will just use the GPT as normal.

  Simply enter 1 and confirm.

• Next, gdisk wants to know at what position in the MBR it should put the protective 0xEE partition.

  That’s the partition that’s normally the only one in a protective MBR, as a placeholder. We do not want to put that first, because the RPi bootloader only finds the boot partition if it’s first in the MBR, so we answer n and continue.

• gdisk will ask for the “MBR hex code” for the boot partition. MBR partitions have codes that designate what’s on the partition. As we’ll be using this partition as ESP, we’ll be formatting it with a FAT file system.

  For a FAT32 LBA file system, which is what we created earlier, the hex code is 0c, so we enter that and continue on.

  By the way, for a FAT16 LBA file system the code would’ve been 0e instead.

• When asked whether the partition should be set as bootable, we can safely answer n. It’ll work either way though, as the partition isn’t used as boot partition in the MBR way by an MBR bootloader, so it doesn’t actually matter what the MBR’s boot flag is set to.

• gdisk will offer to use unused partition slots in the MBR to protect more partitions. There is no need for that, and I’d recommend just not doing it to keep things simple, but you can try it anyway if you’d like.

• Check your layout with o. Finally, write your changes and finish up with w.

Again, I recommend leaving your fingers off the partitioning now that we’re done. The hybrid MBR in combination with the somewhat finicky RPi bootloader really is quite fragile. For example, at one point in a test run I had the disk flag pmbr_boot shown on the SD card in parted. I guess that stemmed from me setting the bootable flag when hybridizing, I’m not 100% sure. Anyway, I tried to disable it with disk_toggle pmbr_boot in parted, which promptly broke the boot – the UEFI firmware didn’t load anymore, and toggling the flag again didn’t fix it.

Ian: Told ya!

Installing the UEFI Firmware

Download the latest release of the RPi 3 UEFI firmware from the GitHub repo. Mount the EFI system partition from the SD card, unzip the firmware and just copy the files onto the ESP.

That should be it: if you made no mistakes on the way, the RPi should now boot and show the interface of the UEFI firmware. You should be able to enter the UEFI menu and play around with the settings.

Installing the OS

Now that the card is prepared and the UEFI firmware is ready, we can get to the installation itself. This should work out of the box and mostly the same as it did for the RPi4 in my previous post, take a look there for some more info.

The short version is: you should now be able to install the Linux distribution of your choice from a USB stick as you would on any other device, as long as they offer a 64-bit ARM ISO image.

For NixOS, that means downloading the NixOS ISO image for 64-bit ARM from the NixOS download page, putting it onto a USB stick and selecting it in the UEFI firmware’s boot menu.

Just be careful that nothing touches the partition table! I haven’t tried the graphical installer, so I can’t guarantee that that won’t mess it up.

However, running nixos-install directly with my personal NixOS config worked just fine. systemd-boot installs without issues onto the hybridized boot partition! The UEFI firmware still loaded fine after the installation, I could select my NixOS installation from there, and it booted without problems.

Wrapping Up

Hailey: That’s cool and all, but was there actually any point to this whole maneuver, anyway? Didn’t the U-Boot setup you had before work just fine?

Well… I guess it did work fine, yes. If you use the “standard” way of installing NixOS on the RPi via the SD card image as I did before it does work.

Still, I like that both my RPi 3 and RPi 4 now use the same uniform way of booting, with the same bootloader and bootloader config from my desktop and laptop. I don’t have to maintain another of boot configuration just for the RPi 3.

The UEFI firmware itself is also just really nice, e.g. it allows you to boot from a USB stick easily. It even supports network boot and some other cool things.

Also, I did have some problems with U-Boot before (see the post), and with this setup I can avoid U-Boot altogether. I really didn’t like having U-Boot and some other stuff lying around on an unmounted firmware partition with no automated updates. The UEFI firmware is on the normal ESP and more accessible for updates and changes, and my bootloader is just systemd-boot which is updated and configured by NixOS as it would be on any other device as well.

All in all: on the RPi 4 where you don’t have to do the annoying hybrid MBR stuff I can fully recommend using the UEFI firmware stuff for your Linux install. It’s easy, works well and should work with any distribution.

On the RPI 3 though… to be honest, I would not actually recommend to anyone else jumping through the hoops of hybridizing your SD card MBR. Just use the SD card image, it’s easier to set up and probably less weird. Still, it was a fun experiment. Personally, I don’t regret it and would do it again!