This is part 4 in my series of posts learning how to setup NixOS on my Framework 16" AMD laptop. It is not meant as a guide, necessarily, but more as a captain’s log of my thoughts and processes along the way.

• Framework and NixOS - Day One
• Framework and NixOS - Day Two
• Framework and NixOS - Secure Boot

Full Disk Encryption

As a self-respecting security person, I can’t very well be walking around with a laptop with an unencrypted disk.

But here’s my dilemma. I’ve already partitioned my disk when I did the original install and I’m not using luks. Additionally, my disk partition isn’t particularly declarative – it’s not actually tracked in my configuration at all, so if I want to replicate my operating system in the future, I’d have to remember to set it all up just the right way.

Since I only have the one root partition right now, I need to back up a few files that I don’t want to lose before I do any partitioning. In particular, I’m going to save the contents of my /etc/secureboot directory that I generated earlier, as well as just save my whole /home/dade directory. In the future I’d like to avoid saving this whole directory all willy nilly like this, but right now I’d prefer to not lose any of the state that I’ve accumulated.

It turns out I can’t copy a lot of things from my home directory over to my flash drive, because a bunch of things are symlinks to my nix store. That’s okay, though – I have most everything I need, and anything in my nix store should be recoverable by just building my config again on the new disk layout when I’m done.

Disk partitioning

I’ll be using disko to setup a declarative disk partitioning scheme, as well as indicate my preference for a luks encrypted volume. I also want to be wary of my future intentions to use impermanence.

I think to get started, I’m going to use the luks btrfs subvolumes example from the disko repository. This looks like it’ll be similar to what I want to do with impermanence and at least get me started.

One thing of note is that the disko nix configuration used here uses a key file, or I can comment it out and specify a password file for an interactive password entry when I boot up my laptop. You’ll see a bit later on that I opted for a password entry, since this volume will contain my entire operating system, so using a key file is a bit harder than I’d like to figure out.

Adding disko as an input

First up, let’s add disko as an input to our flake.nix file.

{
  inputs = {
  ...
    disko = {
      url = "github:nix-community/disko";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  ouputs = { self, nixpkgs, disko, lanzaboote, ...}@inputs: {
	  nixosConfigurations.serenity = nixpkgs.lib.nixosSystem {
	    specialArgs = { inherit inputs; };
	    modules = [
	      disko.nixosModules.disko
	      lanzaboote.nixosModules.lanzaboote
	      ./hosts/serenity/configuration.nix
	      inputs.home-manager.nixosModules.default
	    ];
	  };
  };
}

I’m going to do a nixos-rebuild switch here, just to make sure the disko input gets picked up correctly. If all goes well, my next rebuild is going to reformat my drive, so this will likely be my last time rebuilding before the switch. I anticipate needing to boot into a live media to do the next rebuild, otherwise the rebuild will destroy the contents of the disk and nix will immediately stop working. But for now we’re going to stay in our nice environment and setup the disko configuration.

Since we’re going to base our config off the luks-btrfs-subvolumes example mentioned above, let’s get that onto our disk to start making it match what we want. To do this, I used nix-shell -p wget to open a shell with wget installed, then wget https://raw.githubusercontent.com/nix-community/disko/master/example/luks-btrfs-subvolumes.nix -O ~/nixcfg/hosts/serenity/disko-config.nix.

Setting up my disko config

Based on the next steps of the Disko Quickstart, it’s supposed to be time to boot into our installer image so we can modify the disk without issue. But I’m going to skip that just a little bit and make some tweaks to my disko-config.nix and commit it to my repo. It’s kind of a pain to use ssh to pull my nixcfg from a private repository from a live installer, so I’m going to try to get it as close to setup as possible and then do a little mount trick after I boot into the live media.

At this point we can get the disk name, nvme0n1, which should remain consistent for the lifecycle of my Framework laptop. Even if I add an additional drive, nvme0n1 should be a consistent name to refer to this drive. We’re also going to comment out the keyFile and additionalKeyFiles settings, and uncomment the passwordFile so that we can set a password for our lukscrypt volume when we run disko.

{
  disko.devices = {
    disk = {
      main = {
        type = "disk";
        device = "/dev/nvme0n1";
        content = {
          type = "gpt";
          partitions = {
            ESP = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
                mountOptions = [
                  "defaults"
                ];
              };
            };
            luks = {
              size = "100%";
              content = {
                type = "luks";
                name = "crypted";
                # disable settings.keyFile if you want to use interactive password entry
                passwordFile = "/tmp/secret.key"; # Interactive
                settings = {
                  allowDiscards = true;
                  #keyFile = "/tmp/secret.key";
                };
                #additionalKeyFiles = [ "/tmp/additionalSecret.key" ];
                content = {
                  type = "btrfs";
                  extraArgs = [ "-f" ];
                  subvolumes = {
                    "/root" = {
                      mountpoint = "/";
                      mountOptions = [ "compress=zstd" "noatime" ];
                    };
                    "/persist" = {
                      mountpoint = "/persist";
                      mountOptions = [ "compress=zstd" "noatime" ];
                    };
                    "/nix" = {
                      mountpoint = "/nix";
                      mountOptions = [ "compress=zstd" "noatime" ];
                    };
                    "/swap" = {
                      mountpoint = "/.swapvol";
                      swap.swapfile.size = "16G";
                    };
                  };
                };
              };
            };
          };
        };
      };
    };
  };
}

At this point, I’m stressing out because I’m worried I’ll do it wrong. Which is silly because the whole point of my nix config is that if I messed up, I just reinstall my config and I’m good to go. But nevertheless, I was paralyzed by indecision for a couple days before taking the next step.

Get on with it already

I made sure I had a copy of my nixcfg repo, complete with the disko config, on a flash drive where I could easily mount it, make changes and persist them, in case I need to reboot from my live installer a few times to get it working. Then I rebooted into my live installer, created a file at /tmp/secret.key so that disko will be able to read the passphrase I want to use, and connected the live environment to my WiFi so nix can pull packages.

Before I am ready to run the installation, I noticed that Disko recommends running nixos-generate-config --no-filesystems --root /mnt. This made me realize that my existing nix configuration probably contains filesystem information that I need to remove. Sure enough, in my nixcfg/hosts/serenity/hardware-configuration.nix, there are two filesystems defined. I’m going to just comment these out for now, since the disko config will supply filesystem information (hopefully). I’m also going to comment out swapDevices since disko has a swap volume defined.

Note: I’m commenting these out because at this point, I am working off my flash drive. I technically have the whole git repo on the flash drive and could make commits, but I don’t have my whole git config setup and I’ll just do a commit once I know things are working.

Time to install the disk partition – I did this via the recommended command in the quick start guide, tweaking the path to where my disk config is. (Side note: It’s both super cool and kind of guano-insane that you can just run the github repo with some parameters and it just works.)

sudo nix --experimental-features "nix-command flakes" run github:nix-community/disko -- --mode disko /run/media/nixos/Samsung USB/serenity-backup/dade/nixcfg/hosts/serenity/disko-config.nix

After it downloaded disko, it ran a bunch of commands and formatted my drive – I didn’t get any errors, so that’s a good sign. I ran mount | grep /mnt to see if it correctly mounted /dev/nvme0n1, and it looks like it did create my crypted volume and then mounted /, /nix, and /persist into /mnt/.

At this point, it seems like I’ve done everything right, I just have to finish the new nixos install to be able to reboot into it. But since my whole system config is a flake, I need to use a modified version of the install command.

I copied my whole nixcfg into /mnt/etc/nixos and then ran nixos-install --flake /mnt/etc/nixos#serenity. This started to work, but I got errors that the fileSystems option does not specify my root file system. This is because I forgot to include my disko-config.nix in my hosts/serenity/configuration.nix imports.

Once I did this, I re-ran the install command and it began building my whole system again. It mostly succeeded, but it gave me a “Failed to install generation: No such file or directory” after the “Installing Lanzaboote to /boot” line. Then it prompted me for my root password, and I was done. I looked at /mnt and it looked like a typical linux filesystem, so I think it’s time to reboot and see if it works…

It didn’t work…

Looks like I did something wrong. I guess that “failed to install generation” error makes plenty of sense. If the first generation couldn’t be installed into /boot then it’s not really surprising that I couldn’t boot.

To see if I can fix it without having to redo the whole process, I booted back up into my live media, nix-shell -p cryptsetup to get a shell that has the necessary packages to decrypt my encrypted volume, and then ran the following:

sudo cryptsetup luksOpen /dev/nvme0n1p2 crypted

This prompted me for my password from the disko config /tmp/secret.key and then successfully decrypted the volume. This then allowed me to mount the btrfs subvolumes with mount --options subvol=<volume> /dev/mapper/crypted /mnt, so I mounted the root subvolume to /mnt, persist subvol to /mnt/persist, nix subvol to /mnt/nix, and then mounted the boot partition to /mnt/boot with mount /dev/nvme0n1p1 /mnt/boot. Now I’m basically set back up the way I was after disko ran. In hindsight, I am pretty sure disko has a command that will just mount your disko config for you, so I didn’t have to do it by hand.

After digging around in the mounted drive and double checking my configuration, I couldn’t really figure out what was going on. I re-ran the install and got the same error about a file not being found during the lanzaboote install stage. I wondered if it might be the /etc/secureboot folder that lanzaboote is configured for, so I copied that off my backup drive into /mnt/etc/secureboot.

I re-ran the install, again, nixos-install --flake /mnt/etc/nixos#serenity, and it didn’t error the same way this time! It did give me some errors trying to read files from the boot directory that didn’t exist, but then it said it replaced the missing files with signed binaries.

I think it’s time to try to reboot again?

It boots!

We have NixOS generations, so our boot partition must be working now! Even better, we get prompted to enter our lukscrypt volume password…

It booted up into my login window, with hyprland installed and everything! All this time working on these config files wasn’t wasted.

But one problem…

Turns out, I never set the password for my regular user. A better linux user than I would have switched to another tty, logged in as root, and then set the password without rebooting.

Couldn’t be me, dawg. I rebooted back into my live NixOS environment, re-launched a nix-shell -p cryptsetup, re-opened my luks volume, re-mounted all my subvolumes to their appropriate place in /mnt, and then learned that chroot doesn’t work the way I expected it to when I tried to chroot /mnt.

Turns out, NixOS has a tool called nixos-enter, which is specifically designed to enter into NixOS installations from rescue systems. So I went into /mnt and ran nixos-enter, then ran passwd dade. I set my password, and now I should be able to reboot and get into my user environment.

Sure enough, I repeat the reboot process and am able to login. I’m in. This time with 100% more disk encryption, and 100% more disk partition declarations.

What’s next?

This was a bit of a painful experience, if I’m being honest. I think disko isn’t as well documented as it could/should be, and I think my lack of experience with btrfs also put me at a slight disadvantage, because I wasn’t sure what was necessary as part of disko and what was necessary for btrfs. But the good news is that it’s done, the config works, which means I can use it on any system if I want.

But I did run into a couple issues that I’d love to solve next. The first, my luks volume password isn’t managed in my configuration. As a self-respecting security professional, I’m not about to just commit the password to my git repo, though. The second happens to be very similar – I went through all this work and then failed to login because I forgot to set my user password. Finally, the third issue, is that my wifi password wasn’t saved and I had to type it out again. I’d like all of these passwords to just be automatically set as part of my nix config.

I think my next exploration will be into sops-nix, so that I can manage my secrets in a safe way, as part of a separate, private repository. I should be able to manage my secrets for home in a repository only available in my home, and then manage secrets for servers in a private github repo.

I’m also going to make a backup of my luks header, which I was reminded to do by this lovely askubuntu post. I might also look into adding a backup key on a flash drive in my safe, as a recovery mechanism in case I forget my password or in case something happens to me and someone needs to unravel the madness of my laptop.

After I get my secrets managed, it’ll probably be time to start configuring Impermanence and seeing if I did this disk configuration correctly.