M1 dev setup using a NixOS virtual machine

2024 UPDATE: I ended up ditching this in favor of one of the Docker Desktop clients and dev containers. Sometimes I still think of integrating dev containers and nix using one the tools like devenv.sh and Devbox, but I haven’t taken a look at it yet.

For the last two months I have been doing dev work on an M1 MacBook Air. Kind of a big change in retrospective, having used Linux in bare metal almost uninterruptedly for 10 years, give or take. Needless to say at this point, the hardware is great and the main reason that made me go with it, and I did not find any blockers on the software side for the things I normally do.

One of the reasons I suspect made everything a bit smoother is being prepared to leverage my comfort using Linux to set up a virtual machine with UTM (a frontend for QEMU in macOS) or Docker containers using the Docker Desktop app for my work. Given that Docker in macOS uses virtualization to run a Linux kernel anyway, I decided to go with the “bare” VM approach for more tinkering control.

NixOS and Nix were also tools I wanted to start using more instead of on-and-off experiments as I was doing. I feel like giving even an intro to these projects would make this post quite long because they can do a lot! But I can mention that I felt attracted to try them, among others, because I can:

• handle multiple development environments with nix-shell;
• backup my configuration.nix file and treat my VM as something I can torch whenever I want - for the most part, the VM “state” and configuration including programs are encapsulated there in a declarative form;
• have a “reasonable” level of reproducibility within reach by pinning nixpkgs to a commit hash;
• start with a console-only NixOS VM that I SSH into, and use its configuration as a baseline in case I want to run NixOS in the Air natively by the time Linux is ready for it (soon™), adding a graphical session and stuff like that.

When I started looking into running NixOS using UTM, I could not find a proper ISO to boot with, so I went with Ubuntu + Nix to hit the ground running. This week I decided to search again and I finally could find an image in the UEFI section of NixOS on ARM. I do not know why I did not find it earlier, but eh. Incidentally I also found another reference in a blog post by a NixOS dev this week.

So the release and unstable ISOs can be found at Hydra, NixOS’ CI system. Search for “nixos.iso_minimal” there and you will get the latest built ones. I went with the nixos.iso_minimal_new_kernel.aarch64-linux from the unstable branch, mainly because I can roll back to a previous machine state easily from the boot menu if something breaks, but I imagine going with the release version would be OK as well. You can always pick single packages from unstable if you want to.

After downloading and mounting the ISO in a newly created VM, I followed the installation section of the manual pretty much verbatim. I did not have to configure GRUB as stated in the ARM section of the wiki for this setup. Also, take into account that you have to forward port 22 (or the one you use for SSH) to some other local port in your machine in the configuration of the VM in UTM to access it from macOS.

A big downside of committing to “the Nix way” at this point in time is that fixing broken stuff might need learning the language and digging through docs and issues which are not always clear or complete. So far though, working with Node.js, Docker, Docker Compose and VSCode through SSH has been flawless. The VM is really snappy and, as a small curiosity, it does not show up in the list of apps using “significant battery” where Docker Desktop would be at all times! For the occasional container that only has a x86_64 version I can always set up another VM with NixOS amd64 to replace the one I already have with Alpine, but I do not see the urgent need for it.

Keep in mind running Nix in macOS is also an option, one that is being nicely shaped thanks to the Nix 🖤 macOS Open Collective. As far as I read, it can work as a replacement for Homebrew. Some nice improvements might still need to wait for a backport or a new Nix release though. For me, needing Docker already made me go with the VM approach instead, and after some time I find the logical separation of integrated OS apps and the dev environment “cozy”, if that makes any sense.

A couple of elements I recommend complementing this setup with:

• The Nix Environment Selector extension to make VSCode aware of the development environment defined in shell.nix files, so that it can use tooling declared there and not in the general system configuration.
• Code Server support for NixOS (you will need this if you plan to use VSCode as your editor with the remote SSH extension).

Finally, here’s my current configuration.nix file in case it helps with anything. Not much added from the base one though.