Partition Layout

[ajeddeloh]

In converstations we had recently, we think that FCOS should have a default partition layout, similar to how CL has a standard fs layout since it provides consistency across bare metal and clouds and well as making the image "dd-able" directly to a drive (which makes installation trivial). Any further disk modification should be done via Ignition.

What should that partition layout look like?

My (quick and not fully thought out) proposal:

1 - EFI-SYSTEM (needed for all EFI systems)
2 - BIOS-BOOT  (holds grub for systems that need bios booting)
3 - ROOT       (i.e. everything else)

Ideally we'd be able to move ROOT around using Ignition and re-deploying the OSTree to where we moved it to between the disks and files stages. If you're on a EFI system you could even wipe away BIOS-BOOT to make more room (not that its terribly large). There's some tricky cases with that which we're still exploring, but it should be possible at very least in simple cases.

[dustymabe]

1 - EFI-SYSTEM (needed for all EFI systems)
2 - BIOS-BOOT  (holds grub for systems that need bios booting)
3 - ROOT       (i.e. everything else)

sounds like a reasonable default.. just to be clear people would be able to add a separate mounted /var/ or /home/, etc, via ignition on boot but everyone would start from the same place ?

[ajeddeloh]

Yeah.
Isn't /home actually just part of /var on ostree systems?

[dustymabe]

Isn't /home actually just part of /var on ostree systems?

it is but I think we've enabled (because people asked for it) to have /home/ (or rather /var/home/?) be a separate block device mount.

[cgwalters]

Do you see us adopting the GPT generator and avoiding the need to have the OS mounts in /etc/fstab too?

[cgwalters]

it is but I think we've enabled (because people asked for it) to have /home/ (or rather /var/home/?) be a separate block device mount.

Yeah, we support that; it was the primary rationale for the systemd fstab symlink patch.

[ajeddeloh]

We could say that all the standard top level directories on / except /var and /boot needs to be on the same partition (kinda like how we say you can't move USR-{A.B} on container linux). Since /var is empty by default and can be populated by systemd-tmpfiles users can add a mount unit for it and partition it however they want. You want a seperate partition for /var, /var/home and /var/srv? Go for it.

I'm not a huge fan of the GPT generator. It's pretty limited since it only supports a few partitions and reeks magic that users may not be aware of. Much better to have explicit mount units imho.

[cgwalters]

I'm not a huge fan of the GPT generator. It's pretty limited since it only supports a few partitions and reeks magic that users may not be aware of. Much better to have explicit mount units imho.

Yeah; agree with the magic aspect. Though the specific thing I don't like about it is that if one plugs in a backup drive into a system, you might end up having the old /home partition mounted into your current OS. The scheme actually says explicitly:

Since the GPT partition table scheme cannot express which sets of partitions belong to a single OS installation, and to avoid the risk of accidentally mixing and matching incorrect combinations of these partitions, we decided to not define auto-discovery of these partition types within this specification.

Having a "full installer" that auto-generates UUIDs for each partition and binds those in /etc/fstab avoids all issues like this (except when LVM is in use).

The only way I can think of to address this in a "dd install" is to generate the machine id at install time, walk the target disk post-install and change the GUID/mount units. Ug. I guess CL has gone a long time with the current approach, and while the "foreign partition problem" does occur in the real world, it's definitely possible to work around it.

[ajeddeloh]

CL makes use of labels instead of uuids for partitions*. Ignition can bake in a specified uuid to your Ignition config but that means your machines will all have the same one. Since everyone is running off the same dd'd image anyway there's already duplicate uuids across machines (hasn't been a problem as far as I've seen). Labels actually work pretty well for this case. It means that while different machines will have partitions with different uuids, all of your config files and such (e.g. /etc/fstab, mount units, etc) are the same across machines and contain no machine-specific bits.

We could use uuids if Ignition supported templating but thats a rabbit hole I really don't want to go down, especailly since I see the "no machine specific bits" to be a feature.

*for the root-on-raid case we actually have special type guids for raid devices containing the root. This allows us know which devices to start in the initramfs. FWIW I'd like to avoid this for FCOS. Hell, stick a config file in /boot some tool in the initramfs knows how to read for all I care.

[JasonGiedymin]

I have to chime in on this portion. Soon I will be open sourcing some lvm encryption work done on Atomic LVM managed volumes. We store keys remotely basically. During that exercise the part which stopped the system startup and decryption from being more elegant and robust were UUIDs (we're on baremetal with diverse set of disks). If I had a more generic interface like labels to work with, live would have been easier.

[cgwalters]

One thing that's strongly implied by this is that Anaconda is probably not the primary path for bare metal installs; a whole goal of this is that "install" is just dd. By default we boot to a live image just like CL?

Side note: It'd be a nice twist to have the installer come as a privileged container.

However, a whole interesting question is how we generate that "base disk image" - I'd probably at least initially start with using Anaconda for it personally, but there's also the libguestfs-style disk image creation.

One question though; do we still support people using Anaconda (say that I want dm-crypt for everything, or XFS reflink=1, or...)? And there's a tension between Ignition for disk provisioning vs kickstart here.

[bgilbert]

@cgwalters Off-topic for this issue, but I don't see any reason to support installing via Anaconda, even optionally. If you want dm-crypt or XFS or whatever, we should be supporting that through Ignition.

@ajeddeloh Why don't you like the type GUID approach? It seems to work pretty well for CL.

[ajeddeloh]

Re installer in a container: we could package it as such, but it should also be so simple you don't need to. The current one is just a few hundred lines of bash (most of which is a ascii armored pubkey).

Re using anaconda bare metal installation: I really really really don't like this. Your Ignition config is the declaration of what your machine looks like; there shouldn't be anything else controlling that. It's also yet another thing to support. If we want to support dmcrypt or other filesystem/partition weirdness we should implement that in Ignition.

@bgilbert It's a hack from when we were thinking we shouldn't use the boot partition. We're going to need to store some config on the boot partition anyway for supporting encryption. It'd be much cleaner to just store a "map" for mounting the root partition in a config file. That would also eliminate the need to do GPT on RAID on GPT on Disks (and instead of GPT on RAID on Disks)

[cgwalters]

@cgwalters Off-topic for this issue, but I don't see any reason to support Anaconda, even optionally. If you want dm-crypt or XFS or whatever, we should be supporting that through Ignition.

I'd say it's a distinct thread but the installer path is pretty intertwined with our default partitioning.
OK, with "full" Ignition-disks where we support re-locating the OS to the new root - yes. I know I keep getting hung up on this; the concept is foreign to me. I have concerns about the amount of overlap with Anaconda this is going to imply going forward, but...I also definitely see the value in the simplification here.