FCOS config versioning

[ajeddeloh]

Background

CL has the Container Linux Config Transpiler or ct for short. Since users should not be writing Ignition configs by hand, they write Container Linux Configs (CLCs) which ct then renders into an Ignition config. This scheme has a few advantages:

• Forces users to validate their configs
• Allows easy addition of sugar for common operations
• Allows for adding OS specific bits without making the Ignition spec OS specific
• Allows users to write configs in yaml but keeps the configs that Ignition ingests in json

CLCs are currently unversioned. This is a problem since users don't know what the backwards compatibility promises are for their configs. So far we haven't needed to break compatibility and with CL's EOL in ~1 year we don't plan to need to. FCOS however should get this right from the start.

Cost of supporting multiple versions

Supporting more than one ct config version at a time is not free. There are a couple approaches:

• Translation to newest config version: This is what Ignition does. The translation code is very verbose but it avoids duplicating the translation logic.
• Carry logic for multiple versions independently: This makes it much easier to make a new config version but means that changes to the translation logic must be backported to older versions

Targeting Ignition versions

Another question is what version of the Ignition spec to target. CLCT currently targets whatever is the most recent version supported by CL's stable channel. This is also not ideal since new versions in alpha don't gain support until the update propagates to stable. Ideally we would support targeting any version (and error out if the config requires a newer version) but it's unclear if that is worth the effort or not.

A few proposals:

1. Don't version it: We break compat arbitrarily when we need to and tell users to "fix your configs". I'm not a fan of this one.
2. Version in lock-step with the Ignition config: Have a version field that maps directly to the version of the Ignition config that will be generated. New features/sugar would go in the most recent spec version. This is easier to implement but means that users would need to keep their ct version up to date to take advantage of new features and that just because a version of fcosct accepts an fcosc with version X it wouldn't necessarily accept all configs with version X.
3. Do the Ignition-style translation dance: Have the fcosc spec version be independent of the Ignition spec version. When a user hands fcosct an old config it is translated up to the latest version.

[bgilbert]

Do we want FCOSCs to also support RHEL (or multiple RHCOS branches)? If so, we'll need to be able to support older Ignition spec versions for a long time. In that case, I'm thinking the right approach is to 1) translate the FCOSC to the latest version, and then 2) render it down to an appropriate Ignition config version. That approach maximizes implementation pain, though.

[ajeddeloh]

Do we want FCOSCs to also support RHEL (or multiple RHCOS branches)?

Now that dust has settled a bit, I think we do no have to support RHCOS, since they are both on a different Ignition spec version and have their own format (Machine Configs).

For RHEL I think we should only target them if they're running Ignition 2.0.0+ with spec 3.0.0+. I can't see justifying the development effort to support both wildly different specs. From the proposal here we can start with supporting "ignition as yaml with inlined files" (aka just the distro agnostic bits) and add sugar as we see fit.

Dealing with targeting multiple 3.0.0 specs is still up in the air. It would bring more implementation pain but it would be nice to be able to say "generate a 3.0.0 config please".

As for versioning the FCOS spec that's also another can of worms. If we support multiple distros with multiple sets of sugar I almost think the distro-agnostic bit and distro specific bit should be seperate. Like:

ct:
    version: 1.0.0
    distroVersion: 1.2.0
storage:
    <stuff in ct base spec 1.0.0>
systemd:
    <stuff in ct base spec 1.0.0>
distro_sugar1:
    <stuff in that distro's spec 1.2.0>
distro_sugar2:
    <stuff in that distro's spec 1.2.0>

then run with ct --target-ignition 3.0.0 --target-distro fcos < config.fcc > config.ign

[ajeddeloh]

Proposal (thanks @lucab):
Note I will use "spec" to reference to the fcos config language spec and "Ignition spec" or "target spec" to refer to Ignition's spec.

Have CT the binary versioned independent of the spec. Each CT release will accept the current spec and all older specs. Each spec will target only one ignition spec. This means:

• Old fcos configs keep working with new CT
• Since Ignition up-translates old Ignition configs, old CT still generates configs that work with new FCOS/Ignition
• Multiple specs can target the same Ignition spec.

Example:
ct v1.0.0 is released with spec v1.0.0 targeting Ignition spec v3.0.0.
We add some distro-specific sugar
ct v1.1.0 is released with spec v1.1.0 targeting Ignition spec v3.0.0.
We do some bugfixes, golang API changes
ct v2.0.0 is released with spec v1.1.0 targeting Ignition spec v3.0.0
We refactor the ct spec
ct v3.0.0 is released with spec v2.0.0 targeting Ignition spec v3.0.0
New Ignition spec v3.1.0 is released (minor bump), we add those fields to the ct spec
ct v3.1.0 is released with spec v2.1.0 targeting Ignition spec v3.1.0

ct v3.1.0 would accept ct configs with versions v1.0.0, v1.1.0, v2.0.0 and v2.1.0. Each of those configs would only emit one Ignition spec version. This also allows us to update CT for new Ignition releases without needing to wait for that Ignition release to work it's way into stable.

WRT versioning distro and core bits separately, I'm leaning towards no, and just making it easy to consume the core bits as a library for CTs for other distros.

[ajeddeloh]

Closed via #175