diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md deleted file mode 100644 index ef46026..0000000 --- a/.github/ISSUE_TEMPLATE.md +++ /dev/null @@ -1,4 +0,0 @@ - diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md deleted file mode 100644 index 94e2abf..0000000 --- a/.github/ISSUE_TEMPLATE/bug-report.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -name: Report a bug -about: Report an issue with Fedora CoreOS -title: '' -labels: 'kind/bug' -assignees: '' - ---- - -**Describe the bug** -A clear and concise description of what the bug is. - -**Reproduction steps** -Steps to reproduce the behavior: -1. -2. -3. - -**Expected behavior** -A clear and concise description of what you expected to happen. - -**Actual behavior** -A clear and concise description of what actually happened. - -**System details** - - Bare Metal/QEMU/AWS/GCP/etc. - - Fedora CoreOS version - -**Ignition config** -Please attach the Butane config or Ignition config used to provision your system. Be sure to sanitize any private data. If not using Butane to generate your Ignition config, does the Ignition config pass validation using [ignition-validate](https://coreos.github.io/ignition/getting-started/#config-validation)? - -**Additional information** -Add any other information about the problem here. diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml new file mode 100644 index 0000000..c0699a7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug-report.yml @@ -0,0 +1,73 @@ +name: Report a bug +description: Report an issue with Fedora CoreOS +labels: ["kind/bug"] +assignees: [] +body: + - type: textarea + id: bug-description + attributes: + label: Describe the bug + description: A clear and concise description of what the bug is. + placeholder: I'm using foo on bar and it fails with foobar. + validations: + required: true + + - type: textarea + id: bug-reproduction + attributes: + label: Reproduction steps + description: Steps to reproduce the behavior. + placeholder: | + 1. + 2. + 3. + validations: + required: true + + - type: textarea + id: bug-expected + attributes: + label: Expected behavior + description: A clear and concise description of what you expected to happen. + placeholder: Foo should succeed without errors. + validations: + required: true + + - type: textarea + id: bug-actual + attributes: + label: Actual behavior + description: A clear and concise description of what actually happened. + placeholder: Foo failed with ... + validations: + required: true + + - type: textarea + id: bug-system + attributes: + label: System details + description: Version (`rpm-ostree status -b`) and platform (Bare Metal/QEMU/AWS/GCP/etc.) where you've seen the issue. + placeholder: | + - Bare Metal/QEMU/AWS/GCP/etc. + - Fedora CoreOS version + validations: + required: true + + - type: textarea + id: bug-config + attributes: + label: Butane or Ignition config + description: The Butane config or Ignition config used to provision your system. Be sure to sanitize any private data. If not using Butane to generate your Ignition config, ensure the Ignition config passes validation using [ignition-validate](https://coreos.github.io/ignition/getting-started/#config-validation). + # Might be Butane YAML or Ignition JSON, which is upward-compatible + # with YAML + render: yaml + validations: + required: false + + - type: textarea + id: bug-additional + attributes: + label: Additional information + description: Add any other information about the problem here. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/enhancement.md b/.github/ISSUE_TEMPLATE/enhancement.md deleted file mode 100644 index f89404b..0000000 --- a/.github/ISSUE_TEMPLATE/enhancement.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -name: Request an enhancement -about: Request a new feature in Fedora CoreOS -title: '' -labels: 'kind/enhancement' -assignees: '' - ---- - -**Describe the enhancement** -A clear and concise description of the desired feature. - -**System details** - - Bare Metal/QEMU/AWS/GCP/etc. - - Fedora CoreOS version - -**Additional information** -Add any other information here. diff --git a/.github/ISSUE_TEMPLATE/enhancement.yml b/.github/ISSUE_TEMPLATE/enhancement.yml new file mode 100644 index 0000000..47a4bb0 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/enhancement.yml @@ -0,0 +1,32 @@ +name: Request an enhancement +description: Request a new feature in Fedora CoreOS +labels: ["kind/enhancement"] +assignees: [] +body: + - type: textarea + id: enhancement-description + attributes: + label: Describe the enhancement + description: A clear and concise description of the desired feature. + placeholder: I want to use foo with bar on Fedora CoreOS. + validations: + required: true + + - type: textarea + id: enhancement-system + attributes: + label: System details + description: Platform (Bare Metal/QEMU/AWS/GCP/etc.) where you'd want to see this feature. Version you've tried that does not have it. + placeholder: | + - Bare Metal/QEMU/AWS/GCP/etc. + - Fedora CoreOS version + validations: + required: false + + - type: textarea + id: enhancement-additional + attributes: + label: Additional information + description: Add any other information about the problem here. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/implementing-new-emerging-platform.md b/.github/ISSUE_TEMPLATE/implementing-new-emerging-platform.md new file mode 100644 index 0000000..de334f4 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/implementing-new-emerging-platform.md @@ -0,0 +1,41 @@ +# Implementing a new emerging platform + +This template is a simplified version of the +[full template](https://github.com/coreos/fedora-coreos-tracker/blob/main/.github/ISSUE_TEMPLATE/implementing-new-platform.md) +that only includes what is strictly needed to get initial support for a new +platform in Fedora CoreOS. This simplified version notably does not include the +steps needed to add new boot images to the release process. + +Platforms added via this process are labelled "emerging" and users will have to +get boot images for them by converting existing images in the right format and +changing the `ignition.platform.id=` command line parameter. + +This process will be documented using `guestfish` as an example. + +## During Development + +Create PRs addressing the following: + +- [ ] [Ignition](https://github.com/coreos/ignition/) ([example PR](https://github.com/coreos/ignition/pull/918)) + - [ ] Add userdata fetch + - [ ] If the platform supports it (unlikely), add userdata deletion +- [ ] [Afterburn](https://github.com/coreos/afterburn/) ([example PR](https://github.com/coreos/afterburn/pull/451)) + - [ ] (Cloud Only) Add relevant attributes + - [ ] (Cloud Only) Add SSH key support if available + - [ ] (Cloud Only) Add hostname support if available + - [ ] (Cloud Only) Add check-in if needed (unlikely) +- [ ] [fedora-coreos-docs](https://github.com/coreos/fedora-coreos-docs) ([example PR](https://github.com/coreos/fedora-coreos-docs/pull/377)) + - [ ] Add a `provisioning-.adoc` that walks through how to setup the new platform + - [ ] Add an entry in the `modules/ROOT/nav.adoc` that points to new documentation +- [ ] (Optional but recommended) Add support for the platform to [kola](https://github.com/coreos/coreos-assembler) to simplify testing +- Create or ask for a new upstream releases for: + - [ ] Ignition + - [ ] Afterburn +- Wait for new images with updated Ignition and Afterburn to reach stable then + merge documentation with `guestfish` commands: + - [ ] fedora-coreos-docs + +## At Release + +There are no "At Release" steps as we do not produce new boot images for +emerging platforms/ diff --git a/.github/ISSUE_TEMPLATE/implementing-new-platform.md b/.github/ISSUE_TEMPLATE/implementing-new-platform.md new file mode 100644 index 0000000..4ede3f2 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/implementing-new-platform.md @@ -0,0 +1,63 @@ +# Implementing a new supported platform + +## During Development + +Create PRs addressing the following: + +- [ ] [Ignition](https://github.com/coreos/ignition/) ([example PR](https://github.com/coreos/ignition/pull/918)) + - [ ] Add userdata fetch + - [ ] If the platform supports it (unlikely), add userdata deletion +- [ ] [Afterburn](https://github.com/coreos/afterburn/) ([example PR](https://github.com/coreos/afterburn/pull/451)) + - [ ] (Cloud Only) Add relevant attributes + - [ ] (Cloud Only) Add SSH key support if available + - [ ] (Cloud Only) Add hostname support if available + - [ ] (Cloud Only) Add check-in if needed (unlikely) +- [ ] [stream-metadata-go](https://github.com/coreos/stream-metadata-go) ([example PR](https://github.com/coreos/stream-metadata-go/pull/45/)) + - [ ] Add platform to the `Media` struct in `release/release.go` + - [ ] Add supporting code for new platform to `toStreamArch` func in `release/translate.go` + - [ ] (Cloud Only) Cloud Images need to have an `Images` field +- [ ] (Cloud Only) [stream-metadata-rust](https://github.com/coreos/stream-metadata-rust/) ([example PR](https://github.com/coreos/stream-metadata-rust/pull/16)) +- [ ] [fedora-coreos-tracker](https://github.com/coreos/fedora-coreos-tracker/) ([example PR](https://github.com/coreos/fedora-coreos-tracker/pull/1213)) + - [ ] Update the metadata for the new platform +- [ ] [coreos-assembler](https://github.com/coreos/coreos-assembler) ([example PR](https://github.com/coreos/coreos-assembler/pull/2489)) + - [ ] Updated `cmd-generate-release-meta` + - [ ] `cosa osbuild ` works +- [ ] [fedora-coreos-config](https://github.com/coreos/fedora-coreos-config/) + - [ ] Add a stanza to `platforms.yaml` if the system should use a serial console, or both serial and graphical consoles +- [ ] [fedora-websites-3.0](https://gitlab.com/fedora/websites-apps/fedora-websites/fedora-websites-3.0/) + - [ ] Add friendly name for platform to `components/utilities/FpDownloadItem.vue` + - [ ] Add artifact to `pages/coreos/download.vue` + - [ ] Possibly add logo to `content/editions/coreos/home.yml` +- [ ] [fedora-coreos-browser](https://github.com/coreos/fedora-coreos-browser) ([example PR](https://github.com/coreos/fedora-coreos-browser/pull/35)) + - [ ] Add a list element for new platform in `browser/index.html` +- [ ] [build pipeline](https://github.com/coreos/fedora-coreos-pipeline) ([example PR](https://github.com/coreos/fedora-coreos-pipeline/pull/815)) + - [ ] Add platform to the list found in `config.yaml` for building the new artifact +- [ ] [fedora-coreos-docs](https://github.com/coreos/fedora-coreos-docs) ([example PR](https://github.com/coreos/fedora-coreos-docs/pull/377)) + - [ ] Add a `provisioning-.adoc` that walks through how to setup the new platform + - [ ] Add an entry in the `modules/ROOT/nav.adoc` that points to new documentation + +## At Release + +1. Merge upstream changes and put out a release: + - [ ] Ignition + - [ ] Afterburn +1. Merge metadata changes: + - [ ] stream-metadata-go + - [ ] stream-metadata-rust + - [ ] fedora-coreos-tracker + - [ ] fedora website + - [ ] fedora-coreos-browser +1. Release updated components + - [ ] Create and follow release checklist for [stream-metadata-go](https://github.com/coreos/stream-metadata-go/blob/main/docs/development.md#release-process) + - [ ] Ensure that Dependabot has PRed stream-metadata-go into fedora-coreos-stream-generator and coreos-assembler. Merge the update PRs. + - This can be triggered manually by navigating to [fedora-coreos-stream-generator's Dependabot](https://github.com/coreos/fedora-coreos-stream-generator/network/updates/) and [coreos-assembler's Dependabot](https://github.com/coreos/coreos-assembler/network/updates) respectively, then clicking "Check for updates". + - This might need to be done a few times, as Dependabot might not pick up tag changes for a few attempts after initial tagging. + - [ ] Create and follow release checklist for [fedora-coreos-stream-generator](https://github.com/coreos/fedora-coreos-stream-generator/blob/main/docs/development.md#release-process) +1. Merge the following changes: + - [ ] coreos-assembler +1. Wait for updates made to coreos-assembler to be propagated to latest container + - [ ] Download latest version of coreos-assembler container. Verify platform support functionality. +1. Merge changes for: + - [ ] Build pipeline +1. Wait for new images to reach stable then merge documentation. + - [ ] fedora-coreos-docs merged diff --git a/.github/ISSUE_TEMPLATE/new-feature.md b/.github/ISSUE_TEMPLATE/new-feature.md new file mode 100644 index 0000000..808e7b6 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/new-feature.md @@ -0,0 +1,59 @@ +--- +name: Implement a feature +about: Propose a design for a new feature +--- + +# Feature proposal + +## Description + + + +## Implementation PRs + + + +## Did you consider? + + + +- Storage + - [ ] Disk space usage + - [ ] Behavior on 4Kn disks + - [ ] Compatibility with multiple ESPs (Butane `boot_device.mirror`) +- First boot + - [ ] Behavior on first boot vs. second boot + - [ ] initrd networking requirements + - [ ] Reprovisioned systems that reused existing storage devices +- OS update + - [ ] Behavior after an OS rollback + - [ ] Compatibility with old bootloaders +- Architectures + - aarch64 + - [ ] Compatibility with non-UEFI boot + - ppc64le + - [ ] Whether new GRUB directives are supported by petitboot + - s390x + - [ ] Endianness issues + - [ ] Need to rerun `zipl` to update kernel or kargs + - [ ] ECKD/MBR lack of partition labels + - [ ] ECKD maximum partition count +- Implementation + - [ ] How interlocking PRs will be ratcheted into repos + +## Implementation steps + +- [ ] Create tracker ticket with initial design (above) +- [ ] Initial discussion and refinement in the ticket +- [ ] Add `meeting` label +- [ ] Discuss at community meeting +- [ ] Further refinement + - [ ] Post draft [fedora-coreos-docs](https://github.com/coreos/fedora-coreos-docs/) PR, ideally before doing any implementation, to help identify design problems. +- [ ] Update issue description with final proposal and post a comment saying that you did +- [ ] Verify that rough consensus exists +- [ ] Implement. Post PR links in the section above. In the description of each PR, link to this issue and specify the prerequisites for merging. + - [ ] Add kola test(s) for new feature +- [ ] Land implementation PRs, in order +- [ ] Wait for the functionality to reach FCOS stable +- [ ] Land docs PR +- [ ] Remove any ratcheting glue (e.g. workarounds in coreos-assembler) diff --git a/.github/ISSUE_TEMPLATE/new-package.md b/.github/ISSUE_TEMPLATE/new-package.md deleted file mode 100644 index 1c2baa3..0000000 --- a/.github/ISSUE_TEMPLATE/new-package.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -name: Request a new package -about: Ask for a new package to be added to Fedora CoreOS -title: 'New Package Request: ' -labels: 'kind/enhancement' -assignees: '' - ---- - -Please try to answer the following questions about the package you are requesting: - -1. What, if any, are the additional dependencies on the package? (i.e. does it pull in Python, Perl, etc) - -2. What is the size of the package and its dependencies? - -3. What problem are you trying to solve with this package? Or what functionality does the package provide? - -4. Can the software provided by the package be run from a container? Explain why or why not. - -5. Can the tool(s) provided by the package be helpful in debugging container runtime issues? - -6. Can the tool(s) provided by the package be helpful in debugging networking issues? - -7. Is it possible to layer the package onto the base OS as a day 2 operation? Explain why or why not. - -8. In the case of packages providing services and binaries, can the packaging be adjusted to just deliver binaries? - -9. Can the tool(s) provided by the package be used to do things we’d rather users not be able to do in FCOS? (e.g. can it be abused as a Turing complete interpreter?) - -10. Does the software provided by the package have a history of CVEs? diff --git a/.github/ISSUE_TEMPLATE/new-package.yml b/.github/ISSUE_TEMPLATE/new-package.yml new file mode 100644 index 0000000..f545e01 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/new-package.yml @@ -0,0 +1,84 @@ +name: Request a new package +description: Ask for a new package to be added to Fedora CoreOS +title: "New Package Request: " +labels: ["kind/enhancement"] +assignees: [] +body: + - type: markdown + attributes: + value: | + Please try to answer the following questions about the package you are requesting. + + - type: textarea + id: newpackage-dependencies + attributes: + label: What, if any, are the additional dependencies on the package? (i.e. does it pull in Python, Perl, etc) + description: Paste here the output of `rpm-ostree install --dry-run ` from a fresh Fedora CoreOS node. + validations: + required: true + + - type: textarea + id: newpackage-size + attributes: + label: What is the size of the package and its dependencies? + description: Paste here the output of `rpm -qi ` for each package mentioned above. + validations: + required: true + + - type: textarea + id: newpackage-solution + attributes: + label: What problem are you trying to solve with this package? Or what functionality does the package provide? + validations: + required: true + + - type: textarea + id: newpackage-container + attributes: + label: Can the software provided by the package be run from a container? Explain why or why not. + validations: + required: true + + - type: textarea + id: newpackage-debug-container + attributes: + label: Can the tool(s) provided by the package be helpful in debugging container runtime issues? + validations: + required: true + + - type: textarea + id: newpackage-debug-network + attributes: + label: Can the tool(s) provided by the package be helpful in debugging networking issues? + validations: + required: true + + - type: textarea + id: newpackage-day2 + attributes: + label: Is it possible to layer the package onto the base OS as a day 2 operation? Explain why or why not. + description: Can the package be installed on first boot or later with `rpm-ostree install `? + validations: + required: true + + - type: textarea + id: newpackage-service + attributes: + label: In the case of packages providing services and binaries, can the packaging be adjusted to just deliver binaries? + validations: + required: true + + - type: textarea + id: newpackage-interpreter + attributes: + label: Can the tool(s) provided by the package be used to do things we’d rather users not be able to do in FCOS? + description: E.g. can it be abused as a Turing complete interpreter? + validations: + required: true + + - type: textarea + id: newpackage- + attributes: + label: Does the software provided by the package have a history of CVEs? + validations: + required: true diff --git a/.github/ISSUE_TEMPLATE/new-platform.md b/.github/ISSUE_TEMPLATE/new-platform.md deleted file mode 100644 index c484e47..0000000 --- a/.github/ISSUE_TEMPLATE/new-platform.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -name: Request a new platform -about: Ask for Fedora CoreOS to support a new cloud environment -title: 'Platform Request: ' -labels: 'area/platforms, kind/enhancement' -assignees: '' - ---- - -In order to implement support for a new cloud platform in Fedora CoreOS, we need to know several things about the platform. Please try to answer as many questions as you can. - -- [ ] Why is the platform important? Who uses it? - -- [ ] What is the official name of the platform? Is there a short name that's commonly used in client API implementations? - -- [ ] How can the OS retrieve instance userdata? What happens if no userdata is provided? - -- [ ] Does the platform provide a way to configure SSH keys for the instance? How can the OS retrieve them? What happens if none are provided? - -- [ ] How can the OS retrieve network configuration? Is DHCP sufficient, or is there some other network-accessible metadata service? - -- [ ] In particular, how can the OS retrieve the system hostname? - -- [ ] Does the platform require the OS to have a specific console configuration? - -- [ ] Is there a mechanism for the OS to report to the platform that it has successfully booted? Is the mechanism required? - -- [ ] Does the platform have an agent that runs inside the instance? Is it required? What does it do? What language is it implemented in, and where is the source code repository? - -- [ ] How are VM images uploaded to the platform and published to other users? Is there an API? What disk image format is expected? - -- [ ] Are there any other platform quirks we should know about? diff --git a/.github/ISSUE_TEMPLATE/new-platform.yml b/.github/ISSUE_TEMPLATE/new-platform.yml new file mode 100644 index 0000000..35ad0ac --- /dev/null +++ b/.github/ISSUE_TEMPLATE/new-platform.yml @@ -0,0 +1,87 @@ +name: Request a new platform +description: Ask for Fedora CoreOS to support a new cloud environment +title: "Platform Request: " +labels: ["area/platforms", "kind/enhancement"] +assignees: [] +body: + - type: markdown + attributes: + value: | + In order to implement support for a new cloud platform in Fedora CoreOS, we need to know several things about the platform. Please try to answer as many questions as you can. + + - type: textarea + id: newplatform-user + attributes: + label: Why is the platform important? Who uses it? + validations: + required: false + + - type: textarea + id: newplatform-name + attributes: + label: What is the official name of the platform? Is there a short name that's commonly used in client API implementations? + validations: + required: false + + - type: textarea + id: newplatform-userdata + attributes: + label: How can the OS retrieve instance userdata? What happens if no userdata is provided? + validations: + required: false + + - type: textarea + id: newplatform-sshkeys + attributes: + label: Does the platform provide a way to configure SSH keys for the instance? How can the OS retrieve them? What happens if none are provided? + validations: + required: false + + - type: textarea + id: newplatform-network + attributes: + label: How can the OS retrieve network configuration? Is DHCP sufficient, or is there some other network-accessible metadata service? + validations: + required: false + + - type: textarea + id: newplatform-hostname + attributes: + label: In particular, how can the OS retrieve the system hostname? + validations: + required: false + + - type: textarea + id: newplatform-console + attributes: + label: Does the platform require the OS to have a specific console configuration? + validations: + required: false + + - type: textarea + id: newplatform-boot-success + attributes: + label: Is there a mechanism for the OS to report to the platform that it has successfully booted? Is the mechanism required? + validations: + required: false + + - type: textarea + id: newplatform-agent + attributes: + label: Does the platform have an agent that runs inside the instance? Is it required? What does it do? What language is it implemented in, and where is the source code repository? + validations: + required: false + + - type: textarea + id: newplatform-image-upload + attributes: + label: How are VM images uploaded to the platform and published to other users? Is there an API? What disk image format is expected? + validations: + required: false + + - type: textarea + id: newplatform-quirks + attributes: + label: Are there any other platform quirks we should know about? + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/rebase.md b/.github/ISSUE_TEMPLATE/rebase.md index cfb9a80..64ad4b4 100644 --- a/.github/ISSUE_TEMPLATE/rebase.md +++ b/.github/ISSUE_TEMPLATE/rebase.md @@ -7,6 +7,55 @@ - [ ] Fedora Changes Considerations ([example](https://github.com/coreos/fedora-coreos-tracker/issues/1222)) - [ ] Package Additions/Removals ([example](https://github.com/coreos/fedora-coreos-tracker/issues/1221)) - [ ] Test Week ([template](https://github.com/coreos/fedora-coreos-tracker/issues/new?template=test-week.md&title=tracker:+FN+Test+Week)) +- [ ] Communications Tracker ([example](https://github.com/coreos/fedora-coreos-tracker/issues/1655)) + +## At the first change checkpoint + +### Untag old packages + +`koji untag` N-2 packages from the pool (at some point we'll have GC in place to do this for us, but for now we must remember to do this manually or otherwise distRepo will fail once the signed packages are GC'ed). For example the following snippet finds all RPMs signed by the Fedora 32 key and untags them. Use this process: + +- [ ] Find the key short hash. Usually found [here](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2/backend/templates/pungi.rpm.conf.j2). Then: + +``` +f32key=12c944d0 +key=$f32key +echo > untaglist # create or empty out file +for build in $(koji list-tagged --quiet coreos-pool | cut -f1 -d' '); do + if koji buildinfo $build | grep -i $key 1>/dev/null; then + echo "Adding $build to untag list" + echo "${build}" >> untaglist + fi +done +``` + +Now we have a list of builds to untag. But we need a few more sanity checks. + +- [ ] Make sure none of the builds are used in `N` based FCOS. Check by running: + +``` +f32key=12c944d0 +key=$f32key +podman run -it --rm quay.io/fedora/fedora-coreos:testing-devel rpm -qai | grep -i -B 9 $key +podman rmi quay.io/fedora/fedora-coreos:testing-devel +``` + +If there are any RPMs signed by the old key they'll need to be investigated. Maybe they shouldn't be used any longer. Or maybe they're still needed. One example of this is the shim RPM where the same build could be used for many Fedora releases. In this case you'll need to untag the RPM from `coreos-pool`, run a `koji distrepo`, which will remove that RPM from the repo metadata, and then re-tag it into the pool. The RPM in the repo will now be signed with a newer signing key. + + + +- [ ] After verifying the list looks good, untag: + +``` +# use xargs so we don't exhaust bash string limit +cat untaglist | xargs -L50 koji untag-build -v coreos-pool +``` + +- [ ] Now that untagging is done, give a heads up to rpm-ostree developers that N-2 packages have been untagged and that they may need to update their CI compose tests to freeze on a newer FCOS commit. + +- [ ] Remove the N-2 signing key from the tag info for the coreos-pool tag. The following commands view the current settings and then update the list to the 33/34/35 keys. You'll most likely have to get someone from releng to run the second command (`edit-tag`). + - `koji taginfo coreos-pool` + - `koji edit-tag coreos-pool -x tag2distrepo.keys="9570ff31 45719a39 9867c58f"` ## At Branching @@ -27,122 +76,120 @@ Branching is when a new stream is "branched" off of `rawhide`. This eventually b - `koji tag-build f${N+1}-coreos-continuous $BUILD` - example: `koji tag-build f36-coreos-continuous fedora-release-36-0.16` -- [ ] Add the N+1 signing key short hash (usually found [here](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2/backend/templates/pungi.rpm.conf.j2)) to the tag info for the coreos-pool tag. The following commands view the current settings and then update the list to the 32/33/34/35 keys. You'll most likely have to get someone from releng to run the second command (`edit-tag`). +- [ ] Add the N+1 signing key short hash (usually found [here](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2/backend/templates/pungi.rpm.conf.j2)) to the tag info for the coreos-pool tag. The following commands view the current settings and then update the list to the 32/33/34/35 keys. You'll most likely have to get someone from releng to run the second command (`edit-tag`). An example request looks [like this](https://pagure.io/releng/issue/10635). - `koji taginfo coreos-pool` - `koji edit-tag coreos-pool -x tag2distrepo.keys="12c944d0 9570ff31 45719a39 9867c58f"` ### coreos-installer changes - [ ] Update coreos-installer to know about the signing key used for the future new major version of Fedora (N+1). + - The current set of trusted signing keys is available at https://fedoraproject.org/security/. + - If the Fedora (N+1) signing key isn't available yet at that site, you can also get it from https://src.fedoraproject.org/rpms/fedora-repos/tree/rawhide. - [ ] Drop the signing key for the obsolete stable release (N-2). +Example PR: https://github.com/coreos/coreos-installer/pull/1113 + ### Update `rawhide` stream -- [ ] Update [manifest.yaml](https://github.com/coreos/fedora-coreos-config/blob/rawhide/manifest.yaml) to list N+1 as the releasever. +- [ ] Update `VERSION`, `MUTATE_OS_RELEASE`, `BUILDER_IMG` in [build-args.conf](https://github.com/coreos/fedora-coreos-config/blob/rawhide/build-args.conf) ([example PR](https://github.com/coreos/fedora-coreos-config/pull/4003)) ### Enable `branched` stream -- [ ] Update [manifest.yaml](https://github.com/coreos/fedora-coreos-config/blob/branched/manifest.yaml) to list N as the releasever. -- [ ] Update [streams.groovy](https://github.com/coreos/fedora-coreos-pipeline/blob/main/streams.groovy) to include the `branched` stream in the list of mechanical refs. - +- [ ] Update `VERSION`, `MUTATE_OS_RELEASE`, `BUILDER_IMG` in [build-args.conf](https://github.com/coreos/fedora-coreos-config/blob/branched/build-args.conf) ([example PR](https://github.com/coreos/fedora-coreos-config/pull/4005)) +- [ ] Update [config.yaml](https://github.com/coreos/fedora-coreos-pipeline/blob/main/config.yaml) to un-comment out the `branched` stream definition ([example PR](https://github.com/coreos/fedora-coreos-pipeline/pull/904)) ## At Fedora (N) Beta ### Update [fedora-coreos-config](https://github.com/coreos/fedora-coreos-config/) `next-devel` -- [ ] Bump `releasever` in `manifest.yaml` +- [ ] Bump `VERSION`, `MUTATE_OS_RELEASE`, `BUILDER_IMG` in `build-args.conf` +- [ ] Add the `fedora-candidate-compose` repo in `manifest.yaml` ([example PR](https://github.com/coreos/fedora-coreos-config/pull/2706)) - [ ] Update the repos in `manifest.yaml` if needed -- [ ] Run `cosa fetch --update-lockfile` +- [ ] Run `cosa fetch --dry-run --update-lockfile` + - this updates the x86_64 lockfile - the others will get updated when `bump-lockfile` runs. + - in the future we may support [this](https://github.com/coreos/coreos-assembler/issues/3088) in `cosa fetch` directly - [ ] PR the result -### Ship rebased `next` +- [ ] Re-enable `next-devel` if needed ([docs](https://github.com/coreos/fedora-coreos-pipeline/tree/main/next-devel)) +- [ ] Disable `branched` stream since it is no longer needed. + - Update [config.yaml](https://github.com/coreos/fedora-coreos-pipeline/blob/main/config.yaml) to comment out the `branched` stream definition. -- [ ] Ship `next` -- Set a new update barrier for N-2 on `next`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-631724629). +### Prepare Fedora CoreOS (N) Beta announcement +- [ ] Draft an announcement that contains information found in the "Communications Tracker", created in a step above, to inform users of Fedora CoreOS of upcoming changes in the Fedora (N) version. [(example)](https://lists.fedoraproject.org/archives/list/coreos@lists.fedoraproject.org/thread/GK4RMQ3UFMLJGKMBUVTTWGNFVFNNSH5E/) -## Preparing for Fedora (N) GA - -### Update [fedora-coreos-config](https://github.com/coreos/fedora-coreos-config/) `testing-devel` +### Ship rebased `next` -- [ ] Bump `releasever` in `manifest.yaml` -- [ ] Update the repos in `manifest.yaml` if needed -- [ ] Run `cosa fetch --update-lockfile` -- [ ] Bump the base Fedora version in `ci/buildroot/Dockerfile` -- [ ] PR the result +- [ ] Ship `next` +- [ ] Set a new update barrier for the final release of N-1 on `next`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-1247314065) -## At Fedora (N) GA +## Preparing for Fedora (N) GA -### Ship rebased `testing` +Do these steps as soon as we have a Go confirmation for GA, usually the Thursday of the week before GA. -- [ ] Ship `testing` -- Set a new update barrier for N-2 on `testing`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-631724629). +### Ship a final `next` release -### Disable `branched` stream +If the packages in `next-devel` don't exactly match the last `next` release that was done, we need to do a release with the final GA content. This ensures that what we'll promote to `testing` has the exact content in GA (plus version fast-tracks). This usually happens on the Thursday of the announcement of Go. -- [ ] Update [streams.groovy](https://github.com/coreos/fedora-coreos-pipeline/blob/main/streams.groovy) to remove the `branched` stream in the list of mechanical refs. +- [ ] Ensure final `next` release has GA content -### Untag old packages +### Build rebased `testing` and final `stable` release on N-1 -`koji untag` N-2 packages from the pool (at some point we'll have GC in place to do this for us, but for now we must remember to do this manually or otherwise distRepo will fail once the signed packages are GC'ed). For example the following snippet finds all RPMs signed by the Fedora 32 key and untags them. Use this process: +- [ ] Build `stable`; promote it from the `testing` branch, which should still be on N-1. Don't release it yet (i.e. don't run the `release` job). +- [ ] Build `testing`; promote it from the `next` branch instead of `testing-devel`. Don't release it yet (i.e. don't run the `release` job). -- [ ] Find the key short hash. Usually found [here](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2/backend/templates/pungi.rpm.conf.j2). Then: +### Update [fedora-coreos-config](https://github.com/coreos/fedora-coreos-config/) `testing-devel` -``` -f32key=12c944d0 -key=$f32key -untaglist='' -for build in $(koji list-tagged --quiet coreos-pool | cut -f1 -d' '); do - if koji buildinfo $build | grep $key 1>/dev/null; then - untaglist+="${build} " - echo "Adding $build to untag list" - fi -done -``` +- [ ] Bump `VERSION`, `MUTATE_OS_RELEASE`, `BUILDER_IMG` in `build-args.conf` +- [ ] Update the repos in `manifest.yaml` if needed +- [ ] Sync the lockfiles for all arches from `next-devel` +- [ ] Bump the base Fedora version in `ci/buildroot/Dockerfile` +- [ ] Bump the Fedora version for the test containers in `tests/kola/data/commonlib.sh` +- [ ] PR the result -- [ ] Now we have a list of builds to untag. But we need one more sanity check. Let's make sure none of those are actually being used. Fire up the latest FCOS `testing-devel` and run: -``` -f32key=12c944d0 -key=$f32key -rpm -qai | grep -B 8 $key -``` +## At Fedora (N) GA -If there are any RPMs signed by the old key they'll need to be investigated. Maybe they shouldn't be used any longer. Or maybe they're still needed. +Do these steps on GA day. -- [ ] After verifying the list looks good: +### Release rebased `testing` and final `stable` release on N-1 -``` -koji untag-build coreos-pool $untaglist -``` +- [ ] Run the `release` job for the staged `testing` and `stable` builds and start rollout. +- [ ] Set a new update barrier for the final release of N-1 on `testing`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-1247314065) -- [ ] Now that untagging is done, give a heads up to rpm-ostree developers that N-2 packages have been untagged and that they may need to update their CI compose tests to freeze on a newer FCOS commit. +### Disable `next-devel` stream if not needed -- [ ] Remove the N-2 signing key from the tag info for the coreos-pool tag. The following commands view the current settings and then update the list to the 33/34/35 keys. You'll most likely have to get someone from releng to run the second command (`edit-tag`). - - `koji taginfo coreos-pool` - - `koji edit-tag coreos-pool -x tag2distrepo.keys="9570ff31 45719a39 9867c58f"` +We prefer to disable `next-devel` when there is no difference between `testing-devel` and `next-devel`. This allows us to prevent wasting a bunch of resources (bandwidth, storage, compute) for no reason. After the switch to N if `next-devel` and `testing-devel` are in lockstep, then disable `next-devel`. +- [ ] Follow the instructions [here](https://github.com/coreos/fedora-coreos-pipeline/tree/main/next-devel) to disable `next-devel` -### Disable `next-devel` stream +### Switch upstream packages to shipping release binaries from Fedora (N) -We prefer to disable `next-devel` when there is no difference between `testing-devel` and `next-devel`. This allows us to prevent wasting a bunch of resources (bandwidth, storage, compute) for no reason. After the switch to N if `next-devel` and `testing-devel` are in lockstep, then disable `next-devel`. +- [ ] Update [repo-templates](https://github.com/coreos/repo-templates) [config.yaml](https://github.com/coreos/repo-templates/blob/main/config.yaml) with the version number and GPG key ID for Fedora (N). -- [ ] Follow the instructions [here](https://github.com/coreos/fedora-coreos-pipeline/tree/main/next-devel) to disable `next-devel` +### Disable the `fedora-candidate-compose` repo +- [ ] Remove from the `manifest.yaml` of `next-devel` the `fedora-candidate-compose` repo ## After Fedora (N) GA ### Ship rebased `stable` - [ ] Ship `stable` -- Set a new update barrier for N-2 on `stable`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-631724629). +- [ ] Set a new update barrier for the final release of N-1 on `stable`. In the barrier entry set a link to [the docs](https://docs.fedoraproject.org/en-US/fedora-coreos/update-barrier-signing-keys/). See [discussion](https://github.com/coreos/fedora-coreos-tracker/issues/480#issuecomment-1247314065) ### Open ticket for the next Fedora rebase - [ ] Create a new ticket from the [rebase template](https://github.com/coreos/fedora-coreos-tracker/issues/new?assignees=&labels=area%2Fplatforms%2C+kind%2Fenhancement&template=rebase.md&title=tracker:+Rebase+onto+Fedora+N) - label with `FN` label where `N` is the Fedora version. +### Update the FCOS Meeting-Action Template + +Now that Fedora N is GA, we need to start tracking the release schedule of Fedora N+1 during the weekly [Fedora CoreOS Community Meeting](https://github.com/coreos/fedora-coreos-tracker/blob/main/README.md#meetings). + +- [ ] Update the "Review Fedora N Release Schedule" topic and link to point to Fedora N+1 in the [FCOS meeting-action template](https://github.com/coreos/fcos-meeting-action/blob/main/static/meeting-template.md) + ## Miscellaneous container updates @@ -150,6 +197,8 @@ These are various containers in use throughout our ecosystem. We should update o - [ ] Update coreos-assembler or open ticket to update: - [Dockerfile](https://github.com/coreos/coreos-assembler/blob/main/Dockerfile) + - [Dockerfiles for kola test containers](https://github.com/coreos/coreos-assembler/tree/main/tests/containers) + - [Dockerfile for the OpenShift CI buildroot image](https://github.com/openshift/release/blob/master/ci-operator/config/coreos/coreos-assembler/coreos-coreos-assembler-main.yaml) - [ ] Update coreos-installer - [Dockerfile](https://github.com/coreos/coreos-installer/blob/main/Dockerfile) - [ ] Update Ignition @@ -158,6 +207,9 @@ These are various containers in use throughout our ecosystem. We should update o - [Dockerfile](https://github.com/coreos/butane/blob/main/Dockerfile) - [ ] Update fedora-coreos-cincinnati - [Dockerfile](https://github.com/coreos/fedora-coreos-cincinnati/blob/main/dist/fedora-infra/Dockerfile) + - [ImageStream](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/coreos-cincinnati/templates/imagestream.yml) + - [BuildConfig](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/coreos-cincinnati/templates/buildconfig.yml) + - [Git Hash Variables (Optional)](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/coreos-cincinnati/vars) - [ ] Update config-bot - [Dockerfile](https://github.com/coreos/fedora-coreos-releng-automation/blob/main/config-bot/Dockerfile) - [ ] Update coreos-koji-tagger @@ -168,3 +220,7 @@ These are various containers in use throughout our ecosystem. We should update o - [Dockerfile](https://github.com/coreos/fedora-coreos-releng-automation/blob/main/coreos-ostree-importer/Dockerfile) - [ImageStream](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/coreos-ostree-importer/templates/imagestream.yml) - [BuildConfig](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/coreos-ostree-importer/templates/buildconfig.yml) +- [ ] Update fedora-ostree-pruner + - [Dockerfile](https://github.com/coreos/fedora-coreos-releng-automation/blob/main/fedora-ostree-pruner/Dockerfile) + - [ImageStream](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/fedora-ostree-pruner/templates/imagestream.yml) + - [BuildConfig](https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/fedora-ostree-pruner/templates/buildconfig.yml) diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 74e699d..0439425 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -13,6 +13,6 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out repository - uses: actions/checkout@v2 + uses: actions/checkout@v3 - name: Verify meeting-people.txt is sorted run: awk '!/^$/ {if (name) print} /^exit 0$/ { name = 1 }' meeting-people.txt | sort -c diff --git a/Design.md b/Design.md index af85d0c..6b83bb1 100644 --- a/Design.md +++ b/Design.md @@ -18,7 +18,7 @@ conclusion should be summarized here with a link to the issue. ## OSTree Delivery Format -- Originally discussed in issue [#23](https://github.com/coreos/fedora-coreos-tracker/issues/23). +- Originally discussed in issue [#23](https://github.com/coreos/fedora-coreos-tracker/issues/23). ### Summary: @@ -29,7 +29,7 @@ end user systems: repo) on a server and fetched via HTTP requests. - rojig: uses a special rojig RPM and re-assembles OSTree commit from RPMs already on mirrors. -- OCI: OSTree commits are packaged up in OCI container images and delivered +- OCI: OSTree commits are packaged up in OCI container images and delivered via a container registry. Currently the plan in Fedora CoreOS is to deliver content via a plain @@ -87,6 +87,7 @@ The release process integrates with Fedora's release milestones in the following - Week -1 (Fedora "Go" Decision): `next` release: - `next` release with final Fedora GA content - Week 0 (GA release): triple release: + - `stable` release promoted from previous `testing` (on N-1) - `testing` release promoted from previous `next` - `next` release contains latest Fedora N content, including Bodhi updates - Week 2: triple release: @@ -101,7 +102,7 @@ Because production refs are unversioned, users will seamlessly upgrade between F ## Disk Layout -- Originally discussed in issue [#18](https://github.com/coreos/fedora-coreos-tracker/issues/18). +- Originally discussed in issue [#18](https://github.com/coreos/fedora-coreos-tracker/issues/18). See also [dustymabe's comment](https://github.com/coreos/fedora-coreos-tracker/issues/18#issuecomment-409668929) summarizing the discussion in the FCOS meeting. - Filesystem details were discussed in [#33](https://github.com/coreos/fedora-coreos-tracker/issues/33). @@ -227,16 +228,7 @@ Originally discussed in [#71](https://github.com/coreos/fedora-coreos-tracker/is Originally discussed in [#68](https://github.com/coreos/fedora-coreos-tracker/issues/68). - OpenStack environments do not require a cloud agent -- We will provide any base level of functionality with ignition and coreos-metadata - -### Packet: - -Originally discussed in [#69](https://github.com/coreos/fedora-coreos-tracker/issues/69). - -- On the first boot, Packet requires the machine to phone home to report a successful boot. This will be [handled by coreos-metadata](https://github.com/coreos/coreos-metadata/issues/120). -- Packet provides the IPv4 public address via DHCP, allowing a machine to acquire network via standard mechanisms. However, to obtain a private IPv4 address or a public IPv6 address (on the same interface), networking must be configured using metadata from an HTTP metadata service. This can be handled by coreos-metadata in the initramfs, but it [may need to learn to configure NetworkManager or nm-state](https://github.com/coreos/fedora-coreos-tracker/issues/111) depending on the outcome of [#24](https://github.com/coreos/fedora-coreos-tracker/issues/24). -- Packet needs the serial console on x86 to be directed to `ttyS1`, not `ttyS0`, requiring [cloud-specific bootloader configuration](https://github.com/coreos/fedora-coreos-tracker/issues/110). A different serial console configuration is required on ARM64. -- On many Linux OSes, Packet sets a randomized root password which is then available from the Packet console for 24 hours. This allows the serial (SOS) console to be used for interactive debugging. Container Linux, instead, enables autologin on the console by default. To avoid surprising users, Fedora CoreOS will do neither. For interactive console access, users can use Ignition to enable autologin or to set a password on the `core` account, and we'll document how to do that. +- We will provide any base level of functionality with ignition and coreos-metadata ### Open questions: @@ -274,7 +266,7 @@ This means: Originally discussed in [#114](https://github.com/coreos/fedora-coreos-tracker/issues/114). -We will not enable autologin on serial or VGA consoles by default, even on platforms (e.g. Azure, DigitalOcean, GCP, Packet) which provide authenticated console access. Doing so would provide an access vector that could surprise users unfamiliar with their platform's console access mechanism and access control policy. For users who wish to use the console for debugging, we will provide documentation for using Ignition to enable autologin or to set a user password. +We will not enable autologin on serial or VGA consoles by default, even on platforms (e.g. Azure, DigitalOcean, GCP) which provide authenticated console access. Doing so would provide an access vector that could surprise users unfamiliar with their platform's console access mechanism and access control policy. For users who wish to use the console for debugging, we will provide documentation for using Ignition to enable autologin or to set a user password. ### Automatically disable SMT when needed to address vulnerabilities @@ -344,8 +336,6 @@ next-devel | 10 testing-devel | 20 rawhide | 91 branched | 92 -bodhi-updates-testing | 93 -bodhi-updates | 94 For developer builds (those not produced by the official pipeline), Z is always `dev`. @@ -364,8 +354,6 @@ next-devel | 31.20191018.10.10 | 11th build of the day testing-devel | 31.20191018.20.0 | rawhide | 33.20191018.91.0 | F33-based, first build of the day branched | 32.20191018.92.0 | -bodhi-updates-testing | 31.20191018.93.0 | -bodhi-updates | 31.20191018.94.0 | (any developer build) | 31.20191018.dev.2 | Third build of the day We are not committing to this version scheme indefinitely, and may change it in future if it proves unworkable. A new Fedora major release (X bump) would be a good time to make such a change. We don't intend Fedora CoreOS version numbers to be parsed by machine; they're meant to help humans quickly determine the salient properties of a release. diff --git a/PRD.txt b/PRD.txt index 1cd5a63..a09dd46 100644 --- a/PRD.txt +++ b/PRD.txt @@ -117,7 +117,7 @@ All artifacts will be downloadable from the getfedora.org website. Similarly, fo === Delivery Format === -Artifacts will be delivered as cloud images on Amazon EC2, Azure, DigitalOcean, Google Compute Engine, and Packet; as downloadable images for OpenStack, QEMU, VirtualBox, and VMware; and as ISO images, netboot images, and installable raw images for bare metal systems. We may add other public cloud images and downloadable formats to meet demand or anticipated need. +Artifacts will be delivered as cloud images on Amazon EC2, Azure, DigitalOcean, and Google Compute Engine; as downloadable images for OpenStack, QEMU, VirtualBox, and VMware; and as ISO images, netboot images, and installable raw images for bare metal systems. We may add other public cloud images and downloadable formats to meet demand or anticipated need. === Architectures === diff --git a/README.md b/README.md index d01c229..decca97 100644 --- a/README.md +++ b/README.md @@ -25,13 +25,13 @@ technologies and produce Fedora CoreOS. # Communication channels for Fedora CoreOS -- main mailing list: [coreos@lists.fedoraproject.org](https://lists.fedoraproject.org/archives/list/coreos@lists.fedoraproject.org/) -- status mailing list: [coreos-status@lists.fedoraproject.org](https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/) (announcements/important messages) -- IRC/Matrix: [`#fedora-coreos` on Libera.Chat](https://web.libera.chat/#fedora-coreos) (ircs://irc.libera.chat:6697/#fedora-coreos) or [`#coreos:fedoraproject.org` on Matrix](https://matrix.to/#/#coreos:fedoraproject.org) -- forum at [https://discussion.fedoraproject.org/tag/coreos](https://discussion.fedoraproject.org/tag/coreos) -- feature planning and important issue tracking at [github.com/coreos/fedora-coreos-tracker](https://github.com/coreos/fedora-coreos-tracker) -- website at [https://getfedora.org/coreos/](https://getfedora.org/coreos/) -- documentation at [https://docs.fedoraproject.org/en-US/fedora-coreos/](https://docs.fedoraproject.org/en-US/fedora-coreos/) +- Main mailing list: [coreos@lists.fedoraproject.org](https://lists.fedoraproject.org/archives/list/coreos@lists.fedoraproject.org/) +- Status mailing list: [coreos-status@lists.fedoraproject.org](https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/) (announcements/important messages) +- Chat room: [`#coreos:fedoraproject.org` on Matrix](https://chat.fedoraproject.org/#/room/#coreos:fedoraproject.org) +- Forum at [https://discussion.fedoraproject.org/tag/coreos-wg](https://discussion.fedoraproject.org/tag/coreos-wg) +- Feature planning and important issue tracking at [github.com/coreos/fedora-coreos-tracker](https://github.com/coreos/fedora-coreos-tracker) +- Website at [https://getfedora.org/coreos/](https://getfedora.org/coreos/) +- Documentation at [https://docs.fedoraproject.org/en-US/fedora-coreos/](https://docs.fedoraproject.org/en-US/fedora-coreos/) - Twitter: [@fedoracoreos](https://twitter.com/fedoracoreos) # Roadmap/Plans @@ -49,7 +49,7 @@ Thus, new package requests are carefully scrutinized to weigh the benefits and drawbacks of adding an additional package. If you would like to propose the inclusion of a new package in the base set of packages, -please file a [new package request](https://github.com/coreos/fedora-coreos-tracker/issues/new?labels=kind/enhancement&template=new-package.md&title=New+Package+Request%3A+%3Cpackage+name%3E). +please file a [new package request](https://github.com/coreos/fedora-coreos-tracker/issues/new/choose). # Releases @@ -58,56 +58,72 @@ See [RELEASES.md](RELEASES.md). # Meetings The Fedora CoreOS Working Group has a weekly meeting. The meeting usually -happens in `#fedora-meeting-1` on irc.libera.chat and the schedule for the -meeting can be found here: https://calendar.fedoraproject.org/CoreOS/ -Currently, meetings are at `16:30 UTC` on Wednesdays. +happens in +[#meeting-1:fedoraproject.org](https://matrix.to/#/#meeting-1:fedoraproject.org) +on Matrix and the schedule for the meeting can be found here: +https://calendar.fedoraproject.org/CoreOS/ Currently, meetings are at +[`15:30 UTC`](https://time.is/15:30+UTC) on Wednesdays. + +As the +[Matrix bridge to Libera Chat is shutdown](https://matrix.org/blog/2023/11/28/shutting-down-bridge-to-libera-chat/), +you can not attend the meeting from IRC and you have to join using Matrix. ## Steps to run the meeting +The fedora meeting host can follow the guide which is curated by the [fcos-meeting-action](https://github.com/coreos/fcos-meeting-action) repo. +Every Wednesday a new checklist will be available in the form of a issue in the fcos-meeting-action repo, which can be used to run the meeting. + +If the action meeting repo is not available for some reason, the host can follow the below steps to run the meeting. + +
+Legacy Meeting steps + - `cd` to a local checkout of this repo and `git pull` - Ping [meeting people](https://github.com/coreos/fedora-coreos-tracker/blob/main/meeting-people.txt) in `#fedora-coreos` on libera.chat - `bash meeting-people.txt` - - copy lines of output and paste into `#fedora-coreos` channel -- Navigate to `#fedora-meeting-1` on libera.chat -- Type `#startmeeting fedora_coreos_meeting` -- `#topic roll call` + - copy lines of output and paste into + [`#coreos:fedoraproject.org`](https://chat.fedoraproject.org/#/room/#coreos:fedoraproject.org) + on Matrix +- Navigate to + [`#meeting-1:fedoraproject.org`](https://matrix.to/#/#meeting-1:fedoraproject.org) + on Matrix +- Type: + - `!startmeeting fedora_coreos_meeting` + - `!topic roll call` Wait for 2-4 minutes for people to check in for the roll call. -- `#chair` all the people present for the meeting -- `#topic Action items from last meeting` +- `!topic Action items from last meeting` -Find the last meeting log from -[meetbot](https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting) -and post the action items in the meeting for people to -update the status of. +Find the last meeting log from [meetbot](https://meetbot.fedoraproject.org/) +and post the action items in the meeting for people to update the status of. - After they are done move to each `meeting` ticket from [this tracker](https://github.com/coreos/fedora-coreos-tracker/labels/meeting) Do the following for each ticket -- `#topic` Ticket subject -- `#link` link\_to\_the\_ticket +- `!topic` Ticket subject +- `!link ` During the meeting, you can give people action items for them to complete: -- `#action ` description of what needs to be done +- `!action ` description of what needs to be done When all topics are over, go for open floor: -- `#topic Open Floor` +- `!topic Open Floor` After open floor, end the meeting. -- `#endmeeting` +- `!endmeeting` Then, when convenient: - Remove `meeting` labels from [tickets that were discussed](https://github.com/coreos/fedora-coreos-tracker/labels/meeting) - Send an email to [coreos@lists.fedoraproject.org](mailto:coreos@lists.fedoraproject.org) with the -details of the meeting from [meetbot page](https://meetbot.fedoraproject.org/sresults/?group_id=fedora_coreos_meeting&type=team). +details of the meeting from [meetbot page](https://meetbot.fedoraproject.org/). Minutes in textual format are directly available using `.txt` as URL extension. It's easiest to get the Minutes/Minutes (text)/Log URLs by copying the footer that Meetbot prints after `#endmeeting`. You can see examples in the @@ -125,6 +141,7 @@ Log: ``` +
# Voting @@ -160,3 +177,23 @@ Working days: non-holiday weekdays. Relevant holidays are the national holidays # Working Group Members and Points of Contact Please see [meeting-people.txt](https://github.com/coreos/fedora-coreos-tracker/blob/main/meeting-people.txt). + +# Metrics + +To view CountME stats you can use a tool called +[sqlitevis](https://sqliteviz.com/) to view the +CountME database and make graphs. This can easily be done with a +single URL but due to [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS) +you have to run your browser in a specific mode to allow the +application to download the database and the inquiries file: + +``` +chromium-browser --disable-web-security --user-data-dir=~/chrome-disable-web-security/ +# OR +google-chrome-stable --disable-web-security --user-data-dir=~/chrome-disable-web-security/ +``` + +Now navigate to +[this](https://sqliteviz.com/app/#/load?data_url=https%3A%2F%2Fdata-analysis.fedoraproject.org%2Fcsv-reports%2Fcountme%2Ftotals-countme.db&data_format=sqlite&inquiry_url=https%3A%2F%2Fraw.githubusercontent.com%2Fcoreos%2Ffedora-coreos-tracker%2Frefs%2Fheads%2Fmain%2Fmetrics%2Ffcos-sqlitevis.json) +URL in the browser and it should autoload the database and the inquiries. This +URL was generated from the [sqlitevis docs](https://sqliteviz.com/docs/sharing/). diff --git a/docs/ci-and-builds.md b/docs/ci-and-builds.md index e06f704..aeffbf0 100644 --- a/docs/ci-and-builds.md +++ b/docs/ci-and-builds.md @@ -51,7 +51,7 @@ Examples: ## quay.io/coreos-assembler namespace -A key aspect of Fedora CoreOS as well as RHEL CoreOS is [coreos-assembler](https://github.com/coreos/coreos-assembler). As of today, we build it in quay.io and deliver it that way in the `quay.io/coreos-assembler` namespace. The list of administrators for this namespace is managed independently of anything else. If you think you need administrator access, file a ticket or ask on #fedora-coreos IRC. +A key aspect of Fedora CoreOS as well as RHEL CoreOS is [coreos-assembler](https://github.com/coreos/coreos-assembler). As of today, we build it in quay.io and deliver it that way in the `quay.io/coreos-assembler` namespace. The list of administrators for this namespace is managed independently of anything else. If you think you need administrator access, file a ticket or ask on [`#coreos:fedoraproject.org` on Matrix](https://chat.fedoraproject.org/#/room/#coreos:fedoraproject.org). ### The buildroot container: quay.io/coreos-assembler/fcos-buildroot:testing-devel diff --git a/docs/fedora-coreos-kernel-bisect.md b/docs/fedora-coreos-kernel-bisect.md index 9f7de5c..0742cfc 100644 --- a/docs/fedora-coreos-kernel-bisect.md +++ b/docs/fedora-coreos-kernel-bisect.md @@ -37,7 +37,7 @@ Here's a summary of what those branches are used for: - `os-build` - The latest bits that track the under development yet to be release kernel. -- `fedora-5.16` +- `fedora-6.3` - Follows a particular released kernel stream. This is where things are merged before they are fed into dist-git. If you want a commit reverted this is where it will land first. @@ -54,7 +54,7 @@ environment directly in the VM. If not you'll probably want to use a container for your kernel builds. Here's how to start up a container: ``` -podman run -it --name=kbuild -v /path/to/kernel/git/:/path/to/kernel/git/ registry.fedoraproject.org/fedora:35 +podman run -it --name=kbuild -v /path/to/kernel/git/:/path/to/kernel/git/ registry.fedoraproject.org/fedora:38 ``` NOTE: try to use the same Fedora Cloud or Fedora container version as @@ -64,20 +64,22 @@ Once inside the VM or container we need to install some software to build the ke ``` sudo dnf update -y && \ -sudo dnf install -y rpm-build rsync 'dnf-command(builddep)' && \ +sudo dnf install -y make rpm-build rsync 'dnf-command(builddep)' && \ sudo dnf builddep -y kernel # reboot here if in a VM ``` We can now make changes to the git repo (revert commits, etc) and run a few commands to build the kernel. Before building we need to copy down the config -from the kernel dist-git repo and disable DEBUG symbols if they were enabled -(makes very large files): +from the kernel dist-git repo and disable making a DEBUG kernel if it was enabled, +which makes very large files: ``` cd /path/to/kernel/git/ -curl https://src.fedoraproject.org/rpms/kernel/raw/f35/f/kernel-x86_64-fedora.config > .config -sed -i 's/CONFIG_DEBUG_INFO=y/CONFIG_DEBUG_INFO=n/' .config +RELEASE=f38 # or RELEASE=rawhide +curl "https://src.fedoraproject.org/rpms/kernel/raw/${RELEASE}/f/kernel-x86_64-fedora.config" > .config.fedora +cp .config.fedora .config +sed -i 's/CONFIG_DEBUG_KERNEL=y/CONFIG_DEBUG_KERNEL=n/' .config ``` ## 1. Directly Building and Installing the Kernel from Kernel Source git repo @@ -86,8 +88,10 @@ To build and install the kernel directly on the system (i.e. on Fedora Cloud Bas you can run the following: ``` +# Set make target. See https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel.spec +make_target=bzImage # for x86_64 or vmlinux(ppc64le) or vmlinuz.efi(aarch64) make olddefconfig -make -j$(nproc) bzImage +make -j$(nproc) $make_target make -j$(nproc) modules sudo make modules_install sudo make install @@ -109,7 +113,12 @@ Then run the following script to build and install the kernel: cat build.sh #!/bin/bash set -eux -o pipefail -make -j$(nproc) bzImage +cp .config.fedora .config +sed -i 's/CONFIG_DEBUG_KERNEL=y/CONFIG_DEBUG_KERNEL=n/' .config +# Set make target. See https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel.spec +make_target=bzImage # for x86_64 or vmlinux(ppc64le) or vmlinuz.efi(aarch64) +make olddefconfig +make -j$(nproc) $make_target make -j$(nproc) modules sudo make modules_install sudo make install @@ -127,6 +136,12 @@ sudo rm -vf /boot/initramfs*bisect* /boot/vmlinuz-*bisect* /boot/System.map-*bis sudo rm -rf /lib/modules/*bisect* ``` +Then you can automate with: + +``` +bash clean.sh && bash build.sh +``` + ## 2. Directly Building and Creating an RPM from the Kernel Source git repo In this scenario we're creating an RPM that can either then be package @@ -151,40 +166,11 @@ sudo rpm-ostree override replace ./kernel-5.17.0_rc8-1.x86_64.rpm --remove=kerne ### Doing a Build with COSA Then copy the built RPM into the `overrides/rpm` folder under the COSA build directory. -Update the `manifest-lock.overrides.yaml` to specify the kernel and also update the manifest -to not specify `kernel-core` and `kernel-modules`. Here is an example: - - -```diff -diff --git a/manifest-lock.overrides.yaml b/manifest-lock.overrides.yaml -index 62cfbe5..81de60f 100644 ---- a/manifest-lock.overrides.yaml -+++ b/manifest-lock.overrides.yaml -@@ -8,4 +8,6 @@ - # in the `metadata.reason` key, though it's acceptable to omit a `reason` - # for FCOS-specific packages (ignition, afterburn, etc.). - --packages: {} -+packages: -+ kernel: -+ evr: 5.17.0_rc8+-2 -diff --git a/manifests/bootable-rpm-ostree.yaml b/manifests/bootable-rpm-ostree.yaml -index 784acd4..734f374 100644 ---- a/manifests/bootable-rpm-ostree.yaml -+++ b/manifests/bootable-rpm-ostree.yaml -@@ -7,7 +7,8 @@ - packages: - # Kernel + systemd. Note we explicitly specify kernel-{core,modules} - # because otherwise depsolving could bring in kernel-debug. -- - kernel kernel-core kernel-modules systemd -+ - kernel systemd - # linux-firmware now a recommends so let's explicitly include it - # https://gitlab.com/cki-project/kernel-ark/-/commit/32271d0cd9bd52d386eb35497c4876a8f041f70b - # https://src.fedoraproject.org/rpms/kernel/c/f55c3e9ed8605ff28cb9a922efbab1055947e213?branch=rawhide -``` - After that you should be able to `cosa fetch --with-cosa-overrides && cosa build` like normal. +While iterating you should be able to skip the `cosa fetch` step. Just delete the old +RPM out of `overrides/rpm`, put the new one in place and then `cosa build`. + ## Performing a Kernel Bisect @@ -192,3 +178,52 @@ Now that we know how to build and use a kernel in various ways the bisect is the easy part. Just follow the [upstream kernel documentation](https://www.kernel.org/doc/html/latest/admin-guide/bug-bisect.html) for doing a `git bisect` and repeat the build/test steps in between each step. + +## Reporting issues upstream + +Unfortunately the kernel doesn't have any git forge structure. It's +mostly email and mailing lists. If you want to report an issue +upstream you can run a command to give you what people/lists to email: + +``` +commit=abcdef +git format-patch --stdout "${commit}^..${commit}" | \ + ./scripts/get_maintainer.pl --norolestats +``` + +example: + +``` +$ commit=a09b314 +$ git format-patch --stdout "${commit}^..${commit}" | ./scripts/get_maintainer.pl --norolestats +Jens Axboe +linux-block@vger.kernel.org +linux-kernel@vger.kernel.org +``` + +## Testing out fixes with Fedora's kernel + +Once you have a proposed fix/patch you can easily build a Fedora kernel RPM by +adding your patch to the [`linux-kernel-test.patch` file](https://docs.fedoraproject.org/en-US/quick-docs/kernel/testing-patches/#_applying_the_patch) +in the [kernel distgit repo](https://src.fedoraproject.org/rpms/kernel). + +After adding your patch you can then use `fedpkg` to build a new +kernel for your target architecture. For example: + +``` +fedpkg scratch-build --srpm --arch=x86_64 +``` + +Once the build is complete you can grab the RPMs using the `koji` CLI: + +``` +koji download-task +``` + +Placing these RPMs into the `overrides/rpm` directory and do a new COSA build +will give you a CoreOS build with the patched kernel. + +After the tested patch looks good you can then open a PR to the `fedora-X.Y` +branch in the `kernel-ark` repo. See the above +[Kernel Source git Repos](#kernel-source-git-repos) +section for more details. diff --git a/docs/fedora-coreos-systemd-bisect.md b/docs/fedora-coreos-systemd-bisect.md new file mode 100644 index 0000000..673804a --- /dev/null +++ b/docs/fedora-coreos-systemd-bisect.md @@ -0,0 +1,66 @@ + +# Systemd regressions need bisecting + +Similar to the kernel, systemd is often a core component of our +stack that has regressions that aren't easy to identify just by +inspecting a changelog. + +## Systemd Source git Repos + +There are a few kernel source git repositories to know about: + +- `https://github.com/systemd/systemd.git` + - Where the latest upstream development happens +- `https://github.com/systemd/systemd-stable.git` + - Where stable/LTS tags are handled (backports to stable branches happen here) + +There is also the [Fedora dist-git repo](https://src.fedoraproject.org/rpms/systemd). + +## Creating a Kernel Build Environment + +You can use a container to build systemd from upstream. + +``` +SHARED=/path/to/shared/directory/ +RELEASE=38 +podman run -it --name=systemdbuild -v "${SHARED}:${SHARED}" "registry.fedoraproject.org/fedora:${RELEASE}" +``` + +``` +sudo dnf update -y && \ +sudo dnf install -y make rpm-build rsync 'dnf-command(builddep)' && \ +sudo dnf builddep -y systemd +``` + +We can now make changes to the git repo (revert commits, etc) and run a few +commands to build systemd. If doing a +[`git` bisect](https://www.kernel.org/doc/html/latest/admin-guide/bug-bisect.html) +run the commands needed to start the bisect. + +## Doing the systemd build/test + +To build systemd you can run the following commands. These commands +were adapted from the notes in the +[Systemd README](https://github.com/systemd/systemd/blob/579fbe5b789cbee10546f6274c39be311e71e49c/README#L233-L247). + + +``` +meson setup build/ +``` + +And then the following can be iterated upon for each commit to test: + +``` +export DESTDIR=/path/to/shared/directory/fcos/overrides/rootfs/ +ninja -C build && ninja -C build install +``` + +NOTE: If you run into `permission denied` errors when copying the files around check for SELinux denails. + +Now you can run COSA to build/test. From the COSA directory: + +``` +cosa build && cosa kola run mytest +``` + +Now you can iterate until you find the problematic commit. diff --git a/meeting-people.txt b/meeting-people.txt index 955c7f0..967e3cf 100644 --- a/meeting-people.txt +++ b/meeting-people.txt @@ -1,22 +1,25 @@ # List of people to ping before the Fedora CoreOS community meetings. # Please keep this list in alphabetical order. -tail -n +5 $0 | tr '\n' ' ' && echo -e '\nFCOS community meeting in #fedora-meeting-1' && echo "If you don't want to be pinged remove your name from this file: https://github.com/coreos/fedora-coreos-tracker/blob/main/meeting-people.txt" -exit 0 - -aaradhak -davdunc -dustymabe -gurssing -jaimelm -jbrooks -jcajka -jdoss -jlebon -jmarrero -lorbus -miabbott -nasirhm -ravanelli -saqali -skunkerk -walters +@aaradhak:matrix.org +@apiaseck:matrix.org +@bipinbn:fedora.im +@bri:transfem.dev +@davdunc:fedora.im +@dustymabe:matrix.org +@guidon:guidon.ems.host +@gurssing:matrix.org +@jaimelm:fedora.im +@jbrooks:matrix.org +@jbtrystram:matrix.org +@jdoss:fedora.im +@jlebon:fedora.im +@jmarrero:matrix.org +@marmijo:fedora.im +@miabbott:fedora.im +@quentin9696:matrix.org +@rapneset:matrix.org +@ravanelli:fedora.im +@tlbueno:fedora.im +@walters:fedora.im +@ydesouza:fedora.im +@yves:siegrist.io diff --git a/metadata/README.md b/metadata/README.md index 495495d..8fd7f1d 100644 --- a/metadata/README.md +++ b/metadata/README.md @@ -80,4 +80,3 @@ RPMs and our configuration into images and ostree commits. Projects: - https://github.com/coreos/coreos-assembler - - https://github.com/coreos/fedora-coreos-releng-automation/blob/main/coreos-meta-translator/trans.py diff --git a/metadata/release/sample.json b/metadata/release/sample.json index 2a1c1d8..23818b4 100644 --- a/metadata/release/sample.json +++ b/metadata/release/sample.json @@ -36,6 +36,17 @@ } } }, + "applehv": { + "artifacts": { + "raw.gz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-applehv.raw.gz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-applehv.raw.gz.sig", + "sha256": "a889159d661339e635372b807f0a98bb93c64aabfaf89a801b2f03491488f0ef" + } + } + } + }, "azure": { "artifacts": { "vhd.xz": { @@ -91,6 +102,28 @@ } } }, + "hetzner": { + "artifacts": { + "raw.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-hetzner.raw.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-hetzner.raw.xz.sig", + "sha256": "a889159d661339e635372b807f0a98bb93c64aabfaf89a801b2f03491488f0ef" + } + } + } + }, + "hyperv": { + "artifacts": { + "vhdx.zip": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-hyperv.vhdx.zip", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-hyperv.vhdx.zip.sig", + "sha256": "a889159d661339e635372b807f0a98bb93c64aabfaf89a801b2f03491488f0ef" + } + } + } + }, "ibmcloud": { "artifacts": { "qcow2.xz": { @@ -104,10 +137,10 @@ }, "kubevirt": { "artifacts": { - "qcow2.xz": { + "ociarchive": { "disk": { - "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-kubevirt.qcow2.xz", - "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-kubevirt.qcow2.xz.sig", + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-kubevirt.ociarchive", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-kubevirt.ociarchive.sig", "sha256": "2accf3a1385b9eb435224bb0e1595f4b3e4344c1cc05b1d2484e163c66f35bca" } } @@ -170,6 +203,28 @@ } } }, + "oraclecloud": { + "artifacts": { + "qcow2.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-oraclecloud.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-oracecloud.qcow2.xz.sig", + "sha256": "868da197ae9179aded982ea6445d7d5e30acf8d03cdcdc32acfe2003d2c65491" + } + } + } + }, + "proxmoxve": { + "artifacts": { + "raw.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-proxmoxve.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-proxmoxve.qcow2.xz.sig", + "sha256": "394cd6431b19c82a46a7215ebead15960faf9814092203456d56960a1b4d8777" + } + } + } + }, "qemu": { "artifacts": { "qcow2.xz": { @@ -181,6 +236,17 @@ } } }, + "qemu-secex": { + "artifacts": { + "qcow2.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-qemu-secex.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/30.20190801.0/x86_64/fedora-coreos-30.20190801.0-qemu-secex.qcow2.xz.sig", + "sha256": "2afbb0ac4a19f58a55db35db0a690d488f065664e9bcba1b802966f0ae6aad57" + } + } + } + }, "virtualbox": { "artifacts": { "ova": { diff --git a/metadata/stream/rationale.yaml b/metadata/stream/rationale.yaml index 54e8567..4072f34 100644 --- a/metadata/stream/rationale.yaml +++ b/metadata/stream/rationale.yaml @@ -10,8 +10,8 @@ architectures: artifacts: # Some of these will be useful for many users, such as qemu or # openstack. Some will likely only be useful for cloud operators, - # such as digitalocean or packet. Some, such as aws, are useful - # for users in special situations. + # such as digitalocean. Some, such as aws, are useful for users + # in special situations. aliyun: release: 30.1.2.3 formats: @@ -21,6 +21,14 @@ architectures: signature: https://artifacts.example.com/g0xah6aenvaaVosh.qcow2.xz.sig sha256: 149afbf4c8996fb92427ae3b0c44298fc1ce41e4649b934ca495991b7852b855 uncompressed-sha256: d02d5ac0f2a2789602e9df950c38acb15380d2799b4bdb59394e4eeabdd3a662 + applehv: + release: 30.1.2.3 + formats: + "raw.gz": + disk: + location: https://artifacts.example.com/quohgh8ei0uzaD5a.raw.gz + signature: https://artifacts.example.com/quohgh8ei0uzaD5a.raw.gz.sig + sha256: 4c8996fb92427ae41e4649b934ca4e3b0c44298fc1c149afbf95991b7852b855 aws: release: 30.1.2.3 formats: @@ -78,6 +86,22 @@ architectures: signature: https://artifacts.example.com/ais7tah1aa7Ahvei.tar.gz.sig sha256: 96fb92427ae41e4649b934ca495991b7852b85e3b0c44298fc1c149afbf4c895 uncompressed-sha256: 38acb15d02d5ac0f2a2789602e9df950c380d2799b4bdb59394e4eeabdd3a662 + hetzner: + release: 30.1.2.3 + formats: + "raw.xz": + disk: + location: https://artifacts.example.com/quohgh8ei0uzaD5a.raw.xz + signature: https://artifacts.example.com/quohgh8ei0uzaD5a.raw.xz.sig + sha256: 4c8996fb92427ae41e4649b934ca4e3b0c44298fc1c149afbf95991b7852b855 + hyperv: + release: 30.1.2.3 + formats: + "vhdx.zip": + disk: + location: https://artifacts.example.com/quohgh8ei0uzaD5a.vhdx.zip + signature: https://artifacts.example.com/quohgh8ei0uzaD5a.vhdx.zip.sig + sha256: 4c8996fb92427ae41e4649b934ca4e3b0c44298fc1c149afbf95991b7852b855 ibmcloud: release: 30.1.2.3 formats: @@ -90,12 +114,11 @@ architectures: kubevirt: release: 30.1.2.3 formats: - "qcow2.xz": + "ociarchive": disk: - location: https://artifacts.example.com/Kiejeeb6ohpu8Eel.qcow2.xz - signature: https://artifacts.example.com/Kiejeeb6ohpu8Eel.qcow2.xz.sig + location: https://artifacts.example.com/Kiejeeb6ohpu8Eel.ociarchive + signature: https://artifacts.example.com/Kiejeeb6ohpu8Eel.ociarchive.sig sha256: 2427ae41e4649b934ca495991b7852b85e3b0c44298fc1c149afbf4c8996fb95 - uncompressed-sha256: 38acb15d02d5ac0f2a2789602e9df950c380d2799b4bdb59394e4eeabdd3a662 metal: release: 30.1.2.3 formats: @@ -140,15 +163,23 @@ architectures: signature: https://artifacts.example.com/oKooheogobofai8l.qcow2.xz.sig sha256: ae41e4649b934ca495991b785e3b0c44298fc1c149afbf4c8996fb924272b855 uncompressed-sha256: 38acb15d02d5ac0f2a2789602e9df950c380d2799b4bdb59394e4eeabdd3a662 - packet: + oraclecloud: release: 30.1.2.3 formats: - "raw.xz": + "qcow2.xz": disk: - location: https://artifacts.example.com/Oofohng0xo2phai5.raw.xz - signature: https://artifacts.example.com/Oofohng0xo2phai5.raw.xz.sig - sha256: e41e4649b934ca495991b7852b85e3b0c44298fc1c149afbf4c8996fb92427a5 - uncompressed-sha256: 38acb15d02d5ac0f2a2789602e9df950c380d2799b4bdb59394e4eeabdd3a662 + location: https://artifacts.example.com/oKooheogobofai8l.qcow2.xz + signature: https://artifacts.example.com/oKooheogobofai8l.qcow2.xz.sig + sha256: 868da197ae9179aded982ea6445d7d5e30acf8d03cdcdc32acfe2003d2c65491" + uncompressed-sha256: 75a5c30bf84a605cc9fa617e856d9523d8d4c50607837a7d33e4d81e9809891a + proxmoxve: + release: 30.1.2.3 + formats: + "qcow2.xz": + disk: + location: https://artifacts.example.com/TieshohWah0aewai/.qcow2.xz + signature: https://artifacts.example.com/TieshohWah0aewai/.qcow2.xz.sig + sha256: 394cd6431b19c82a46a7215ebead15960faf9814092203456d56960a1b4d8777 qemu: release: 30.1.2.3 formats: @@ -158,6 +189,15 @@ architectures: signature: https://artifacts.example.com/Siejeeb6ohpu8Eel.qcow2.xz.sig sha256: b0c44298fc1c149afbf4c8996fb9242e37ae41e4649991b7852b855b934ca495 uncompressed-sha256: 38acb15d02d5ac0f2a2789602e9df950c380d2799b4bdb59394e4eeabdd3a662 + qemu-secex: + release: 30.1.2.3 + formats: + "qcow2.xz": + disk: + location: https://artifacts.example.com/6d5814250381013f.qcow2.xz + signature: https://artifacts.example.com/6d5814250381013f.qcow2.xz.sig + sha256: 2afbb0ac4a19f58a55db35db0a690d488f065664e9bcba1b802966f0ae6aad57 + uncompressed-sha256: 2b1cb667f3468ef7b462e5ec8395fcd2982e424d1727336e95f74c611d8bbd53 virtualbox: release: 30.1.2.3 formats: @@ -234,7 +274,3 @@ architectures: # currently recommended image, and its release. release: 30.1.2.3 digest-ref: exampleregistry.io/fcos/fcos@sha256:67a81539946ec0397196c145394553b8e0241acf27b14ae9de43bc56e167f773 - packet: - # Images don't have addressable versions, so an operating system - # slug is the best we can do. - image: fedora_coreos_stable diff --git a/metadata/stream/sample.json b/metadata/stream/sample.json index 81c3dd0..134ebb0 100644 --- a/metadata/stream/sample.json +++ b/metadata/stream/sample.json @@ -20,6 +20,18 @@ } } }, + "applehv": { + "release": "33.20210412.3.0", + "formats": { + "raw.gz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-applehv.x86_64.raw.gz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-applehv.x86_64.raw.gz.sig", + "sha256": "728e876d87ec71de27fc1d882840e6877346423433339a2b8606fa28e57413fd" + } + } + } + }, "aws": { "release": "33.20210412.3.0", "formats": { @@ -96,6 +108,30 @@ } } }, + "hetzner": { + "release": "33.20210412.3.0", + "formats": { + "raw.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-hetzner.x86_64.raw.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-hetzner.x86_64.raw.xz.sig", + "sha256": "728e876d87ec71de27fc1d882840e6877346423433339a2b8606fa28e57413fd" + } + } + } + }, + "hyperv": { + "release": "33.20210412.3.0", + "formats": { + "vhdx.zip": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-hyperv.x86_64.vhdx.zip", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-hyperv.x86_64.vhdx.zip.sig", + "sha256": "728e876d87ec71de27fc1d882840e6877346423433339a2b8606fa28e57413fd" + } + } + } + }, "ibmcloud": { "release": "33.20210412.3.0", "formats": { @@ -112,12 +148,11 @@ "kubevirt": { "release": "33.20210412.3.0", "formats": { - "qcow2.xz": { + "ociarchive": { "disk": { - "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-kubevirt.x86_64.qcow2.xz", - "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-kubevirt.x86_64.qcow2.xz.sig", - "sha256": "6343b99ca70975bd821050f274aa1db0898fb88aae95a79f63d18a2e2a489e26", - "uncompressed-sha256": "744f25cf86927fe4780b57cd75c2d5b979e15336e4c9bd02fe4f71827d820d4c" + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-kubevirt.x86_64.ociarchive", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-kubevirt.x86_64.ociarchive.sig", + "sha256": "6343b99ca70975bd821050f274aa1db0898fb88aae95a79f63d18a2e2a489e26" } } } @@ -192,6 +227,31 @@ } } }, + "oraclecloud": { + "release": "33.20210412.3.0", + "formats": { + "qcow2.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-oraclecloud.x86_64.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-oraclecloud.x86_64.qcow2.xz.sig", + "sha256": "868da197ae9179aded982ea6445d7d5e30acf8d03cdcdc32acfe2003d2c65491", + "uncompressed-sha256": "75a5c30bf84a605cc9fa617e856d9523d8d4c50607837a7d33e4d81e9809891a" + } + } + } + }, + "proxmoxve": { + "release": "33.20210412.3.0", + "formats": { + "qcow2.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-proxmoxve.x86_64.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-proxmoxve.x86_64.qcow2.xz.sig", + "sha256": "394cd6431b19c82a46a7215ebead15960faf9814092203456d56960a1b4d8777" + } + } + } + }, "qemu": { "release": "33.20210412.3.0", "formats": { @@ -205,6 +265,19 @@ } } }, + "qemu-secex": { + "release": "33.20210412.3.0", + "formats": { + "qcow2.xz": { + "disk": { + "location": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-qemu-secex.x86_64.qcow2.xz", + "signature": "https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/33.20210412.3.0/x86_64/fedora-coreos-33.20210412.3.0-qemu-secex.x86_64.qcow2.xz.sig", + "sha256": "2afbb0ac4a19f58a55db35db0a690d488f065664e9bcba1b802966f0ae6aad57", + "uncompressed-sha256": "2b1cb667f3468ef7b462e5ec8395fcd2982e424d1727336e95f74c611d8bbd53" + } + } + } + }, "virtualbox": { "release": "33.20210412.3.0", "formats": { diff --git a/metrics/README.md b/metrics/README.md new file mode 100644 index 0000000..24a4941 --- /dev/null +++ b/metrics/README.md @@ -0,0 +1,2 @@ + +See [README.md](../README.md#metrics). diff --git a/metrics/fcos-sqlitevis.json b/metrics/fcos-sqlitevis.json new file mode 100644 index 0000000..1c3c311 --- /dev/null +++ b/metrics/fcos-sqlitevis.json @@ -0,0 +1,711 @@ +{ + "version": 2, + "inquiries": [ + { + "id": "WUPD4gZdu-j4mFgxjHG0P", + "query": "SELECT os_variant FROM countme_totals \n WHERE weeknum = (SELECT MAX(weeknum) FROM countme_totals)\n AND os_variant REGEXP ''\n GROUP BY os_variant;", + "viewType": "chart", + "viewOptions": { + "data": [], + "layout": { + "autosize": true, + "xaxis": { + "range": [ + -1, + 6 + ], + "autorange": true + }, + "yaxis": { + "range": [ + -1, + 4 + ], + "autorange": true + } + }, + "frames": [] + }, + "name": "All OS Variants", + "createdAt": "2025-05-12T20:54:27.120Z" + }, + { + "id": "tcIRiJz5gn5ci4DJyHgqU", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, weeknum, os_variant, repo_arch, SUM(hits) FROM countme_totals \n WHERE os_variant IS 'coreos'\n AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' \n AND weeknum = (SELECT MAX(weeknum) FROM countme_totals)\n GROUP BY repo_arch;", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "pie", + "mode": "markers", + "values": null, + "valuessrc": "SUM(hits)", + "meta": { + "columnNames": { + "values": "SUM(hits)", + "labels": "repo_arch", + "text": "" + } + }, + "labels": null, + "labelssrc": "repo_arch", + "opacity": 1, + "textinfo": "label+value+percent", + "textfont": { + "size": 26, + "family": "sans-serif" + }, + "hoverinfo": "percent+label+value", + "hoverlabel": { + "align": "auto" + }, + "direction": "counterclockwise", + "rotation": 0, + "hole": 0.52, + "pull": 0, + "marker": { + "line": { + "width": 1 + } + }, + "insidetextorientation": "radial" + } + ], + "layout": { + "xaxis": { + "range": [ + -1, + 6 + ], + "autorange": true + }, + "yaxis": { + "range": [ + -1, + 4 + ], + "autorange": true + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "title": { + "text": "Fedora CoreOS Node Architecture Breakdown Week of 2025-05-04", + "font": { + "size": 25 + } + }, + "hiddenlabels": [ + "ppc64le", + "s390x" + ], + "legend": { + "x": 0.7407924239291469, + "y": 0.8257272143643333, + "font": { + "size": 20 + }, + "yanchor": "middle" + }, + "annotations": [], + "meta": [ + "2023-10-08", + "2023-10-08", + "2023-10-08", + "2023-10-08" + ], + "metasrc": "date", + "extendpiecolors": true + }, + "frames": [] + }, + "name": "FCOS Architectures", + "createdAt": "2025-05-12T20:55:12.622Z" + }, + { + "id": "wEj338NrufIRE-3UBDXPK", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, weeknum, SUM(transient_hits), SUM(static_hits), SUM(transient_hits + static_hits) FROM (\n SELECT weeknum, SUM(hits) AS transient_hits, 0 AS static_hits FROM countme_totals WHERE os_variant IS 'coreos' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age = 1 GROUP BY weeknum\n UNION\n SELECT weeknum, 0 AS transient_hits, SUM(hits) AS static_hits FROM countme_totals WHERE os_variant IS 'coreos' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age > 1 GROUP BY weeknum\n) WHERE date > '2020-01-01' GROUP BY weeknum", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "scatter", + "mode": "lines", + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(static_hits)" + } + }, + "y": null, + "ysrc": "SUM(static_hits)", + "stackgroup": 1, + "name": "Static Nodes", + "hoveron": "points" + }, + { + "type": "scatter", + "mode": "lines", + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(transient_hits)" + } + }, + "y": null, + "ysrc": "SUM(transient_hits)", + "stackgroup": 1, + "name": "Transient Nodes", + "fillcolor": "rgba(205, 96, 52, 0.5)", + "line": { + "color": "rgb(180, 38, 5)" + } + }, + { + "type": "scatter", + "mode": "lines", + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(transient_hits + static_hits)", + "text": "" + } + }, + "y": null, + "ysrc": "SUM(transient_hits + static_hits)", + "name": "Total Nodes", + "line": { + "width": 3, + "color": "rgb(95, 100, 96)" + }, + "hovertemplate": "", + "error_x": { + "_template": null, + "visible": false, + "type": "percent", + "symmetric": true, + "value": 10, + "color": "rgb(95, 100, 96)", + "thickness": 2, + "width": 4 + } + } + ], + "layout": { + "xaxis": { + "range": [ + "2021-09-19 23:38:29.3717", + "2025-05-22 01:49:05.1712" + ], + "autorange": false, + "rangeselector": { + "visible": false, + "buttons": [ + {} + ] + }, + "showspikes": false, + "rangeslider": { + "visible": false, + "yaxis": {}, + "autorange": true, + "range": [ + "2020-05-03", + "2023-11-28 23:10:02.9513" + ] + }, + "type": "date", + "tickfont": { + "size": 28 + }, + "title": { + "font": { + "size": 17 + } + } + }, + "yaxis": { + "range": [ + -1419.2166576673771, + 123989.902612791 + ], + "autorange": false, + "ticks": "", + "showspikes": false, + "showline": false, + "zeroline": true, + "type": "linear", + "tickfont": { + "size": 28 + } + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "dragmode": "pan", + "title": { + "text": "Fedora CoreOS Node Count", + "font": { + "size": 33 + } + }, + "legend": { + "font": { + "size": 28 + }, + "orientation": "h", + "x": 0.2471859552265083, + "y": 0.9623782823483056 + } + }, + "frames": [] + }, + "name": "FCOS Node Count", + "createdAt": "2025-05-12T20:55:20.874Z" + }, + { + "id": "rcnNNlpFRfhIaX27ownZa", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, upper(trim(repo_tag, 'updates-releaseed-')) as repo_tag, os_variant, SUM(hits) FROM countme_totals\n WHERE os_variant IS 'coreos'\n AND repo_tag REGEXP 'updates-released-f[3-4][0-9]'\n AND weeknum = (SELECT MAX(weeknum) FROM countme_totals)\n GROUP BY repo_tag;", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "pie", + "mode": "markers", + "values": null, + "valuessrc": "SUM(hits)", + "meta": { + "columnNames": { + "values": "SUM(hits)", + "labels": "repo_tag" + } + }, + "labels": null, + "labelssrc": "repo_tag", + "hole": 0.5, + "pull": 0, + "marker": { + "line": { + "width": 2 + } + }, + "textinfo": "label", + "textfont": { + "size": 25 + }, + "sort": false, + "direction": "clockwise", + "rotation": -90, + "legendgroup": 1, + "showlegend": true, + "hoverinfo": "percent+label+value", + "opacity": 1, + "textposition": "inside" + } + ], + "layout": { + "xaxis": { + "range": [ + -1, + 6 + ], + "autorange": true + }, + "yaxis": { + "range": [ + -1, + 4 + ], + "autorange": true + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "title": { + "text": "Fedora CoreOS Release Breakdown", + "x": 0.5, + "font": { + "size": 31 + } + }, + "showlegend": true, + "legend": { + "font": { + "family": "monospace", + "size": 22 + }, + "title": { + "text": "
", + "font": { + "size": 34 + } + }, + "y": 0.04329087951849141, + "x": 0.20084040421902638, + "yanchor": "bottom", + "orientation": "v" + }, + "hiddenlabels": [], + "hoverlabel": { + "align": "auto" + }, + "uniformtext": { + "mode": false + }, + "modebar": { + "orientation": "h" + }, + "margin": { + "pad": 0, + "r": 80 + }, + "extendpiecolors": true, + "piecolorway": [ + "#1b9e77", + "#d95f02", + "#7570b3", + "#e7298a", + "#66a61e", + "#e6ab02", + "#a6761d", + "#666666" + ] + }, + "frames": [] + }, + "name": "FCOS Release Breakdown", + "createdAt": "2025-05-12T20:55:31.761Z" + }, + { + "id": "BZlflOgPAYGoaKwjBpIe8", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, weeknum, SUM(coreos_hits), SUM(cloud_hits), SUM(server_hits) FROM (\n SELECT weeknum, 0 AS server_hits, 0 AS cloud_hits, SUM(hits) AS coreos_hits FROM countme_totals WHERE os_variant IS 'coreos' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age > 1 GROUP BY weeknum\n UNION\n SELECT weeknum, 0 AS server_hits, SUM(hits) AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'cloud' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age > 1 GROUP BY weeknum\n UNION\n SELECT weeknum, SUM(hits) AS server_hits, 0 AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'server' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age > 1 GROUP BY weeknum\n) WHERE date > '2022-01-01' GROUP BY weeknum", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "scatter", + "mode": "lines", + "stackgroup": null, + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(cloud_hits)" + } + }, + "y": null, + "ysrc": "SUM(cloud_hits)", + "name": "Cloud", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(coreos_hits)" + } + }, + "y": null, + "ysrc": "SUM(coreos_hits)", + "name": "CoreOS", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "stackgroup": null, + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(server_hits)" + } + }, + "y": null, + "ysrc": "SUM(server_hits)", + "name": "Server", + "line": { + "width": 5 + } + } + ], + "layout": { + "xaxis": { + "range": [ + "2023-05-18 21:13:31.5607", + "2025-05-04" + ], + "autorange": false, + "type": "date", + "tickfont": { + "size": 22 + } + }, + "yaxis": { + "range": [ + -4149.444444444446, + 85743.15981948335 + ], + "autorange": false, + "type": "linear", + "tickfont": { + "size": 22 + } + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "title": { + "text": "Static Node Count for Fedora Cloud/CoreOS/Server" + }, + "dragmode": "zoom", + "legend": { + "font": { + "size": 28 + }, + "orientation": "h", + "x": 0.4185161699429296, + "y": 0.988780487804878 + } + }, + "frames": [] + }, + "name": "Static Node Count By Edition", + "createdAt": "2025-05-12T20:56:19.303Z" + }, + { + "id": "r6KJ-g1sxjbqtOYuWRJoK", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, weeknum, SUM(coreos_hits), SUM(cloud_hits), SUM(server_hits) FROM (\n SELECT weeknum, 0 AS server_hits, 0 AS cloud_hits, SUM(hits) AS coreos_hits FROM countme_totals WHERE os_variant IS 'coreos' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age = 1 GROUP BY weeknum\n UNION\n SELECT weeknum, 0 AS server_hits, SUM(hits) AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'cloud' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age = 1 GROUP BY weeknum\n UNION\n SELECT weeknum, SUM(hits) AS server_hits, 0 AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'server' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' AND sys_age = 1 GROUP BY weeknum\n) WHERE date > '2022-01-01' GROUP BY weeknum", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "scatter", + "mode": "lines", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(cloud_hits)" + } + }, + "x": null, + "xsrc": "date", + "name": "Cloud", + "y": null, + "ysrc": "SUM(cloud_hits)", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(coreos_hits)" + } + }, + "x": null, + "xsrc": "date", + "y": null, + "ysrc": "SUM(coreos_hits)", + "name": "CoreOS", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "stackgroup": null, + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(server_hits)" + } + }, + "y": null, + "ysrc": "SUM(server_hits)", + "x": null, + "xsrc": "date", + "name": "Server", + "line": { + "width": 5 + } + } + ], + "layout": { + "xaxis": { + "range": [ + "2023-05-14 20:40:13.8728", + "2025-05-04" + ], + "autorange": false, + "type": "date", + "tickfont": { + "size": 22 + } + }, + "yaxis": { + "range": [ + -7660.833333333334, + 155839.15818339647 + ], + "autorange": false, + "type": "linear", + "tickfont": { + "size": 22 + } + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "title": { + "text": "Transient Node Count for Fedora Cloud/CoreOS/Server" + }, + "dragmode": "zoom", + "legend": { + "font": { + "size": 28 + }, + "orientation": "h", + "x": 0.388712745719721, + "y": 1.0030674846625767 + } + }, + "frames": [] + }, + "name": "Transient Node Count By Edition", + "createdAt": "2025-05-12T20:56:34.411Z" + }, + { + "id": "8C93FoFqg3Zpw4wcxyS2e", + "query": "SELECT date(julianday('1970-01-05')+weeknum*7 + 6) AS date, weeknum, SUM(coreos_hits), SUM(cloud_hits), SUM(server_hits) FROM (\n SELECT weeknum, 0 AS server_hits, 0 AS cloud_hits, SUM(hits) AS coreos_hits FROM countme_totals WHERE os_variant IS 'coreos' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' GROUP BY weeknum\n UNION\n SELECT weeknum, 0 AS server_hits, SUM(hits) AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'cloud' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' GROUP BY weeknum\n UNION\n SELECT weeknum, SUM(hits) AS server_hits, 0 AS cloud_hits, 0 AS coreos_hits FROM countme_totals WHERE os_variant IS 'server' AND repo_tag REGEXP 'updates-released-f[3-4][0-9]' GROUP BY weeknum\n) WHERE date > '2022-01-01' GROUP BY weeknum", + "viewType": "chart", + "viewOptions": { + "data": [ + { + "type": "scatter", + "mode": "lines", + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(cloud_hits)" + } + }, + "y": null, + "ysrc": "SUM(cloud_hits)", + "name": "Cloud", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "stackgroup": null, + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(coreos_hits)" + } + }, + "y": null, + "ysrc": "SUM(coreos_hits)", + "name": "CoreOS", + "line": { + "width": 5 + } + }, + { + "type": "scatter", + "mode": "lines", + "stackgroup": null, + "x": null, + "xsrc": "date", + "meta": { + "columnNames": { + "x": "date", + "y": "SUM(server_hits)" + } + }, + "y": null, + "ysrc": "SUM(server_hits)", + "name": "Server", + "line": { + "width": 5 + } + } + ], + "layout": { + "xaxis": { + "range": [ + "2023-05-17 21:05:12.1387", + "2025-05-04" + ], + "autorange": false, + "type": "date", + "tickfont": { + "size": 22 + } + }, + "yaxis": { + "range": [ + -10802.38888888889, + 229205.70535714284 + ], + "autorange": false, + "type": "linear", + "tickfont": { + "size": 22 + } + }, + "autosize": true, + "mapbox": { + "style": "open-street-map" + }, + "title": { + "text": "Total Node Count for Fedora Cloud/CoreOS/Server" + }, + "legend": { + "orientation": "h", + "x": 0.4578313253012048, + "y": 0.9863986313088109, + "font": { + "size": 28 + } + } + }, + "frames": [] + }, + "name": "Total Node Count for Fedora Cloud/CoreOS/Server", + "createdAt": "2025-05-12T20:56:53.783Z" + } + ] +} \ No newline at end of file diff --git a/stream-tooling.md b/stream-tooling.md index c0e1a06..f8455b8 100644 --- a/stream-tooling.md +++ b/stream-tooling.md @@ -13,8 +13,6 @@ FCOS will have multiple streams: | Development | next-devel | annex | | Mechanical | rawhide | annex | | Mechanical | branched | annex | -| Mechanical | bodhi-updates | annex | -| Mechanical | bodhi-updates-testing | annex | Development and mechanical streams are subject to change. @@ -32,8 +30,6 @@ We need a way to both (1) fix the content set for a particular stream release, a **Mechanical** streams are not curated; they're automated nightly snapshots of the underlying repos. They source their RPMs from the regular Fedora repos (using 30 here to mean `$currentrelease`): 1. **rawhide** <- f32 2. **branched** <- f31 when a branch exists, otherwise tracks **rawhide** -3. **bodhi-updates** <- f30-stable + f30-updates -4. **bodhi-updates-testing** <- f30-stable + f30-updates + f30-updates-testing **Production** streams are intended for production use. They source their RPMs from a _single_ Koji tag, `coreos-pool`, from which we create a yum repo: 1. **next** <- coreos-pool @@ -52,7 +48,7 @@ There is also a second Koji tag, `coreos-release`, for packages which have been We maintain a git repository containing the rpm-ostree treefile and lockfiles. This could be [fedora-coreos-config](https://github.com/coreos/fedora-coreos-config). We have one branch for each stream, and no main branch. -For the mechanical streams, a nightly job will run the compose from the corresponding yum repos and SCM refs. This job will output a lockfile for each CPU architecture. Those lockfiles will be committed to Git to preserve a record of the build's contents, and the builds will be pushed to the corresponding ostree refs. The {bodhi-updates, branched} lockfile will also be PR'd to the {testing-devel, next-devel} branch, the latter only during the part of the cycle where next-devel is maintained. We want to keep the development branches ready to release, so those PRs are not merged unless green. +For the mechanical streams, a nightly job will run the compose from the corresponding yum repos and SCM refs. This job will output a lockfile for each CPU architecture. Those lockfiles will be committed to Git to preserve a record of the build's contents, and the builds will be pushed to the corresponding ostree refs. The branched lockfile will also be PR'd to the {testing-devel, next-devel} branch, the latter only during the part of the cycle where next-devel is maintained. We want to keep the development branches ready to release, so those PRs are not merged unless green. The lockfiles produced from the automatic snapshot will never be hand-modified, and in the next/testing/stable branches will never be modified at all except during promotions. Instead, pins (to older NEVRAs) and updates (to newer ones) will be hand-maintained in the Git branches in a separate lockfile that overrides the autogenerated ones. These overrides will be the major distinction between the mechanical refs and the "curated" (development/production) refs. Each curated branch will have one override file, which can carry both CPU-architecture-independent and architecture-specific overrides. @@ -74,7 +70,7 @@ Update the development treefile as usual. On the next bot push, the lockfile wil To focus development effort, there will be one base treefile shared across all branches, whose canonical copy will live in the testing-devel branch. Changes will automatically be mirrored to next-devel and to the mechanical branches. To address divergence across Fedora releases, each branch will also have an overlay treefile (possibly empty): -- **testing-devel** -> automatically mirrored to bodhi-updates and bodhi-updates-testing +- **testing-devel** - **next-devel** -> automatically mirrored to branched - **rawhide**