diff --git a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md similarity index 98% rename from docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md rename to docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md index 1412770ed..ad98e419a 100644 --- a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md +++ b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md @@ -4,7 +4,7 @@ issue-type: task status: open priority: p1 github-issue: 1841 -spec-path: docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md +spec-path: docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md branch: "1841-1840-workflow-performance-baseline-analysis" related-pr: null last-updated-utc: 2026-05-28 00:00 @@ -15,7 +15,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md - contrib/dev-tools/workflow-benchmarks/run-container-baseline.sh - contrib/dev-tools/workflow-benchmarks/run-testing-baseline.sh - .github/skills/dev/planning/create-issue/SKILL.md diff --git a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md similarity index 99% rename from docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md rename to docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md index 26644f7c8..9e09bbb08 100644 --- a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md +++ b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md @@ -1,7 +1,7 @@ --- semantic-links: related-artifacts: - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md - .github/workflows/container.yaml - .github/workflows/testing.yaml - contrib/dev-tools/workflow-benchmarks/run-container-baseline.sh @@ -51,7 +51,7 @@ cargo build --timings --all-targets --release --workspace --all-features ``` Evidence logs are stored under -`docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/`. +`docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/`. ## Cache Reset Procedure (Cold Run) @@ -139,7 +139,7 @@ Notes: Source: `cargo build --timings --all-targets --release --workspace --all-features` run on 2026-05-28. Full HTML report: -`docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html` +`docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html` Total `cargo build` wall time reported by `--timings`: **188 s** (warm incremental, local machine). diff --git a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html similarity index 100% rename from docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html rename to docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/cargo-timing-release-20260528T074109Z.html diff --git a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/container-baseline-20260527T210123Z.log b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/container-baseline-20260527T210123Z.log similarity index 100% rename from docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/container-baseline-20260527T210123Z.log rename to docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/container-baseline-20260527T210123Z.log diff --git a/docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/testing-baseline-20260527T211129Z.log b/docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/testing-baseline-20260527T211129Z.log similarity index 100% rename from docs/issues/open/1841-1840-workflow-performance-baseline-analysis/evidence/testing-baseline-20260527T211129Z.log rename to docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/evidence/testing-baseline-20260527T211129Z.log diff --git a/docs/issues/open/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md b/docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md similarity index 94% rename from docs/issues/open/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md rename to docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md index 28fa08312..c930d7315 100644 --- a/docs/issues/open/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md +++ b/docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md @@ -4,7 +4,7 @@ issue-type: task status: open priority: p1 github-issue: 1853 -spec-path: docs/issues/open/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md +spec-path: docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md branch: "1853-containerfile-target-scope" related-pr: null last-updated-utc: 2026-06-02 00:00 @@ -16,7 +16,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md - docs/issues/open/1726-1840-workflow-performance-sccache/ISSUE.md --- @@ -118,7 +118,7 @@ Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`. | ID | Scenario | Command/Steps | Expected Result | Status | Evidence | | --- | ------------------------------ | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| M1 | Bottleneck confirmation | Use baseline report to compare phase timings and identify unneeded target build/link cost. | Decision to proceed is backed by measured data. | DONE | [benchmark-results-baseline.md](../../open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md): 27/30 top compile units unrelated to runtime image; benches+examples ~250-300s link cost per profile | +| M1 | Bottleneck confirmation | Use baseline report to compare phase timings and identify unneeded target build/link cost. | Decision to proceed is backed by measured data. | DONE | [benchmark-results-baseline.md](../1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md): 27/30 top compile units unrelated to runtime image; benches+examples ~250-300s link cost per profile | | M2 | Reduced-scope build validation | Build tracker image with narrowed Containerfile target scope. | Required executables are present and image build succeeds. | DONE | Cold build (`--no-cache`) succeeded in 5m23s; `docker run --rm torrust-tracker:1853-test ls /usr/bin/torrust-tracker /usr/bin/http_health_check` → both present; image size 173MB | | M3 | E2E compatibility check | Run Docker E2E flow against the reduced-scope image. | E2E tests pass with no functional regression. | TODO | {log/output/path} | | M4 | Performance comparison | Compare before/after container and testing workflow runtimes. | Improvement is measurable and documented. | TODO | {log/output/path} | @@ -127,7 +127,7 @@ Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`. | AC ID | Status (`TODO`/`DONE`) | Evidence | | ----- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| AC1 | DONE | [benchmark-results-baseline.md](../../open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md): 27/30 top compile units unrelated to runtime image | +| AC1 | DONE | [benchmark-results-baseline.md](../1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md): 27/30 top compile units unrelated to runtime image | | AC2 | DONE | Cold build in 5m23s; `/usr/bin/torrust-tracker` and `/usr/bin/http_health_check` confirmed in release image (173MB) | | AC3 | TODO | {workflow timing comparison} | | AC4 | TODO | {e2e results link} | diff --git a/docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md b/docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md index 70830b2fb..cce8c6bde 100644 --- a/docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md +++ b/docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md @@ -17,7 +17,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- @@ -35,7 +35,7 @@ reduce the dominant per-binary link time recorded in the baseline report. ### The baseline finding The baseline profiling report -(`docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md`) +(`docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md`) identified the build as **linker-dominated**: > "Individual crate compilation (frontend + codegen): ≤ 8 s per crate. diff --git a/docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md b/docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md index e4be0269b..75e8f410f 100644 --- a/docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md +++ b/docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md @@ -15,7 +15,7 @@ semantic-links: - Containerfile - .github/workflows/container.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md - docs/issues/open/1726-1840-workflow-performance-sccache/ISSUE.md --- diff --git a/docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md b/docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md index d388323b2..07a133e14 100644 --- a/docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md +++ b/docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md @@ -16,7 +16,7 @@ semantic-links: - .github/workflows/testing.yaml - Containerfile - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- diff --git a/docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md b/docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md index 98c9d5768..f3e7ceb7e 100644 --- a/docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md +++ b/docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md @@ -15,7 +15,7 @@ semantic-links: - Containerfile - .github/workflows/container.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- diff --git a/docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md b/docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md index 745d6928f..2985e563b 100644 --- a/docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md +++ b/docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md @@ -18,7 +18,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md - docs/issues/open/1669-overhaul-packages/EPIC.md --- diff --git a/docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md b/docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md index 545ba251c..d15e291e9 100644 --- a/docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md +++ b/docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md @@ -4,7 +4,7 @@ status: planned github-issue: 1840 spec-path: docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md epic-owner: josecelano -last-updated-utc: 2026-06-01 00:00 +last-updated-utc: 2026-06-03 00:00 semantic-links: skill-links: - create-issue @@ -66,20 +66,21 @@ Ordering policy: - Subissue 1 (baseline analysis) is mandatory first. - All later subissues are provisional and may be reordered based on baseline findings. -| Order | Issue | Local Spec | Status | Notes | -| ----- | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 1 | #1841 - Baseline workflow profiling and bottleneck analysis | `docs/issues/open/1841-1840-workflow-performance-baseline-analysis/ISSUE.md` | DONE | Merged in PR #1848. Baseline report at `docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md`. | -| 2 | #1852 - Restrict recipe stage to manifest-only COPY | `docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md` | TODO | Replace `COPY . /build/src` in the `recipe` stage with per-manifest COPY lines so the cook (dependency) layers are only invalidated when `Cargo.toml` or `Cargo.lock` changes, not on every `.rs` edit. High expected impact. | -| 3 | #1851 - Audit `.dockerignore` to minimize Docker build context | `docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md` | TODO | Systematically exclude tracked repo paths not needed in any Containerfile stage to reduce context transfer size and reduce spurious cache invalidation of `build` and `test` stages. | -| 4 | #1853 - Narrow Containerfile build targets to tracker image needs | `docs/issues/open/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md` | TODO | Execute only if baseline confirms significant time spent compiling or linking targets not required for the final tracker image. | -| 5 | #1726 - Reduce Build Times with `sccache` | `docs/issues/open/1726-1840-workflow-performance-sccache/ISSUE.md` | TODO | Existing GitHub issue; link it as a child issue after the EPIC is published. Order is provisional after baseline. | -| 6 | #1854 - Evaluate test execution policy in container image build | `docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md` | TODO | Assess whether test execution inside container build is redundant, evaluate separating validation from packaging across multiple artifact types, and define safer gating plus optional debug-image paths for failing commits. | -| 7 | #[To be assigned] - Improve dependency-layer cache reuse within each workflow | `docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md` | TODO | Ensure dependency layers are reused reliably inside each workflow when Cargo dependencies are unchanged. Defer optional cross-workflow cache-sharing and sequencing trade-offs to follow-up once this is working. | -| 8 | #[To be assigned] - Evaluate removing duplicate container build from container workflow | `docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md` | TODO | Assess whether PR-time container build in container workflow is redundant because testing workflow already builds an image for Docker E2E, and keep publish paths intact. | -| 9 | #[To be assigned] - Switch to a faster linker (mold or lld) to reduce link time | `docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md` | TODO | Baseline shows 35–117 s link time per binary (sections: null). Fair local relink: BFD = mold (54 s each) — compile dominates incremental builds. mold docs: 10–31× faster than BFD in cold builds (MySQL: 10.8 s → 0.46 s). 20+ binaries linked in container build. | -| 10 | #[To be assigned] - Investigate splitting cook layer to isolate external dependency cache (p4, deferred) | `docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md` | TODO | Low priority. C build scripts dominate cook time; workspace stub cost is near-zero. Revisit once other bottlenecks are resolved and workspace shrinks via EPIC #1669. | -| 11 | #[To be assigned] - Publish stable base stages as pre-built Docker Hub images (p3, deferred) | `docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md` | TODO | Low priority. Base stages (`chef`, `tester`, `gcc`) are fast (3–7 min cold). Compile dominates (35+ min). Revisit if base stages grow or if CI runner cold-cache frequency increases. | -| 12 | #[To be assigned] - Pass Cargo registry/git caches into BuildKit cook stages | `docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md` | TODO | Adds `--mount=type=cache` for registry/git to cook stages. Local benefit: saves ~7 s download per cook rebuild (cold fetch 6.9 s → warm 0.16 s; registry 823 MB). CI benefit: none with ephemeral GitHub Actions runners (`type=gha` layer cache does not persist cache mount volumes). Evaluate target-dir cache mount variant as T5. | +| Order | Issue | Local Spec | Status | Notes | +| ----- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 1 | #1841 - Baseline workflow profiling and bottleneck analysis | `docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/ISSUE.md` | DONE | Merged in PR #1848. Baseline report at `docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md`. | +| 2 | #1852 - Restrict recipe stage to manifest-only COPY | `docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md` | TODO | Replace `COPY . /build/src` in the `recipe` stage with per-manifest COPY lines so the cook (dependency) layers are only invalidated when `Cargo.toml` or `Cargo.lock` changes, not on every `.rs` edit. High expected impact. | +| 3 | #1851 - Audit `.dockerignore` to minimize Docker build context | `docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md` | TODO | Systematically exclude tracked repo paths not needed in any Containerfile stage to reduce context transfer size and reduce spurious cache invalidation of `build` and `test` stages. | +| 4 | #1853 - Narrow Containerfile build targets to tracker image needs | `docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md` | DONE | Merged in PR #1867. Removed `--benches --examples --all-targets` from all cargo commands. | +| 4.1 | #1868 - Exclude irrelevant workspace members from container build | `docs/issues/open/1868-1840-workflow-performance-exclude-irrelevant-workspace-members/ISSUE.md` | TODO | Follow-up to #1853. Post-merge CI analysis showed `workspace-coupling` and `torrust-tracker-torrent-repository-benchmarking` are still compiled despite being unneeded in the tracker image. Add `--exclude` flags to all cargo commands. Baseline: 19m03s build step after #1853. | +| 5 | #1726 - Reduce Build Times with `sccache` | `docs/issues/open/1726-1840-workflow-performance-sccache/ISSUE.md` | TODO | Existing GitHub issue; link it as a child issue after the EPIC is published. Order is provisional after baseline. | +| 6 | #1854 - Evaluate test execution policy in container image build | `docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md` | TODO | Assess whether test execution inside container build is redundant, evaluate separating validation from packaging across multiple artifact types, and define safer gating plus optional debug-image paths for failing commits. | +| 7 | #1869 - Improve dependency-layer cache reuse within each workflow | `docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md` | TODO | Ensure dependency layers are reused reliably inside each workflow when Cargo dependencies are unchanged. T3 must also evaluate whether the cargo-chef cook/build split delivers meaningful benefit given workspace-package churn. Defer optional cross-workflow cache-sharing and sequencing trade-offs to follow-up once this is working. | +| 8 | #[To be assigned] - Evaluate removing duplicate container build from container workflow | `docs/issues/drafts/1840-workflow-performance-container-workflow-build-deduplication/ISSUE.md` | TODO | Assess whether PR-time container build in container workflow is redundant because testing workflow already builds an image for Docker E2E, and keep publish paths intact. | +| 9 | #[To be assigned] - Switch to a faster linker (mold or lld) to reduce link time | `docs/issues/drafts/1840-workflow-performance-alternative-linker/ISSUE.md` | TODO | Baseline shows 35–117 s link time per binary (sections: null). Fair local relink: BFD = mold (54 s each) — compile dominates incremental builds. mold docs: 10–31× faster than BFD in cold builds (MySQL: 10.8 s → 0.46 s). 20+ binaries linked in container build. | +| 10 | #[To be assigned] - Investigate splitting cook layer to isolate external dependency cache (p4, deferred) | `docs/issues/drafts/1840-workflow-performance-split-external-dep-cache-layer/ISSUE.md` | TODO | Low priority. C build scripts dominate cook time; workspace stub cost is near-zero. Revisit once other bottlenecks are resolved and workspace shrinks via EPIC #1669. | +| 11 | #[To be assigned] - Publish stable base stages as pre-built Docker Hub images (p3, deferred) | `docs/issues/drafts/1840-workflow-performance-prebuilt-base-images/ISSUE.md` | TODO | Low priority. Base stages (`chef`, `tester`, `gcc`) are fast (3–7 min cold). Compile dominates (35+ min). Revisit if base stages grow or if CI runner cold-cache frequency increases. | +| 12 | #[To be assigned] - Pass Cargo registry/git caches into BuildKit cook stages | `docs/issues/drafts/1840-workflow-performance-buildkit-cargo-cache-mounts/ISSUE.md` | TODO | Adds `--mount=type=cache` for registry/git to cook stages. Local benefit: saves ~7 s download per cook rebuild (cold fetch 6.9 s → warm 0.16 s; registry 823 MB). CI benefit: none with ephemeral GitHub Actions runners (`type=gha` layer cache does not persist cache mount volumes). Evaluate target-dir cache mount variant as T5. | ## Delivery Strategy @@ -141,6 +142,8 @@ Append one line per meaningful update. - 2026-06-01 00:00 UTC - GitHub Copilot - Added sub-issue: buildkit-cargo-cache-mounts (p2, row 12); local benchmark: cold fetch 6.9 s → warm 0.16 s; CI limitation documented - 2026-06-01 00:00 UTC - GitHub Copilot - Promoted rows 2/3/4/6 from drafts to open: #1851 dockerignore-audit, #1852 recipe-manifest-only-copy, #1853 containerfile-target-scope, #1854 container-test-gating - 2026-06-01 00:00 UTC - GitHub Copilot - PR #1855 merged; all sub-issue specs (rows 2–12) are now in develop; renamed #1726 folder to match EPIC sub-issue naming convention +- 2026-06-03 00:00 UTC - GitHub Copilot - Marked #1853 DONE (merged PR #1867); added follow-up subissue #1868 (row 4.1) for `--exclude` fix based on post-merge CI analysis showing `workspace-coupling` still compiled (~840s gap, 19m03s total build step) +- 2026-06-03 00:00 UTC - GitHub Copilot - Created GitHub issue #1869 and promoted dependency-layer-cache-reuse spec to `docs/issues/open/` (row 7) ## Acceptance Criteria diff --git a/docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md b/docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md index e07c06d70..5dc04c3a6 100644 --- a/docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md +++ b/docs/issues/open/1851-1840-workflow-performance-dockerignore-audit/ISSUE.md @@ -17,7 +17,7 @@ semantic-links: - Containerfile - .github/workflows/container.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- diff --git a/docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md b/docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md index 2605e60c1..79ce4176e 100644 --- a/docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md +++ b/docs/issues/open/1852-1840-workflow-performance-recipe-stage-manifest-only-copy/ISSUE.md @@ -18,7 +18,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- diff --git a/docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md b/docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md index cab57d06c..1b4f10210 100644 --- a/docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md +++ b/docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md @@ -16,7 +16,7 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- diff --git a/docs/issues/open/1868-1840-workflow-performance-exclude-irrelevant-workspace-members/ISSUE.md b/docs/issues/open/1868-1840-workflow-performance-exclude-irrelevant-workspace-members/ISSUE.md new file mode 100644 index 000000000..3878d7f06 --- /dev/null +++ b/docs/issues/open/1868-1840-workflow-performance-exclude-irrelevant-workspace-members/ISSUE.md @@ -0,0 +1,175 @@ +--- +doc-type: issue +issue-type: task +status: open +priority: p1 +github-issue: 1868 +spec-path: docs/issues/open/1868-1840-workflow-performance-exclude-irrelevant-workspace-members/ISSUE.md +branch: "1868-1840-exclude-irrelevant-workspace-members" +related-pr: null +last-updated-utc: 2026-06-03 00:00 +semantic-links: + skill-links: + - create-issue + related-artifacts: + - Containerfile + - .github/workflows/container.yaml + - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md + - docs/issues/closed/1853-1840-workflow-performance-containerfile-target-scope/ISSUE.md + - docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md + - docs/issues/open/1854-1840-workflow-performance-container-test-gating/ISSUE.md +--- + + + +# Issue #1868 - Exclude irrelevant workspace members from container build + +## Goal + +Reduce container image build time by excluding workspace packages that are not needed to +produce or validate the tracker runtime image from all cargo commands in the Containerfile. + +## Background + +Issue [#1853](https://github.com/torrust/torrust-tracker/issues/1853) removed `--benches +--examples --all-targets` from all cargo commands. However, `--workspace` still causes two +workspace members that are unrelated to the tracker runtime to be compiled on every container +build: + +- `workspace-coupling` (`contrib/dev-tools/analysis/workspace-coupling`) — a local analysis + tool with unique dependencies (`regex`, `serde_json`) not shared by any other package. It + has no relationship to the tracker runtime image. +- `torrust-tracker-torrent-repository-benchmarking` — a benchmark harness with 17 inline + unit tests. It is not depended on by any other workspace member. + +### What the CI log revealed + +A recent CI run (after #1853 merged) showed the following in the `build-tracker-image` step: + +```text +#60 [dependencies 3/4] cargo chef cook --tests --workspace --all-features ... +#60 278.9 Compiling workspace-coupling v0.0.1 ... +#60 304.1 Finished in 5m 04s + +#61 [dependencies 4/4] cargo nextest archive ... (warmup) +#61 71.63 Finished in 1m 10s <- fast: stubs still in place + +#64 [build 3/3] cargo nextest archive --tests --workspace --all-features ... +#64 253.4 Compiling torrust-tracker v3.0.0-develop +#64 1094.3 Compiling workspace-coupling v3.0.0-develop <- 840s after tracker +#64 1144.0 Finished in 19m 03s +``` + +`workspace-coupling` is compiled twice: once in the cook stage (with stub source, ~5 min), +and again in the build stage (with real source, after an ~840s gap). The 840s gap is the +compilation cost of `workspace-coupling`'s unique transitive dependencies (`regex-automata`, +`regex-syntax`, and `serde_json` internals) from scratch — these were not pre-cooked because +the cook layer for `workspace-coupling` was built with stubs, and the real dep graph for +those crates is only triggered when the actual source is compiled. + +The total build step time was 19m03s; removing these two packages is expected to cut it +significantly. + +## Scope + +### In Scope + +- Add `--exclude workspace-coupling --exclude torrust-tracker-torrent-repository-benchmarking` + to all cargo commands in the Containerfile (`cargo chef cook` × 2, `cargo nextest archive` + × 4). +- Determine whether `cargo chef prepare` should also receive `--exclude` flags, and if so, + remove the corresponding `COPY`/stub lines from the recipe stage. +- Measure the impact on CI build time with evidence from a full CI run after the change. + +### Out of Scope + +- Removing tests from the container build (tracked in #1854). +- Implementing cross-workflow cache sharing (tracked in #1869). +- Changing which packages are part of the workspace `[members]` list. +- Broad Containerfile restructuring unrelated to the exclusion change. + +## Implementation Plan + +Status values: `TODO`, `IN_PROGRESS`, `BLOCKED`, `DONE`. + +| ID | Status | Task | Notes / Expected Output | +| --- | ------ | --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| T1 | TODO | Verify `cargo nextest archive` supports `--exclude` | Confirm the flag works for cargo-nextest; check docs and local test. | +| T2 | TODO | Add `--exclude` flags to all Containerfile cargo commands | Six commands: `cargo chef cook` × 2 and `cargo nextest archive` × 4. Both excluded packages named explicitly. | +| T3 | TODO | Decide on `cargo chef prepare` exclusion | Determine if `--exclude` is supported/needed for `cargo chef prepare`. If yes, remove the COPY/stub entries for the two packages from the recipe stage. If no, document why the COPY/stub lines must stay. | +| T4 | TODO | Run full CI build and record timing evidence | CI log showing build time after exclusion. Compare against pre-fix baseline (19m03s build step, 38m total). | + +## Progress Tracking + +### Workflow Checkpoints + +- [x] Spec drafted in `docs/issues/drafts/` +- [x] Spec reviewed and approved by user/maintainer +- [x] GitHub issue created and issue number added to this spec +- [ ] (Optional, recommended for complex issues) Spec-only PR merged into `develop` before implementation +- [ ] Implementation completed +- [ ] Automatic verification completed (`linter all`, relevant tests, and any pre-push checks) +- [ ] Manual verification scenarios executed and recorded (status + evidence) +- [ ] Acceptance criteria reviewed after implementation and updated with evidence +- [ ] Reviewer validated acceptance criteria and updated checkboxes +- [ ] Committer verified spec progress is up to date before commit +- [ ] Issue closed and spec moved from `docs/issues/open/` to `docs/issues/closed/` + +### Progress Log + +- 2026-06-03 00:00 UTC - GitHub Copilot - Drafted issue spec based on post-merge CI analysis of #1853 - draft file created +- 2026-06-03 00:00 UTC - GitHub Copilot - Narrowed scope to `--exclude` fix only; layer split analysis moved to dependency-layer-cache-reuse draft - draft updated + +## Acceptance Criteria + +- [ ] AC1: `workspace-coupling` and `torrust-tracker-torrent-repository-benchmarking` do not appear in the container build compilation output. +- [ ] AC2: The final `cargo nextest archive` step in the CI build completes in measurably less time than the 19m03s baseline recorded after #1853. +- [ ] AC3: The tracker runtime image is produced correctly and all unit tests still pass inside the container build. +- [ ] AC4: The decision on `cargo chef prepare` exclusion is documented with rationale (either in the Containerfile comments or in this spec). +- [ ] `linter all` exits with code `0` +- [ ] Relevant tests pass +- [ ] Manual verification scenarios are executed and documented (status + evidence) +- [ ] Acceptance criteria are re-reviewed after implementation and reflect actual behavior +- [ ] Documentation is updated when behavior/workflow changes + +## Verification Plan + +Define verification before implementation starts and execute it before closing the issue. + +### Automatic Checks + +- `linter all` +- Pre-push checks pass for changed Containerfile and spec files + +### Manual Verification Scenarios + +Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`. + +| ID | Scenario | Command/Steps | Expected Result | Status | Evidence | +| --- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | ------ | ---------------------- | +| M1 | Confirm excluded packages absent from build output | Run CI or `docker build --target release` locally; grep build log for `workspace-coupling` and `torrust-tracker-torrent-repository-benchmarking` | Neither package name appears in compilation output | TODO | {CI log link} | +| M2 | Measure build step timing improvement | Compare CI log for `[build 3/3] cargo nextest archive` step before and after change | Step completes in significantly less than 19m03s | TODO | {CI run link + timing} | +| M3 | Verify runtime image correctness | Build release image locally; run `docker run --rm torrust-tracker --version` or equivalent health-check | Image starts correctly; expected binaries present | TODO | {command output} | +| M4 | Verify tests still pass inside container | Review CI test stage output; confirm no test regressions | All unit tests pass in the container `test` stage | TODO | {CI log link} | + +### Acceptance Verification + +| AC ID | Status (`TODO`/`DONE`) | Evidence | +| ----- | ---------------------- | ------------------------------------ | +| AC1 | TODO | {CI log link} | +| AC2 | TODO | {timing comparison} | +| AC3 | TODO | {CI test stage log} | +| AC4 | TODO | {Containerfile comment or spec note} | + +## Risks and Trade-offs + +- Risk: `cargo nextest archive` may not support `--exclude` in the same way as `cargo build`. Mitigation: T1 verifies support before implementation. +- Risk: Excluding packages from `cargo chef prepare` may require additional changes to keep `cargo metadata` happy (recipe.json must still be valid). Mitigation: test locally with `cargo chef prepare --exclude ...` before removing COPY/stub lines. +- Risk: The change may interact with #1854 (test gating). If tests are later removed from the container build, the `--exclude` optimization becomes less relevant but is still correct. Mitigation: implement independently; document the relationship in the spec. + +## References + +- Related issues: #1840 (EPIC), #1853, #1854 +- Related drafts: `docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md` +- Related PRs: #1867 (merged, implemented #1853) +- Related ADRs: none diff --git a/docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md b/docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md similarity index 66% rename from docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md rename to docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md index 32a88fc85..72225156d 100644 --- a/docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md +++ b/docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md @@ -1,13 +1,13 @@ --- doc-type: issue issue-type: task -status: draft +status: open priority: p1 -github-issue: null -spec-path: docs/issues/drafts/1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md +github-issue: 1869 +spec-path: docs/issues/open/1869-1840-workflow-performance-dependency-layer-cache-reuse/ISSUE.md branch: "{issue-number}-dependency-layer-cache-reuse" related-pr: null -last-updated-utc: 2026-05-27 00:00 +last-updated-utc: 2026-06-03 00:00 semantic-links: skill-links: - create-issue @@ -16,12 +16,12 @@ semantic-links: - .github/workflows/container.yaml - .github/workflows/testing.yaml - docs/issues/open/1840-improve-pr-workflow-performance-epic/EPIC.md - - docs/issues/open/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md + - docs/issues/closed/1841-1840-workflow-performance-baseline-analysis/benchmark-results-baseline.md --- -# Issue #[To be assigned] - Improve dependency-layer cache reuse within each workflow +# Issue #1869 - Improve dependency-layer cache reuse within each workflow ## Goal @@ -35,6 +35,8 @@ Current workflows use isolated cache scopes to avoid conflicts and race conditio This issue should determine whether current cache misses are caused by layer invalidation inputs, cache configuration, or both, and then propose a safe strategy to improve reuse within workflow boundaries. +A further concern emerged from post-#1853 CI analysis: in this repository, most logic lives in in-repo workspace packages (not external crates), and those packages change on nearly every PR. The `cargo-chef` cook stage can only pre-compile external dependencies; workspace members must always be compiled from source in the build stage. This raises the question of whether the cook/build split provides meaningful cache benefit at all given this churn pattern, or whether an alternative scoping strategy — for example, limiting the cook stage to external-only packages via `--package` selectors — would be more effective. This issue must include that evaluation as part of T3. + ## Scope ### In Scope @@ -42,6 +44,7 @@ This issue should determine whether current cache misses are caused by layer inv - Measure dependency-layer cache hit and miss behavior for unchanged dependency inputs. - Identify invalidation triggers for dependency stages in the Containerfile and workflow build configuration. - Preserve current workflow concurrency while improving cache effectiveness. +- Evaluate whether the current `cargo-chef` cook/build split strategy delivers meaningful cache benefit given typical PR churn on workspace packages, and document findings with evidence. If the split is not effective, propose an alternative (for example, scoping the cook stage to external-only packages via `--package` selectors, or eliminating the split in favour of a single build step). - Propose a practical cache policy and expected impact. - Prepare follow-up scope for optional cross-workflow cache reuse only after in-workflow behavior is reliable. @@ -57,21 +60,21 @@ This issue should determine whether current cache misses are caused by layer inv Status values: `TODO`, `IN_PROGRESS`, `BLOCKED`, `DONE`. -| ID | Status | Task | Notes / Expected Output | -| --- | ------ | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -| T1 | TODO | Reproduce current cache behavior | Demonstrate dependency-layer misses when dependencies are unchanged and only app code differs. | -| T2 | TODO | Identify invalidation inputs | Document which files, build args, or stage structure invalidate dependency layers. | -| T3 | TODO | Propose in-workflow reuse strategy | Recommendation for container and testing workflows independently, keeping current cache-scope isolation and concurrency. | -| T4 | TODO | Validate impact on PR wait time | Before/after evidence for dependency-stage reuse and effect on end-to-end check completion time. | -| T5 | TODO | Draft follow-up scope | Outline a separate follow-up issue for optional cross-workflow cache reuse, including race and sequencing trade-offs. | +| ID | Status | Task | Notes / Expected Output | +| --- | ------ | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| T1 | TODO | Reproduce current cache behavior | Demonstrate dependency-layer misses when dependencies are unchanged and only app code differs. | +| T2 | TODO | Identify invalidation inputs | Document which files, build args, or stage structure invalidate dependency layers. | +| T3 | TODO | Propose in-workflow reuse strategy | Recommendation for container and testing workflows independently, keeping current cache-scope isolation and concurrency. The strategy must also assess whether the `cargo-chef` cook/build split is appropriate given workspace-package churn: if the split provides little reuse benefit, propose an alternative (for example, scoping cook to external-only packages or eliminating the split). | +| T4 | TODO | Validate impact on PR wait time | Before/after evidence for dependency-stage reuse and effect on end-to-end check completion time. | +| T5 | TODO | Draft follow-up scope | Outline a separate follow-up issue for optional cross-workflow cache reuse, including race and sequencing trade-offs. | ## Progress Tracking ### Workflow Checkpoints - [ ] Spec drafted in `docs/issues/drafts/` -- [ ] Spec reviewed and approved by user/maintainer -- [ ] GitHub issue created and issue number added to this spec +- [x] Spec reviewed and approved by user/maintainer +- [x] GitHub issue created and issue number added to this spec - [ ] (Optional, recommended for complex issues) Spec-only PR merged into `develop` before implementation - [ ] Implementation completed - [ ] Automatic verification completed (`linter all`, relevant tests, and any pre-push checks) @@ -87,6 +90,8 @@ Append one line per meaningful update. - 2026-05-27 00:00 UTC - GitHub Copilot - Drafted dependency-layer cache reuse issue from EPIC discussion - draft file created - 2026-05-27 00:00 UTC - GitHub Copilot - Refocused this issue on in-workflow cache reuse first and moved cross-workflow sharing to follow-up scope - draft updated +- 2026-06-03 00:00 UTC - GitHub Copilot - Added workspace-churn angle: T3 now requires evaluating whether the cook/build split itself is effective, not only whether cache config is correct - draft updated +- 2026-06-03 00:00 UTC - GitHub Copilot - Created GitHub issue #1869 and promoted spec to `docs/issues/open/` ## Acceptance Criteria @@ -139,6 +144,7 @@ Status values: `TODO`, `IN_PROGRESS`, `DONE`, `FAILED`, `BLOCKED`. - Risk: reducing per-workflow runtime may still not improve total wait time if critical-path behavior is ignored. Mitigation: measure and optimize end-to-end wait until all required checks complete. - Risk: forcing sequential workflows for cache reuse can increase total wait time despite lower compute usage. Mitigation: keep this issue focused on in-workflow reuse and evaluate sequential orchestration only in follow-up. - Risk: measured gains may be lower than expected if invalidation is driven by unavoidable inputs. Mitigation: validate root causes before implementation. +- Risk: even with correct cache configuration, workspace-package churn on most PRs may mean the cook stage provides little reuse benefit, making the overall optimization marginal. Mitigation: T3 explicitly evaluates this and proposes an alternative strategy if the current split is not effective. ## References