Skip to content

fix(scheduler): only pause a tracker on credential failures - #9

Merged
patrickdundas merged 1 commit into
mainfrom
fix/transient-poll-failures-no-pause
Aug 17, 2026
Merged

fix(scheduler): only pause a tracker on credential failures#9
patrickdundas merged 1 commit into
mainfrom
fix/transient-poll-failures-no-pause

Conversation

@patrickdundas

Copy link
Copy Markdown
Owner

The incident

On 2026-08-16 a home internet outage auto-paused all six trackers at once. Nothing resumed them. The container stayed up throughout, so the container-level health check stayed green and the fault went unnoticed for 33.5 hours.

In that window the MyAnonaMouse balance hit its 99,999 point cap, where accrual stops dead, and burned roughly 5,000 points — about 10 GiB of upload credit.

Why 20 minutes of downtime was enough

Two details compound:

  • A failed poll leaves lastPolledAt untouched, so a failing tracker is permanently "overdue" and gets retried on every 5-minute scheduler tick, not on the hourly poll interval.
  • POLL_FAILURE_THRESHOLD is 4. Four ticks is 20 minutes.

So any outage longer than 20 minutes permanently disabled monitoring, and only a human clicking Resume could bring it back.

The change

New src/lib/poll-failure-policy.ts inverts the default. Only a failure a human must actually fix can pause a tracker:

Failure Behaviour
Authentication failed, Session expired, Invalid credentials pause at threshold, as before
everything else, including the unclassified Poll failed never pauses; retries forever under backoff
rate limit / IP ban never pauses; jumps straight to the hourly cap

Backoff is 5m, 10m, 20m, 40m, then hourly. Failure counts still increment, so the UI still shows the fault.

Notes

  • Classification runs on the output of sanitizeNetworkError, so it matches a small fixed set of phrases rather than driver-specific text. The fallback "Poll failed" is deliberately transient — that is what the real outage produced, and treating an unknown error as permanent is what caused the incident.
  • On the transient path pausedAt is a column self-assign (paused_at = paused_at), never a literal that could clobber a genuine pause. The existing test asserting this is updated rather than dropped.
  • The backoff gate lives in the pollAllTrackers overdue filter, keyed off lastErrorAt. Without it, removing the pause would mean retrying every 5 minutes forever.

Tests

21 new tests in poll-failure-policy.test.ts, including every error phrase the real outage produced. Full suite: 2932 passing. The one failure in chart-transforms.test.ts is pre-existing on main and unrelated, verified by stashing.

🤖 Generated with Claude Code

A failed poll leaves lastPolledAt untouched, so a failing tracker stays
permanently overdue and is retried on every 5-minute scheduler tick. Four
ticks is 20 minutes, so any outage longer than that auto-paused the tracker
for good, and a paused tracker only resumes when a human clicks Resume.

On 2026-08-16 a home internet outage paused all six trackers at once. The
container stayed up, so container-level health stayed green and the fault
went unseen for 33.5 hours. The MyAnonaMouse balance hit its 99,999 cap in
that window and burned ~5,000 points (~10 GiB of upload credit).

Invert the default: only Authentication failed / Session expired / Invalid
credentials can pause a tracker. Everything else keeps its failure count for
visibility but retries forever under exponential backoff (5m, 10m, 20m, 40m,
then hourly), so connectivity faults heal on their own. Rate-limit and IP-ban
errors go straight to the hourly cap, since retrying hard is what causes them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@patrickdundas
patrickdundas merged commit bfe0ac9 into main Aug 17, 2026
6 checks passed
@patrickdundas
patrickdundas deleted the fix/transient-poll-failures-no-pause branch August 17, 2026 18:50
@github-actions

Copy link
Copy Markdown

Knip Code Analysis

Found 9 total issues

Category Count
Unused Dependencies 1
Unused Dev Dependencies 3
Unused Exports 5
View details

Run pnpm knip locally to see the full report.

Use pnpm knip:filter pattern to filter results by file path.


Use /** @public */ JSDoc tags to mark intentionally exported symbols.

@github-actions

Copy link
Copy Markdown

🚫 Security audit failed

Critical Failures

Check File Details
External fetch calls have timeouts src/lib/adapters/btn.ts:69 fetch() call without a timeout signal — add signal: AbortSignal.timeout(ms)

Warnings

Check File Details
No swallowed errors in catch blocks src/lib/adapters/iptorrents.ts:108 catch block swallows error without logging or re-throwing
No swallowed errors in catch blocks src/lib/adapters/iptorrents.ts:114 catch block swallows error without logging or re-throwing
No swallowed errors in catch blocks src/lib/adapters/iptorrents.ts:175 catch block swallows error without logging or re-throwing
No swallowed errors in catch blocks src/lib/adapters/iptorrents.ts:181 catch block swallows error without logging or re-throwing

Passed (36/38)

  • ✅ Auth enforcement on protected routes (per-handler)
  • ✅ No dangerous functions (eval, innerHTML, etc.)
  • ✅ No hardcoded secrets in source
  • ✅ Security headers in next.config.ts
  • ✅ Cookie security (httpOnly, sameSite, secure)
  • ✅ No sensitive fields in API responses
  • ✅ No .env files committed to repo
  • ✅ No raw SQL in API routes
  • ✅ No fetch/redirect with unvalidated URLs in routes
  • ✅ Timing-safe comparison for secret values
  • ✅ No raw SQL migration files (schema-first only)
  • ✅ Docker container runs as non-root user
  • ✅ Public routes match proxy allowlist
  • ✅ File delete operations have path traversal defense
  • ✅ Password hashing uses Argon2 (not SHA-256/bcrypt)
  • ✅ Encrypted columns written via encrypt()
  • ✅ TOTP 2FA flow integrity
  • ✅ Emergency lockdown flow integrity
  • ✅ Scrub & delete (nuke) flow integrity
  • ✅ Backup restore flow integrity
  • ✅ Login flow integrity
  • ✅ Auth result checked before proceeding
  • ✅ Backup password inputs bounded before key derivation
  • ✅ Webhook delivery fetch uses redirect: "error"
  • ✅ SESSION_SECRET minimum-length guard in auth/crypto modules
  • ✅ Notification URL validators include SSRF protection
  • ✅ Dockerfile does not COPY sensitive files
  • ✅ No secret env vars in client components
  • ✅ Adapter Cookie headers guard against injection
  • ✅ Adapter files do not log credential values
  • ✅ No console.log in API routes
  • ✅ No TODO/FIXME in security-critical files
  • ✅ JSON.parse wrapped in try-catch
  • ✅ Request body size validation on upload routes
  • ✅ BigInt fields use string serialization
  • ✅ No raw error messages in API responses

Summary: 36/38 checks passed
1 critical failure(s)
— 1 warning(s)

See scripts/security-audit.ts for check definitions and SECURITY.md for the full security architecture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant