fix(scheduler): only pause a tracker on credential failures - #9
Merged
Conversation
A failed poll leaves lastPolledAt untouched, so a failing tracker stays permanently overdue and is retried on every 5-minute scheduler tick. Four ticks is 20 minutes, so any outage longer than that auto-paused the tracker for good, and a paused tracker only resumes when a human clicks Resume. On 2026-08-16 a home internet outage paused all six trackers at once. The container stayed up, so container-level health stayed green and the fault went unseen for 33.5 hours. The MyAnonaMouse balance hit its 99,999 cap in that window and burned ~5,000 points (~10 GiB of upload credit). Invert the default: only Authentication failed / Session expired / Invalid credentials can pause a tracker. Everything else keeps its failure count for visibility but retries forever under exponential backoff (5m, 10m, 20m, 40m, then hourly), so connectivity faults heal on their own. Rate-limit and IP-ban errors go straight to the hourly cap, since retrying hard is what causes them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Knip Code AnalysisFound 9 total issues
View details
Use |
|
🚫 Security audit failed Critical Failures
Warnings
Passed (36/38)
Summary: 36/38 checks passed See |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The incident
On 2026-08-16 a home internet outage auto-paused all six trackers at once. Nothing resumed them. The container stayed up throughout, so the container-level health check stayed green and the fault went unnoticed for 33.5 hours.
In that window the MyAnonaMouse balance hit its 99,999 point cap, where accrual stops dead, and burned roughly 5,000 points — about 10 GiB of upload credit.
Why 20 minutes of downtime was enough
Two details compound:
lastPolledAtuntouched, so a failing tracker is permanently "overdue" and gets retried on every 5-minute scheduler tick, not on the hourly poll interval.POLL_FAILURE_THRESHOLDis 4. Four ticks is 20 minutes.So any outage longer than 20 minutes permanently disabled monitoring, and only a human clicking Resume could bring it back.
The change
New
src/lib/poll-failure-policy.tsinverts the default. Only a failure a human must actually fix can pause a tracker:Authentication failed,Session expired,Invalid credentialsPoll failedBackoff is 5m, 10m, 20m, 40m, then hourly. Failure counts still increment, so the UI still shows the fault.
Notes
sanitizeNetworkError, so it matches a small fixed set of phrases rather than driver-specific text. The fallback"Poll failed"is deliberately transient — that is what the real outage produced, and treating an unknown error as permanent is what caused the incident.pausedAtis a column self-assign (paused_at = paused_at), never a literal that could clobber a genuine pause. The existing test asserting this is updated rather than dropped.pollAllTrackersoverdue filter, keyed offlastErrorAt. Without it, removing the pause would mean retrying every 5 minutes forever.Tests
21 new tests in
poll-failure-policy.test.ts, including every error phrase the real outage produced. Full suite: 2932 passing. The one failure inchart-transforms.test.tsis pre-existing onmainand unrelated, verified by stashing.🤖 Generated with Claude Code