feat: update rfc json - #10951
Merged
Merged
Conversation
rjsparks
marked this pull request as draft
May 29, 2026 21:05
jennifer-richards
left a comment
jennifer-richards
left a comment
Member
There was a problem hiding this comment.
A few comments inline. Generally looks good to me.
A high-level thing (touched on by the comment about rfc_number not in pub_levels) is that can fail semi-silently if there's a problem generating one of the json files. There are at least a couple probably transient situations that could cause this (the pub_levels issue, or a temporary failure accessing the red bucket are the ones I can think of). It might be worth retrying in those cases so we don't have to count on noticing the logged exception.
Comment on lines
+82
to
+99
| if stream_slug == "ise": | ||
| source = "INDEPENDENT" | ||
| elif stream_slug == "iab": | ||
| source = "IAB" | ||
| elif stream_slug == "ietf" and ( | ||
| group_acronym in ("none", "gen") or not area_acronym | ||
| ): | ||
| source = "IETF - NON WORKING GROUP" | ||
| elif group_acronym not in ("none", ""): | ||
| source = group_acronym | ||
| if stream_slug == "ietf" and area_acronym: | ||
| source += f" ({area_acronym})" | ||
| elif stream_slug: | ||
| source += f" ({stream_slug})" | ||
| elif stream_slug: | ||
| source = "Legacy" if stream_slug == "legacy" else stream_slug.upper() | ||
| else: | ||
| source = "" |
Member
There was a problem hiding this comment.
This differs substantially from the json production at purple. I believe this code is correct and purple's is incorrect for some non-IETF WG documents. We could fix purple to match, but long-term I think we should make datatracker responsible for producing these json files on initial publication as well as update. (The important things being that it's the same code generating the original and updated contents, and that it's the same place that the RFC index is generated)
(In the short term it may be easier to fix purple rather than adjust the plumbing for document publication.)
@rudimatz fyi
jennifer-richards
left a comment
jennifer-richards
left a comment
Member
There was a problem hiding this comment.
A couple typos - but this looks good
rjsparks
marked this pull request as ready for review
June 2, 2026 14:00
jennifer-richards
previously approved these changes
Jun 2, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #10951 +/- ##
=======================================
Coverage 88.51% 88.51%
=======================================
Files 332 333 +1
Lines 44970 45065 +95
=======================================
+ Hits 39805 39890 +85
- Misses 5165 5175 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
jennifer-richards
approved these changes
Jun 2, 2026
kesara
added a commit
that referenced
this pull request
Jun 10, 2026
* ci: numeric settings -> int for searchindex cfg (#10904) * fix: remove no longer needed notification to the rfc-editor (#10911) * fix: meeting_stats -> 404 for nonexistent meeting (#10896) * fix: meeting_stats->404 for nonexistent number * test: add test case * chore: fix lint * test: fix new test * fix: set searchindex hiddenDefault flag (#10916) * fix: set searchindex hiddenDefault flag * chore: adjust docstring comments * ci: pin xym until we can adapt to new required arguments (#10939) * ci: update base image target version to 20260527T1529 * fix: adjust searchindex abstract sanitiziation (#10950) * fix: adjust searchindex abstract sanitiziation * test: leading/trailing whitespace to be stripped * fix: strip each line of the abstract * chore: add items to tracked yarn cache (#10964) * chore: configurable I-D submit timeout for k8s (#10971) * feat: update rfc json (#10951) * feat: update rfc json * fix: resilience around fetch of publication status * fix: guards against malformed RFC document objects * chore: fix log msg typos * test: mock more async calls * chore: nginx -> 1.30 (#10978) * chore: "nginxinc" GH org is now "nginx" (#10981) * chore: use redis for django caches (#10975) * chore: drop memcache, add redis (WIP) (#10940) * chore(dev): use redis for dev sessions Adds a redis container with a persistent volume and uses this for sessions instead of the database. Results in the same general behavior as before - logins should be persistent until the docker compose stack is torn down. * chore(deps): install redis/hiredis python pkgs * chore: remove memcached (mostly) Still keeps some config references and a custom cache backend * chore: switch to django-redis for sentinel support * chore: production redis cache config Uses sentinel. Untested. * chore: telemetry: add redis, drop memcache * chore: remove now-unused cache.py * chore: adjust imports in gunicorn.conf.py * chore: adjust redis sentinel settings * feat: size limit on redis values * style: ruff ruff * chore(dev): SizeLimitingRedisClient for dev Does not make a functional difference in most cases, but might as well exercise the class. * test: use db-backed sessions for tests * chore: typing nit * ci: remove DEPLOY_STRATEGY "strategery" * ci: remove strategy from dt manifests altogether * ci: update base image target version to 20260604T1454 * ci: remove guard against missing CACHES in prod (#10986) * chore: revert switch to redis (#10991) * ci: update base image target version to 20260605T2314 * feat: add group type to search index (#11001) * feat: self-serve queries for inputs to reporting and survey purposes (#10976) * feat: utilities to count I-D submitters and authors by year * chore: fix comment typo * fix: return querysets and sets rather than lists. Improve docstrings. * refactor: don't flatten sets to lists * fix: filter to posted submissions and produce report * chore: black * chore: boilerplate * fix: re-re-re fix the ^M problem in the issue template * feat: self-serve queries for inputs to reporting and survey purposes * Update ietf/utils/tests_reports.py Co-authored-by: Jennifer Richards <jeni@borkbork.org> --------- Co-authored-by: Jennifer Richards <jeni@borkbork.org> * ci: remove strategy from dt manifests (#11004) * fix: tweak logging settings to avoid spurious keyerrors (#11009) * fix: improved source and keywords for rfc json (#11008) * test: make check for April 1 resilient across line-breaks (#11006) * test: make check for April 1 resilient across line-breaks * test: be resilient against faker creating a 63 character long title * chore(dev): fix nginx config (#11014) keepalive 0 is not supported by the version of nginx in the dev container --------- Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org> Co-authored-by: Robert Sparks <rjsparks@nostrum.com> Co-authored-by: jennifer-richards <19472766+jennifer-richards@users.noreply.github.com> Co-authored-by: Jennifer Richards <jeni@borkbork.org>
This was referenced Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RFC JSON files (
json/rfc{N}.json) live in the"rfc"blob bucket. Purple generates theinitial file on publication and uploads it via
RfcPubFilesView. After that, nothing updatesthem. Datatracker must regenerate them — by combining data from its own DB (which purple keeps
current) and
errata.json(produced by the errata system) — whenever any of three triggers fire.