-
Notifications
You must be signed in to change notification settings - Fork 799
feat: Celery task to generate I-D BibXML #10974
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: feat/bibgen
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -81,6 +81,25 @@ def get_fyi_bibxml(fyi_number): | |||||||||
| return f"""<referencegroup anchor="FYI{fyi_number}" target="{fyi_link}">{rfc_bibxml}</referencegroup>""" | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def get_id_bibxml(draft_name, doc): | ||||||||||
| """Return BibXML entry for the given I-D doc""" | ||||||||||
| name = "-".join(draft_name.split("-", 2)[1:]) | ||||||||||
| date = "" | ||||||||||
| if doc.is_dochistory(): | ||||||||||
| latest_event = doc.latest_event(type="new_revision", rev=doc.rev) | ||||||||||
| if latest_event: | ||||||||||
| doc.pub_date = latest_event.time | ||||||||||
| date = doc.pub_date.strftime('<date day="%-d" month="%B" year="%Y"/>') | ||||||||||
|
Comment on lines
+91
to
+92
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I recommend against pseudo-shadowing I think it's just a temp variable and you don't actually need to attach it to the
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, happy to improve. I was using the same logic as in datatracker/ietf/doc/views_doc.py Lines 1292 to 1295 in 1cccb08
IMO there should be easier way to get the revisions.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suspect it's done there for use in a template (where |
||||||||||
| else: | ||||||||||
| date = doc.pub_date().strftime('<date day="%-d" month="%B" year="%Y"/>') | ||||||||||
| link = f"https://datatracker.ietf.org/doc/html/{draft_name}-{doc.rev}" | ||||||||||
| authors = "" | ||||||||||
| for author in doc.author_persons_or_names(): | ||||||||||
| authors += f"""<author fullname={qa(author.person.name)} />""" | ||||||||||
|
|
||||||||||
| return f"""<reference anchor="I-D.{name}" target="{link}"><front><title>{doc.title}</title>{date}{authors}<abstract><t>{doc.abstract}</t></abstract></front><seriesInfo name="Internet-Draft" value="{draft_name}-{doc.rev}"/></reference>""" | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def save_bibxml(bibxml, filename): | ||||||||||
| """Prettify and save given BibXML""" | ||||||||||
|
|
||||||||||
|
|
@@ -156,3 +175,31 @@ def recreate_rfcsubseries_bibxml(): | |||||||||
| filename = f"bibxml-rfcsubseries/fyi{fyi_number}.xml" | ||||||||||
| bibxml = get_fyi_bibxml(fyi_number) | ||||||||||
| save_bibxml(bibxml, filename) | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def recreate_id_bibxml_by_draft_name(draft_name): | ||||||||||
| """Creates BibXML for given draft_name.""" | ||||||||||
| doc = Document.objects.get(name=draft_name) | ||||||||||
| name = "-".join(draft_name.split("-", 2)[1:]) | ||||||||||
|
|
||||||||||
| # revision less BibXML | ||||||||||
| bibxml = get_id_bibxml(draft_name, doc) | ||||||||||
| filename = f"bibxml-ids/reference.I-D.{name}.xml" | ||||||||||
| save_bibxml(bibxml, filename) | ||||||||||
|
|
||||||||||
| # draft BibXML for each revision | ||||||||||
| for revision in reversed(doc.revisions_by_newrevisionevent()): | ||||||||||
| doc_rev = doc.history_set.order_by("-time").filter(rev=revision).first() | ||||||||||
| bibxml = get_id_bibxml(draft_name, doc_rev) | ||||||||||
| filename = f"bibxml-ids/reference.I-D.{draft_name}-{revision}.xml" | ||||||||||
| save_bibxml(bibxml, filename) | ||||||||||
|
|
||||||||||
|
|
||||||||||
| def recreate_id_bibxml(): | ||||||||||
| """Creates BibXML for all Internet Drafts.""" | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we probably want to have a way to throttle this, run on subsets, etc. There are many drafts.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would imagine this to be a one time thing. So kind of hoping we run this manually as required because it's resource consuming.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, and that resource consumption is actually my worry. This is going to tie up a celery worker for the entire run, and if it fails part way through we have to start over. Do you have a sense for how long it takes? If it's five minutes, then fine. If it's hours, then we want to be prepared to manage it in more detail |
||||||||||
| for draft_name in ( | ||||||||||
| Document.objects.filter(type_id="draft") | ||||||||||
| .values_list("name", flat=True) | ||||||||||
| .order_by("-time") | ||||||||||
| ): | ||||||||||
| recreate_id_bibxml_by_draft_name(draft_name) | ||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should implement
DocHistory.pub_date()so that you don't have to branch hereThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that'll make things easier. But this is the sort of same logic that has been used elsewhere in DT get revisions.