Skip to content

feat: use icalendar instead manual template#9187

Merged
rjsparks merged 12 commits intoietf-tools:mainfrom
rudimatz:feat/use-ical-package
Aug 7, 2025
Merged

feat: use icalendar instead manual template#9187
rjsparks merged 12 commits intoietf-tools:mainfrom
rudimatz:feat/use-ical-package

Conversation

@rudimatz
Copy link
Copy Markdown
Contributor

@rudimatz rudimatz commented Jul 19, 2025

fixes #5393 for route
/meeting/{meeting-number}/session/{session-id}.ics or any route that uses views.agenda_ical

creates .ics files using icalendar package instead of hand-made template

Uses simplyfied format using UTC, like

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//IETF//datatracker.ietf.org ical agenda//EN
METHOD:PUBLISH
BEGIN:VEVENT
SUMMARY:JOSE - Javascript Object Signing and Encryption
DTSTART:20250724T150000Z
DTEND:20250724T170000Z
DTSTAMP:20250620T140021Z
UID:ietf-123-19248-jose
CLASS:PUBLIC
DESCRIPTION:Session IV\n\nOnsite tool: https://meetings.conf.meetecho.com/
 onsite123/?session=34370\n\nMeetecho: https://meetings.conf.meetecho.com/i
 etf123/?session=34370\n\nWebex: None\n\nSession materials: https://datatra
 cker.ietf.org/meeting/123/session/jose\n\nSee in schedule: https://datatra
 cker.ietf.org/meeting/123/agenda#row-123-2025-07-24-thu-1500-sec-jose
LOCATION:Patio 3
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR

instead of

BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
PRODID:-//IETF//datatracker.ietf.org ical agenda//EN
BEGIN:VTIMEZONE
TZID:Europe/Madrid
X-LIC-LOCATION:Europe/Madrid
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19810329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19961027T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
BEGIN:STANDARD
TZOFFSETFROM:-001444
TZOFFSETTO:+0000
TZNAME:WET
DTSTART:19010101T000000
RDATE:19010101T000000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:WEST
DTSTART:19170505T230000
RDATE:19170505T230000
RDATE:19180415T230000
RDATE:19190405T230000
RDATE:19240416T230000
RDATE:19260417T230000
RDATE:19270409T230000
RDATE:19280414T230000
RDATE:19290420T230000
RDATE:19370522T230000
RDATE:19380322T230000
RDATE:19390415T230000
RDATE:19400316T230000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:WET
DTSTART:19171007T000000
RDATE:19171007T000000
RDATE:19181007T000000
RDATE:19191007T000000
RDATE:19241005T000000
RDATE:19261003T000000
RDATE:19271002T000000
RDATE:19281007T000000
RDATE:19291006T000000
RDATE:19371003T000000
RDATE:19381002T000000
RDATE:19391008T000000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:WEMT
DTSTART:19420502T230000
RDATE:19420502T230000
RDATE:19430417T230000
RDATE:19440415T230000
RDATE:19450414T230000
RDATE:19460413T230000
END:DAYLIGHT
BEGIN:DAYLIGHT
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:WEST
DTSTART:19420902T000000
RDATE:19420902T000000
RDATE:19431004T000000
RDATE:19441011T000000
RDATE:19450930T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19460930T000000
RDATE:19460930T000000
RDATE:19490930T010000
RDATE:19741006T010000
RDATE:19751005T010000
RDATE:19760926T010000
RDATE:19770925T010000
RDATE:19781001T010000
RDATE:19790930T030000
RDATE:19800928T030000
RDATE:19810927T030000
RDATE:19820926T030000
RDATE:19830925T030000
RDATE:19840930T030000
RDATE:19850929T030000
RDATE:19860928T030000
RDATE:19870927T030000
RDATE:19880925T030000
RDATE:19890924T030000
RDATE:19900930T030000
RDATE:19910929T030000
RDATE:19920927T030000
RDATE:19930926T030000
RDATE:19940925T030000
RDATE:19950924T030000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19490430T230000
RDATE:19490430T230000
RDATE:19740413T230000
RDATE:19750419T230000
RDATE:19760327T230000
RDATE:19770402T230000
RDATE:19780402T230000
RDATE:19790401T020000
RDATE:19800406T020000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19790101T000000
RDATE:19790101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:ietf-123-19070-detnet
SUMMARY:detnet - Deterministic Networking
LOCATION:Auditorio
STATUS:CONFIRMED
CLASS:PUBLIC
DTSTART;TZID=Europe/Madrid:20250725T143000
DTEND;TZID=Europe/Madrid:20250725T163000
DTSTAMP:20250616T025426Z
DESCRIPTION:Session III\n
 \n
 Onsite tool: https://meetings.conf.meetecho.com/onsite123/?session=34273\n
 \n
 Meetecho: https://meetings.conf.meetecho.com/ietf123/?session=34273\n
 \n
 Session materials: https://datatracker.ietf.org/meeting/123/session/detnet\n
 \n
 See in schedule: https://datatracker.ietf.org/meeting/123/agenda#row-123-2025-07-25-fri-1230-rtg-detnet\n
END:VEVENT
END:VCALENDAR

rjsparks
rjsparks previously approved these changes Jul 19, 2025
@rudimatz
Copy link
Copy Markdown
Contributor Author

this function replaces usage of agenda.ics

Copy link
Copy Markdown
Member

@jennifer-richards jennifer-richards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See inline - I think you can lean more heavily on icalendar to handle the formatting.

Also, for the new code it'd be good to run ruff (or at least change ' to ", which I see has been mixed a bit in the new code)

Edit to add: I think working in UTC is a good choice.We could add the meeting time zone to the description as an informational detail. (Just the name, not the full definition.) I don't know whether anyone cares though.

Comment thread ietf/meeting/views.py Outdated

# LOCATION: only if location should be shown
if item.timeslot.show_location and item.timeslot.get_location():
event.add('location', vText(item.timeslot.get_location()))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere: the event.add() method converts strings to vText internally, so I think you can just pass a string instead of doing it yourself. I don't think it's harmful, though.

(According to the icalendar docs, you do need to manually wrap things in vText for parameter values, but that's when you're doing things like organizer.params["cn"] = vText(...).)

Comment thread ietf/meeting/views.py Outdated
event.add('url', item.session.agenda.get_versionless_href())

# DESCRIPTION: build comprehensive description
description_parts = [ics_esc(item.timeslot.name)]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need the ics_esc - icalendar does this itself. E.g., the newlines/semicolons/commas/backslashes in this test are all escaped:

>>> e.add("description", "This is a description\nwith multiple lines\nand other weirdnesses; this is a \\ (backslash), or something isn't it nice!?")

results in

DESCRIPTION:This is a description\nwith multiple lines\nand other weirdnes
 ses\; this is a \\ (backslash)\, or something isn't it nice!?

Comment thread ietf/meeting/views.py
pass

# Join all description parts with 2 newlines (not escaped)
description = "\n\n".join(description_parts)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the "not escaped" something you're doing for a particular purpose or is it to replicate the previous behavior? Similar to the ical_esc comment above, I think icalendar is doing escaping for you and you just want to build up the literal string that you want and let it do the rest.

Comment thread ietf/meeting/views.py Outdated
"updated": updated
}, content_type="text/calendar")
ical_content = generate_agenda_ical(schedule, assignments)
ical_content = parse_ical_line_endings(ical_content)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(broken record) I think that icalendar handles the newlines correctly for you so we shouldn't need to call the fixup method on its output

@rudimatz
Copy link
Copy Markdown
Contributor Author

quick profiling comparison of rendering:

  • old method using template: 2.8 secs
  • new method using ical: 1.0 secs

@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 25, 2025

Codecov Report

❌ Patch coverage is 90.27778% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.76%. Comparing base (f380b1a) to head (dcc8910).
⚠️ Report is 40 commits behind head on main.

Files with missing lines Patch % Lines
ietf/meeting/views.py 90.27% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9187      +/-   ##
==========================================
+ Coverage   88.74%   88.76%   +0.01%     
==========================================
  Files         321      320       -1     
  Lines       41853    41754      -99     
==========================================
- Hits        37144    37063      -81     
+ Misses       4709     4691      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Member

@jennifer-richards jennifer-richards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the test probably needs another tweak. I also pointed at what I think is a long-standing issue with the URL property in the icalendar files we generate. We could change that here, but might be better to deal with it separately.

Comment thread ietf/meeting/tests_views.py Outdated
break

self.assertContains(r, f"LOCATION:{slot.location.name}")
self.assertContains(r, f"URL:{session.agenda().get_href()}")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a new issue, as you're preserving existing behavior, but I think we may be violating RFC5545 with the URL parameter. It says

   Description:  This property may be used in a calendar component to
      convey a location where a more dynamic rendition of the calendar
      information associated with the calendar component can be found.
      This memo does not attempt to standardize the form of the URI, nor
      the format of the resource pointed to by the property value.  If
      the URL property and Content-Location MIME header are both
      specified, they MUST point to the same resource.

I'm pretty sure the intent is that URL point a place to get the calendar event itself, not just "some handy URL for it". I.e., it should probably contain ical_url.

The RFC doesn't define a format for the URL or the resource it points to, so it's hard to say we're technically wrong, but I don't think this property is affecting calendars the way it is intended

Copy link
Copy Markdown
Contributor Author

@rudimatz rudimatz Aug 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jennifer-richards so you think every event's "URL" should point to its own session's ics URL, like
URL:https://datatracker.ietf.org/meeting/123/session/34370.ics

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's more clearly in line with the intended purpose. See the example in 3.8.4.6

https://datatracker.ietf.org/doc/html/rfc5545

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(echoing comments from slack): I finally actually reviewed how a couple calendar tools use the URL field and I think we should point to the "session materials" link. That's the most "dynamic rendition of the calendar information" we have.

Comment thread ietf/meeting/tests_views.py Outdated
cal = Calendar.from_ical(r.content)

for component in cal.walk():
if component.name == "VEVENT":
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is anything ensuring that a VEVENT is encountered? The test should verify that the expected number of VEVENTs is present.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added test to count events and test specifically the values of first event

Copy link
Copy Markdown
Member

@jennifer-richards jennifer-richards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rjsparks rjsparks merged commit 86bce86 into ietf-tools:main Aug 7, 2025
10 checks passed
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Aug 11, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor generation of iCalendar files

3 participants