import from notes.ietf.org: Inaccurate rendition

[chrysn]

What happened?

The data import from notes.ietf.org is a great addition, thanks for that.

Trying it out, I still had to resort to the old process (menu / download /Raw HTML, upload resulting file) because of some issues with the rendering, where apparently the notes server and the data tracker use subtly different variations of Markdown. (Biting my tongue not to start into a longer rant on that topic in general).

Concrete issues are:

• Indentation levels of bullet lists are ignored. This may be due to the precise number of spaces used, but 4 spaces is what a tab in the notes editor expands to. In the imported preview, they all show up in a single <ul>.
• A horizontal line (---) gets expanded to a literal <hr /> in the data tracker (maybe an overzealous HTML sanitizer?)
• Dead things-that-look-like-links (eg. [CB]: Carsten Bormann on a line of its own, maybe also footnotes) do not get rendered. I'm not sure what any Markdown spec says on them, the behavior might be OK -- we started using these when in some version of the minutes tool, having such links resulted in every occurrence of CB being rendered as <abbr title="Carsten Bormann">CB</abbr> (or whatever the precise HTML is), which is useful for minutes. (This issue is primarily about the discrepancy with notes though, which renders all these as separate lines).

The CBOR group's latest minutes at https://notes.ietf.org/notes-ietf-interim-2022-cbor-06-cbor?both provide examples.

What browser(s) are you seeing the problem on?

Firefox

Code of Conduct

• I agree to follow the IETF's Code of Conduct

[cabo]

Dead things-that-look-like-links (eg. [CB]: Carsten Bormann on a line of its own, maybe also footnotes) do not get rendered. I'm not sure what any Markdown spec says on them, the behavior might be OK -- we started using these when in some version of the minutes tool, having such links resulted in every occurrence of CB being rendered as CB (or whatever the precise HTML is), which is useful for minutes. (This issue is primarily about the discrepancy with notes though, which renders all these as separate lines).

I think that a bare

[foo]: bar

cannot be rendered as it defines a link.

The abbrev processing is triggered by

*[foo]: bar

(There are also other "declaration" like lines in markdown such as footnotes.)

[cabo]

Just checked the notes -- de-gfm delivers perfect (soft-break) markdown, so the problem must be somewhere else.

[cabo]

• A horizontal line (---) gets expanded to a literal <hr /> in the data tracker (maybe an overzealous HTML sanitizer?)

What is wrong about this? How do want your horizontal lines today?

[chrysn]

The abbrev processing is triggered by

Thanks, that resolves that subissue (given that it's basically invalid markdown, so impls can be expected to behave differently).

Just checked the notes

The notes I actually uploaded went through raw HTML export; to see the diverging behavior, you'd have to look at the import form (for which you might be authorized solely in the sandbox).

As for the indentation and the horizontal rule (which, I should have emphasized, is expanded and escaped to a literal displayed <hr />), see also these screenshots:

Rendering on notes:

Rendering on import:

These also show difference that's new since I've started doing the abbreviations right: The colons after the abbreviations are rendered with a \ (backslash) before them in the import.

[cabo]

The <hr /> is hilarious.

The backslashes are an upstream bug in kramdown that has already been fixed but did not make it into a release yet: gettalong/kramdown#751

[jennifer-richards]

The <hr /> is likely related to #3525

[rjsparks]

We also have de-gfm version lag, for reasons that we're working on addressing.

[cabo]

I just generated a version of de-gfm that creates 4 space indents for lists, which should solve the other problem.
That will be in kramdown-rfc 1.6.7, but it will become effective only when gettalong/kramdown#753 has been merged and released in upstream kramdown (we are also waiting for gettalong/kramdown#751 for the backslash issue, which has been merged but not yet released).

[cabo]

kramdown 2.4.0 is released, and kramdown-rfc 1.6.7 has a de-gfm -4 option.
Please check whether that solves the problem with the Python markdown interpreter.

[cabo]

So, to clarify what needs to be done to accommodate the markdown renderer currently in use in datatracker:

• update kramdown-rfc (currently 1.6.8)
• call de-gfm with the -4 option (in addition to any other options in use)