[Question] HTML Entities

[andrewetchen]

Context

Hi Jon,

We are trying decipher how your action "sends" Guru card content. All of our cards are written in markdown and have an .md extension. We are also trying to determine how incoming HTML code is interpreted on Guru's end when receiving these 'submitted' Guru cards from your sync action.

Problem / Expected vs Actual

Code spans

When writing instructional information for users, whenever HTML is added, Guru does not render it as expected.

This is an example of a code span with HTML:
Step 2: Find this `</div>` and this `<div class="container">`

Expected Guru rendering:

Actual Guru rendering:

Fenced code blocks

Similar behaviour occurs when providing instructional information for users in the form of fenced code blocks. The following only occurs when using Liquid "inside" HTML code.

This is an example of a fenced code block with Liquid "inside" HTML code:

```
<option {% if collection.handle == "all" %} selected="selected"{% endif %}>{{ 'collections.all_types' | t }}</option>
{% for type in shop.types %}
  <option{% if collection.id contains type %} selected{% endif %} value="{{ type | url_for_type }}">{{ type }}</option>
{% endfor %}
```

Expected Guru rendering:

Actual Guru rendering:

More context regarding fenced code blocks, we've tried a variety of workarounds these were the only ones that worked albeit, 'hacky' and not ideal from a user experience perspective:

(Workaround 1)

Manually replacing < and > with their respective HTML entities and adding <pre> and <code> tags:

<pre><code>
&lt;option {% if collection.handle == "all" %} selected="selected"{% endif %}&gt;{{ 'collections.all_types' | t }}&lt;/option&gt;
{% for type in shop.types %}
  &lt;option{% if collection.id contains type %} selected{% endif %} value="{{ type | url_for_type }}"&gt;{{ type }}&lt;/option&gt;
{% endfor %}
</code></pre>

Will render on Guru as expected: (don't mind the extra space at the top)

(Workaround 2)

Adding an opening <script> and closing </script> tag:

```
<script>
<option {% if collection.handle == "all" %} selected="selected"{% endif %}>{{ 'collections.all_types' | t }}</option>
{% for type in shop.types %}
  <option{% if collection.id contains type %} selected{% endif %} value="{{ type | url_for_type }}">{{ type }}</option>
{% endfor %}
</script>
```

Guru rendering:

(Workaround 3)

Adding an opening <style> and closing </style> tag:

```
<style>
<option {% if collection.handle == "all" %} selected="selected"{% endif %}>{{ 'collections.all_types' | t }}</option>
{% for type in shop.types %}
  <option{% if collection.id contains type %} selected{% endif %} value="{{ type | url_for_type }}">{{ type }}</option>
{% endfor %}
</style>
```

Guru rendering:

---

Thoughts

My gut tells me that the problem lies on Guru's end since all of the above has never been a problem when creating cards from within Guru itself using their editor.

I'm thinking that I could run some sort of function to automatically add and apply HTML entities to all Cards before your Github-to-Guru-sync action runs but I'd like to hear your thoughts as I could potentially be missing something simple. Any thoughts or ideas?

[andrewetchen]

Hey @peckjon

I ended up creating our own solution for this. I added a Pandoc dockerfile to convert all of our markdown files to HTML before running your action. It adds all of the HTML entities. I then uploaded them as artifacts and passed that through into your action - so far so good 😄

Consider this issue closed.

[peckjon]

Hi @andrewetchen ! Thanks so much for opening this up; you're correct that the MD-to-HTML conversion happens entirely w/in Guru's walls by default, and there are definitely some significant issues (and not just restricted to HTML entities!)

We'll be meeting with them next week and I'll bring this issue along. Hoping to see some improvements in that converter (or at least clarity on which one they are using, and known pitfalls). Meanwhile -- you've immediately come to the approach I'd suggest, that of preconverting the Markdown.

I presume you followed https://github.com/pandoc/pandoc-action-example ? Is it running efficiently? If not, we could consider using a Node package directly to lightening the processing cost:

• https://www.npmjs.com/package/showdown
• https://www.npmjs.com/package/markdown-it
• https://www.npmjs.com/package/markdown-to-html (stale?)

...come to think of it, perhaps I'll even add "preconvert HTML" as an option directly in the Action. Any thoughts on which of those Node packages might be best?

[andrewetchen]

Thanks for the reply, @peckjon !

We'll be meeting with them next week and I'll bring this issue along. Hoping to see some improvements in that converter (or at least clarity on which one they are using, and known pitfalls). Meanwhile -- you've immediately come to the approach I'd suggest, that of preconverting the Markdown.

I'd be very curious to know what the outcome of that meeting in terms of what needs to be tweaked on our end to play nicely with their API.

I presume you followed https://github.com/pandoc/pandoc-action-example ? Is it running efficiently? If not, we could consider using a Node package directly to lightening the processing cost:

Indeed! It is running pretty well. I have it setup in a test environment right now with a few markdown files and its pretty quick. I would imagine that when we run it on our ~380 markdown files, that it'll be much longer. As long as we're in the less that ~5 minutes mark, we're good.

...come to think of it, perhaps I'll even add "preconvert HTML" as an option directly in the Action. Any thoughts on which of those Node packages might be best?

That would be amazing!

---

I'm currently working on a pandoc conversion and everything is rendering on Guru's end aside from the the syntax highlighting and indentation:

[1]

Markdown [Before] HTML conversion:

Markdown [After] HTML conversion:

...and how it's rendered on Guru:

[2]

Markdown [Before] HTML conversion:

Markdown [After] HTML conversion:

...and how it's rendered on Guru:

[3] - And of course, if I don't convert from Markdown to HTML:

This:

Will render as expected:

[peckjon]

@andrewetchen: preconversion is now available: https://github.com/peckjon/github-to-guru/releases/tag/1.7.1

Please let me know if you discover any formatting issues... I can tweak the markdown-it options or make it configurable if needed.

IMPORTANT: turning this feature on might or might not result in a change to the URLs of any cards previously published using Markdown. Id appreciate it if you could take note of a few Card URLs before turning this on, and let me know if they change after.

[andrewetchen]

Thanks @peckjon !

I did a quick test on one of our test repos a couple of days ago and this is what it ended up looking like:

Please keep in mind however, I'll have to re-test to make sure that it wasn't an issue with our markdown though.

I built out a workflow that does exactly what we need (for now). Having the ability to download the artifacts to confirm how everything is converted to HTML is a requirement for us and it'll help in the future when it comes to troubleshooting or when/if team members leave the project. Another important thing is that our project that utilizes your action is still fairly new so I'd expect other potential GURU quirks to come up - hopefully not though 🤞🏼

After all the debugging and testing I've been doing for markdown > HTML the past few days, when I have some downtime, I'll plan to re-test things.

Again, you've been an absolute ⭐  throughout all of this and I've learned a thing or two. Cheers!

[peckjon]

Hi again @andrewetchen !

Just wanted to let you know that most of these issues should now be resolved in https://github.com/peckjon/github-to-guru/releases/tag/1.7.2, along with a few other issues such as links not being allowed under images, and certain lists not rendering correctly.

[andrewetchen]

Hi @peckjon

Thanks for this release. I had a look at 1.7.1...1.7.2 and How to use markdown in Guru cards.

This is great news. I will have to do so more testing to see if this fixes our original Liquid issue and some of our other markdown related issues (not related to your action). I'll report back here next week - thanks again!