New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Complete the offline renderer #35
Comments
Relabeling this as an enhancement because this was never completed in the first place. |
it would be helpful if the Markdown documents could be rendered offline |
Just a small update / thinking out loud. Now that Grip has tests (#150) which includes a Markdown file containing all the features of GitHub Flavored Markdown and its GitHub API output, ensuring the correctness of an offline renderer implementation should be easier. Once it's complete, it can become the default (although the styles and assets should still come from GitHub). The GitHub API can still be used to access Markdown features that Grip's offline renderer doesn't implement right away. |
I just received a "github API rate limit reached" error randomly: it's my first time running grip for the day and I wanted to view a single document, but I'm at a busy tech campus where 100+ people are using Github on a routine basis. I hadn't even realized that Grip didn't render offline. I'm confused as to why I would use Grip at all, if it's no more efficient than viewing the Markdown document in the original repo. Seems like a high priority. |
@mileserickson I understand your frustration. The reason it works this way is that from the beginning, correctness of rendering was chosen over offline availability. It's absolutely a limitation, but with the recent test improvements in #148 and 2c37cc4, it'll be easier to make sure correctness is preserved in an offline renderer.
Grip solves the problem of viewing Markdown before committing to the repo. Offline mode solves the additional problem of viewing without an internet connection. (That's still necessary for viewing the Markdown document in the original repo, btw). I'm planning on addressing this at some point, unless someone else jumps in. However, I have fewer large windows of time for big changes these days. Being aware of that actually influenced the original decision. (Nobody likes waiting for a busy maintainer to change and deploy code every time GitHub tweaks or updates their styles.) In the meantime, could you try the workaround? That'll prevent others on your network from using up your own rate limit. Here's how:
Eventually this auth experience will be also improved (see #82). Thanks for trying out Grip and for being patient. Let me know if I can help in any other way. |
The solution above worked for me. |
Is it still the plan to implement offline renders? I don't need a perfect render, just something usable for reading markdown books. |
@astrodad It is. For me personally, I probably won't start until late summer or in the fall. Thanks for checking in! |
Great. I'll take a look at the code but my Python is as good as my mastery of the Force. |
Any updates on this matter? 😊 |
What is holding up the offline renderer? |
I just hit the rate limit, and followed the instructions above to set up a personal access token, which worked great. True offline access would be great, but this is a fine workaround. Am I correct in thinking that this would be the superior solution to supplying If you'd like, I can create a separate issue. Also, I can take this on and issue a PR when ready. |
FWIW, I wonder if the initial offline renderer could be documented as not-exact. That is, for the sake of getting offline rendering off the ground, just state that the offline renderer won't be perfect. My guess is that many of the people needing that support don't care that it isn't exact, they will be happy to just have offline rendering that is close. Then, over time, the offline renderer could be brought up to par. |
It's going to be hard to find a render that does things exactly offline like GitHub unless you want to use their Markdown parser...which I think is Redcarpet? Maybe I'm wrong. You can get githubish though with Python Markdown. I have a branch over here where I enabled offline mode: https://github.com/facelessuser/grip. It's using my dev branch of extensions here: https://github.com/facelessuser/pymdown-extensions/tree/dev. These extensions don't aim for 1:1 exactness for Github (nor do I think I would ever go that far due to inherent differences in parsers), but much of it is inspired by Github. It handles emojis (uses a literal index from their Gemoji repo), githubish emphasis, tasklists, auto-linking links and emails etc. It's more than workable for a test offline rendering. Just saying there are closish things. Not saying you should use anything from here, just letting you know some options that are available. Anyways, this is a nice project that I've started using on my Github readmes and such. EDIT: If I had a place to upload the results I would. |
First of all, thanks for grip. I just discovered it and now I use this script to use the offline renderer:
I use python 3.6. To make this script work I have to comment out 4 lines from line 122 in ~/.local/lib/python3.6/site-packages/grip/renderers.py
Is this normal? The most obvious rendering problem is that with lists there must be a preceding empty line. This also results in TOCs without indentation. Any idea for a quick fix? |
It is a limitation (or I should say according to spec) of the Python Markdown library which is used to render the HTML from Markdown. There is no easy way fix/change. You would have to override the default extensions to rework then in a way that is closer to GFM. To be honest closer is all I think you could ever get with Python Markdown. You can achieve GitHub-ish results, but you will never get 1:1 results. For 1:1 results in Python, you would have to port GFM directly to Python. |
@facelessuser thanks for the assessment. Then I think it will be better to change https://github.com/jonschlinkert/markdown-toc to produce a Python Markdown compatible TOC output. |
I agree. I did a little bit of searching and found a question on SO about GFM in Python with an accepted answer that states the following:
So in case anyone has the time, perhaps it would be fruitful for them to look into Misaka? The official repo for Misaka is on GitHub; https://github.com/FSX/misaka. The GitHub repo description of Misaka reads "A Python binding for Hoedown". Hoedown, in turn (also on GitHub) is described in the README as "a revived fork of Sundown". Misaka is published on PyPi, so it can be installed simply by issuing |
If anyone decides to go the Misaka route, note also one of the comments on the accepted answer in the SO thread:
The issue that is pointed out was closed but doesn't link a commit. I would assume that it means that the XSS issue has been resolved but rather than assume such to be the case one should test and see that it really is. |
I have the same problem with python 2.7 |
It's the end of 2018, is there any progress on this? |
@davestewart The API rate limit bit me repeatedly - especially when trying to 'live edit' a markdown file - so I wrote a markdown converter that also works offline. It's not pixel-perfect compared to It's called @joeyespo Feel free to snatch code from me for |
@OleMussmann - I created a ticket on GitLab. |
Subscribing for interest in this feature. As far as an MVP of offline rendering goes, my usage of this program is to export |
@ianfixes — for HTML documentation, I strongly encourage you to check out Markdeep. In addition to standard Markup, it supports diagrams, mathematical notation, and a whole host of very useful additions. For an example, the Ray Tracing in One Weekend series is written entirely in Markdeep: https://raytracing.github.io/. Not only do you have a much larger palette of documentation features, you also have complete freedom to style as necessary with standard CSS. Also, take a look at the |
That seems interesting, but anything beyond what Github would be rendering on their site is wasted in my particular use case. I ended up using GitHub's own ruby gem called require 'redcarpet'
puts "Reading README.md"
input = File.read("README.md")
puts "Initializing GFM -> HTML converter"
renderer = Redcarpet::Render::HTML.new(
no_images: true,
# no_styles: true,
)
markdown = Redcarpet::Markdown.new(
Redcarpet::Render::HTML,
no_intra_emphasis: true,
autolink: true,
tables: true,
fenced_code_blocks: true,
lax_spacing: true,
)
puts "Rendering markdown to HTML in #{OVERVIEW_FILE}"
intro = "via [README.md](https://github.com/my/repo):\n"
output = markdown.render(intro + input)
File.write(OVERVIEW_FILE, output) |
grip, https://github.com/joeyespo/grip, can very quickly hit the 60 unaunthenticated req/h if a token is not used for the GitHub API, this helps with a smooth grip experience re joeyespo/grip#35
Any update on this? :) |
Is there any reason the API-returned data isn't simply cached? I don't see why it needs to be loaded for every single page render. |
@astrodad Try using something like Typora, it gives you a preview of how it will look like. I just use Grip in conjunction with Typora because they really work well together with Typora being the editor and Grip being the viewer/exporter. |
I can confirm that Typora is a bloody brilliant MD editor: Their GitHub theme needs a bit of work (it's not exact) but is pretty close. You can also set up your own theme (I couldn't be bothered TBH) |
In case anyone is looking to implement this, GitHub's renderer is available at https://github.com/github/cmark-gfm and there are various Python bindings. You'll have to figure out the right combination of flags, something like |
I personally think this would be easy-ish to implement and won't break too many things, will it? The cache can be refreshed when it gets older than an hour or a day maybe. And it can also be forcefully refreshed if the user wishes to do so. |
Personally I'd argue for never updating the cache unless the user requests it, but yeah your thoughts match mine. |
Ended up with uninstalling grip and using glow |
I have been using However, lately, I've started running into GitHub API limits. I found this quite interrupting of my flow, so I looked into other options. I learned that Atom has a nice Markdown preview package built in now (available by default, and on GitHub here). So, my current approach is to build my GitHub-flavored Markdown document in Atom with the preview function running, then to do a 'final' run with I also tried glow (mentioned above). It is cool but not as nice looking and doesn't really seem to work well for my purposes, so I can't recommend it. Additionally, I tried the editR R package with poor results. I could get the shiny app to run but I could not get it to correctly display my markdown, so I cannot recommend it. Hope this helps someone. ~J |
Thanks to @justincbagley for elaborating on his experience in trying several other options and kindly sharing with the community. Unfortunately, it looks like |
It's a shame, this is a really neat package when combined with vim-grip |
I just use pandoc now. Get the style sheet for github, and use it with vim/zathura. Bind f5 to rebuild the document
-------- Original message --------From: Zeioth ***@***.***> Date: 12/17/21 10:16 (GMT-05:00) To: joeyespo/grip ***@***.***> Cc: Fred Frey ***@***.***>, Manual ***@***.***> Subject: Re: [joeyespo/grip] Complete the offline renderer (#35)
It's a shame, this is a really neat package when combined with vim-grip
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
In the end I took this strategy on vim-grip:
This runs grip to render the document in the browser, and then stops grip. This way even 60 requests is enough. My main problem was that grip is real-time and I was running into the rate limit after a couple minutes. |
That's actually an underrated comment. Accompanied with the right CSS this comes pretty close to the original. Only if there was a way to extract all the necessary CSS from GitHub. Luckily there is already a project doing exactly that: https://github.com/sindresorhus/github-markdown-css. So inspired by this I hacked together a script which pretty much replaces |
If it's close enough for 95% of cases maybe ship it with a warning? |
The
--render-offline
didn't make the 2.0 release, because it doesn't render the same content as GitHub.The functionality is there. The CLI option is added back again in the fix-render-offline branch, where work can continue.
The text was updated successfully, but these errors were encountered: