New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Literate CoffeeScript #1786
Comments
Yes -- I think this is very interesting. Please separate out the pull requests into two separate branches so we can look at them in isolation. |
A couple things: If this goes in to the core Are you planning to tie this directly to Markdown? Or do you want to have the ability to use other markup languages? How will multiple files be organized together? Are you planning to generate an index page, with some sort of navigation for browsing around? Is the output format of the final document just HTML? Do you want to make it possible to generate a PDF of all your source code? Would it be possible to add syntax highlighting for all the CoffeeScript snippets, in all output formats? |
Hello; This thing has been inspired almost entirely by Haskell, which happens to be my mother tongue in programming. So most things are going to be just like in Literate Haskell, enabling what it does, and really little more than that. Regarding the first one (magic combination): it is doable, and I think the idea is great because it thinks outside the Literate Haskell box in which I am. We would just have to agree on how literate programs should start. I guess using
at the top of the file should help us detect magically. I am not planning to tie this directly to markdown. They just accidentally happen to be perfect fits. Regarding file concatenation, the files joined together had better have the same syntax. Mixing literate with illiterate and parsing it as one file would not work. If they are treated each file on its own, even if sent into The output of the final format is what your Docco produces. The text transformations do not usurp the literate programming that has already been used to great (beautiful!) effect in your code; they just provide another way to write literate CoffeeScript. They just make it so that it reads as beautifully in source code as it does in your generated docs. I like to think that literate CoffeeScript is what you have not. But when it is capitalised, it is what is in the commit: Literate CoffeeScript. On syntax highlighting, since it doesn’t usurp your Docco system, it will do what Docco does. |
Awesome work @revence27! @jashkenas one nice thing about having a distinct extension ( |
@quackingduck We could have both; it is, after all, still one extra line of code. (Long line.) :-) |
This is pretty cool. |
I love the idea. |
I like @geraldalewis's idea of using a |
I was actually hoping for auto-detection ... purely automatic > file extension > magic comments > string directives, in my book. |
string directives > file extension > magic comments > automagic |
@jashkenas agree that purely automatic is nice, just concerned that this code might not show up highlighted on github pages. Is there a precedent where github starts using a lexer based on a file extension and then switches to a different one based on auto-detection? |
Hmm... Generating an AST for a snippet, and seeing if it generates an error? |
@jashkenas |
@geralalewis you say “Some kind of scoring system like @josh uses for language detection on GitHub?” |
Totally agree with @revence27 here. |
No worries -- let's use continue to use I don't think it should use the Docco format, as you'll have a much higher ratio of prose : code. |
What is the syntax of the proposed input format? Standard markdown? I have been using sort-of-literate coffeescript in markdown with pandoc for some upcoming stuff. Pandoc can render to an epub or pdf ebook, to html, rtf, odt etc. Pandoc accepts both standard markdown code blocks indented with at least 4 spaces or code blocks delimited by bird tracks:
I like the pandoc birdtracks because they make it possible to mix languages and to see that a document is literate without using a file extension; assuming that ~~~ is not valid as code. Pandoc has built-in support for coffeescript syntax highlighting for HTML. Extracting the code sections is simple in an editor with regex support (or possibly with sed/awk). I have been using it without any special support from the coffeescript compiler. In my editor (acme) it is It would be nice (but not required for me) if this was built into the compiler. For example: if a file contain birdtracks then it is literate and everything outside of the birdtrack sections is discarded before lexing/parsing. As an example here is a rendered version of Mark Hahn's A Beginner's Introduction to CoffeeKup. It isn't a literate document but shows what it can look like augmented by TeX template. |
@jashkenas Well, then, it will use a modified Docco format that takes into consideration the potential for a prose:code ratio that is steeped hard in the direction of prose. I am leaning towards adding a test for “Literate CoffeeScript” at the top of the code to start the mode; I think it is excellent, considering that it is literate programming we are talking about. It is a fine preamble. |
@autotelicum It could be any syntax you are going to be using for your output. In the case of this issue, it was Markdown. But it could even be the syntax you speak of above. |
Literate CoffeeScript I have added an alternative way to express that a script is Literate CoffeeScript. If the script starts with the line: “Literate CoffeeScript”, as does this comment, it is considered to be Literate CoffeeScript, regardless of the file extension. I did this because there is no chance that these two tokens will start a valid CoffeeScript program, yet they add a lot of beauty and rhetorical fluidity (as preamble) to the code. In fact, I think that even if that particular test is rejected from the code, the convention should be that Literate CoffeeScript start out as did this comment. (And we all know that code conventions are just missing compiler features.) So, there it is. console.log 'Merge me!' I just had to put that there because I can, and you cannot stop me, and this comment will still compile. :-p |
Hmm, 3 months since there was any activity on this issue. Is this still up for debate or has this idea died with the issue? Would love to see this in the next version of CoffeeScript. :-) |
It's not dead, there's just a lot of things to get done, and this isn't at the top of the list (or at least not mine). |
Hey folks. I've pushed an initial implementation of literate CoffeeScript to the literate branch, here: ... I've initially tested it by formatting the Before merging it, there are a number of things that need to be done, the most important of which is ... It would be lovely if we can figure out a way to not have to add an additional file extension ( Any ideas? |
Try first to parse it through coffee and fall back to litcoffee ? Add an annotation ? (#LITERATE?) |
What about a compromise solution, with a required marker that is also functional? For example, the literate.litcoffee example has a header at the top using the Given that a large number of lines of CoffeeScript are also valid lines of Markdown, it seems unlikely that unaided autodetection is going to work without significant and confusing edge cases. It would be a bad situation if, for example, a typo like a missing |
... you'd think it wouldn't be to hard too detect CoffeeScript, and fall back to markdown ... but unfortunately code like this:
|
I don't think so http://is.gd/NuBxgb |
Whoa -- nice! |
@superjoe30 literate CoffeeScript in general, or reusing the same extension and autodetecting? |
Literate CoffeeScript in general. |
@superjoe30 The Wikipedia article on literate programming is decent reading for understanding the basic purpose, but the main point is better code documentation. |
Really glad to see this happening. If it is of any help, here is how I implemented literate programming for PHP: http://bergie.iki.fi/blog/literate_programming_with_php/ |
Why would it be undesirable to add a new file extension for Literate CS files? Isn't that the standard mechanism that text editors and syntax highlighting tools (like github/linguist) use to detect the language? It's my understanding that Haskell also uses this strategy to difference from normal If a separate extension is indeed undesirable, the proposal of adding a |
From the doctest.js discussion on HN: My main concern with it is that it forces you to write the document in the same order you want the code to be extracted, which may not be the best order for explaining things. That is why classic Literate Programming tools like noweb allow you to name the chunks of code and then arrange them into the generated files as you wish. You can see an example of this in action when I'm assembling noweb.php. One way to work with named chunks of code in Literate CoffeeScript would be to use the fenced code blocks syntax from Github-flavored Markdown. This way we could do stuff like:
We would only need to figure out an appropriate chunk inclusion syntax. Noweb uses
|
... and replied on HN: http://news.ycombinator.com/item?id=4608260 |
@jashkenas well argumented. I guess the way proposed in this issue makes Literate Programming a lot more approachable. The next question with this approach is editor support, as then you'd have to understand both the Markdown syntax and CoffeeScript, and the relations between the two. Emacs probably makes this easy with major and minor modes, but with other editors this can be trickier. |
Yes -- I think that's actually not so hard. I'm going to attempt it for textmate / sublime. Those editors are already able to combine parse modes ... for example, JavaScript is correctly highlighted within an HTML document. Here's it's easier -- any block that's indented more than four whitespace characters should be passed to the CoffeeScript highlighter. |
For a different take on editors, I've been toying with the idea of combining collection support in Create.js, Hallo's Markdown mode and something like CodeMirror to make a web-based WYSIWYG editor for literate programming. On longer term you could then utilize features like image insertion, or even connect it with a web-based graph editor so you could produce really nice documents while you code. The less separation there is between your documentation and the code, the more likely they'll both remain up-to-date. |
is there any support in gedit / gtksourceview for this? That would be great! |
I quickly wrote my own Literate Coffeescript syntax definition for gedit, see my gist. It is far from perfect, but at least Markdown / CoffeeScript highlighting works. You need gedit-markdown (included by default) and gedit-coffeescript. Place the file in |
Cool. I actually had the same idea earlier and wanted to actually start work on it but now I don't have to. Thanks! |
This looks like a great addition. However, after trying a few syntaxes, it appears that there isn't support for "fenced code blocks." Is this the case? In an already white-space sensitive language like CoffeeScript, adding an extra layer of indentation for Markdown code blocks is problematic. Fenced code blocks, for me, would be a must before using this. |
You're correct. Fenced code blocks aren't a syntax of ordinary Markdown that I'm aware of. http://daringfireball.net/projects/markdown/syntax#precode Aren't they a special GitHub thing? |
They are not part of standard Markdown, that's correct, but they are a common addition by many Markdown libraries, including Showdown.js, Github-flavored markdown via Sundown, Ruby's Kramdown, etc. For the purposes of literate CoffeeScript, adding more indentation makes things tougher to read and edit, especially if one is using something like RequireJS wherein most of the file is already indented to live inside the callback to |
Right now the literate syntax is bare minimum--far from full-blown Markdown.
|
First of all, to see a little experimentation of mine with Literate CoffeeScript and the I too have been toying with some ideas I had on these, and related fields, around literate programming and UI design, as a response to bergie. I sent you an email actually to see if we can do something with it. Note that there are, although few, several tools that can take very different approaches. Someone interested in LP should also checkout [Codnar](https://github.com/orenbenkiki/codnar](https://github.com/orenbenkiki/codnar). There are |
I'm not sure if this was considered, but it is possible to append the fenced code block triple backticks ````coffeescript (filename)` where you can easily sequentally parse/concat those into seperate files. I could imagine that explaining why you have the file/directory structure could partially be together in a 'story' and then, we moving back to the main idea, keep writing chunks of files as a coherent red-line throughout, and branch off as the flow of thoughts come along. Donald Knuth and other usually do also have a outline that goes from 'require' to the 'execution' etc. which is still a bit too much towards computers and not humans, but does make a lot more sense when they get explained.
And indeed, with functions and requirements being scanned before execution, we have all we need to write a entire book in 1 file. Just don't forget the Updated: example that does get parsed properly, having the file name on the first carriage return after coffeescript. It could even be easily done with additional tab for aesthetics |
@supersym I suggested something similar in this Hacker News thread: http://news.ycombinator.com/item?id=4608165 Here is the response from @jashkenas back then:
|
Right. And he's probably right :) But we're still talking 1 file? I didn't see that pass in the discussion of 1 litcoffee or 10 or 20 .litcoffee files. One file makes sense from a literary standpoint, and for easy handling / maintenance (debatable, enough author make a seperate file for each chapter because 200 pages aint fun) but e.g. what they did with literate clojure and such is packing it all in 1 file. And after that, the classes get extracted etc. Using the above suggestion I made, and ^(`{3}coffeescript( \r|\r)()(.*)) regex expression I have enough to do this . Updated because file on the same line doesnt get parsed http://www.gridlinked.info/oop-with-coffeescript-javascript/ somekind of namespaces is something I should investigate a bit better too |
Yeah exactly I noticed something like that and, although I find the notation ugly, its a simple and elegant way of working with this style. Personally I like that I can stay on the same line with coffeescript, since the fences are there to stay I guess and the Last thing I noticed, that the used inline #macro() more or less inside the paragraphs to do stuff. Any thoughts if we could call our methods in a non-significant whitespace manner? Probably, the best candidates might be to use the markdown |
Btw, if you have a customer who knows exactly what they want, have them write down the story and put code in between :) Or just comments there where you are unsure or want questions answered. My personal dialect will probably provide some conventions/enhanchements in that order of communicating messages as footnotes etc. Also I like to have some convention to autoindex and number illustrations etc like tex does. Shame that Markdown/HTML doesn't handle 1., 1.1, 1.2 and that kind of enumerations. Or does it? |
Well can't clean up so it seems... In short, perhaps implicit He is another example of a literate program on GitHub: a Clojure to C++ compiler And Emacs org-mode example http://doc.norang.ca/org-mode.org.html and http://orgmode.org/worg/org-contrib/babel/intro.html Another example of Lisp dialect LP (Scheme/Racket) here and a blog post here Maybe some want to learn from other language implementations. |
Should this have closed with 1.5? |
Looks like it. Closing. |
This file, as it is, has been taken from
tests/literate.literatecoffee
in my fork.Without making any changes to it whatsoever, it is valid literate CoffeeScript, if you just copy and paste it into an editor, and make my fork of CoffeeScript run on it. Make the file end in
.literatecoffee
to make it go into literate mode.I wonder if Mr. Jeremy Ashkenas thinks it is a good idea. It would be good to know, before I send a pull request. (Oops! Earlier pull request for binary literals was pending, and it subsumed this one. Oh, well.)
The change is only one line in only one file, and it is bound to be faster than fast in use. It is in my commit revence27/coffee-script@132d306 with an accompanying modification (different commit) of the
Cakefile
, that it may look for.literatecoffee
files in the tests.This, of course, is inspired by the Haskell programming language, but it is neater that the Haskell version of literate programming.
Beautiful Literate Programming
If this file parses at all, that is the test, and it has passed.
Originally, literate programming did not mean heavily-commented code. However, owing to the System, that is what it evolved to mean.
Literate programming is supposed to be where code invades the commentary, not lots of comments invading the code.
Under this scheme, everything is a comment.
Except the bits that are indented. If a line does not start with whitespace, it is a comment.
Everything else is code.
This is how we shall proceed with writing a string reverse in CoffeeScript.
And then we will use it.
See? Wasn't ugly, was it?
Work on a syntax mode should not be difficult, in my opinion. (Although the only
syntax things I know how to do are Vim, and even those, not too perfectly.)
That will be all.
The text was updated successfully, but these errors were encountered: