Newlines in markdown files aren't turned into spaces

[mdinger]

Go here and CTRL-F liketo. This is a mangling of this file where the newline between like and to is removed without any space being added. It shows up in issues like rust-lang/book#39. The newline should be parsed into a space. Maybe this needs to go upstream but this is a good jumpoff point.

[azerupi]

Ow, that's bad 😞
That should definitely be handled in the markdown parser though. I really hope someone picks-up pulldown-cmark or that an alternative pops up.

In the mean time, I am not sure what I could do about it

[steveklabnik]

In the mean time, I am not sure what I could do about it

Well, do you know why pulldown-cmark hasn't been developed? Did they just drop off the map?

[azerupi]

Yes the author had stated his interest in replacing the use of hoedown by pulldown-cmark in rustdocs and there was a lot of enthusiasm about it.

I'd like for this to become the engine in rustdoc. Right now it implements CommonMark (an effort to standardize Markdown) with no extensions, but if extensions are necessary I'm open to adding them.

There were a couple of features missing though before it could replace hoedown. And some pull-request were made. The pull request adding support for tables was the last one that got merged in, I think. After that the author made one more commit, changing the license to MIT.

He hasn't made any comments, he just stopped responding (on github). I am not sure if he is coming back or not.

---

Since I started with Rust I have been interested in creating a markdown parser. Mainly because there wasn't any and because I wanted to learn about parsers. But I don't have enough time nor the experience to develop / maintain one on my own.

If however a couple of people would be interested to team up, I would be glad to be one of them ;)

[steveklabnik]

Well, let's see if we can get him to at least acknowledge what's up, maybe if not we can fork.

[azerupi]

If you can contact him, that would be sweet! Then at least we know what we should expect :)

[asolove]

@azerupi @steveklabnik I'm pretty sure this isn't a problem with pulldown-cmark, but with a little bit of logic here in mdBook.

I noticed the problem only occurs in the index files and not in other ones. Specifically, all the other generated html has newlines in it, but index.html has none. Sure enough, only for the index, there is some code that concatenates all the lines together without newlines: https://github.com/azerupi/mdBook/blob/master/src/renderer/html_handlebars/hbs_renderer.rs#L117

I think if you change that code to add newlines between each item in the collection when collecting them, everything will work. I tried doing this with itertools::Itertools#intersperse but I couldn't get the types to work out. Would be nice if core::str::Lines had an unlines method like in Haskell.

[azerupi]

I noticed the problem only occurs in the index files and not in other ones. Specifically, all the other generated html has newlines in it, but index.html has none. Sure enough, only for the index, there is some code that concatenates all the lines together without newlines

Ow my bad, I didn't even think of that. I jumped to conclusions without really taking the time to look into it...

Thanks for spotting that! Very nice 👍