Fix lampepfl/dotty-feature-requests#28: make Dottydoc responsive #7153

TheElectronWill · 2019-09-01T17:17:32Z

This PR implements an overhaul of the Dottydoc website in order to make it responsive. This is mostly front-end work, very few scala files have been modified.

Bootstrap theme

The new design is based on a customized version of the Bootstrap framework, located in the bootstrap-theme folder. This has several advantages:

it ensures consistent settings (such as the fonts, the colors and the content width) for all the pages.
it simplifies the development of custom dottydoc themes: modifying the custom bootstrap, i.e. changing a few variables, is enough to alter the look and feel of the website.
it greatly reduces the size of Bootstrap’s CSS file, mainly because the features we don’t need can be removed.

Downside: the inclusion of the long file package-lock.json (4500+ lines!) makes it look like this PR increases the size and the complexity of the codebase, whereas many html and css files have actually been simplified 😃

Scala changes

In addition to some tweaks to the page generation code, I've added a new option -project-logo to easily change the toolbar's logo without touching any code.

Visual changes

The most visible changes are the toolbar and sidebar, which have been reworked. Also, the fonts have been harmonized across all the elements.

Below are some screenshots of the updated Dottydoc website.

Front page

Let’s start small. Updating Boostrap broke the front page menu. I fixed it and took the opportunity to simplify the code and improve the fonts a bit.

Before

After

I can do something about the "scroll down for more info" but that will be another PR 😃

Blog page

Before

and

After

and

Sidebar and toolbar

The sidebar is now bigger, more readable and toggleable. The toolbar is a bit smaller and has a better contrast.

Here is what it looks like on desktop:

Here is what it looks like on small screens:

API Search

The search page is responsive too! And it generates links to the relevant members.

Before

After

API documentation

Of course the documentation looks better on desktop than on mobile. But it's still possible to do nice things:

Before

After

Unfortunately I’ve been limited by liqp, which (among others) doesn’t support iterating on Maps. More content to the API pages could be added once Dottydoc has a more powerful Liquid engine.

There is a feature request for implementing Liquid inside of Dottydoc. I wonder if it should be a separate library instead, since it could be useful to other projects. Is it possible to make Dottydoc depend on an external dotty library?

dottybot

Hello, and thank you for opening this PR! 🎉

All contributors have signed the CLA, thank you! ❤️

Commit Messages

We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).

Please stick to these guidelines for commit messages:

Separate subject from body with a blank line

When fixing an issue, start your commit message with Fix #<ISSUE-NBR>:

Limit the subject line to 72 characters

Capitalize the subject line

Do not end the subject line with a period

Use the imperative mood in the subject line ("Add" instead of "Added")

Wrap the body at 80 characters

Use the body to explain what and why vs. how

adapted from https://chris.beams.io/posts/git-commit

Have an awesome day! ☀️

TheElectronWill · 2019-09-01T21:53:18Z

rebased on master

julienrf

Awesome work @TheElectronWill, thanks!

I’ve had a quick look to the diff and the changes look good to me except for the usage of const and let, which might be incompatible with old browsers.

julienrf · 2019-09-02T07:40:22Z

doc-tool/resources/_layouts/search.html

-        var insertMember = function(member, li) {
-            var div = document.createElement("div");
+        var insertMember = function(member, li, parentLink) {
+            const div = document.createElement("div");


Unfortunately, by using const we might make the website unreadable on some old browsers: https://caniuse.com/#feat=const. I would suggest reverting to div.

Did you mean reverting to var? I've reverted to var.

julienrf · 2019-09-02T07:44:48Z

doc-tool/src/dotty/tools/dottydoc/model/comment/Comment.scala

@@ -85,7 +85,7 @@ trait MarkupConversion[T] extends MemberLookup {
    short                   = stringToShortHtml(parsed.body),
    authors                 = filterEmpty(parsed.authors).map(markupToHtml),
    see                     = filterEmpty(parsed.see).map(markupToHtml),
-    result                  = single("@result", parsed.result).map(markupToHtml),
+    result                  = single("@return", parsed.result).map(markupToHtml),


Good catch!

julienrf · 2019-09-02T07:50:08Z

Unfortunately I’ve been limited by liqp, which (among others) doesn’t support iterating on Maps. More content to the API pages could be added once Dottydoc has a more powerful Liquid engine.

Is it possible to workaround this issue by converting the Map into another type of collection that liqp knows how to iterate on?

TheElectronWill · 2019-09-02T22:46:42Z

Unfortunately I’ve been limited by liqp, which (among others) doesn’t support iterating on Maps. More content to the API pages could be added once Dottydoc has a more powerful Liquid engine.

Is it possible to workaround this issue by converting the Map into another type of collection that liqp knows how to iterate on?

I've just tried to convert the Map to a list of list. Because of a bug in nested for loops it's impossible to display the element that I need. And unfortunately, index-based access doesn't seem to work properly either...

OlivierBlanvillain · 2019-09-06T13:02:04Z

I just compiled the website locally and it looks great! However the pages seam to glitch quite a bit until they are fully loaded, do you have any idea how to fix that? Here is a video illustrating what I'm seeing on my machine: 2019-09-06 14-52-02.flv.zip

TheElectronWill · 2019-09-07T15:10:46Z

@OlivierBlanvillain Can reproduce on Chrome, thank you for letting me know! For some reason there is much less glitching on Firefox...

I think there are multiple factors here:

Default local fonts are replaced by remote fonts once they are available.
Code blocks are fully processed by hilight.js after the rest of the page - and it doesn't affect just the color but also the padding and margin (you can see this by disabling javascript).
The sidebar's active entry is expanded by javascript, causing it to flash a bit.
The navbar only glitches when the files are opened manually. With a web server the browser is able to keep unchanged parts and it looks better.

I should be able to solve point 3 and to improve point 1. For point 2 I'll take a closer look at what hljs does and adjust the css and/or reconfigure hljs.

TheElectronWill · 2019-09-14T23:50:00Z

@OlivierBlanvillain @julienrf I've fixed the aforementioned issues. Chrome still flashes when navigating the local files but it's fluid with a web server.

I've also fixed the code blocks that have been broken by fec2ae5. Basically, this markdown syntax to create code blocks:

    def someCode = ...  // indented with 4 spaces

is now unsupported and must be replaced by:

```scala
def someCode = ...
```

The replacements have been done in e17d964.

This PR now looks complete 😃

OlivierBlanvillain

LGTM! I'll merge after resolution of conflicts on docs pages.

Thanks a lot for the PR, it looks like a lot of work went into this and the result is a really nice improvement over the current state.

OlivierBlanvillain · 2019-09-17T13:24:22Z

docs/docs/reference/metaprogramming/macros-spec.md

-      SimpleType      ::=  ...
-                        |  ‘$’ ‘{’ Block ‘}’
-
+```none


Do we gain anything by using none instead of just leaving it empty? If no I would rather not have it as it adds noise when reading the md files directly...

If no language is declared, hljs will try to guess it, which leads to incorrect and annoying highlighting, especially for ASCII graphs or syntax definitions.

I've found a better solution: 685f906
Leaving it empty now works as expected, ie it disables highlighting.

TheElectronWill · 2019-09-17T16:47:59Z

@OlivierBlanvillain conflict resolved

TheElectronWill · 2019-09-19T12:37:29Z

ping @OlivierBlanvillain before more conflicts show up. I've replaced the previous merges by a much cleaner rebase.

OlivierBlanvillain · 2019-09-19T13:03:56Z

The conflicts are coming in faster than the time it takes for a CI build to complete, sorry about that... If you're tiered of rebasing the entire thing you could also try isolating the superficial changes too docs into separate PR.

TheElectronWill · 2019-09-19T13:16:18Z

Rebasing is actually pretty fast and I don't mind doing it multiple times :)
If I can't keep up by tomorrow I'll open another PR but hopefully it should be fine.

An excessive ".." in baseUrl caused errors when using the generated website locally

With bootstrap, popper should be used instead of tether. If required the bootstrap bundle can be added later. See https://getbootstrap.com/docs/4.3/getting-started/introduction/#js

FontAwesome: 4.7.0 -> 5.9.0 Bootstrap: 4.0.0-alpha5 -> 4.3.1 jQuery: 3.1.1 -> 3.4.1 hljs: 9.9.0 -> 9.15.9 Note: fontawesome's css requires several fonts so it seems useless to put the .css alone in the resources. Note 2: some (unused and unlikely to be used) languages have been removed from highlight.pack.js. They can be added as needed in javascript (see https://highlightjs.readthedocs.io/en/latest/api.html#registerlanguage-name-language)

- add plain scala logo - allow caching - prevent css modifications

The condition `{% if site.projectUrl %}` only filters out `null` values, not empty ones.

For instance, in `reference/other-new-features`, `opaques-details.html` doesn't appear in the sidebar but is "linked" to `opaques.html` and therefore should expand the same hierarchy.

A previous commit disabled indentation-based code blocks because it's incompatible with liquid template inclusion. This commit replaces all indented code blocks by backticks-surrounded code blocks. It also adds missing language declarations (for syntax highlighting) and removes some annoying extra indentation.

TheElectronWill · 2019-09-19T20:37:11Z

@OlivierBlanvillain all green

smarter · 2019-09-19T21:14:51Z

Awesome contribution, thank you! ✨ The blog post for the 0.19.0-RC1 release is currently being written at #7269, would you mind contributing a section on the dottydoc redesign ? (since you can't push to the PR, you can just send a markdown comment)

There is a feature request for implementing Liquid inside of Dottydoc. I wonder if it should be a separate library instead, since it could be useful to other projects. Is it possible to make Dottydoc depend on an external dotty library?

Unfortunately not, we cannot allow external scala dependencies in the dotty project because we want to be able to break binary compatibility, which could break these dependencies. This could be possible if we made dottydoc an independent project instead of a part of the compiler, this is complicated right now because dottydoc uses internal compiler APIs.

To fix this a proof of concept replacement for dottydoc has been developed: https://github.com/BryanAbate/dotty/tree/tastydoc/tastydoc. Tastydoc takes .tasty files as input (which means it should be much faster since it doesn't have to run the typechecker!) and currently generates .md, but should be extended to generate HTML. It's implemented using tasty-reflect, the same API used to write macros, this should make it possible to develop it independently of the compiler.

No one is actively working on tastydoc/dottydoc currently, so if you're interested in contributing more, we'd be more than happy to guide you !

TheElectronWill · 2019-09-20T09:42:57Z

I've written a small section for the blog post.

To fix this a proof of concept replacement for dottydoc has been developed: https://github.com/BryanAbate/dotty/tree/tastydoc/tastydoc. Tastydoc takes .tasty files as input (which means it should be much faster since it doesn't have to run the typechecker!) and currently generates .md, but should be extended to generate HTML.

Very interesting, using TASTy instead of the compiler internals seems future-proof. I'll take a look at the APIs. Is tasty-reflection-exercise up-do-date?

No one is actively working on tastydoc/dottydoc currently, so if you're interested in contributing more, we'd be more than happy to guide you !

I'll be happy to contribute more!

IIUC, tastydoc is intended to eventually replace dottydoc, so I should focus on tastydoc?
In that case, maybe it could have its own repository lampepfl/tasty-doc?

smarter · 2019-09-20T10:22:04Z

Very interesting, using TASTy instead of the compiler internals seems future-proof. I'll take a look at the APIs. Is tasty-reflection-exercise up-do-date?

I don't think so, @nicolasstucki can you point @TheElectronWill to up-to-date tasty-reflect resources ?

IIUC, tastydoc is intended to eventually replace dottydoc, so I should focus on tastydoc?

I think that's a good idea yes.

In that case, maybe it could have its own repository lampepfl/tasty-doc?

We could do that yes, what do you think @AleksanderBG ?

nicolasstucki · 2019-09-20T11:37:22Z

I will try to update the exercise soon.

The best examples are probably in the community build

https://github.com/lampepfl/dotty/tree/master/community-build/community-projects
- XML interpolator
- scalatest
- Source code
- Semantic db
f interpolator https://github.com/lampepfl/dotty/blob/master/library/src-bootstrapped/dotty/internal/StringContextMacro.scala

TheElectronWill · 2019-09-23T10:54:36Z

Thank you! I'll look at these examples and play with TASTy to get used to it.

abgruszecki · 2019-09-23T11:01:55Z

@smarter @TheElectronWill Tastydoc is tied to Dotty's build, so I'm not sure if having it in a separate repo is the best approach. Last version can be found here https://github.com/BryanAbate/dotty/tree/tastydoc. Since AFAIU the entire codebase is in the tastydoc directory I'd say if we want to cooperate on it, we might as well just merge it with main Dotty repo and worry about properly integrating it later on. WDYT?

smarter · 2019-09-23T11:03:21Z

I'd say if we want to cooperate on it, we might as well just merge it with main Dotty repo and worry about properly integrating it later on. WDYT?

Sounds good to me.

TheElectronWill · 2019-09-23T11:15:04Z

LGTM too

BryanAbate · 2019-11-23T16:40:11Z

I know this PR is 2 month old but I just fell upon it now, I hope it does not bother anyone I am commenting now.

@TheElectronWill (and @AleksanderBG ) I don't know if you are still working on this but if you need any help with my proof of concept for Tastydoc I am more than happy to help. Also you should not need to understand how Tasty works to produce html as I made sure to offer a complete API on top of Tasty especially for people who want to work on a doctool without worrying to understand how Tasty works. You just need to call the TastyConsumer and then you are left with the representation API: https://github.com/BryanAbate/dotty/blob/tastydoc/tastydoc/src/dotty/tastydoc/representations.scala. You can give a look to the main class for a complete example.

Also, more oriented to Alex, if you need help with integration/something else, I am still more than happy to help to be able to merge the work with dotty.

TheElectronWill · 2019-11-24T16:42:28Z

Thank you @BryanAbate! I didn't have time to work on this since September but it's still on my TODO list. It's great that you've made an API that abstracts Tastydoc! That will make things easier, I'll definitely check it out.

dottybot reviewed Sep 1, 2019

View reviewed changes

TheElectronWill force-pushed the responsive-docs branch 2 times, most recently from d4ca271 to 5655153 Compare September 1, 2019 21:52

julienrf reviewed Sep 2, 2019

View reviewed changes

TheElectronWill force-pushed the responsive-docs branch from 7acbee1 to c92e0a5 Compare September 2, 2019 22:22

TheElectronWill force-pushed the responsive-docs branch 2 times, most recently from 0e971b0 to e17d964 Compare September 14, 2019 23:46

TheElectronWill mentioned this pull request Sep 15, 2019

Dottydoc needs to be responsive lampepfl/dotty-feature-requests#28

Closed

OlivierBlanvillain approved these changes Sep 17, 2019

View reviewed changes

TheElectronWill force-pushed the responsive-docs branch from c8dc3f5 to 6925f4c Compare September 19, 2019 12:33

TheElectronWill force-pushed the responsive-docs branch from 6925f4c to 3ace390 Compare September 19, 2019 13:13

TheElectronWill added 9 commits September 19, 2019 19:06

Fix site base url

9b67b1a

An excessive ".." in baseUrl caused errors when using the generated website locally

Remove unused tether js library

8904c2d

With bootstrap, popper should be used instead of tether. If required the bootstrap bundle can be added later. See https://getbootstrap.com/docs/4.3/getting-started/introduction/#js

Detect gitlab repositories and use the appropriate icon

edf0511

Add Scala and Dotty logos to resources

02315db

- add plain scala logo - allow caching - prevent css modifications

Add dottydoc setting -project-logo

54fcdb6

Hide empty/unspecified projectUrl and projectLogo

efc6664

The condition `{% if site.projectUrl %}` only filters out `null` values, not empty ones.

Customize Bootstrap for dottydoc

66fbd68

Rework the toolbar

fe70297

TheElectronWill added 12 commits September 19, 2019 19:07

Display search results in tabs on small screens

0c86711

Fix search

766746c

Update dottydoc tests

be2f01b

Replace const by var in js script

65e7630

Fix supertypes and annotations lists

9f4c1ad

Statically expand the sidebar

ac9fd79

Prevent code hilighting from modifying the padding of blocks

53f72e2

Fix api table on webkit and align it to the right

3281402

Add many fallback fonts to reduce the flash of unstyled text

ab61ae2

Add a script to easily update the default Bootstrap theme

5498efe

Fix sidebar expansion for "details" doc pages

c801013

For instance, in `reference/other-new-features`, `opaques-details.html` doesn't appear in the sidebar but is "linked" to `opaques.html` and therefore should expand the same hierarchy.

TheElectronWill force-pushed the responsive-docs branch from 3ace390 to 052c2cf Compare September 19, 2019 17:12

Disable code block highlighting when no language is specified

685f906

TheElectronWill force-pushed the responsive-docs branch from 2102ca1 to 685f906 Compare September 19, 2019 19:26

smarter merged commit 6eb5404 into scala:master Sep 19, 2019

TheElectronWill mentioned this pull request Sep 20, 2019

0.19.0-RC1 Blog Article #7269

Merged

anatoliykmetyuk added this to the 0.19 Tech Preview milestone Sep 20, 2019

smarter mentioned this pull request Nov 20, 2019

Please Revert Dotty Doc Style #7595

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix lampepfl/dotty-feature-requests#28: make Dottydoc responsive #7153

Fix lampepfl/dotty-feature-requests#28: make Dottydoc responsive #7153

TheElectronWill commented Sep 1, 2019

dottybot left a comment

TheElectronWill commented Sep 1, 2019

julienrf left a comment

julienrf Sep 2, 2019

TheElectronWill Sep 2, 2019

julienrf Sep 2, 2019

julienrf commented Sep 2, 2019

TheElectronWill commented Sep 2, 2019 •

edited

Loading

OlivierBlanvillain commented Sep 6, 2019

TheElectronWill commented Sep 7, 2019 •

edited

Loading

TheElectronWill commented Sep 14, 2019 •

edited

Loading

OlivierBlanvillain left a comment

OlivierBlanvillain Sep 17, 2019

TheElectronWill Sep 17, 2019 •

edited

Loading

TheElectronWill Sep 19, 2019 •

edited

Loading

TheElectronWill commented Sep 17, 2019

TheElectronWill commented Sep 19, 2019

OlivierBlanvillain commented Sep 19, 2019

TheElectronWill commented Sep 19, 2019

TheElectronWill commented Sep 19, 2019

smarter commented Sep 19, 2019 •

edited

Loading

TheElectronWill commented Sep 20, 2019

smarter commented Sep 20, 2019

nicolasstucki commented Sep 20, 2019

TheElectronWill commented Sep 23, 2019

abgruszecki commented Sep 23, 2019

smarter commented Sep 23, 2019

TheElectronWill commented Sep 23, 2019

BryanAbate commented Nov 23, 2019

TheElectronWill commented Nov 24, 2019

Fix lampepfl/dotty-feature-requests#28: make Dottydoc responsive #7153

Fix lampepfl/dotty-feature-requests#28: make Dottydoc responsive #7153

Conversation

TheElectronWill commented Sep 1, 2019

Bootstrap theme

Scala changes

Visual changes

Front page

Blog page

Sidebar and toolbar

API Search

API documentation

dottybot left a comment

Choose a reason for hiding this comment

Commit Messages

TheElectronWill commented Sep 1, 2019

julienrf left a comment

Choose a reason for hiding this comment

julienrf Sep 2, 2019

Choose a reason for hiding this comment

TheElectronWill Sep 2, 2019

Choose a reason for hiding this comment

julienrf Sep 2, 2019

Choose a reason for hiding this comment

julienrf commented Sep 2, 2019

TheElectronWill commented Sep 2, 2019 • edited Loading

OlivierBlanvillain commented Sep 6, 2019

TheElectronWill commented Sep 7, 2019 • edited Loading

TheElectronWill commented Sep 14, 2019 • edited Loading

OlivierBlanvillain left a comment

Choose a reason for hiding this comment

OlivierBlanvillain Sep 17, 2019

Choose a reason for hiding this comment

TheElectronWill Sep 17, 2019 • edited Loading

Choose a reason for hiding this comment

TheElectronWill Sep 19, 2019 • edited Loading

Choose a reason for hiding this comment

TheElectronWill commented Sep 17, 2019

TheElectronWill commented Sep 19, 2019

OlivierBlanvillain commented Sep 19, 2019

TheElectronWill commented Sep 19, 2019

TheElectronWill commented Sep 19, 2019

smarter commented Sep 19, 2019 • edited Loading

TheElectronWill commented Sep 20, 2019

smarter commented Sep 20, 2019

nicolasstucki commented Sep 20, 2019

TheElectronWill commented Sep 23, 2019

abgruszecki commented Sep 23, 2019

smarter commented Sep 23, 2019

TheElectronWill commented Sep 23, 2019

BryanAbate commented Nov 23, 2019

TheElectronWill commented Nov 24, 2019

TheElectronWill commented Sep 2, 2019 •

edited

Loading

TheElectronWill commented Sep 7, 2019 •

edited

Loading

TheElectronWill commented Sep 14, 2019 •

edited

Loading

TheElectronWill Sep 17, 2019 •

edited

Loading

TheElectronWill Sep 19, 2019 •

edited

Loading

smarter commented Sep 19, 2019 •

edited

Loading