Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add & update official documentation #40

Closed
wants to merge 4 commits into from
Closed

Add & update official documentation #40

wants to merge 4 commits into from

Conversation

IlnarSelimcan
Copy link
Member

@IlnarSelimcan IlnarSelimcan commented Mar 4, 2019 •
edited

TL,DR several prospective GSoC students already have asked me how-up-to date the official documentation [1] was. There were a few additions/changes over the years the docs don't mention, so I set out to re-read and update it. Any help with this effort is appreciated. To see how it looks currently, visit https://taruen.github.io/apertium/ https://taruen.github.io/organisation/index/ (see discussion below why this change). If this PR gets approved (and https://github.com/apertium/apertium/settings https://github.com/apertium/organisation/settings gets changed to serve documentation from the docs/ folder), it will become available at https://apertium.github.io/apertium/ https://apertium.github.io/organisation/)

I plan to "port"/copy-paste the official documentation from the .tex files in the coming days. It's better idea to merge this PR after that. Before that, here we can discuss it.

The official documentation has been published in 2010. That was quite a while ago, and although the core things are still the same, there were quite a few additions since then. Off the top of my head the most notable are:

  • apertium-lex-tools,
  • many languages started using Constraint Grammar-based disambiguators, or HFST
  • monolingual modules were separated out and put in into apertium-languages/
  • apertium-separable,
  • nightly binary packages are available now,
  • apertium-apy,
  • other GSoC projects like Python library for Apertium and weighted automata.

There isn't a document which would describe these changes (or at least give pointers to their documentation, if any) in a single place.

Without such, it's rather difficult to keep overview of the code and possibilities available.

"Current limitations" would also be a great chapter to have, so that it's clear where to go next, and easier for prospective GSoC students to understand the problems they want to solve.

Wiki has a ton of great info, I'm not saying that it is bad. But I think that the usefulness of a book-like, "you-should-read-this-and-this-chapters first, and then this other chapter"-kind-of-documentation, with "if you need more info on this topic, see this and this wiki pages" notes is generally acknowledged.

The documentation tool I've chosen is called Scribble. [2] Scribble documents can compile into html, latex, pdf, markdown and probably several other formats. It is relatively easy to pick up, but quite powerful. Its power comes from the fact that a whole programming language is available to the writer anywhere in the document. You can think of it as a a modern texinfo [3], with the difference that you can insert arbitrary code (for logic, automation etc.) into the documentation itself.

Another great feature of it is that it can figure out where a function being mentioned in the documentation was defined, and add a cross-reference to its documentation automatically. But this feature is not very relevant, unless you write Racket code, so I'll spare the details.

The choice of the documentation tool/format is debatable, of course. The main output format, though, in my opinion has to be in html. The reason for that is that in html we'll be able to insert things like videos or asciicinemas. [4] A good example (at least visually) I'm following as a model is spacy.io's documentation. [5]

[1] http://xixona.dlsi.ua.es/~fran/apertium2-documentation.pdf
[2] https://docs.racket-lang.org/scribble/index.html
[3] https://www.gnu.org/software/texinfo/
[4] https://asciinema.org/
[5] https://spacy.io/

@TinoDidriksen
Copy link
Member

I would prefer if this was not part of a code repository. https://github.com/apertium/organisation would be better, or a dedicated documentation repo.

@unhammer
Copy link
Member

unhammer commented Mar 5, 2019

This is great @IlnarSelimcan ! The scribble output looks nice (and even without trying it I feel 99% sure it has better error messages than latex).

The current documentation also doesn't include

  • the compounding feature of lttoolbox
  • lt-trim
  • unicode support(!)
  • tracing (-t) in transfer

@mlforcada
Copy link
Member

This is excellent, thanks for the initiative. I agree with Tino that this should be a different repository as he suggests.

@flammie
Copy link
Member

flammie commented Mar 5, 2019

Looks cool, this is definitely a needed improvement.
Why not use a repo named "apertium.github.io" for the docs that affect multiple packages, this is pretty much the standard for other github projects aiui?

I actually do also like having gh-pages in docs directory, but that is more for stuff that is tightlier coupled to the current code version and the specific package and details that really need to change with the code. Like you could separate a stable overview of what apertium-lex-tools is in these new main doc, and all the nittygritty details and hacky scripts there are in the apertium-lex-tools in its own gh-pages docs.

(I don't want to bikeshed on implementations details but what is this racket scribble and is anyone using it? It doesn't seem too scary but is there a risk that it's gonna be dead in 3 months and nobody can update docs anymore =)

@unhammer
Copy link
Member

unhammer commented Mar 5, 2019

I don't want to bikeshed either, but™ scribble is almost exactly ten years older than apertium (at least according to initial git commits). If it makes selimcan feel happy while writing documentation, I'm all for it.

@IlnarSelimcan
Copy link
Member Author

IlnarSelimcan commented Mar 5, 2019 •
edited

Thanks everyone for the feedback and encouragement!

In short, the consensus is clear: this "project-wide" documentation -- since it's not only about the apertium program proper -- belongs to a different repo.

https://github.com/apertium/organisation seems to be the right place.

Therefore, I close this pull request, and then (if you don't mind) will:

  • fork https://github.com/apertium/organisation
  • put the documentation into the docs/ folder
  • make another PR there (in a non-master branch initially, as soon as the first draft with an outline of new parts will be ready).

Once pushed to the master branch, the documentation will show up at https://apertium.github.io/organization/

Then it's minor details for admins to decide whether to rename the organization repo to apertium.github.io (I like that suggestion), or setup some custom subdomain like docs.apertium.org which points to https://apertium.github.io/organization/. I think these thoughts fall under the category of "bikeshedding", a new cool word with a history that I have just learned.

I also like @flammie's point that particular packages can have their own documentation, coupled to their particular release. Apertium-tat has a sketch of one at https://apertium.github.io/apertium-tat/.

As for the Scribble, I think that until now it has mostly been used to document Racket packages or to write books about that language (which btw started out as a Scheme dialect in the early 90s and was called PLT Scheme initially). Which means that the community (although rather small compared to mainstream languages) has been around for a while and I don't think that they will disappear.

Anyway, I'm quite open-minded and don't insist on Scribble too strongly. Let's see how it goes. A relevant detail about it is that it can output Markdown. Which means that if we decide to ditch Scribble and switch to, say, Markdown, we will be able to do so easily. Upon request, I can keep both versions in the repo and in sync.

@unhammer unhammer mentioned this pull request Mar 15, 2019
Closed
This was referenced Apr 8, 2019
Merged
Open
@TinoDidriksen
Copy link
Member

Because of how Github Pages works, documentation repo is: https://github.com/apertium/apertium.github.io

@hectoralos
Copy link
Member

hectoralos commented Apr 10, 2019 via email

@IlnarSelimcan
Copy link
Member Author

That's my bad, @hectoralos . It should not refer to apertium-lex-tools the way it does, and this would fail no matter where this repo is. I'll fix it.

@IlnarSelimcan
Copy link
Member Author

IlnarSelimcan commented Apr 10, 2019 •
edited

@hectoralos This has been fixed now.

Just this one time, you will need to run raco pkg remove docs-source.

Then cd docs-source ; make docs should generate html files without any issues.

If any issues, please report at https://github.com/apertium/apertium.github.io/issues

The documentation is deployed to https://apertium.github.io

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@IlnarSelimcan @TinoDidriksen @unhammer @mlforcada @flammie @hectoralos