Skip to content
Permalink
Browse files
Write some basic site development instructions
  • Loading branch information
djencks committed Jul 22, 2021
1 parent d9197ee commit f5ab68cda86bb06a8bac1859258cefe081a87df8
Showing 2 changed files with 87 additions and 157 deletions.
@@ -17,7 +17,7 @@ The basic steps for releasing are:
* <<#_create_an_announcement,Announce the release>>

The Apache Felix PMC appreciates all committers playing release manager for the projects they maintain.
Due to restrictions imposed on us, only members of the PMC are actually able to commit the releases to the distribution repository (See <<promoting-the-release,Publish the release candidate>>).
Due to restrictions imposed on us, only members of the PMC are actually able to commit the releases to the distribution repository (See <<_promoting_the_release,Publish the release candidate>>).
In this case please ask a member of the PMC to actually publish the release artifacts.
Thanks.

@@ -280,10 +280,12 @@ NOTE: with Maven 3, you must add an extension to your `<build>` providing the SC

=== Update the Site

. Update the news section on the website at https://cms.apache.org/redirect?uri=http%3A//felix.apache.org/news.html[news];
. Update the download page on the website at https://cms.apache.org/redirect?uri=http%3A//felix.apache.org/downloads.list[downloads] to point to the new release;
. Commit your changes (click the commit link);
. https://cms.apache.org/felix/publish[Publish the site].
The appropriate parts of the website are in the https://github.com/apache/felix-antora-site repository or the Apache "original" copy.

. Update the news section on the website at xref:news.adoc[];
. Update the appropriate entry in at modules/ROOT/examples/downloads.yml[] to point to the new release;
. Commit your changes and push to GitHub (or the Apache mirror);
. The published site will be updated by the Jenkins site build jobs.

For the last two tasks, it's better to give the mirrors some time to distribute the uploaded artifacts (one day should be fine).
This ensures that once the website (news and download page) is updated, people can actually download the artifacts.
@@ -294,6 +296,8 @@ Go to https://issues.apache.org/jira/secure/project/ViewProject.jspa?pid=1231010

== Publish Generated Documentation

NOTE: This doesn't seem to relate well to the state of the CMS site.

This procedure applies currently only to the `maven-bundle-plugin`:

. Checkout the release tag of the plugin
@@ -1,181 +1,107 @@
= Site How To

WARNING: This page is totaly wrong and needs to be rewritten.
The website is written in https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc], built using https://docs.antora.org/[Antora], and published using an https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-WebSiteDeploymentServiceforGitRepositories[`.asf.yaml`] file in a git repo.

The site is managed with the https://www.apache.org/dev/cms.html[Apache CMS] where the source is kept in SVN at https://svn.apache.org/repos/asf/felix/site/trunk/content.
== Overall organization

== To update the documentation using the CMS system
For simplicity, this references the GitHub mirrors of the Apache gitbox repos.

* Install the bookmarklet from the https://cms.apache.org/[cms] page.
You only have to do this once.
* Navigate to the page you wish to edit (on the live site, not in the cms).
* Click the bookmarklet.
There will be a short pause while the CMS system is initialised for you.
* Click on `Edit` (to skip this step hack the bookmarklet to add an 'action=edit' param to the query string)
* The page editor should then be displayed.
* Click `Submit` to save your edit to the workarea
* Click `Commit` to save the updated file to SVN and trigger a staged build.
(to skip this step click on the "Quick Commit" checkbox in the `Edit` form).
* The results should appear shortly on the http://felix.staging.apache.org/content/documentation.html[staging] site.
(You may have to force the page to refresh in order to see the updated content)
* Once you are happy with the updated page, click on `Publish Site` to deploy.
General content::
Content not specific to a particular subproject and unmigrated content is in the https://github.com/apache/felix-antora-site[felix-antora-site] repository.
At the moment there is only one Antora component, version, and module, so the content is all under `modules/ROOT/pages`.

There is also a https://www.apache.org/dev/cmsref.html[Reference Manual] of the Apache CMS as well as a video tutorial at http://s.apache.org/cms-tutorial.
Subproject-specific content::
Content specific to a subproject may be put in the subproject repo in the standard Antora layout.
For instance, SCR content could be in `felix-dev` under `scr/docs/modules/ROOT/pages`.
This feature is not yet used.

== Features of the Apache Felix Site
Antora playbook::
The Antora playbook together with the Node project to fetch and set up the required dependencies is in the https://github.com/apache/felix-antora[felix-antora] repository.

This section lists some Apache Felix features to help with the maintenance of the site, such as automatic link generation.
Antora UI::
The UI for this website is in the https://github.com/apache/felix-antora-ui[felix-antora-ui] repository.
For simplicity this is currently set up as an extension of the Antora default UI using https://gitlab.com/djencks/antora-ui-builder[].

Start the file with a `Title:` line to define the page title and the first H1 tag:
Published site content::
The website build publishes the content to https://github.com/apache/felix-site-pub[felix-site-pub] in the `asf-site` branch.
The `.asf.yaml` file is configured to publish the content to Apache servers.

----
= Page Title
Here comes the content separated with a blank like from the
header ...
----

The last modification information from SVN (revision, committer, and date/time) is automatically added when the page is rendered

Excerpts can be added to a page using the `Excerpt:` header:

= Page Title
Excerpt: Summary of the page for inclusion in other pages;
continuation of the excerpt must be indented

Here comes the content separated with a blank like from the
header ...

Metadata from child pages can be referred to in the content with the Django variable reference notation using the child page name (without extension) as its container;
e.g.
for the child page named `childpage`:

:::django
{{ y|default:"{{" }} children.childpage.headers.excerpt }}
{{ y|default:"{{" }} children.childpage.headers.title }}

Content Pages can contain Django templates of the form `+{{ y|default:"{{" }}...}}+` and `+{{ y|default:"{%" }}...%}+`.
If so, the page content is evaluated as a Django template before running it through the page template.

Any page in the site can be referenced with refs.pagename returning properties:

`.path`:: the absolute path of the page on the site

`.headers`::
page headers (e.g.
`.title`, `.excerpt`)

`.content`:: the raw page content

All pages in the children namespace are also available in the refs namespace

Some usefull hints:

Printing title of another page "handler":

:::django
{{ y|default:"{{" }} refs.handler.headers.title }}

Printing excerpt of another page "handler":
Automated site build::
The site is built using Jenkins jobs https://ci-builds.apache.org/job/Felix/job/website-build[Felix/website-build] and https://ci-builds.apache.org/job/Felix/job/website-content-trigger[Felix/website-content-trigger].
Committing to the `felix-antora-site` or `felix-antora` repositories will trigger a website build and publish.
Changes to the UI will not trigger an automated build, but PMC members may trigger a build manually with the https://ci-builds.apache.org/job/Felix/job/website-build/build?delay=0sec[Build Now] button.
If subproject content is moved to subproject repositories more trigger jobs will be needed.

:::django
{{ y|default:"{{" }} refs.handler.headers.excerpt }}
== Extensions currently in use

Linking to another page "handler":
https://github.com/Mogztter/asciidoctor-kroki[asciidoctor-kroki]::
This provides access to numerous text-to-diagram converters.
Currently it is used for some UMLet diagrams in the DependencyManager documentation.

:::django
({{ y|default:"{{" }} refs.handler.path }})
https://gitlab.com/djencks/asciidoctor-antora-indexer[asciidoctor-antora-indexer]::
This queries the Antora content catalog and renders the results as lists, tables, etc.
It is currently used in the xref:site-map.adoc[] page to partially automate generation.

Printing title as a link to another page "handler":
https://gitlab.com/djencks/asciidoctor-jsonpath[asciidoctor-jsonpath]::
This queries a json, yaml, or toml document and renders the results as lists, tables, etc.
It is currently used to generate the content on the downloads page.

:::django
[{{ y|default:"{{" }} refs.handler.headers.title }}]({{ y|default:"{{" }} refs.handler.path }})
== Basic AsciiDoc

Printing excerpt as a link to another page "handler":
The Intellij AsciiDoc plugin provides extensive syntax checking, side-by-side rendering, and a lot of other help.

:::django
[{{ y|default:"{{" }} refs.handler.headers.excerpt }}]({{ y|default:"{{" }} refs.handler.path }})
Consult the Asciidoctor and Antora documentation for complete coverage of syntax and features.

Print a bullet pointed child page list:
Please use "ventilated prose", with one sentence per line.

:::django
{{ y|default:"{%" }} for label, page in children %}* [{{ y|default:"{{" }} page.headers.title }}]({{ y|default:"{{" }} page.path }})
{{ y|default:"{%" }} endfor %}

It is important to have the first part as a single line, otherwise the Django/Markdown combo will create a list for each entry.

== Code Highlighting

Code Highlighting works by indenting code by four blanks.
To indicate the type of highlighting preced the code style text with either `:::<lexer>` to get high lighted code using the given `<lexer>` or `#!<lexer>` to get high lighted code with line numbers using the given `<lexer>`.
See http://www.apache.org/dev/cmsref.html#code-hilighter for main info and http://pygments.org/docs/lexers/ for supported lexers

== HTML and Markdown

Markdown supports embedding HTML.
But be aware that contents of HTML elements are not further handled by the Markdown converter.
Thus it is not possible to embed Markdown syntax inside of HTML elements to have them converted.

== Manual Generation

When commiting changes to pages into SVN the pages are automatically generated in http://felix.staging.apache.org[the staging site].<div class="info">To generate the site locally, you must have checked out the complete SVN to access the tools:<div class="codehilite">$ svn https://svn.apache.org/repos/asf/felix/site/ felix-site</div></div>

To manually generate the site or single pages the http://svn.apache.org/repos/asf/felix/site[site] can be checked out from SVN.
In addition Perl and Python must be installed for the build tools to work.

To prepare for site build, the Markdown daemon has to be started:

[source,sh]
$ export MARKDOWN_SOCKET="$PWD/tools/build/../markdown.socket"
$ export PYTHONPATH="$PWD/tools/build"
$ python "$PWD/tools/build/markdownd.py"

The `MARKDOWN_SOCKET` environment variables is also required by the `build_site.pl` and `build_file.pl` scripts to connect to the Markdown daemon.

To build the complete site use the `build_site.pl` script:

[source,sh]
$ tools/build/build_site.pl --source-base $PWD/trunk \
--target-base $PWD/trunk/target

To build a single page use the `build_file.pl` script:

[source,sh]
$ tools/build/build_site.pl --source-base $PWD/trunk \
--target-base $PWD/trunk/target \
--source content/documentation.mdtext

The argument to the `--source` parameter is relative to the `--source-base` folder.
* A blank line indicates a new paragraph.
* A page starts with a title, indicated by `= Title`.
* Sections are marked with `==`, `===`, etc, the number of `=` indicating section nesting.
* Sections are automatically assigned ids that can be used as the fragment in links.
To manually specify an id, precede the section line with `[#my-id]` on a separate line.
* Code blocks look like this:
[source,adoc]
------
[source,java]
----
class Foo {
...
}
----
------

== Configuring site generation on Mac
== Local build

Those instructions were computed on Mountain Lion.
You need Node and npm installed.
You also need git to clone the repositories locally, but the build itself does not use git.

A couple of Python and Perl libraries are required and need to be installed
I do not recommend following the instructions on the Antora site, which advise installing Antora globally.
The playbook project is instead set up to use an npm script to install Antora and all required dependencies locally
At a minimum:

[source,sh]
$ sudo easy_install Pygments
$ sudo easy_install Markdown
* Clone the `felix-antora` playbook project.
* Run `npm run plain-install`.
* Run `npm run build`.

And for the Perl modules:
This will build the site locally into build/site, using the remote content sources, such as `felix-antora-site`.

[source,sh]
$ sudo cpan install XML::Atom::Feed
$ sudo cpan install XML::RSS::Parser
$ sudo cpan install XML::Parser::Lite
$ sudo cpan install XML::RSS::Parser::Lite
$ sudo cpan install Net::Twitter
$ sudo cpan install SVN::Client
Most likely you will want to modify content sources locally, and build using your local sources, before committing and pushing your changes.
To enable this:

Be careful that some of those commands require time...
Once done, launch the mardown daemon with the following command from the SVN root:
* Clone the `felix-antora-site` repository and any other content repositories you wish to work with next to the playbook project.
* Modify the `antora-playbook.yml` to point to the local clones rather than the remote repositories.
Be sure to specify `branches: HEAD` to use the worktree content rather than the local committed repository content.
The appropriate changes are already in the playbook, commented out, with comments, for the `felix-antora-site` repository.
* Build the site as above.

[source,sh]
$ export MARKDOWN_SOCKET="$PWD/tools/build/../markdown.socket"
$ export PYTHONPATH="$PWD/tools/build"
$ python "$PWD/tools/build/markdownd.py"
=== Working with the UI

And finally, generate the web site from the svn root with:
* Clone the `felix-antora-ui` project next to the playbook project.
* Run `npm i`.
* Run `./node_modules/gulp/bin/gulp.js` to assemble the UI.
* To see the results, run `cp build/felix-antora-ui-bundle.zip ../felix-antora/node_modules/@apache-felix/felix-antora-ui/build/felix-antora-ui-bundle.zip` and build the playbook project.
* When you are satisfied with your changes, commit both source changes and the assembled UI bundle and push to the remote repository.
* UI bundle changes do not trigger a site build; you'll need to start one manually.

[source,sh]
tools/build/build_site.pl --source-base $PWD/trunk --target-base $PWD/trunk/target
Most likely making changes to src/partials/* templates will be fairly self-explanatory.
Other changes are likely to need a fairly detailed understanding of the Antora default UI.

0 comments on commit f5ab68c

Please sign in to comment.