Skip to content

Commit

Permalink
[docs] The Markdown document generator parses the pure HTML "a" hrefs.
Browse files Browse the repository at this point in the history 
Signed-off-by: Stéphane Galland <galland@arakhne.org>
  • Loading branch information
@gallandarakhneorg
gallandarakhneorg committed Sep 12, 2017
1 parent 374ca21 commit b809dee
Show file tree
Hide file tree
Showing 7 changed files with 181 additions and 25 deletions.
5 changes: 4 additions & 1 deletion docs/io.sarl.docs.markdown/pom.xml
View file
Expand Up @@ -17,6 +17,8 @@
<properties>
<!-- URL of the maven repository of Janus -->
<janus.maven.repository.url>http://maven.janusproject.io</janus.maven.repository.url>
<!-- Are the links' addresses converted from a Markdown format to a HTML format? -->
<convert.links>true</convert.links>
</properties>

<dependencies>
Expand Down Expand Up @@ -85,7 +87,8 @@
<addYamlHeader>true</addYamlHeader>
<githubExtension>true</githubExtension>

<md2html>true</md2html>
<md2html>${convert.links}</md2html>
<transformPureHtmlLinks>${convert.links}</transformPureHtmlLinks>

<autoSectionNumbering>true</autoSectionNumbering>
<addSectionLabels>true</addSectionLabels>
Expand Down
2 changes: 1 addition & 1 deletion docs/io.sarl.docs.markdown/src/main/documentation/faq/GeneralFAQ.md
View file
Expand Up @@ -29,7 +29,7 @@ Unlike other JVM languages, SARL has zero interoperability issues with Java: eve
__The language is platform- and architecture-independent.__

For a brief comparison between SARL, Java and Xtend languages, see the Section
"[Comparison between SARL, Java and Xtend](../reference/OOP.md#comparison-between-sarl-java-and-xtend)".
"[Comparison between SARL and other languages](../reference/OOP.md#comparison-between-sarl-and-other-languages)".


### Can I use SARL to make agent-based software?
Expand Down
14 changes: 7 additions & 7 deletions docs/io.sarl.docs.markdown/src/main/documentation/reference/OOP.md
View file
Expand Up @@ -16,7 +16,7 @@ complete support of the OOP, you should use a dedicated language, such as Java,
<note>The SARL Eclipse product includes the tools for programming with the Java and Xtend languages.</note>


## Comparison between SARL, Java and Xtend
## Comparison between SARL and Other Languages

<a href="https://en.wikipedia.org/wiki/Java_(programming_language)">Java</a>, [Xtend](https://www.eclipse.org/xtend/) and
[Scala](http://scala-lang.org/) are object-oriented programming languages.
Expand All @@ -34,14 +34,14 @@ environment (Eclipse, IntelliJ...)
<td style="background: green; color: white;">Yes</td>
<td style="background: orange; color: white;">No</td>
<td style="background: orange; color: white;">No</td>
<td style="background: yellow; color: darkgray;">Partial: actor paradigm</td></tr>
<td style="background: yellow; color: black;">Partial: actor paradigm</td></tr>
<tr><td colspan=5><strong>Object-oriented programming</strong></td></tr>
<tr><td><a href="./OOP.md">Definition of class and interface types</a></td>
<td style="background: green; color: white;">Yes</td>
<td style="background: green; color: white;">Yes</td>
<td style="background: green; color: white;">Yes</td>
<td style="background: green; color: white;">Yes</td></tr>
<tr><td><a href="#Enumeration>Object-oriented enumeration</a></td>
<tr><td><a href="#Enumeration">Object-oriented enumeration</a>
<td style="background: orange; color: white;">No, only constants could be defined</td>
<td style="background: green; color: white;">Yes, constants and functions could be defined</td>
<td style="background: orange; color: white;">No, only constants could be defined</td>
Expand All @@ -55,7 +55,7 @@ environment (Eclipse, IntelliJ...)
<td style="background: green; color: white;">Yes</td>
<td style="background: green; color: white;">Yes</td>
<td style="background: orange; color: white;">No</td>
<td style="background: yellow; color: darkgray;">See companion object</td></tr>
<td style="background: yellow; color: black;">See companion object</td></tr>
<tr><td>Automatic generation of <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#equals-java.lang.Object-"><code>equals()</code></a>
and <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#hashCode--"><code>hashCode()</code></a></td>
<td style="background: green; color: white;">Yes</td>
Expand All @@ -73,7 +73,7 @@ environment (Eclipse, IntelliJ...)
<td style="background: green; color: white;">Yes</td>
<td style="background: orange; color: white;">No</td>
<td style="background: orange; color: white;">No</td>
<td style="background: yellow; color: darkgray;">Manual with <code>@SerialVersionUID</code></td></tr>
<td style="background: yellow; color: black;">Manual with <code>@SerialVersionUID</code></td></tr>
<tr><td colspan=5><strong>Functions, Procedures and Operators</strong></td></tr>
<tr><td><a href="./general/FuncDecls.md#dispatch-function">Definition of dispatch functions</a></td>
<td style="background: green; color: white;">Yes</td>
Expand All @@ -95,12 +95,12 @@ environment (Eclipse, IntelliJ...)
<td style="background: orange; color: white;">No</td>
<td style="background: orange; color: white;">No</td>
<td style="background: green; color: white;">Yes</td></tr>
<tr><td><a href="./general/Operators.md##operator-overloading">Operator overloading</a></td>
<tr><td><a href="./general/Operators.md#operator-overloading">Operator overloading</a></td>
<td style="background: green; color: white;">Yes</td>
<td style="background: orange; color: white;">No</td>
<td style="background: green; color: white;">Yes</td>
<td style="background: green; color: white;">Yes</td></tr>
<tr><td>Automatic detection of and marking <a href="http://download.eclipse.org/modeling/tmf/xtext/javadoc/2.9/org/eclipse/xtext/xbase/lib/Pure.html">pure functions</a></td>
<tr><td>Automatic detection of <a href="http://download.eclipse.org/modeling/tmf/xtext/javadoc/2.9/org/eclipse/xtext/xbase/lib/Pure.html">pure functions</a> and marking</td>
<td style="background: green; color: white;">Yes</td>
<td style="background: orange; color: white;">No</td>
<td style="background: orange; color: white;">No</td>
Expand Down
5 changes: 4 additions & 1 deletion main/externalmaven/io.sarl.maven.docs.generator/pom.xml
View file
Expand Up @@ -22,7 +22,10 @@
<dependency>
<groupId>com.vladsch.flexmark</groupId>
<artifactId>flexmark</artifactId>
<version>0.18.7</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
</dependency>

<dependency>
Expand Down
16 changes: 15 additions & 1 deletion ...rnalmaven/io.sarl.maven.docs.generator/src/main/java/io/sarl/maven/docs/GenerateMojo.java
View file
Expand Up @@ -56,11 +56,24 @@ public class GenerateMojo extends AbstractDocumentationMojo {
protected String outputDirectory;

/**
* Indicates if the Markdown references should be transformed to references to HTML files.
* Indicates if the Markdown references (addresses in links) should be transformed to
* their equivalent HTML addresses. For example "./repo/file.md" is transformed
* to "./repo/file.html".
*
* @see #transformPureHtmlLinks
*/
@Parameter(defaultValue = "true", required = false)
protected boolean md2html;

/** Indicates if the pure html references (addresses in the "a" tags) should be
* transformed to their equivalent HTML addresses. For example "./repo/file.md" is transformed
* to "./repo/file.html".
*
* @see #md2html
*/
@Parameter(defaultValue = "true", required = false)
protected boolean transformPureHtmlLinks;

/**
* Indicates if the sections should be automatically numbered.
*/
Expand Down Expand Up @@ -107,6 +120,7 @@ protected AbstractMarkerLanguageParser createLanguageParser(File inputFile) thro
mdParser.setAutoSectionNumbering(this.autoSectionNumbering);
mdParser.setOutlineDepthRange(AbstractMarkerLanguageParser.parseRange(this.outlineDepth, 1));
mdParser.setMarkdownToHtmlReferenceTransformation(this.md2html);
mdParser.setPureHtmlReferenceTransformation(this.transformPureHtmlLinks);
mdParser.setOutlineStyleId(this.outlineStyleId);
}
return parser;
Expand Down
154 changes: 140 additions & 14 deletions ...o.sarl.maven.docs.generator/src/main/java/io/sarl/maven/docs/markdown/MarkdownParser.java
View file
Expand Up @@ -23,6 +23,7 @@

import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.text.MessageFormat;
import java.util.ArrayList;
Expand Down Expand Up @@ -55,6 +56,9 @@
import org.eclipse.xtext.xbase.compiler.output.ITreeAppendable;
import org.eclipse.xtext.xbase.lib.Functions.Function2;
import org.eclipse.xtext.xbase.lib.IntegerRange;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

import io.sarl.maven.docs.bugs.FileSystemAddons;
import io.sarl.maven.docs.parser.AbstractMarkerLanguageParser;
Expand Down Expand Up @@ -133,6 +137,8 @@ public class MarkdownParser extends AbstractMarkerLanguageParser {

private boolean transformMdToHtmlReferences = true;

private boolean transformPureHtmlReferences = true;

@Override
@Inject
public void setDocumentParser(SarlDocumentationParser parser) {
Expand Down Expand Up @@ -207,6 +213,7 @@ public void setOutlineStyleId(String id) {
/** Replies if the references to the Markdown files should be transform to references to HTML pages.
*
* @return {@code true} if the references should be validated.
* @see #isPureHtmlReferenceTransformation()
*/
public boolean isMarkdownToHtmlReferenceTransformation() {
return this.transformMdToHtmlReferences;
Expand All @@ -215,11 +222,30 @@ public boolean isMarkdownToHtmlReferenceTransformation() {
/** Change the flag that indicates if the references the Markdown files should be transform to references to HTML pages.
*
* @param transform {@code true} if the references should be validated.
* @see #setPureHtmlReferenceTransformation(boolean)
*/
public void setMarkdownToHtmlReferenceTransformation(boolean transform) {
this.transformMdToHtmlReferences = transform;
}

/** Replies if the pure HTML references (in "a" tags) should be transform to references to HTML pages.
*
* @return {@code true} if the references should be validated.
* @see #isMarkdownToHtmlReferenceTransformation()
*/
public boolean isPureHtmlReferenceTransformation() {
return this.transformPureHtmlReferences;
}

/** Change the flag that indicates if the pure html references should be transform to references to HTML pages.
*
* @param transform {@code true} if the references should be validated.
* @see #setMarkdownToHtmlReferenceTransformation(boolean)
*/
public void setPureHtmlReferenceTransformation(boolean transform) {
this.transformPureHtmlReferences = transform;
}

/** Replies if the references to the local files should be validated.
*
* @return {@code true} if the references to the local files should be validated.
Expand Down Expand Up @@ -411,23 +437,75 @@ public boolean isAutoSectionNumbering() {
@Override
protected String postProcessingTransformation(String content) {
String result = updateOutline(content);
result = transformLinks(result);
result = transformMardownLinks(result);
result = transformHtmlLinks(result);
return result;
}

/** Apply link transformation.
/** Apply link transformation on the HTML links.
*
* @param content the original content.
* @return the result of the transformation.
*/
protected String transformHtmlLinks(String content) {
if (!isPureHtmlReferenceTransformation()) {
return content;
}

// Prepare replacement data structures
final Map<String, String> replacements = new TreeMap<>();

// Visit the links and record the transformations
final org.jsoup.select.NodeVisitor visitor = new org.jsoup.select.NodeVisitor() {
@Override
public void tail(org.jsoup.nodes.Node node, int index) {
//
}

@Override
public void head(org.jsoup.nodes.Node node, int index) {
if (node instanceof Element) {
final Element tag = (Element) node;
if ("a".equals(tag.nodeName()) && tag.hasAttr("href")) { //$NON-NLS-1$ //$NON-NLS-2$
final String href = tag.attr("href"); //$NON-NLS-1$
if (!Strings.isEmpty(href)) {
URL url = FileSystem.convertStringToURL(href, true);
url = transformURL(url);
if (url != null) {
replacements.put(href, convertURLToString(url));
}
}
}
}
}
};
final Document htmlDocument = Jsoup.parse(content);
htmlDocument.traverse(visitor);

// Apply the replacements
if (!replacements.isEmpty()) {
String buffer = content;
for (final Entry<String, String> entry : replacements.entrySet()) {
final String source = entry.getKey();
final String dest = entry.getValue();
buffer = buffer.replaceAll(Pattern.quote(source), Matcher.quoteReplacement(dest));
}
return buffer;
}
return content;
}

/** Apply link transformation on the Markdown links.
*
* @param content the original content.
* @return the result of the transformation.
*/
protected String transformLinks(String content) {
protected String transformMardownLinks(String content) {
if (!isMarkdownToHtmlReferenceTransformation()) {
return content;
}
final MutableDataSet options = new MutableDataSet();
final Parser parser = Parser.builder(options).build();
final Node document = parser.parse(content);

// Prepare replacement data structures
final Map<BasedSequence, String> replacements = new TreeMap<>((cmp1, cmp2) -> {
final int cmp = Integer.compare(cmp2.getStartOffset(), cmp1.getStartOffset());
if (cmp != 0) {
Expand All @@ -436,20 +514,21 @@ protected String transformLinks(String content) {
return Integer.compare(cmp2.getEndOffset(), cmp1.getEndOffset());
});

// Visit the links and record the transformations
final MutableDataSet options = new MutableDataSet();
final Parser parser = Parser.builder(options).build();
final Node document = parser.parse(content);
final NodeVisitor visitor = new NodeVisitor(
new VisitHandler<>(Link.class, (it) -> {
final URL url = FileSystem.convertStringToURL(it.getUrl().toString(), true);
if (URISchemeType.FILE.isURL(url)) {
File filename = FileSystem.convertURLToFile(url);
final String extension = FileSystem.extension(filename);
if (isMarkdownFileExtension(extension)) {
filename = FileSystem.replaceExtension(filename, ".html"); //$NON-NLS-1$
replacements.put(it.getUrl(), FileSystemAddons.convertFileToURL(filename, true).getPath());
}
URL url = FileSystem.convertStringToURL(it.getUrl().toString(), true);
url = transformURL(url);
if (url != null) {
replacements.put(it.getUrl(), convertURLToString(url));
}
}));
visitor.visitChildren(document);

// Apply the replacements
if (!replacements.isEmpty()) {
final StringBuilder buffer = new StringBuilder(content);
for (final Entry<BasedSequence, String> entry : replacements.entrySet()) {
Expand All @@ -461,6 +540,53 @@ protected String transformLinks(String content) {
return content;
}

/** Convert the given URL to its string representation that is compatible with Markdown document linking mechanism.
*
* @param url the URL to transform.
* @return the sting representation of the URL.
*/
static String convertURLToString(URL url) {
if (URISchemeType.FILE.isURL(url)) {
final StringBuilder externalForm = new StringBuilder();
externalForm.append(url.getPath());
final String ref = url.getRef();
if (!Strings.isEmpty(ref)) {
externalForm.append("#").append(ref); //$NON-NLS-1$
}
return externalForm.toString();
}
return url.toExternalForm();
}

/** Transform an URL from Markdown format to HTML format.
*
* <p>Usually, the file extension ".md" is replaced by ".html".
*
* @param link the link to transform.
* @return the result of the transformation. {@code null} if the link should not changed.
*/
@SuppressWarnings("static-method")
protected URL transformURL(URL link) {
if (URISchemeType.FILE.isURL(link)) {
File filename = FileSystem.convertURLToFile(link);
final String extension = FileSystem.extension(filename);
if (isMarkdownFileExtension(extension)) {
filename = FileSystem.replaceExtension(filename, ".html"); //$NON-NLS-1$
final String anchor = link.getRef();
final URL url = FileSystemAddons.convertFileToURL(filename, true);
if (!Strings.isEmpty(anchor)) {
try {
return new URL(url.toExternalForm() + "#" + anchor); //$NON-NLS-1$
} catch (MalformedURLException e) {
//
}
}
return url;
}
}
return null;
}

/** Replies if the given extension is for Markdown file.
*
* @param extension the extension to test.
Expand Down
10 changes: 10 additions & 0 deletions pom.xml
View file
Expand Up @@ -415,6 +415,16 @@
<artifactId>mockito-core</artifactId>
<version>2.0.111-beta</version>
</dependency>
<dependency>
<groupId>com.vladsch.flexmark</groupId>
<artifactId>flexmark</artifactId>
<version>0.18.7</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.10.3</version>
</dependency>
</dependencies>
</dependencyManagement>

Expand Down

0 comments on commit b809dee

Please sign in to comment.