Skip to content
Extensible Android library for both parsing text into Abstract Syntax Trees and rendering those trees as rich text.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app Fix: headers should consume newline (#11) Jul 5, 2018
gradle/wrapper update project gradle deps Jun 14, 2018
simpleast-core Bumped to 1.1.4. Mar 21, 2019
.gitignore removed unused files Mar 16, 2018
LICENSE
README.md Fix: Line Headers default values were overriding class headers (#12) Jul 6, 2018
build.gradle update project gradle deps Jun 14, 2018
gradle.properties initial commit Mar 16, 2018
gradlew initial commit Mar 16, 2018
gradlew.bat initial commit Mar 16, 2018
settings.gradle

README.md

SimpleAST

SimpleAST is a Kotlin/Java library designed to parse text into Abstract Syntax Trees. It is heavily inspired by (and began as a port of) Khan Academy's simple-markdown.

It strives for extensibility and robustness. How text is parsed into nodes in a tree is determined by a set of rules, provided by the client. This makes detecting and rendering your own custom entities in text a breeze.

Besides basic markdown, SimpleAST is what the Discord Android app uses to detect and render various entities in text.

For example:

"<@123456789> has **joined the server**." becomes "@AndyG has joined the server." Read more here: How Discord Renders Rich Messages on the Android App

Using SimpleAST in your application

If you are building with Gradle, simply add the following line to the dependencies section of your build.gradle file:

implementation 'com.discord:simpleast:1.1.1'

Basic Usage with SimpleMarkdownRenderer

If you want to simply render some text with basic markdown, you can use SimpleRenderer:

val source = "here is some bold text: **this is bold**"
val textView = findViewById<TextView>(R.id.textView)

SimpleRenderer.render(source, textView)

SimpleRenderer.render uses the rules provided in SimpleMarkdownRules.kt. These rules currently include:

  • Bold: **bold**
  • Italics 1: *italics*
  • Italics 2: _italics_
  • Underline: __underline__
  • Strikethru: ~~Strikethru~~
  • Escaping: \*Not Italics*

Adding your own Rules

We can create rules which will detect other entities in text. Rules should detect text that begins with symbols or other characters not matched in the plaintext rule.

A few things to keep in mind when building your own Parsers and Rules:

  1. Always include, at the very least, the plaintext rule. Without this rule, you may end up with unmatched text in the source (a fatal error.)
  2. A Pattern that defines a Rule should begin with a symbol that is non-alphanumeric. The plaintext rule is designed so that a non-alphanumeric character will trigger the Parser to consider whether the source matches any other rules first, before consuming it as plaintext.
  3. The Pattern that defines the rule matches only with the beginning of source text (i.e. begins with the '^' character). You will end up with magically-disappearing text if you suddenly match something in the middle of your source.

Simplest example

Let's imagine we want to render all occurrences of <Foo> as Bar, i.e. "This is <Foo> speaking" becomes "This is Bar speaking".

We create a simple Rule that detects and performs the replacement:

class FooRule : Rule<Any?, Node<Any?>>(Pattern.compile("^<Foo>")) {
  override fun parse(matcher: Matcher, parser: Parser<Any?, in Node<Any?>>, isNested: Boolean): ParseSpec<Any?, Node<Any?>{
    return ParseSpec.createTerminal(TextNode("Bar"))
  }
}

Now we create a Parser, add that Rule (and the rest of the basic rules) and render it.

val parser = Parser<Any?, Node<Any?>>()
  .addRule(FooRule())
  .addRules(SimpleMarkdownRules.createSimpleMarkdownRules())
  
resultText.text = SimpleRenderer.render(
    source = input.text,
    parser = parser,
    renderContext = null
)
Input Output
Hello **<Foo>** Hello Bar

Slightly more complex

Suppose we want to replace all occurrences of <1234> (where "1234" is a user id) with UserNode.

We'll create the Rule the same as before.

class UserNode(private val userId: Int) : Node<Any?>() {
  override fun render(builder: SpannableStringBuilder, renderContext: Any?) {
    builder.append("User $userId")
  }
}

class UserMentionRule : Rule<Any?, UserNode>(Pattern.compile("^<(\\d+)>")) {
  override fun parse(matcher: Matcher, parser: Parser<Any?, in UserNode>, isNested: Boolean): ParseSpec<Any?, UserNode> {
    return ParseSpec.createTerminal(UserNode(matcher.group(1).toInt()))
  }
}

The usage is the analogous to the first example.

Input Output
Hello <1234> Hello User 1234

Real-world application: Adding a Render Context

We modify our UserNode thusly to specify that it requires an instance of RenderContext in order to render, which contains a map of Int -> username.

data class RenderContext(val usernameMap: Map<Int, String>)

class UserNode(private val userId: Int) : Node<RenderContext>() {
  override fun render(builder: SpannableStringBuilder, renderContext: RenderContext) {
    builder.append(renderContext.usernameMap[userId] ?: "Invalid User")
  }
}

class UserMentionRule : Rule<RenderContext, UserNode>(Pattern.compile("^<(\\d+)>")) {
  override fun parse(matcher: Matcher, parser: Parser<RenderContext, in UserNode>, isNested: Boolean): ParseSpec<RenderContext, UserNode> {
    return ParseSpec.createTerminal(UserNode(matcher.group(1).toInt()))
  }
}

Now at the call-site, we specify that the parser produces nodes that require the RenderContext, and perform the parse:

val parser = Parser<RenderContext, Node<RenderContext>>()
    .addRule(UserMentionRule())
    .addRules(SimpleMarkdownRules.createSimpleMarkdownRules())

resultText.text = SimpleRenderer.render(
    source = input.text,
    parser = parser,
    renderContext = RenderContext(mapOf(1234 to "CoolDude1234"))
)

Note that we only provide {1234 : "CoolDude1234"} as our map of usernames. As such, we will render the following:

Input Output
Hello <1234> Hello CoolDude1234
Hello <6789> Hello Invalid User
You can’t perform that action at this time.