fixes: #418 (#419)

* fixes: #418

* added 1.12.3 to GHA

* better doc?

.github/workflows/ci.yml

@@ -25,6 +25,9 @@ jobs:
           - pair:
             otp: "24.0.5"
             elixir: "1.12.2"
+          - pair:
+            otp: "24.0.5"
+            elixir: "1.12.3"
     steps:
       - uses: actions/checkout@v2
       - name: Set up Elixir

README.md

@@ -19,9 +19,37 @@ and any changes you make in this file will most likely be lost
 This README contains the docstrings and doctests from the code by means of [extractly](https://hex.pm/packages/extractly)
 and the following code examples are therefore verified with `ExUnit` doctests.
 
-## Dependency
-
-    { :earmark, ">= 1.4.15" }
+## Table Of Content
+
+- [Table Of Content](#table-of-content)
+- [Options](#options)
+  - [Earmark.Cli.Implementation](#earmarkcliimplementation)
+  - [Earmark.Options](#earmarkoptions)
+  - [Earmark.Options.make_options/1](#earmarkoptionsmake_options1)
+  - [Earmark.Options.with_postprocessor/2](#earmarkoptionswith_postprocessor2)
+  - [Earmark](#earmark)
+- [Earmark](#earmark)
+  - [Abstract Syntax Tree and Rendering](#abstract-syntax-tree-and-rendering)
+    - [Earmark.as_ast](#earmarkas_ast)
+    - [Earmark.as_html](#earmarkas_html)
+    - [Earmark.as_html!](#earmarkas_html)
+    - [Options](#options)
+  - [Rendering](#rendering)
+    - [`escape:` defaulting to `true`](#escape-defaulting-to-true)
+    - [`inner_html:` defaulting to `false`](#inner_html-defaulting-to-false)
+    - [`smartypants:` defaulting to `true`](#smartypants-defaulting-to-true)
+  - [Command line](#command-line)
+- [Timeouts](#timeouts)
+- [Security](#security)
+  - [Earmark.Transform](#earmarktransform)
+- [Structure Conserving Transformers](#structure-conserving-transformers)
+  - [`map_ast`](#map_ast)
+  - [`map_ast_with`](#map_ast_with)
+  - [Postprocessors and Convenience Functions](#postprocessors-and-convenience-functions)
+    - [Use case: Modification of Link Attributes depending on the URL](#use-case-modification-of-link-attributes-depending-on-the-url)
+- [Structure Modifying Transformers](#structure-modifying-transformers)
+- [Contributing](#contributing)
+- [Author](#author)
 
 ## Options
 
@@ -39,7 +67,7 @@ This is a superset of the options that need to be passed into `EarmarkParser.as_
 The following options are proper to `Earmark` only and therefore explained in detail
 
 - `compact_output`: boolean indicating to avoid indentation and minimize whitespace
-- `eex`: Coming soon (EEx preprocessing)
+- `eex`: Allows usage of an `EEx` template to be expanded to markdown before conversion
 - `file`: Name of file passed in from the CLI
 - `line`: 1 but might be set to an offset for better error messages in some integration cases
 - `ignore_strings`, `postprocessor` and `registered_processors`: processors that modify the AST returned from
@@ -48,7 +76,6 @@ The following options are proper to `Earmark` only and therefore explained in de
 
 All other options are passed onto EarmarkParser.as_ast/`2`
 
-
 ### Earmark.Options.make_options/1
 
 Make a legal and normalized Option struct from, maps or keyword lists
@@ -94,6 +121,7 @@ And the bang version will raise an `Earmark.Error` as excepted (sic)
 A convenience constructor
 
 
+
 ### Earmark
 
 
@@ -167,17 +195,17 @@ Normally `Earmark` aims to produce _Human Readable_ output.
 This will give results like these:
 
 ```elixir
-    iex(0)> markdown = "# Hello\nWorld"
-    ...(0)> Earmark.as_html!(markdown, compact_output: false)
+    iex(1)> markdown = "# Hello\nWorld"
+    ...(1)> Earmark.as_html!(markdown, compact_output: false)
     "<h1>\nHello</h1>\n<p>\nWorld</p>\n"
 ```
 
 
 But sometimes whitespace is not desired:
 
 ```elixir
-    iex(1)> markdown = "# Hello\nWorld"
-    ...(1)> Earmark.as_html!(markdown, compact_output: true)
+    iex(2)> markdown = "# Hello\nWorld"
+    ...(2)> Earmark.as_html!(markdown, compact_output: true)
     "<h1>Hello</h1><p>World</p>"
 ```
 
@@ -189,20 +217,48 @@ Be cautions though when using this options, lines will become loooooong.
 If set HTML will be properly escaped
 
 ```elixir
-      iex(2)> markdown = "Hello<br />World"
-      ...(2)> Earmark.as_html!(markdown)
+      iex(3)> markdown = "Hello<br />World"
+      ...(3)> Earmark.as_html!(markdown)
       "<p>\nHello&lt;br /&gt;World</p>\n"
 ```
 
 However disabling `escape:` gives you maximum control of the created document, which in some
 cases (e.g. inside tables) might even be necessary
 
 ```elixir
-      iex(3)> markdown = "Hello<br />World"
-      ...(3)> Earmark.as_html!(markdown, escape: false)
+      iex(4)> markdown = "Hello<br />World"
+      ...(4)> Earmark.as_html!(markdown, escape: false)
       "<p>\nHello<br />World</p>\n"
 ```
 
+#### `inner_html:` defaulting to `false`
+
+This is especially useful inside templates, when a block element will disturb the layout as
+in this case
+
+```html
+<span><%= Earmark.as_html!(....)%></span>
+<span><%= Earmark.as_html!(....)%></span>
+```
+
+By means of the `inner_html` option the disturbing paragraph can be removed from `as_html!`'s
+output
+
+```elixir
+      iex(5)> markdown = "Hello<br />World"
+      ...(5)> Earmark.as_html!(markdown, escape: false, inner_html: true)
+      "Hello<br />World\n"
+```
+
+**N.B.** that this applies only to top level paragraphs, as can be seen here
+
+```elixir
+      iex(6)> markdown = "- Item\n\nPara"
+      ...(6)> Earmark.as_html!(markdown, inner_html: true)
+      "<ul>\n  <li>\nItem  </li>\n</ul>\nPara\n"
+```
+
+
 * `postprocessor:` defaults to nil
 
 Before rendering the AST is transformed by a postprocessor.

README.md.eex

@@ -15,18 +15,17 @@
 This README contains the docstrings and doctests from the code by means of [extractly](https://hex.pm/packages/extractly)
 and the following code examples are therefore verified with `ExUnit` doctests.
 
-## Dependency
+## Table Of Content
 
-    { :earmark, ">= <%= Earmark.version %>" }
+<%= xtra.toc :self, gh_links: true, min_level: 2, max_level: 4, remove_gaps: true %>
 
 ## Options
 
 
 <%= xtra.moduledoc "Earmark.Cli.Implementation", wrap_code_blocks: "elixir", headline: 3 %>
 
-<%= xtra.moduledoc "Earmark.Options", wrap_code_blocks: "elixir", headline: 3 %>
+<%= xtra.moduledoc "Earmark.Options", include: :all, wrap_code_blocks: "elixir", headline: 3 %>
 
-<%= xtra.functiondoc :all, module: "Earmark.Options", wrap_code_blocks: "elixir", headline: 3 %>
 
 <%= xtra.moduledoc "Earmark", wrap_code_blocks: "elixir", headline: 3 %>
 

RELEASE.md

@@ -1,3 +1,17 @@
+# 1.4.16 2021-??-??
+
+
+414 Test Coverage
+
+412 Refactoring of `Options`
+
+411 Refactoring of CLI code → Readability and Testability
+
+410 Allow using `.eex` templates as source
+
+407 Configuration of Link Attributes
+
+
 # 1.4.15 2021-04-18
 
 - Pushed `EarmarkParser` dependency to version >= 1.4.13 which fixes a bug wich could crash the parser

lib/earmark.ex

@@ -85,15 +85,15 @@ defmodule Earmark do
 
   This will give results like these:
 
-      iex(0)> markdown = "# Hello\\nWorld"
-      ...(0)> Earmark.as_html!(markdown, compact_output: false)
+      iex(1)> markdown = "# Hello\\nWorld"
+      ...(1)> Earmark.as_html!(markdown, compact_output: false)
       "<h1>\\nHello</h1>\\n<p>\\nWorld</p>\\n"
 
 
   But sometimes whitespace is not desired:
 
-      iex(1)> markdown = "# Hello\\nWorld"
-      ...(1)> Earmark.as_html!(markdown, compact_output: true)
+      iex(2)> markdown = "# Hello\\nWorld"
+      ...(2)> Earmark.as_html!(markdown, compact_output: true)
       "<h1>Hello</h1><p>World</p>"
 
   Be cautions though when using this options, lines will become loooooong.
@@ -103,17 +103,41 @@ defmodule Earmark do
 
   If set HTML will be properly escaped
 
-        iex(2)> markdown = "Hello<br />World"
-        ...(2)> Earmark.as_html!(markdown)
+        iex(3)> markdown = "Hello<br />World"
+        ...(3)> Earmark.as_html!(markdown)
         "<p>\\nHello&lt;br /&gt;World</p>\\n"
 
   However disabling `escape:` gives you maximum control of the created document, which in some
   cases (e.g. inside tables) might even be necessary
 
-        iex(3)> markdown = "Hello<br />World"
-        ...(3)> Earmark.as_html!(markdown, escape: false)
+        iex(4)> markdown = "Hello<br />World"
+        ...(4)> Earmark.as_html!(markdown, escape: false)
         "<p>\\nHello<br />World</p>\\n"
 
+  #### `inner_html:` defaulting to `false`
+
+  This is especially useful inside templates, when a block element will disturb the layout as
+  in this case
+
+  ```html
+  <span><%= Earmark.as_html!(....)%></span>
+  <span><%= Earmark.as_html!(....)%></span>
+  ```
+
+  By means of the `inner_html` option the disturbing paragraph can be removed from `as_html!`'s
+  output
+
+        iex(5)> markdown = "Hello<br />World"
+        ...(5)> Earmark.as_html!(markdown, escape: false, inner_html: true)
+        "Hello<br />World\\n"
+
+  **N.B.** that this applies only to top level paragraphs, as can be seen here
+
+        iex(6)> markdown = "- Item\\n\\nPara"
+        ...(6)> Earmark.as_html!(markdown, inner_html: true)
+        "<ul>\\n  <li>\\nItem  </li>\\n</ul>\\nPara\\n"
+
+
   * `postprocessor:` defaults to nil
 
   Before rendering the AST is transformed by a postprocessor.

lib/earmark/options.ex

@@ -29,6 +29,7 @@ defmodule Earmark.Options do
             gfm: true,
             gfm_tables: false,
             ignore_strings: false,
+            inner_html: false,
             line: 1,
             mapper: &Earmark.pmap/2,
             mapper_with_timeout: &Earmark.pmap/3,

lib/earmark/transform.ex

@@ -167,7 +167,10 @@ defmodule Earmark.Transform do
   def transform(ast, options) when is_map(options) do
     options1 = options
       |> Map.put_new(:indent, 2)
-    to_html(ast, options1)
+    ast
+    # |> IO.inspect
+    |> _maybe_remove_paras(options1)
+    |> to_html(options1)
   end
 
   @doc ~S"""
@@ -399,6 +402,16 @@ defmodule Earmark.Transform do
     [acc | binary_part(original, skip, len)]
   end
 
+  defp _add_trailing_nl(node)
+  defp _add_trailing_nl(text) when is_binary(text), do: [text, "\n"]
+  defp _add_trailing_nl(node), do: node
+
+  defp _maybe_remove_paras(ast, options)
+  defp _maybe_remove_paras(ast, %Options{inner_html: true}) do
+    Enum.map(ast, &_remove_para/1)
+  end
+  defp _maybe_remove_paras(ast, _), do: ast
+
   @pop {:__end__}
   defp _pop_to_pop(result, intermediate \\ [])
   defp _pop_to_pop([@pop, {tag, atts, _, meta}|rest], intermediate) do
@@ -408,6 +421,10 @@ defmodule Earmark.Transform do
     _pop_to_pop(rest, [continue|intermediate])
   end
 
+  defp _remove_para(ele_or_string)
+  defp _remove_para({"p", _, content, _}), do: content |> Enum.map(&_add_trailing_nl/1)
+  defp _remove_para(whatever), do: whatever
+
   defp _walk_ast(ast, fun, ignore_strings, result)
   defp _walk_ast([], _fun, _ignore_strings, result), do: Enum.reverse(result)
   defp _walk_ast([[]|rest], fun, ignore_strings, result) do

mix.exs

@@ -1,7 +1,7 @@
 defmodule Earmark.Mixfile do
   use Mix.Project
 
-  @version "1.4.15"
+  @version "1.4.16-pre"
 
   @url "https://github.com/pragdave/earmark"
 
@@ -13,7 +13,7 @@ defmodule Earmark.Mixfile do
     {:earmark_ast_dsl, "~> 0.2.5", only: [:test]},
     {:excoveralls, "~> 0.14.2", only: [:test]},
     # {:extractly, "~> 0.5.0", git: "https://github.com/RobertDober/extractly.git", tag: "v0.5.0-pre1", only: [:dev]},
-    {:extractly, "~> 0.5.0", only: [:dev]},
+    {:extractly, "~> 0.5.3", only: [:dev]},
     {:floki, "~> 0.21", only: [:dev, :test]},
     {:traverse, "~> 1.0.1", only: [:dev, :test]}
   ]

mix.lock

@@ -6,7 +6,7 @@
   "earmark_parser": {:hex, :earmark_parser, "1.4.15", "b29e8e729f4aa4a00436580dcc2c9c5c51890613457c193cc8525c388ccb2f06", [:mix], [], "hexpm", "044523d6438ea19c1b8ec877ec221b008661d3c27e3b848f4c879f500421ca5c"},
   "erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"},
   "excoveralls": {:hex, :excoveralls, "0.14.2", "f9f5fd0004d7bbeaa28ea9606251bb643c313c3d60710bad1f5809c845b748f0", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "ca6fd358621cb4d29311b29d4732c4d47dac70e622850979bc54ed9a3e50f3e1"},
-  "extractly": {:hex, :extractly, "0.5.0", "28021916e757199bfa9df6746b8a6eee2fe79d318c22415a752f7afb2b5b648a", [:mix], [], "hexpm", "af216e00025a3036b4960b0034dc1da368072be699b224f475730e4ac42c1962"},
+  "extractly": {:hex, :extractly, "0.5.3", "ca6afc3430e63cc9017d50eb03bdf1a3499c7e888037d2279b0ec0ea393af390", [:mix], [], "hexpm", "858f7a285ffb767937e75bb8866f03fecb3b7e5b42728a9677d2ebb6ea885503"},
   "floki": {:hex, :floki, "0.30.0", "22ebbe681a5d3777cdd830ca091b1b806d33c3449c26312eadca7f7be685c0c8", [:mix], [{:html_entities, "~> 0.5.0", [hex: :html_entities, repo: "hexpm", optional: false]}], "hexpm", "a9e128a4ca9bb71f11affa315b6768a9ad326d5996ff1e92acf1d7a01a10076a"},
   "hackney": {:hex, :hackney, "1.17.4", "99da4674592504d3fb0cfef0db84c3ba02b4508bae2dff8c0108baa0d6e0977c", [:rebar3], [{:certifi, "~>2.6.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~>6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~>1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~>1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "de16ff4996556c8548d512f4dbe22dd58a587bf3332e7fd362430a7ef3986b16"},
   "html_entities": {:hex, :html_entities, "0.5.2", "9e47e70598da7de2a9ff6af8758399251db6dbb7eebe2b013f2bbd2515895c3c", [:mix], [], "hexpm", "c53ba390403485615623b9531e97696f076ed415e8d8058b1dbaa28181f4fdcc"},

test/acceptance/html/inner_html_test.exs

@@ -0,0 +1,37 @@
+defmodule Test.Acceptance.Html.InnerHtmlTest do
+  use Support.AcceptanceTestCase
+
+  describe "output with the inner_html option" do
+    test "one para" do
+      markdown = "aaa"
+      html     = "aaa\n"
+      messages = []
+
+      assert as_html(markdown, inner_html: true) == {:ok, html, messages}
+    end
+
+    test "two paras" do
+      markdown = "aaa\n\nbbb\n"
+      html     = "aaa\nbbb\n"
+      messages = []
+
+      assert as_html(markdown, inner_html: true) == {:ok, html, messages}
+    end
+
+    test "a list and a para" do
+      markdown = "1. item\n\npara"
+      html     = "<ol>\n  <li>\nitem  </li>\n</ol>\npara\n"
+      messages = []
+
+      assert as_html(markdown, inner_html: true) == {:ok, html, messages}
+    end
+
+    test "complex content of the para" do
+      markdown = "a **b** `c`"
+      html     = "a \n<strong>b</strong> \n<code class=\"inline\">c</code>"
+      messages = []
+
+      assert as_html(markdown, inner_html: true) == {:ok, html, messages}
+    end
+  end
+end