Indentation and HTML reporting #162

Closed
asflierl opened this Issue May 24, 2013 · 24 comments

Comments

Projects
None yet
2 participants
@asflierl
Contributor

asflierl commented May 24, 2013

Hi Eric,

please forgive me if I'm missing something but I've read the new user guide and your blog post but could not find a solution to this issue, which seems like a bit of an oversight.

Anyway, please consider the following simple spec:

import org.specs2._

class Meep extends SpecificationWithJUnit { def is = s2"""
This is a simple, hierarchial specification
  If things are indented
    and examples are nested
      this should be reflected
        in the console                   $ok
        as well as the HTML report       $ko
      just as expected
        by me                            $ok
        or maybe someone else            $todo
    or if there's only text, indentation
    possibly on multiple following lines
      should also work                   $ok
      and not quote lines as code        $ko
"""
}

Executing it shows a nice result on the console, i.e. all the different indentation levels are carried over just as expected:
console

The HTML report in contrast is lacking many of the layout features and shows some unwanted things:
html

So, my biggest issue is that the indentation is not reflected at all anymore (this works a lot better with the "traditional" acceptance spec).

The second problem is that any text that is indented 4 or more spaces is interpreted as a code block. While this is standard markdown behaviour, it is really unwanted for nested examples.

The third issue is a rather minor one: the "todo" appears as two examples, rather than one "i" icon, followed by the example text and then the "TODO" text all on one line.

So far, these issues would mean I cannot do hierarchial specs with the new interpolated spec feature, which is a real pity because of all the other advantages this style offers.

Also, I kinda have to do things in this hierarchial way if I want to avoid repitition and also because I can only use one line of text per example (without interpolating inside the interpolation... which just reads awful).

It would be cool if you could look into this at some point.

Thanks a lot in advance.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 24, 2013

Owner

Ouch, this is indeed a complete oversight. I'm going to have a look at that.

Owner

etorreborre commented May 24, 2013

Ouch, this is indeed a complete oversight. I'm going to have a look at that.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 24, 2013

Owner

Ok, this is not trivial. Part of the solution is to pass nomarkdown on the command-line to avoid spaces to be interpreted as code blocks. But that's not going to indent the examples properly. I need to think a bit more about the best approach for that during the week-end.

Owner

etorreborre commented May 24, 2013

Ok, this is not trivial. Part of the solution is to pass nomarkdown on the command-line to avoid spaces to be interpreted as code blocks. But that's not going to indent the examples properly. I need to think a bit more about the best approach for that during the week-end.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl May 24, 2013

Contributor

No hurries. I'm just glad you're looking at this. Using nomarkdown would really mean that nothing in the spec would be parsed as markdown? If so, that would be veeeery undesirable. :( We use markdown extensively.

Contributor

asflierl commented May 24, 2013

No hurries. I'm just glad you're looking at this. Using nomarkdown would really mean that nothing in the spec would be parsed as markdown? If so, that would be veeeery undesirable. :( We use markdown extensively.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 24, 2013

Owner

Then how do you solve the indentation problem? Indented text is going to be represented as code in that case.

Owner

etorreborre commented May 24, 2013

Then how do you solve the indentation problem? Indented text is going to be represented as code in that case.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl May 24, 2013

Contributor

I always hated indented code blocks in markdown. Fenced code blocks are much clearer and better delimited IMHO. I'll have to think about if it's possible to turn off only the former somehow.

Contributor

asflierl commented May 24, 2013

I always hated indented code blocks in markdown. Fenced code blocks are much clearer and better delimited IMHO. I'll have to think about if it's possible to turn off only the former somehow.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl May 24, 2013

Contributor

Hmm, I also just found out that if you link a "traditional" acceptance spec from an interpolated one, all auto-indentation is turned off for the linked spec. :(

Contributor

asflierl commented May 24, 2013

Hmm, I also just found out that if you link a "traditional" acceptance spec from an interpolated one, all auto-indentation is turned off for the linked spec. :(

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 24, 2013

Owner

That's just saying that the number of combinations exceed my testing ability... I'll have a look at that as well.

Owner

etorreborre commented May 24, 2013

That's just saying that the number of combinations exceed my testing ability... I'll have a look at that as well.

etorreborre added a commit that referenced this issue May 26, 2013

partial fix for #162: indentation in html files
now the indentation looks better but markdown code blocks are still an issue to be solved
@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 27, 2013

Owner

Hmm, I also just found out that if you link a "traditional" acceptance spec from an interpolated one, all auto-indentation is turned off for the linked spec. :(

This is actually a "feature". All the linked specifications are inheriting the context of their parent specification which is noindent in that case. But this is where the real problem is!

I'm starting to think that the proper way out of this issue is to give more granularity to the formatting of specifications. At the moment this granularity is at the specification level (with the args.report parameters) but if I can put this at the Fragment level we might be able to write something like:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
    * do this
    * do that
""" ^ s2"""
  We should be able to
      do this $e1
      do that $e2
  But also
      do this $e3
      do that $e4     
""".set(nomarkdown = true)
}

The first block would use Markdown (the default) and the second one would deactivate it so that indentation does not get translated to code blocks.

What do you think?

Owner

etorreborre commented May 27, 2013

Hmm, I also just found out that if you link a "traditional" acceptance spec from an interpolated one, all auto-indentation is turned off for the linked spec. :(

This is actually a "feature". All the linked specifications are inheriting the context of their parent specification which is noindent in that case. But this is where the real problem is!

I'm starting to think that the proper way out of this issue is to give more granularity to the formatting of specifications. At the moment this granularity is at the specification level (with the args.report parameters) but if I can put this at the Fragment level we might be able to write something like:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
    * do this
    * do that
""" ^ s2"""
  We should be able to
      do this $e1
      do that $e2
  But also
      do this $e3
      do that $e4     
""".set(nomarkdown = true)
}

The first block would use Markdown (the default) and the second one would deactivate it so that indentation does not get translated to code blocks.

What do you think?

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 29, 2013

Owner

Hi Andreas, here is my proposal, for which I have an almost working implementation. It is based on using tags to specify the formatting of some parts of the specification (an extension of the tags that are used for selection).

For the markdown problem, you can add a section to specify that no markdown should be used:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
    * do this
    * do that

  We should be able to                  $noMarkdownSection
      do this $e1
      do that $e2
  But also
      do this $e3
      do that $e4             
"""
}

This works because, by default, a s2 string will have markdown and flow as formatting tags (flow means no automatic indentation.

Now, if you want the included specification, you can simply link it outside the s2 block

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
...
""" ^ link (new MySpec2)
}

Or, you can link it inside but create a noFlow section:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
...
${link (new MySpec2)}  $noFlowSection
""" 
}

I hope that doesn't sound too complex. I think that this is the most flexible thing to do and it allows both styles to coexist.

Owner

etorreborre commented May 29, 2013

Hi Andreas, here is my proposal, for which I have an almost working implementation. It is based on using tags to specify the formatting of some parts of the specification (an extension of the tags that are used for selection).

For the markdown problem, you can add a section to specify that no markdown should be used:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
    * do this
    * do that

  We should be able to                  $noMarkdownSection
      do this $e1
      do that $e2
  But also
      do this $e3
      do that $e4             
"""
}

This works because, by default, a s2 string will have markdown and flow as formatting tags (flow means no automatic indentation.

Now, if you want the included specification, you can simply link it outside the s2 block

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
...
""" ^ link (new MySpec2)
}

Or, you can link it inside but create a noFlow section:

class MySpec extends Specification { def is = s2"""
  This is an introduction to the system. With this system you can:
...
${link (new MySpec2)}  $noFlowSection
""" 
}

I hope that doesn't sound too complex. I think that this is the most flexible thing to do and it allows both styles to coexist.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl May 29, 2013

Contributor

Hmm,

my problem with that is not that it is too complex but that it doesn't allow markdown in the examples "just" because I don't want the code block quoting.

My proposal therefore is to "just" use bullet points / lists for examples:

class Meep extends SpecificationWithJUnit { def is = s2"""
This is a simple, hierarchical specification

* If things are indented
    * and examples are nested                  $ok
    * this should be reflected
        * in the console                       $ok
        * as well as the HTML report           $ko
            * arbitrary levels
        * inline **markdown** should just work $todo
    * just as expected
        * by me                                $ok
        * or maybe someone else                $todo
    * or if there's only text, indentation
      possibly on multiple following lines
        * should also work                     $ok
        * and not quote lines as code          $ko
"""
}

This would just work like in regular markdown and the ouput be similar to this:

screen1

So nested examples would be properly reflected in the resulting HTML. Ideally, the text of an example would just be all the text on the same indentation level until the next bullet point begins or the next paragraph. Note that your choice to use github-style paragraphs / line-endings interferes with this. (I really oppose that markdown extension btw. since programmers' editors typically don't wrap -- but that is another matter. :D )

This would simply leverage markdown facilities but I have no idea how well that would integrate with other reporters or your existing implementation for the other spec styles.

It should probably still be possible to write an example without a bullet point, with the current behavior (although it would rock if the example text could be the whole paragraph, not just 1 line.... dunno if that's possible).

So... sorry for the lengthy text. I realize this is more like a feature request than a bug and that maybe this conflicts with your views (and possibly that of other specs2 users) and/or implementation. I just wanted to express that in the current form, I won't be able to use the interpolated specs feature for all but the simplest of specs.... and it could be so much more. The current acceptance specs allow me to do this kind of nesting + formatting (at the expense of some editing hassle, a column of symbols that tends to intimidate people and compile time... but I can live with all that... more or less). I also realize that this probably requires a lot of effort to implement and I don't really expect this to happen. I personally don't find the time to give the implementation a try myself at the moment or I'd cook up a pull request. Anyway, thanks for having spent so much time with this issue already.

Contributor

asflierl commented May 29, 2013

Hmm,

my problem with that is not that it is too complex but that it doesn't allow markdown in the examples "just" because I don't want the code block quoting.

My proposal therefore is to "just" use bullet points / lists for examples:

class Meep extends SpecificationWithJUnit { def is = s2"""
This is a simple, hierarchical specification

* If things are indented
    * and examples are nested                  $ok
    * this should be reflected
        * in the console                       $ok
        * as well as the HTML report           $ko
            * arbitrary levels
        * inline **markdown** should just work $todo
    * just as expected
        * by me                                $ok
        * or maybe someone else                $todo
    * or if there's only text, indentation
      possibly on multiple following lines
        * should also work                     $ok
        * and not quote lines as code          $ko
"""
}

This would just work like in regular markdown and the ouput be similar to this:

screen1

So nested examples would be properly reflected in the resulting HTML. Ideally, the text of an example would just be all the text on the same indentation level until the next bullet point begins or the next paragraph. Note that your choice to use github-style paragraphs / line-endings interferes with this. (I really oppose that markdown extension btw. since programmers' editors typically don't wrap -- but that is another matter. :D )

This would simply leverage markdown facilities but I have no idea how well that would integrate with other reporters or your existing implementation for the other spec styles.

It should probably still be possible to write an example without a bullet point, with the current behavior (although it would rock if the example text could be the whole paragraph, not just 1 line.... dunno if that's possible).

So... sorry for the lengthy text. I realize this is more like a feature request than a bug and that maybe this conflicts with your views (and possibly that of other specs2 users) and/or implementation. I just wanted to express that in the current form, I won't be able to use the interpolated specs feature for all but the simplest of specs.... and it could be so much more. The current acceptance specs allow me to do this kind of nesting + formatting (at the expense of some editing hassle, a column of symbols that tends to intimidate people and compile time... but I can live with all that... more or less). I also realize that this probably requires a lot of effort to implement and I don't really expect this to happen. I personally don't find the time to give the implementation a try myself at the moment or I'd cook up a pull request. Anyway, thanks for having spent so much time with this issue already.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre May 31, 2013

Owner

I extended my current refactoring in 2 directions:

  • you can mark sections with some formatting parameters
class Meep extends SpecificationWithJUnit { def is = s2""" ${formatSection(verbatim=false)}
This is a simple, hierarchial specification
  ....
  • then I extended the markdown converter (pegdown) to avoid rendering code blocks as code when verbatim is false (only code blocks, not inline markdown or fenced code blocks)

This should give us enough flexibility for what we want to do. I need a bit more time to polish the code and fix some minor issues, then I'll publish a version for you to try.

Owner

etorreborre commented May 31, 2013

I extended my current refactoring in 2 directions:

  • you can mark sections with some formatting parameters
class Meep extends SpecificationWithJUnit { def is = s2""" ${formatSection(verbatim=false)}
This is a simple, hierarchial specification
  ....
  • then I extended the markdown converter (pegdown) to avoid rendering code blocks as code when verbatim is false (only code blocks, not inline markdown or fenced code blocks)

This should give us enough flexibility for what we want to do. I need a bit more time to polish the code and fix some minor issues, then I'll publish a version for you to try.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl May 31, 2013

Contributor

That looks great.

Contributor

asflierl commented May 31, 2013

That looks great.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Jun 2, 2013

Owner

You can have a go at RC2-SNAPSHOT. I think that I should change the html creation strategy for interpolated specification in order to produce a html that's more respectful of original newlines and indentations but I find that a bit hard for now. Anyway, at least we have more control over the markdown -> html now.

Owner

etorreborre commented Jun 2, 2013

You can have a go at RC2-SNAPSHOT. I think that I should change the html creation strategy for interpolated specification in order to produce a html that's more respectful of original newlines and indentations but I find that a bit hard for now. Anyway, at least we have more control over the markdown -> html now.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl Jun 3, 2013

Contributor

Tried the latest snapshot and things seem to work as intended. I think this is a good progress. Still hoping that you'll find a solution for the indentation issues at some point but I can imagine that that's not easy to do - and it's maybe not the biggest of issues since I can still use the former acceptance spec style if I want to go "crazy" with the nesting.
Thanks a lot, again!

Contributor

asflierl commented Jun 3, 2013

Tried the latest snapshot and things seem to work as intended. I think this is a good progress. Still hoping that you'll find a solution for the indentation issues at some point but I can imagine that that's not easy to do - and it's maybe not the biggest of issues since I can still use the former acceptance spec style if I want to go "crazy" with the nesting.
Thanks a lot, again!

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Jun 3, 2013

Owner

Andreas, can you be more specific about which indentation issue you are referring to? Because now, with formatSection(verbatim = false) you can go crazy with nesting even in interpolated specifications.

Owner

etorreborre commented Jun 3, 2013

Andreas, can you be more specific about which indentation issue you are referring to? Because now, with formatSection(verbatim = false) you can go crazy with nesting even in interpolated specifications.

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl Jun 3, 2013

Contributor

OK, sure, my example is still the same:

class Meep extends SpecificationWithJUnit { def is = s2""" ${formatSection(verbatim = false)}
This is a simple, hierarchial specification

If things are indented
  and examples are nested
    this should be reflected
      in the console                     $ok
      as well as the HTML report         $ko
    just as expected
      by me                              $ok
      or maybe someone else              $todo
    or if there's only text, indentation  
    possibly on multiple following lines
      should also work                   $ok
      and not quote lines as code        $ko
"""
}

Contrast the (IMHO correct) console output...

a

... with the HTML report...

b

... and you'll notice that "and examples are nested" should be a sub-example of "If things are indented" and that "this should be reflected" should be a sub-example of "and examples are nested", but the HTML report shows them as if they were on the top level.

That the 4th-to-last and 3rd-to-last lines are joined together is a bit surprising, too. (Note the two spaces at the end or if there's only text, indentation which usually signals a line break in markdown.) ... but that's a totally minor issue.

Contributor

asflierl commented Jun 3, 2013

OK, sure, my example is still the same:

class Meep extends SpecificationWithJUnit { def is = s2""" ${formatSection(verbatim = false)}
This is a simple, hierarchial specification

If things are indented
  and examples are nested
    this should be reflected
      in the console                     $ok
      as well as the HTML report         $ko
    just as expected
      by me                              $ok
      or maybe someone else              $todo
    or if there's only text, indentation  
    possibly on multiple following lines
      should also work                   $ok
      and not quote lines as code        $ko
"""
}

Contrast the (IMHO correct) console output...

a

... with the HTML report...

b

... and you'll notice that "and examples are nested" should be a sub-example of "If things are indented" and that "this should be reflected" should be a sub-example of "and examples are nested", but the HTML report shows them as if they were on the top level.

That the 4th-to-last and 3rd-to-last lines are joined together is a bit surprising, too. (Note the two spaces at the end or if there's only text, indentation which usually signals a line break in markdown.) ... but that's a totally minor issue.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Jun 3, 2013

Owner

I see, thanks for the detailed report. I remember this issue now. I tried to endlessly tweak the markdown processing + html generation to fix that but nothing really worked. This is why I think that a brand new strategy is necessary. I hope I'll be able to propose something in that direction after ScalaDays.

Owner

etorreborre commented Jun 3, 2013

I see, thanks for the detailed report. I remember this issue now. I tried to endlessly tweak the markdown processing + html generation to fix that but nothing really worked. This is why I think that a brand new strategy is necessary. I hope I'll be able to propose something in that direction after ScalaDays.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Jun 20, 2013

Owner

I need to work again on this issue. 2.0 actually breaks bullet lists in the generated HTML. Trying to do Markdown + keeping the indentation is a bit hard but I'll keep trying to find a solution, hopefully in 2.1.

Owner

etorreborre commented Jun 20, 2013

I need to work again on this issue. 2.0 actually breaks bullet lists in the generated HTML. Trying to do Markdown + keeping the indentation is a bit hard but I'll keep trying to find a solution, hopefully in 2.1.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Nov 6, 2014

Owner

Some things are still a bit tricky but here is what I got tonight on my branch for specs2 3.0:

image

I'm still far for an official 3.0 release but that's encouraging!

Owner

etorreborre commented Nov 6, 2014

Some things are still a bit tricky but here is what I got tonight on my branch for specs2 3.0:

image

I'm still far for an official 3.0 release but that's encouraging!

@asflierl

This comment has been minimized.

Show comment
Hide comment
@asflierl

asflierl Nov 6, 2014

Contributor

that looks veeeery promising, cool stuff

Contributor

asflierl commented Nov 6, 2014

that looks veeeery promising, cool stuff

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Mar 3, 2015

Owner

Hi Andreas, now that specs2 3.0 is out, can you please check that this works for you? Note that you will have to install Pandoc to create Html documents now. I'm not happy about having to impose an external dependency but that was the best thing I could find to make Markdown really work.

Owner

etorreborre commented Mar 3, 2015

Hi Andreas, now that specs2 3.0 is out, can you please check that this works for you? Note that you will have to install Pandoc to create Html documents now. I'm not happy about having to impose an external dependency but that was the best thing I could find to make Markdown really work.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Mar 4, 2015

Owner

Don't spend time on it right now. There was a regression I am currently investigating.

Owner

etorreborre commented Mar 4, 2015

Don't spend time on it right now. There was a regression I am currently investigating.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Mar 5, 2015

Owner

The regression is that you should use Pandoc 1.12.4.1 or Pandoc 1.13.2 with the markdown_attribute extension on.

Owner

etorreborre commented Mar 5, 2015

The regression is that you should use Pandoc 1.12.4.1 or Pandoc 1.13.2 with the markdown_attribute extension on.

@etorreborre

This comment has been minimized.

Show comment
Hide comment
@etorreborre

etorreborre Mar 18, 2015

Owner

Closing this for now. Please test in 3.1.

Owner

etorreborre commented Mar 18, 2015

Closing this for now. Please test in 3.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment