getLineHTMLForExport - Fixes #2486 but breaks plugins

[ilmartyrk]

The key of this fix is fixing the wrong usage of context.lineContent and getLineHTMLForExport.
Previously all getLineHTMLForExport hooks were expected to return lines updated by plugins and all of these lines were concatet into one string resulting mutliple lines with same text but different html wrappers.
The correct use would be to update context.lineContent inside each of these hooks and return true if a line was changed. Because the context is always passed on each plugin will modify the same value with each one adding its own updates rather than modifing the same line from orignal value.

This fix will need updating each plugin using getLineHTMLForExport to work properly!!!

I updated my previous fix cause I found that returning true from multiple plugins made lineContentFromHook value into 'true true true'. But I must add that this fix cannot work if there are updated and not updated plugins in use at the same time.

[JohnMcLear]

I'd merge this but it would make too much work for me. I'd be more inclined to merge if you can also patch my plugins for me please? :)

[ilmartyrk]

Will do that

[ilmartyrk]

@Gared I think it's ok, it doesn't change anything in this fixes behaviour so I added it

[ilmartyrk]

@JohnMcLear Could you merge this now?

[JohnMcLear]

@ilmartyrk can you resolve the merge conflicts please?

[ilmartyrk]

Thank you @JohnMcLear !!! Can you please make a new release on it too. I've also listed all the issues that I know of, that this update fixes.

[muxator]

If releasing this change breaks some plugins we should probably have an upgrade strategy (for example, we could invite the authors to issue a new release for their plugin, starting with the ones for which ilmartyrk has already wrote a pull request).

Current Etherpad version is 1.6.6. Since this is a change that somewhat breaks compatibility, I would propose to bump Etherpad version to 1.7.x (a plugin with a sane dependency spec could make good use of this).

[ilmartyrk]

@muxator I have to say that I downloaded all the plugins, that I could find from the official plugins lisy, that had html export hook and most of them were not working anyway. But I believe that 1.7.x would be the correct way to move forward because update isn't backwards compatible.

[muxator]

Let's investigate #3425 before unleashing 1.7 :)

[ilmartyrk]

@muxator Sorry for not reacting, now I finally created a new test that will blow up in the previous version and works fine in the updated one. This test shows how bad the final result was when two different types of lists were inserted one after another. Hope that you are good with it.

[muxator]

@ilmartyrk, thank you for your work.

The new test is clear, and - as desired - breaks before #3268 (fe08d2a), and works afterwards. Perfect.

For clarity, the test we are talking about is contained in 06cf3ff and is currently part of #3408.

Some nits:

1. I would like to keep this change separate from PR added goToRevisionEvent hook, regexp support for pre and post values in ep.json - WONT MERGE #3408, which is unrelated. For example: it could be applied to the current develop head cherry-picking it to another branch, and a new PR created from there. If you are off put by this request, I can do it by myself, keeping your name & commit data. But, since it's your work, maybe you want to take care of it. Just tell me.

2. The info that this test (and therefore, getLineHTMLForExport - Fixes #2486 but breaks plugins  #3268) address the case of two different types of list inserted one after another is probably worth mentioning in specs/api/pad.js, either in the comments or in the test text itself.

3. I would normally not want to remove an existing test, but it probably makes sense to remove it this time, for the following reasons (that should probably be included in the commit message for the new test):

   • the old list test was introduced in cadb83a with an explicit attempt to import an invalid HTML and see if it could recover from it
   • in 5967e08 the code base finally managed to export a meaningful HTML (but we do not know if other bugs were introduced in pursuing this)
   • that behaviour seems aimed at attaining resiliency, but correctness should be addressed before that

[muxator]

@ilmartyrk, I went on and committed 718b175 on your behalf (merged with b466acd), so this PR is deservedly covered by a test.

Thank you again for the help!

[raybellis]

The documentation for getLineHTMLForExport is now out of date with respect to this change.

http://etherpad.org/doc/v1.7.0/#index_exporthtmladditionaltags

It's also unclear now how (for example) one might even fix the ap_export_authors plugin, since its internal author.chars property is indexed based on the .text property of the context and not the lineContent.

(By the time the hook is called the lineContent already has HTML elements wrapped around the individually styled chunks of text, but the previous method of wrapping a <span> with an author-specific colour uses the author.chars array).