Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Nested table implementation doesn't encode < character to invalid XML produced #267

Closed
ALuckyGuy opened this Issue · 27 comments

6 participants

@ALuckyGuy

I'm using defines to create nested tables. It works well, but I ran into a problem when I used a character that's not allowed in XML. Here's my define - notice the description for item 4:
!define BONUSRatingTbl {| RATING_NBR | DESCR2 |
| 1 | Met 100% of goals |
| 2 | Met 75% of goals |
| 3 | Met 50% of goals |
| 4 | Met < 50% of goals |
}

This resulted in the following text being passed to SLIM:

<table>
    <tr>
        <td>RATING_NBR</td>
        <td>DESCR2</td>
    </tr>
    <tr>
        <td>1</td>
        <td>Met 100% of goals</td>
    </tr>
    <tr>
        <td>2</td>
        <td>Met 75% of goals</td>
    </tr>
    <tr>
        <td>3</td>
        <td>Met 50% of goals</td>
    </tr>
    <tr>
        <td>4</td>
        <td>Met < 50% of goals</td>
    </tr>
</table>

When my fixture tries to convert the string to an XML object for conversion, it fails due to the invalid < character being in the text, which I believe should be encoded as &#x60;

Edit: In reality, it ought to be encoded as &lt;, which still isn't happening in my case.

@stanio

Which version of FitNesse are you using? It doesn't reproduce for me using the latest release (v20130530).

@ALuckyGuy

Sorry I forgot to include that, but I'm using 20130531

@amolenaar
Collaborator

This already happens during rendering of page (before being processed by the test system). I would expect the < to be converted to a &lt; however.

A quick test on my side seems to work out fine (from the wiki page source):

    <td>Met &lt; 50% of goals</td>

Is the problem only in combination with the test system or also showing up then rendering the page?

@amolenaar
Collaborator

Is this issue still valid?

@ALuckyGuy

Yes, I believe so. At least it still occurs in the code from 2 months ago. I haven't updated recently.

I did a little more testing that might help. Just for kicks, I changed the < symbol to &lt; in the table, figuring I could at least do that as a work-around. Surprisingly, I still found the < coming through to my fixture: it appears the escaped &lt; gets converted back to a < char at some point.

Here's the error getting thrown by my slim server. I thinks the < in the body is the start of a tag, thus the error message:

Cannot convert value "<table>
    <tr>
        <td>RATING_NBR</td>
        <td>DESCR2</td>
    </tr>
    <tr>
        <td>1</td>
        <td>Met 100% of goals</td>
    </tr>
    <tr>
        <td>2</td>
        <td>Met 75% of goals</td>
    </tr>
    <tr>
        <td>3</td>
        <td>Met 50% of goals</td>
    </tr>
    <tr>
        <td>4</td>
        <td>Met < 50% of goals</td>
    </tr>
</table>" to type "System.Xml.XmlDocument". Error: "Name cannot begin with the ' ' character, hexadecimal value 0x20. Line 20, 
position 12."
@ALuckyGuy

Updated comment above to itself be escaped properly...

@ALuckyGuy

Just downloaded the Sep 9, 2013 build. Problem still exists.

@ALuckyGuy

Just in case anyone else encounters this before it's fixed, I have found a work-around. I can use &#x3C; to include a < char in my data. FitNesse seems to ignore this and pass it unmolested to the fixture, and then when converted to an XML object, the '<' char ends up in the data correctly.

@amolenaar
Collaborator

I was working on #274 where HTML fragments are pasted in the page output without encoding. In doing so, the reverse (decode HTML when reading cell content) should also work. I applied a fix as part of #274.

Encoding/decoding is not done when cell content starts and ends with a HTML tag (span, div, table). This is what you do when you embed a (wiki formatted) table for example. If that's the case, the content is left as-is.

The advantage over the old code is that this behaviour is consistent over all table types.

@ALuckyGuy

Working with the 20130918 build, there is still something unusual going on... I'm just not quite sure what it it.

I see test FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestHtmlContent, which succeeds. I think the only reason it succeeds is that in the test, the slim server isn't trying to convert the text to valid XML.

Using a similar test on my local PowerSlim server, where my test intentionally attempts to convert the passed text to an XML document, it fails. I've added some debugging logic to see the text as I'm being sent by FitNesse, and it's not valid: the < char is in the text, not &lt;. Below is the data as it appears read directly from the stream:

000109:<table>
    <tr>
        <td><span>< 50%</span></td>
    </tr>
    <tr>
        <td><span>< 50%</span></td>
    </tr>
</table>:
@amolenaar
Collaborator

I updated the test a little to use wiki markup instead of plain HTML. Anyway, the output of the test (logged by the echo fixture says:

<table>
    <tr>
        <td><p class="note">&gt; 50% (note) </p></td>
    </tr>
    <tr>
        <td><span class="meta">&lt; 50% (meta) </span></td>
    </tr>
</table>

So what is PowerSlim doing differently?

@ALuckyGuy

My point is I don't think it's doing anything differently... what I shared above is data read directly from the stream, unmanipulated. It's data as it comes from the FitNesse server. My question is what is the java SLIM server doing differently :)

I'm inclined to believe what's happening is that the java SLIM server receives the data as text - so no error is produced - and then some logic re-encodes the < when the data is sent back, so it appears to be working when in fact it is not. I suspect if the test fixture tried to convert the text to an XML document on the SLIM side it would run into the same error I'm hitting.

I've never tried to test the java slim server and not to familiar with the fixtures that are available. Is there one what will simply save the text received to a file, where we might verify that exactly is being received?

@amolenaar
Collaborator

I'll dig in a little deeper. Awkward it seems...

@amolenaar
Collaborator

I checked the code in the FitNesse workspace. It's still the old code. The code has not been updated with the latest master.

@woodybrood: It looks like not all files are updated on the build server. There is a difference in src/fitnesse/testsystems/slim/HtmlTable.java. Can you see why?

@woodybrood
Collaborator

@amolenaar I'm not sure why that would be. It all looks like it is configured correctly. I'm going to wipe out the workspace and run a fresh build. That should ensure the checkout is clean.

@amolenaar
Collaborator

How is the latest build working out? The code is up to date now.

@ALuckyGuy

Just downloaded the JAR and tried it again - same error. The < character is still in the body of the data and thus isn't valid XML.

@amolenaar
Collaborator

I don't get it. If you compile FitNesse yourself, is the problem still persistent?

@amolenaar amolenaar referenced this issue from a commit
@amolenaar amolenaar Added few extra tests to nail down issue #267 (Nested table implement…
…ation doesn't encode < character to invalid XML produced).
a7dd9ab
@amolenaar
Collaborator

I added a few unit tests to illustrate what I think it should be doing.

@amolenaar
Collaborator

Can I close the issue?

@ALuckyGuy
@ALuckyGuy

Ok, so I've gotten some strange results and not sure where to look.

I learned running the java slim server with the -v flag echoes the instruction information sent to the server and I can see that FitNesse is indeed encoding the < character appropriately. However, as I mentioned before, running the PowerSlim server I've modified the code to echo the data immediately after reading from the server stream and I can see that the < character is not encoded properly. This suggests that something on the server side is causing the difference.

The one significant difference that I can see on the server side is the fact that, running the PowerSlim server, COMMAND_PATTERN is specified so a server other than the java SLIM server is executed. Is this causing another path in the code to be taken, one that bypasses the HTML escaping? That's my theory right now but my java skills are kind of weak and being unfamiliar with the code, my progress is slow. If you have any suggestions on where I might look or otherwise confirm/reject my theory, I'd appreciate it.

@mgaertne
Collaborator
@amolenaar
Collaborator

@mgaertne It's a SLIM server for Windows power shell (http://vlasenko.org/powerslim/)

@velaskec Can you say anything about this?

The COMMAND_PATTERN is only used to build the to-be executed command. It has nothing to do with encoding the content (which is done in the SlimTable class, btw).

@konstantinvlasenko

I can't see the code right now. But I remember some issue related to < caused by Fitnesse side in case of multy string definition. You can try to use !- ... -! inside your !define if it is the same case I have in mind.

@ALuckyGuy

@amolenaar Like I said, the COMMAND_PATTERN is the only significant difference I could see, doesn't mean it is :) I'm going to try to get to the bottom of this one way or the other this week. My work schedule isn't being very friendly right now so it's slow going for me. I just hoped someone might have an idea what to look at based on what I'm seeing.

@konstantinvlasenko I haven't tried putting !- -! inside my define, but I don't think that would help anyway. If you follow the issue from above, you'll see the problem is that the FitNesse server isn't escaping the text in data, not that it is. It's escaped properly when it's sent to the java SLIM server - I've confirmed that with amolenaar's help - but it's not escaped when sent to the PowerSlim server, which doesn't seem to make sense. Use the define at the top of this thread to demonstrate the trouble yourself if you'd like. Either I'm completely off my marbles - possible I suppose - or there's an interesting little bug hiding in the FitNesse server somewhere.

@amolenaar
Collaborator

I'd like to make a new release. For that it would be nice if this issue is resolved.

I'd like to give it a try myself. What would an "echo" fixture look like in Powerslim?

@amolenaar amolenaar removed this from the Next release milestone
@amolenaar amolenaar closed this
@antoine-aumjaud antoine-aumjaud referenced this issue from a commit in antoine-aumjaud/fitnesse
@amolenaar amolenaar Added few extra tests to nail down issue #267 (Nested table implement…
…ation doesn't encode < character to invalid XML produced).
21aad5b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.