While using @before-cell and @after-cell random cells and even rows can be deleted

[nicky1038]

---Sorry, accidendally I have closed issue #37 so this is it once again---

What steps will reproduce the problem?

1. Create a lot of several mergefields in .docx document (e.g. @before-cell#if($p.source) and @after-cell#end in a table cell).
   In my case the larger is the document the more is a probability that some of the mergefields will be divided into different runs in document.xml as it is shown below.
   
   
2. Try to fill this template using java/scala.

What is the expected output? What do you see instead?
I expect that every mergefield will be filled but in practice the other situation happens - those mergefields that were divided into various runs (during creating template) will not be filled. If you try to launch your java code once again you will discover that the same mergefields were not filled because mergefields are stored in the template itself.

What version of the product are you using? On what operating system?

• xDocReport 1.0.6, but I tried to use earlier versions too - it does't matter.
• MS Word 2010 for creating templates - the problem, as I understand, is in it but I would like to ask you to create more smart solution that will take this stupid fault into consideration
• OS Windows 8.1

Please provide any additional information below.
The most interesting thing is that if you try to delete and re-enter bad mergefields with the same information there will be a probability they will not be spoiled again.

I understand it is not a problem of yours and your library. Moreover it's really easy to use and it's a pleasure to work with it, but this trouble is a real headache for me and doesn't allow to work normally. Please add some features so that this problem will not be a problem anymore.

Here you can find a template, scala/java code and resulting output. I need to implement my project in scala so this template was processed using scala code too. But for your convenience I've added equivalent java code.
https://www.dropbox.com/sh/f2iyaiyvqg7wy1m/AAAf7nfNnTuULFstakS-59hUa?dl=0

This issue was also created there: https://code.google.com/p/xdocreport/issues/detail?id=483
UPD:
I've been tried to find a bit more information what exactly is wrong and I've found that as it's shown at screenshots every run has its own RSID parameter. It serves for conveniet merging of documents and can be turned off. It could be helpful, but..

• In the attached template "@before-cell#if($print.variableThree)" mergefield was broken even without any RSIDs and wasn't filled correctly.
• In my project (sorry, I can't attach it) it seems that turnind RSIDs off helped and fresh mergefields (I tested ones with "@before-cell" token - it is the main headache) were really created as a single run. Whatever after processing resulting table didn't contain some cells/rows/columns (I can't guess a rule in which they are being deleted) as before.

Then I've tried to edit document.xml myself to make broken mergefields look like ordinary ones. It didn't affect on template's view but my program began throwing strange parse exceptions even if I've only removed really unnecessary runs.

Thank you very much!
Nick.

[angelozerr]

@nicky1038 as I said you, it should be very cool if you create a JUnit like It should be very cool if you could create a test with your case like https://github.com/opensagres/xdocreport/blob/master/integrationtests/fr.opensagres.xdocreport.core.test/src/test/java/fr/opensagres/xdocreport/document/docx/preprocessor/DocxPreprocessorWith2InstrText.java

It will win me a lot of time if you do that. Thanks!

But I'm very busy, I don't know when I will have time to study your problem.

[nicky1038]

@angelozerr OK, here it is (again in scala but this code almost doesn't differ from java)
https://www.dropbox.com/s/nqywcgxkdmphbld/DocxPreprocessorWithSplittedBeforeCellMergefield.scala?dl=0
I've done the same thing as in your example but with my XML. Preprocessor does cope with splitted mergefields well. Maybe it's good news but the next question is why do we still have a situation where they are not filled correctly.

[nicky1038]

@angelozerr DocxPreprocessor class is really helpful to control output - it's easy just to preprocess document.xml of template and get updated xml, then, if it's OK, try to finally process it.

I've found that if there are multiple subsequent cells in one row with some condition stated like this
{ MERGEFIELD @before-cell#if($print.variableThree) \* MERGEFORMAT }
then after preprocessing this condition will be concatenated with the #end of the previous cell condition:
</w:tc>#end#if($print.variableThree)<w:tc>

With this syntax
{ MERGEFIELD "@before-cell #if($print.variableThree)" \* MERGEFORMAT }
It became possible to split these keywords
</w:tc>#end #if($print.variableThree)<w:tc>

But even such measure didn't help me and my resulting table is incorrect - the whole row is being deleted (in xml).
Please, answer two more questions:

• Can a situation when two keywords were placed into one place between the same tags really be a problem or it's ok?
• Are there any ways to see library's intermediate results between preprocessing and ready output?

Excuse me please for my persistence and wasting your time but it's really important to find out what's wrong.

[angelozerr]

@nicky1038 I have fixed #42, perhaps it's the same problem than you?

[nicky1038]

@angelozerr Sorry, I think I finally confused you. Initially I thought that my problem is in preprocessor but it turned out that it works fine, as I have said in my last reply - #if($..) and #end fields were token out of <w:tc>...</w:tc> block as it should be. Whatever, after preprocessing something goes wrong and the whole row, that contains cells surrounded with #if($..) and #end code, is being deleted.
Thats why, unfortunately, build 682 didn't help.

[angelozerr]

To be honnest with you, I'm a little lost.

I have not a lot of time to understand problem of each users (I have already token time with your case with splitted mergefield and it seems that there are not problem).

It seems that you are using @before-cell which could provides some problem (eg : if you use it with #if, ooxml table column will not updated, perhaps it's your problem).

If you wish I help you, please create a very simple case which doesn't work and share it with your github project.

Thanks

[nicky1038]

@angelozerr I'm really sorry that I have to steal your time with such problem which I can't even properly explain. But it's really critical for me.

I've tried to recreate a situation with a whole row deletion but I only came across a single cell deletion. This case is interesting too, and maybe it is an effect of the same problem.

https://github.com/nicky1038/Java-and-Scala-Tests/tree/master/xDocReport

Let me explain what did I do, what happened and what does my repo contain.

My template contains a table in which rows are records (instances of a class) and columns are class' fields ("variables") that can be optionally printed, as user decides.

• MyBean.java (as the name implies) is a class that represents a single record
• Print.java is a class that contains boolean values. Each of them matches a field of MyBean class and says if user wants to print it.
• xDocReportTest.java is the main application and processes my template.

As soon as I created my template I processed it and saw that the whole column with 9th variable (I don't know why 9th) was deleted. Then I made a backup of this template, re-entered all mergefields in defective column and launched my application again... The column with 9th variable's data appeared, but not completely - its header cell was deleted just as before.
Then I copied this template once again, re-entered mergefields (absolutely with the same text!) ... and got the correct output.

• In "cases" folder you will find these three templates (and extracted document.xml files). It seems these templates are the same! ..but resulting outputs are different just as I described. I've attached these outputs too. But you can try to process each template yourself.

What else?

• CreatePreprocessedXml.java... Creates preprocessed xmls. And for your convenience they were put into "cases" folder too. It's strange but these three preprocessed xmls only differ by rsids and... Bookmarks.

Now you do know the problem and do have its sample. Please do something :(

[angelozerr]

@nicky1038 I'm very busy today and it seems that you are using @before-cell, @after-cell which is not official because the support is basic and I think there are several bugs.

It was a long time that I have developed that and it requires a lot of time for me to improve @before-cell, @after-cell .

I will try to study the problem when I will find time. Any contribution are welcome!

[nicky1038]

@angelozerr Hi, it's me again :D Recently I finished the rest part of my work and decided to look at my template processing code again. And... it is awful. It puts an instance of my bean and a boolean value to IContext with the same names and sends information about EVERY field to FieldsMetadata, even if it isn't necessary.
Even if sometimes some mergefields in lazy loops really remain unfilled after processing, this problem can be avoided by using explicit foreach-loops. Besides, everything works perfectly (so far). Thank you for this exellent library, and sorry that you had to spend your time on me.