SourceCode style and *.docx output

[dymanic]

The "Source Code" style in docx output does not work properly when edited and used as a reference style.

Input file test.markdown:

~~~{.haskell}
putStrLn "hello"
~~~

Command:

pandoc -f markdown -t docx -o test.docx test.markdown

Expected output:

The output file test.docx should contain a single "Source Code" style which can be edited, so that it can be used with --reference-docx and applied to subsequent docx output.

Actual output:

The output docx contains two copies of the "Source Code" style.

This seems to cause further problems using the output as a reference docx. I.e.:

1. Create test.docx as detailed above.

2. Open test.docx in MS Word and select the whole line of source code.

3. Alt-Ctrl-Shift-S to open the Styles box. You will notice two styles named "Source Code", one which is selected and one which isn't.

4. Right click on the selected "Source Code" style (i.e. the one applied to the selected line).

5. Make some modifications to the style. [Note: you will notice if you try to make changes to the other "Source Code" style, you will receive the message "This style name already exists or is reserved for a built-in style" after clicking Modify -> Format]

6. Save the document.

7. Run the following to use test.docx as a style reference for another conversion:

   pandoc -f markdown -t docx -o test2.docx --reference-docx test.docx test.markdown
   

8. You will see that the changes made to the Source Code style in test.docx have not been applied in test2.docx

Versions:

pandoc 1.13.2 Windows 7
MS Word 2013

[mpickering]

Related: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/pandoc-discuss/4YkX-B-3Umk/9ioO4TwnTHwJ

[dymanic]

Thanks Matthew.

Yes, I can edit the reference test.docx so it only contains the Source Code style that I want (either by deleting the wrong Source Code style in Word or by editing styles.xml directly).

However, any further conversions using that as a --reference-docx always add a new Source Code style to the output, and then use that new style.

So, I end up with an unwanted "Select all source code blocks and change to the other Source Code style" step in my conversion process.

[jgm]

The docx writer generates the SourceCode style based on the highlighting
style that was selected with --highlight-style. The generated version
replaces whatever wa in the styles.xml file in the reference.docx. So,
I have removed the SourceCode style from the reference.docx.

This addresses a bug in pandoc, but it may not solve your problem,
depending on the kind of alterations to the SourceCode style you
wanted. First, see if any of the styles pandoc provides are suitable.

If not, we might think about how to make this customizable. A good
solution, I think, would be for the docx writer NOT to replace a
SourceCode style if one is found in the reference.docx, but otherwise
to generate one. Another solution might be to create a means for
generating new custom highlighting styles that would work with
--highlight-style and highlighting-kate. Anyway, tell us more
about the kinds of style modifications you were interested in.

[zgarnog]

I found a different method for working around this.

If you use the --no-highlight option, the "Source Code" style from your docx will be used on your source code blocks.

Maybe pandoc works as intended, you just need to use this option instead?

(I am using pandoc version 1.13.1 under cygwin on Windows 7 x64)

[lierdakil]

Pandoc does not work as expected. It doesn't replace existing token styles, it adds new ones without any regard for their prior existence. This leads to duplicate token styles in resulting docx, which is definitely not what I would expect.

It boils down to styleToOpenXML (in docx writer) implementation in that it blindly adds token styles to existing stylesheet.

Parsing stylesheet for existing styles and skipping those that already exist might be a solution, and we likely need to parse said styles anyway to solve #1607 and #1933

[jgm]

Yes, I think it makes sense to check for token styles, and not replace
them if they're defined already. That will mean that
--highlight-style won't do anything if you've got token styles defined
in the reference docx, and this may produce surprising behavior when
people use a document with highlighted code as a reference docx. But
it does more important to give users more flexibility to generate
token styles.

[lierdakil]

@jgm here's an idea:

1. if --highlight-style is explicitly defined, replace existing *Tok styles
2. if not, keep *Tok styles already defined in reference docx, and add any missing with default properties
3. if --no-highlight is explicitly defined, remove *Tok styles.

I think to make distinction between --highlight-style explicitly set to default (pygments I believe?) and not set at all, I'll have to make modifications to options parser, but this looks as intuitive as possible to me.

[jgm]

Well, writerHighlightStyle in WriterOptions would have to change its
type. That would be a major API change; existing code that defines this
field would break. So, that's a strike against the idea. But I think
it would be okay, probably, just to have 2 and 3 without 1.

+++ Nikolay Yakimov [Feb 21 15 15:27 ]:

@jgm here's an idea:

1. if --highlight-style is explicitly defined, replace existing *Tok styles
2. if not, keep *Tok styles already defined in reference docx, and add any missing with default properties
3. if --no-highlight is explicitly defined, remove *Tok styles.

I think to make distinction between --highlight-style explicitly set to default (pygments I believe?) and not set at all, I'll have to make modifications to options parser, but this looks as intuitive as possible to me.

---

Reply to this email directly or view it on GitHub:
#1872 (comment)

[lierdakil]

@jgm then again, while this will be ugly and I'm not advocating anything, it's not strictly necessary to change writerHighlightStyle type. One might opt to simply add a boolean parameter like writerHighlightDefined and set it in option parser -- this won't be as much of an API change. Anyway, I'll just fiddle with (2,3) for now.

[lierdakil]

This should be fixed (somewhat) by #1968, which is merged.

Token styles from reference.docx now take precedence. If --no-highlight, then token styles are not present in output docx altogether, regardless of whether those were defined in reference.docx or not.

[mpickering]

Please reopen if you are still having problems.

[shreve]

Hey sorry to resurrect such an old issue, but this is all the right context for the issue I'm having now.

If the *Tok styles are already defined in the reference, they won't be added. This is good. However, this isn't true for the SourceCode style which is added in the same way under the same circumstances.

I'm trying to style code blocks and get syntax highlighting in them. I saw that the compiled docx contains all the relevant styles injected at the end. I was able to stop the injection of all the *Tok styles by copying them into the source reference.docx, but the SourceCode style is always injected, overwriting my custom style.

[jgm]

@shreve I'd have to look into why this is happening, but probably you shouldn't be manually injecting Tok styles. Have you considered defining a custom highlight style instead? Then the proper Tok styles would be auto-generated, and would work in other formats besides docx as well. See --highlight-style documentation.

[shreve]

Thank you, especially for the fast reply. The custom style does get me closer because I can change the background color, but I'm concerned with the code wrapper style rather than the tokens. I'm trying to add padding and a border and that doesn't seem to me to be supported by the highlight style.

[agusmba]

Mmm, I've had no problems defining a SourceCode custom style with paragraph background color, border, etc. (not sure about padding, but using paragraph styling it should work).

[shreve]

Since this is unexpected, I'll make a new issue with reproduction details rather than continue on here.

[windmaomao]

I also can make it work with Source Code style for paragraph settings, except the font , which is coded as "consolas".