How to find and replace ALL text in Word Documents with NPOI?

[adamfisher]

I am attempting to find a set of variables (denoted by replacementValues in a Word document. The two issues I am facing are 1) I need the method to return whether a replacement actually happened (the dirty variable) and 2) replace all values in replacementValues dictionary but that doesn't appear to work all of the time due. So I first try calling the NPOI FindAndReplaceText() method but that doesn't appear to catch everything. Some stuff in headers/footers or inside tables within the document body get missed. I was wondering if it had to do with tables being embedded inside the header/footer and body. Perhaps FindAndReplaceText() wouldn't work in those instances so I tried to add extra code to account for it. This caught a few more but not everything.

I created this TestDocument.docx as an example.

The FindAndReplaceValues() Extension Method:

public static class XwpfDocumentExtensionMethods
{
	/// <summary>
	/// Performs a find and replace on a Word document. 
	/// </summary>
	/// <param name="document">The Word document.</param>
	/// <param name="replacementValues">The mapping of values to be replaced within the document.</param>
	/// <returns>True if a value was replaced in the file, false otherwise.</returns>
	public static bool FindAndReplaceValues(this XWPFDocument document, Dictionary<string, string?> replacementValues)
	{
		bool dirty = false;
		
		foreach (KeyValuePair<string, string?> replacementValue in replacementValues)
		{
			// Use built-in NPOI Find and Replace first
			long initialCharacterCount = document.CountCharacterLength();
			document.FindAndReplaceText(document, replacementValue.Key, replacementValue.Value);
			long finalCharacterCount = document.CountCharacterLength();
			dirty = dirty || initialCharacterCount != finalCharacterCount;

			// Do find and replace on elements that NPOI's built-in FindAndReplaceText() method doesn't work on for some reason.
			foreach (XWPFHeader? header in document.HeaderList)
			{
				foreach (XWPFTable? table in header.Tables)
				{
					dirty = dirty || table.FindAndReplaceValue(replacementValue);
				}

				foreach (XWPFParagraph? paragraph in header.Paragraphs)
				{
					dirty = dirty || paragraph.FindAndReplaceValue(replacementValue);
				}
			}

			foreach (XWPFParagraph? paragraph in document.Paragraphs)
			{
				dirty = dirty || paragraph.FindAndReplaceValue(replacementValue);
			}

			foreach (XWPFTable table in document.Tables)
			{
				dirty = dirty || table.FindAndReplaceValue(replacementValue);
			}

			foreach (XWPFFooter? footer in document.FooterList)
			{
				foreach (XWPFTable? table in footer.Tables)
				{
					dirty = dirty || table.FindAndReplaceValue(replacementValue);
				}

				foreach (XWPFParagraph? paragraph in footer.Paragraphs)
				{
					dirty = dirty || paragraph.FindAndReplaceValue(replacementValue);
				}
			}
		}

		return dirty;
	}
	
	private static bool FindAndReplaceValue(this XWPFTable table, KeyValuePair<string, string?> replacementValue)
	{
		bool dirty = false;

		foreach (XWPFTableRow? row in table.Rows)
		{
			foreach (XWPFTableCell? cell in row.GetTableCells())
			{
				foreach (XWPFParagraph? paragraph in cell.Paragraphs)
				{
					if (paragraph.Text.Contains(replacementValue.Key))
					{
						paragraph.ReplaceText(replacementValue.Key, replacementValue.Value);
						dirty = true;
					}
				}
			}
		}

		return dirty;
	}

	private static bool FindAndReplaceValue(this XWPFParagraph paragraph, KeyValuePair<string, string?> replacementValue)
	{
		if (paragraph.Text.Contains(replacementValue.Key))
		{
			paragraph.ReplaceText(replacementValue.Key, replacementValue.Value);
			return true;
		}

		return false;
	}

	private static long CountCharacterLength(this XWPFDocument document)
	{
		long characterCount = 0;

		foreach (XWPFHeader? header in document.HeaderList)
		{
			characterCount += header.Text.Length;
		}

		foreach (XWPFFooter footer in document.FooterList)
		{
			characterCount += footer.Text.Length;
		}

		foreach (XWPFParagraph paragraph in document.Paragraphs)
		{
			characterCount += paragraph.ParagraphText.Length;
		}

		foreach (XWPFTable? table in document.Tables)
		{
			characterCount += table.Text.Length;
		}
		return characterCount;
	}
}

So I end up calling it like this:

var replacementValues = new Dictionary<string, string>()
{
	{"[City]", "Houston"},
	{"[State]", "Texas"},
	{"[ProjectNumber]", "123456"},
	{"[Project Number]", "123456"}
};
var dirty = document.FindAndReplaceValues(replacementValues);

[adamfisher]

It seems like this StackOverflow issue is relevant regarding formatting with runs in Word:

Basically, an XWPFParagraph is composed of multiple XWPFRuns, and XWPFRun is a contagious text that has a fixed same style.

So when you try writing something like "[PLACEHOLDER_NAME]" in MS-Word it will create a single XWPFRun. But if you somehow add a few things more, and then you go back and change "[PLACEHOLDER_NAME]" to something else it is never guaranteed that it will remain a single XWPFRun it is quite possible that it will split to two Runs. AFAIK this is how MS-Word works.

How to avoid splitting of Runs in such cases?

Solution: There are two solutions that I know of:

1. Copy text "[PLACEHOLDER_NAME]" to Notepad or something. Make your necessary modification and copy it back and paste it instead of "[PLACEHOLDER_NAME]" in your word file, this way your whole "[PLACEHOLDER_NAME]" will be replaced with new text avoiding splitting of XWPFRuns.

2. Select "[PLACEHOLDER_NAME]" and then click of MS-Word "Replace" option and Replace with "[Your-new-edited-placeholder]" and this will guarantee that your new placeholder will consume a single XWPFRun.

If you have to change your new placeholder again, follow step 1 or 2.

[tonyqus]

Yes, I know this issue. #666

Word randomly puts splitted text into Runs for some reason. And I'm still looking for a good algorithm to handle this.

[adamfisher]

Here is the code in NPOI for find and replace. I guess one question I have is whether it covers tables in headers and footers based on it iterating through the root document tables property? Tables can also appear in headers and footers. Just based on how the other code is iterating over elements in this method, I'm wondering if that needs to be enhanced? Either way it still didn't help with my sample document above ☝️.

[tonyqus]

This code is being improved in the latest release. #1018

Definitely we can include header and footer. But I suggest we provide a parameter called includeHeaderAndFooter=false for the sake of performance. Because in most cases, developers don't need this FindAndReplaceText to replace header and footer.

And NPOI is actively enhancing XWPF namespace (for Word) in recent releases. Our goal is to become the most powerful Word read/write library in .NET community.

And if you are willing to contribute a PR, it would be great.

[adamfisher]

Yes I can include a PR but I guess I don't know the document standard enough to know whether I need to also process tables and paragraphs separately in the headers and footers or are those included as part of the paragraphs and tables collection in the header and footer collection?

Note that this still doesn't fix my original issue with the test document that I attached. For some reason formatting or something behind the scenes is causing it to not be replaced in every instance.

[adamfisher]

I ended up modifying my extension method to process the tables contained in the headers and footers since that's where most of the replacements needed to happen in my case. As for detecting is a change was made, I use a hash of the document text so I know whether the document changed.

public static class XwpfDocumentExtensionMethods
{
	/// <summary>
	/// Performs find and replace on a Word document. 
	/// </summary>
	/// <param name="document">The Word document.</param>
	/// <param name="replacementValues">The mapping of values to be replaced within the document.</param>
	/// <returns>True if a value was replaced in the file, false otherwise.</returns>
	public static bool FindAndReplaceValues(this XWPFDocument document, Dictionary<string, string?> replacementValues)
	{
		var extractor = new XWPFWordExtractor(document);
		string initialThumbprint = extractor.Text.GenerateHash();

		foreach (KeyValuePair<string, string?> replacementValue in replacementValues)
		{
			// Try built-in NPOI Find and Replace first
			document.FindAndReplaceText(document, replacementValue.Key, replacementValue.Value);

			// Now iterate over other elements the built-in find and replace doesn't scan
			foreach (XWPFHeader? header in document.HeaderList)
			{
				foreach (XWPFTable? table in header.Tables)
				{
					table.FindAndReplaceValue(replacementValue);
				}
			}
			
			foreach (XWPFFooter? footer in document.FooterList)
			{
				foreach (XWPFTable? table in footer.Tables)
				{
					table.FindAndReplaceValue(replacementValue);
				}
			}
		}

		string finalThumbprint = extractor.Text.GenerateHash();
		return !string.Equals(initialThumbprint, finalThumbprint);
	}
	
	private static void FindAndReplaceValue(this XWPFTable table, KeyValuePair<string, string?> replacementValue)
	{
		foreach (XWPFTableRow? row in table.Rows)
		{
			foreach (XWPFTableCell? cell in row.GetTableCells())
			{
				foreach (XWPFParagraph? paragraph in cell.Paragraphs)
				{
					paragraph.ReplaceText(replacementValue.Key, replacementValue.Value);
				}
			}
		}
	}
}