How to maintain backwards compatibility with null vs empty collections when going from 1.4.4 to 1.4.5?

[bmr-exerp]

I'm working on upgrading the XStream version used at my company. We have a 'workflow import' feature that takes an XML file and unmarshals it to a graph of JPA entities. A snippet of the XML:

<workflowData>
	<workflows>
		<workflow id="1">
...
			<steps>
				<step reference="2"/>					
				<step reference="3"/>
			</steps>
		</workflow>
	</workflows>
...

This is expected to be unmarshalled into a Workflow JPA entity. The Workflow entity has two fields of interest:

	@LazyCollection(LazyCollectionOption.FALSE)
	@OneToMany(mappedBy = "workflow", cascade = { CascadeType.PERSIST, CascadeType.MERGE })
	@IndexColumn(name = "workflow_id")
	@Fetch(FetchMode.JOIN)
	private Set<TaskStep> steps = new HashSet<TaskStep>();

	@LazyCollection(LazyCollectionOption.FALSE)
	@OneToMany(mappedBy = "workflow", cascade = { CascadeType.PERSIST, CascadeType.MERGE })
	@IndexColumn(name = "workflow_id")
	@Fetch(FetchMode.JOIN)
	private Set<TaskProgress> taskProgresses = new HashSet<TaskProgress>();

We configure the XStream appropriately, e.g. using xstream.alias("step", TaskStep.class); to map "step" to a TaskStep entity.

When migrating past 1.4.4 of XStream, two issues arise related to these two fields.

1. The TaskStep JPA class, which defines its parent Workflow as a @ManyToOne field (the inverse of the first field in the XML above), no longer sees the field being set. I.e. in 1.4.4, Workflow.steps is a collection of TaskStep, with the TaskStep.workflow of each item in that collection being correctly set to its parent workflow. But in 1.4.5, TaskStep.workflow is being set to null. The step being referenced in the XML (<step reference="3"/>) does not explicitly reference the workflow to which it belongs, but it's correctly inferred in 1.4.4 and not in 1.4.5.
2. The XML snippet mentioned contains no XML element corresponding to the taskProgresses collection. In 1.4.4, this is not a problem, and the value of Workflow.taskProgresses after unmarshalling is an empty hashset. In 1.4.5, the value of this field on the unmarshalled JPA entity is null, rather than an empty collection. This is causing a NullPointerException.

I'm unable to modify the XML provided to this import feature, and therefore I must retain backwards-compatibility. However, in my test, if I add <taskProgresses/> to the workflow XML, then the expected behaviour is preserved.

Can anyone advise?

[joehni]

It seems someone piled up technical debt for more than a decade... 😁 we have 1.4.21 now

1. All these annotations have no meaning for XStream.
2. From a short look over the changes for this release I cannot detect anything that would cause such a behaviour, but I don't know all these changes in detail any more, it's been more than a decade.
3. The values in the reference and id attributes are used by XStream if you configure it to use ids for references. The references 2 and 3 will require two other XML elements (of a compatible type for TaskStep) with id 2 and 3 in the snipped part of the XML ... I can't tell if that's the case. It's the referenced object's value for the workflow element that is of interest here. A reference means XStream will use exactly the same object again, not a copy.
4. By default XStream works on the class layout using reflection. You provided declarations of class members (steps and taskProcesses), but you tell nothing about the class where these members are declared. The converter of that class will do the work here.
5. How is the XML generated? Generate the structure once using the old XStream version and once using the newer one and look for differences. That will give you an idea, what XStream expects to read by default. Only then you're able to tweak it to process the structure differently. Otherwise you're simply guessing with trial and error.
6. You've upgraded other dependencies as well e.g. JPA? Or were the old XML files written also with an old Java version?
7. XStream's reflection converter does not process constructors (like Java serialization), the initializations above will not be executed. Therefore it is normal that a member is null if it is not contained in the incoming XML. Did you modify also your class types during "modernization" e.g. was the type serializable before?

[bmr-exerp]

Hi @joehni ,

Try 15+ years. I'm upgrading us from 1.3.1, and 1.4.4 is just a stepping stone.

I've created a minimal example to avoid any confusion, and to demonstrate the problem verifiably. I've tested it with 1.4.21 (the latest commit), still fails.

Reproduction steps:

1. Check out commit hash f44f8062
2. Run WorkflowConfigServiceBeanTest: confirm test passes
3. Modify the xstream.version in the POM to 1.4.5
4. Rerun WorkflowConfigServiceBeanTest

Expected behaviour: test continues to pass.

Actual behaviour: test fails because Workflow.getTaskProgresses() returns null, whereas in 1.4.4 it returns an empty collection.

5. Check out a5c9354c, which uses the latest XStream version
6. Rerun WorkflowConfigServiceBeanTest
7. Assert test fails for the same reason as 4.

[joehni]

Well, I guess, the difference between 1.4.4 and 1.4.5 is that since then XStream is able to detect from the Java runtime that it can use the enhanced mode, i.e. it behaves like Java Serialization which implies that it calls no constructors hence the taskProcesses in Workflow are not initialized.

From XStream's PoV this is absolutely correct. If you add in your test the following snippet between the unmarshalled workflowData and your assert, you'll see that the provided XML has been perfectly unmarshalled:

final XStream xstream = factory.getWorkflowDataXmlFormat();
String xml = xstream.toXML(workflowData);
System.out.print(xml);

Remember, XStream is designed for serializing an object to XML and back i.e. the generated XML does not contain an element for taskProcesses only if the member had been null which is only achievable by not running the constructors.

The pure Java mode does not support this ability and you may live with the "limitation" that your unmarshalled object might look differently, because XStream has to run its constructor. Strictly spoken, the unmarshalled object no longer equals the marshalled one in any detail.

Now, since your application did run well with XStream 1.4.4 running in pure Java mode, you might simply activate this for the default converter on purpose by registering a ReflectionConverter using the pure Java mode:

xstream.registerConverter(
    new ReflectionConverter(xstream.getMapper(), new PureJavaReflectionProvider()),
    XStream.PRIORITY_VERY_LOW);

BTW: XStream is designed for concurrent execution. Once you've set up the instance, you can keep it static and use it concurrently. The configuration is the most expensive part and reusing the instance will also reuse XStream's internal caches. Just never reconfigure a shared instance.