Private final fields in BigDecimal inspected in Java 17

[phillipuniverse]

I am upgrading from spring-data-mongodb 3.2.3 to 3.3.1, and also to Java 17. Here is a test that worked before:

EDIT: this seems to be a Java 17 problem, the same issue exists in Spring Data Mongo 3.2.9 with Java 17 and Kotlin 1.6. Although this did work ok with Java 11.

package com.shipwell.opus.utils.numerics

import org.assertj.core.api.Assertions.assertThat
import org.bson.types.ObjectId
import org.junit.jupiter.api.Test
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.data.mongo.DataMongoTest
import org.springframework.data.annotation.Id
import org.springframework.data.mongodb.core.mapping.Document
import org.springframework.data.mongodb.repository.MongoRepository
import java.math.BigDecimal

@DataMongoTest
class RateDecimalMongoTest(@Autowired private val repository: RateDecimalMongoTestDocumentRepository) {

    @Test
    fun `test rate decimal saved and loaded properly`() {
        val rateDecimal = RateDecimal("203.0008")
        val objectId = repository.save(RateDecimalMongoTestDocument(someObj = rateDecimal)).id!!

        val savedObject = repository.findById(objectId).get()
        assertThat(savedObject.someObj).isInstanceOf(RateDecimal::class.java)
        assertThat(savedObject.someObj).isEqualTo(BigDecimal("203.0008"))
    }
}

@Document(collection = "RateDecimalMongoTestDocumentRepository")
class RateDecimalMongoTestDocument(
    var someObj: Any,
    @JvmField @Id var id: ObjectId? = null
)

interface RateDecimalMongoTestDocumentRepository : MongoRepository<RateDecimalMongoTestDocument, ObjectId>

This test was fine in 3.2.3 was fine with Java 11, but now fails with:

Unable to make field private final java.math.BigInteger java.math.BigDecimal.intVal accessible: module java.base does not "opens java.math" to unnamed module @7dc36524
java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.math.BigInteger java.math.BigDecimal.intVal accessible: module java.base does not "opens java.math" to unnamed module @7dc36524
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at org.springframework.util.ReflectionUtils.makeAccessible(ReflectionUtils.java:787)
	at org.springframework.data.mapping.context.AbstractMappingContext$PersistentPropertyCreator.doWith(AbstractMappingContext.java:550)
	at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:710)
	at org.springframework.data.mapping.context.AbstractMappingContext.doAddPersistentEntity(AbstractMappingContext.java:424)
	at org.springframework.data.mapping.context.AbstractMappingContext.addPersistentEntity(AbstractMappingContext.java:381)
	at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentEntity(AbstractMappingContext.java:266)
	at org.springframework.data.mapping.context.AbstractMappingContext.getPersistentEntity(AbstractMappingContext.java:92)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readDocument(MappingMongoConverter.java:341)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$ConversionContext.convert(MappingMongoConverter.java:2077)

Where the RateDecimalConfig class looks like this:

public class RateDecimal extends AbstractFixedDecimal implements Serializable {

    private static final long serialVersionUID = 1L;

    private static final int scale = 4;

    /**
     * Necessary constructor that MongoDB will use to create the instance
     * @param val String whose value is used to construct the instance
     */
    @PersistenceConstructor
    public RateDecimal(@Value("#root.amount") String val) {
        super(scaleToString(new BigDecimal(val), scale));
    }

    public RateDecimal(double val) {
        super(scaleToString(new BigDecimal(val), scale));
    }

    public RateDecimal(int val) {
        super(scaleToString(new BigDecimal((val)), scale));
    }

    public RateDecimal(long val) {
        super(scaleToString(new BigDecimal(val), scale));
    }

}

And then AbstractFixedDecimal:

public abstract class AbstractFixedDecimal extends BigDecimal {

    AbstractFixedDecimal(String val) {
        super(val);
    }

    static String scaleToString(BigDecimal unscaledVal, int scale) {
        return unscaledVal.setScale(scale, RoundingMode.HALF_UP).toString();
    }
}

To deal with the serialization of a RateDecimal to a document, I have this converter registered as a mongoCustomConversions bean:

    @Bean
    fun mongoCustomConversions(): MongoCustomConversions {
        return MongoCustomConversions(mutableListOf(
           RateDecimalConverter()
        ))
    }

    private final static class FixedDecimalToDocumentConverter<T extends AbstractFixedDecimal> implements Converter<T, Document> {
        @Override
        public Document convert(@NotNull T source) {
            Document doc = new Document();
            doc.append(DefaultMongoTypeMapper.DEFAULT_TYPE_KEY, source.getClass().getName());

            // Mongo by default has BigDecimal converters to both Decimal128 and String, so there is no obvious
            // correct format to use when storing the value. Choosing String because of spring data Mongo documentation.
            // See https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mapping-conversion
            doc.append("amount", source.toString());
            return doc;
        }
    }

My question is, what changed in 3.3 that caused the process to look at these private fields when before it didn't? It kind of makes me think that my PersistentConstructor is being ignored.

This is the same behavior in 3.2, but has to do with using Java 17.

[phillipuniverse]

I have what I would call a somewhat hacky workaround where I manually register a converter from the document to a RateDecimal:

    @Bean
    fun mongoCustomConversions(): MongoCustomConversions {
        return MongoCustomConversions(mutableListOf(
            RateDecimalConverter(),
           DocumentToRateDecimalConverter()
        ))
    }

    private final static class DocumentToRateDecimalConverter<T extends AbstractFixedDecimal> implements Converter<Document, T> {
    
        @Override
        public T convert(@NotNull Document doc) {
            return (T) new RateDecimal(
                    doc.get("amount", String.class));
        }
    }

The test passes here. But it kind of defeats the purpose because I want a flexible mapping for all subclasses of AbstractFixedDecimal. If I register a custom converter I now have to have dynamic construction of AbstractFixedDecimal subclasses, or register a converter for each one. Am I missing a piece of configuration that would make this work on 3.3 in a similar way that it did before? Or have some way to tell Spring Data to ignore the intVal field from BigDecimal?

[phillipuniverse]

1 more workaround is to add an --add-opens to the jvmArgs in the test task. This obviously won't work at runtime (without the corresponding jvm argument there too) but does fix the issue:

tasks.test {
    systemProperty("spring.profiles.active", "test")
    maxHeapSize = "3096m"
    jvmArgs = listOf("--add-opens=java.base/java.math=ALL-UNNAMED")
    useJUnitPlatform()

    // show standard out and standard error of the test JVM(s) on the console
    testLogging {
        showStandardStreams = true
        exceptionFormat = TestExceptionFormat.FULL
    }
    finalizedBy(tasks.jacocoTestReport)
}

[phillipuniverse]

Hm - I went back to Java 11 and realized that there has been a warning there all along:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.util.ReflectionUtils (file:/Users/phillip/.gradle/caches/modules-2/files-2.1/org.springframework/spring-core/5.3.9/cfef19d1dfa41d56f8de66238dc015334997d573/spring-core-5.3.9.jar) to field java.math.BigDecimal.intVal
WARNING: Please consider reporting this to the maintainers of org.springframework.util.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

So looks like this has always been an issue but Java 17 took the warnings off and hard-failed with the new strong encapsulation changes.

[mp911de]

Java 9++ introduced strong encapsulation and each Java runtime increased the level of encapsulation. Domain models are fully analyzed by the MappingContext and domain models must be accessible for Spring Data to work.

Subclassing BigDecimal no longer considers BigDecimal as a simple type but the class is being decomposed into its Java properties. Instead of subclassing, we recommend composition.