[mongodb] persistence query gives ClassCastException for QuantityTypes

[ulbi]

Expected Behavior

If I query the historic states in analyze, graphing or use the persistence interface, I want to get the stored states and no error ;-)

Current Behavior

In Openhab I query the history for an item and want to get the stored states for a given timerange.
In the MainUI - I navigate to the settings->items and select an item with a type of Number:Energy and want analyze it's value, but nothing gets returned, but in the log I see:

2024-01-20 07:05:05.242 [DEBUG] [b.internal.MongoDBPersistenceService] - Query: { "item" : "Erdgeschoss_Kueche_Kuehlschrank_meter_totalKWH_currentyear" , "timestamp" : { "$gte" : { "$date" : "2024-01-19T06:05:02.691Z"} , "$lte" : { "$date" : "2024-01-20T06:05:02.690Z"}}}
2024-01-20 07:05:05.251 [ERROR] [internal.JSONResponseExceptionMapper] - Unexpected exception occurred while processing REST request.
java.lang.ClassCastException: null

Possible Solution

I checked the code for the MongoDB persistence service and see that in https://github.com/openhab/openhab-addons/blob/9e1f87db86409ec2ea1a36ea0de64bbd14c79608/bundles/org.openhab.persistence.mongodb/src/main/java/org/openhab/persistence/mongodb/internal/MongoDBPersistenceService.java

Method public Iterable query(FilterCriteria filter) {

        while (cursor.hasNext()) {
            BasicDBObject obj = (BasicDBObject) cursor.next();

            final State state;
            if (item instanceof NumberItem) {
                state = new DecimalType(obj.getDouble(FIELD_VALUE));
            } else if (item instanceof DimmerItem) {
                state = new PercentType(obj.getInt(FIELD_VALUE));
            } else if (item instanceof SwitchItem) {
                state = OnOffType.valueOf(obj.getString(FIELD_VALUE));
            } else if (item instanceof ContactItem) {
                state = OpenClosedType.valueOf(obj.getString(FIELD_VALUE));
            } else if (item instanceof RollershutterItem) {
                state = new PercentType(obj.getInt(FIELD_VALUE));
            } else if (item instanceof DateTimeItem) {
                state = new DateTimeType(
                        ZonedDateTime.ofInstant(obj.getDate(FIELD_VALUE).toInstant(), ZoneId.systemDefault()));
            } else {
                state = new StringType(obj.getString(FIELD_VALUE));
            }

Does not deal with quantity types as well as the write operation does not - it uses a private Object convertValue(State state) { with

        Object value;
        if (state instanceof PercentType type) {
            value = type.toBigDecimal().doubleValue();
        } else if (state instanceof DateTimeType type) {
            value = Date.from(type.getZonedDateTime().toInstant());
        } else if (state instanceof DecimalType type) {
            value = type.toBigDecimal().doubleValue();
        } else {
            value = state.toString();
        }
        return value;
    }

Both will handle Quantity Types as "String", but for sure all other components do expect a QuantityType like it is done in the influxdb persistence service:

    public static Object stateToObject(State state) {
        Object value;
        if (state instanceof HSBType) {
            value = state.toString();
        } else if (state instanceof PointType) {
            value = state.toString();
        } else if (state instanceof DecimalType type) {
            value = type.toBigDecimal();
        } else if (state instanceof QuantityType<?> type) {
            value = type.toBigDecimal();
        } else if (state instanceof OnOffType) {
            value = state == OnOffType.ON ? DIGITAL_VALUE_ON : DIGITAL_VALUE_OFF;
        } else if (state instanceof OpenClosedType) {
            value = state == OpenClosedType.OPEN ? DIGITAL_VALUE_ON : DIGITAL_VALUE_OFF;
        } else if (state instanceof DateTimeType type) {
            value = type.getZonedDateTime().toInstant().toEpochMilli();
        } else {
            value = state.toFullString();
        }
        return value;
    }

I'm unsure, whether to fix this in the read or write section of the mongodb persistence service

Steps to Reproduce (for Bugs)

Create a Number:Energy typed item, pump values in and try to graph it.

Context

I'm trying to use the mongodb persistence service instead of influxdb.

Your Environment

• Version used: (e.g., openHAB and add-on versions) 4.1.1-alpine
• Environment name and version - official docker container
• Operating System and version alpine

[ulbi]

Just trying to fix this bug and figured out, that the driver is still 2.3.1 from 2015 so in order to create some unittests, to properly fix this bug. I need to implement this #16310 as well. At least with the 4.4 driver.

[J-N-K]

You should fix it in both, so that QuantityType can be persisted and restored.

[J-N-K]

Upgrading such an old driver sounds good. Do you know if it is backward compatible, so older versions of MongoDB can still be used?

[ulbi]

You should fix it in both, so that QuantityType can be persisted and restored.

Would do similar to the influxdb implementation.

[ulbi]

Upgrading such an old driver sounds good. Do you know if it is backward compatible, so older versions of MongoDB can still be used?

@J-N-K Just did a quick check. When we use the latest Java Sync driver (4.11) we would support all MongoDB versions down to 3.6 see the Compatibility
3.6 is EOL since April 2021 - see the lifecycles

So I recommend to use 4.11.

Let's see whether I get some tests implemented as well ;-)

[ulbi]

@J-N-K Before I search too long - I've updated the driver and get this at the end of the build:

[INFO] --- karaf:4.4.4:verify (karaf-feature-verification) @ org.openhab.persistence.mongodb ---
[INFO] Using repositories: https://openhab.jfrog.io/openhab/libs-release@id=openhab-release,https://openhab.jfrog.io/openhab/libs-snapshot@id=openhab-snapshot@noreleases@snapshots,https://repo1.maven.org/maven2@id=central
[WARNING] Feature resolution failed for [openhab-persistence-mongodb/4.2.0.SNAPSHOT]
Message: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=openhab-persistence-mongodb; type=karaf.feature; version=4.2.0.SNAPSHOT; filter:="(&(osgi.identity=openhab-persistence-mongodb)(type=karaf.feature)(version>=4.2.0.SNAPSHOT))" [caused by: Unable to resolve openhab-persistence-mongodb/4.2.0.SNAPSHOT: missing requirement [openhab-persistence-mongodb/4.2.0.SNAPSHOT] osgi.identity; osgi.identity=org.openhab.persistence.mongodb; type=osgi.bundle; version="[4.2.0.202401232117,4.2.0.202401232117]"; resolution:=mandatory [caused by: Unable to resolve org.openhab.persistence.mongodb/4.2.0.202401232117: missing requirement [org.openhab.persistence.mongodb/4.2.0.202401232117] osgi.wiring.package; filter:="(&(osgi.wiring.package=com.mongodb)(version>=4.11.0)(!(version>=5.0.0)))"]]
Repositories: {
        file:/home/rene/dev_container/openhabdev/openhab-addons/bundles/org.openhab.persistence.mongodb/target/feature/feature.xml
        mvn:org.apache.karaf.features/framework/4.4.4/xml/features
        mvn:org.apache.karaf.features/specs/4.4.4/xml/features
        mvn:org.apache.karaf.features/standard/4.4.4/xml/features
        mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-core/4.2.0-SNAPSHOT/xml/features
        mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-tp/4.2.0-SNAPSHOT/xml/features
        mvn:org.ops4j.pax.web/pax-web-features/8.0.22/xml/features
}

I could not find where to adjust the package dependencies further. Do you have any hint?

[lsiepel]

How did you update the driver? As the latest version on maven is 3.12.14

Newer versions are listed under a different package name: mongodb-driver-sync

When you update the pom.xml it should work, you only have to fix the build errors by adapting the code :-)

    <dependency>
      <groupId>org.mongodb</groupId>
      <artifactId>mongodb-driver-sync</artifactId>
      <version>4.11.1</version>
    </dependency>

[ulbi]

As mentioned there we have to go to https://mvnrepository.com/artifact/org.mongodb/mongodb-driver-sync for the later version. The build and test is successfull, but the last step fails.

[lsiepel]

I can't reproduce that error as with the changed pom.xml it builds up to the part that it fails because some Types do not exist, and that is expected with such a verison bumb. No issue with dependencies.

[J-N-K]

org.mongodb/mongodb-driver-sync/4.11.1 seems to be the correct artifact.

Maybe you need to add org.mongodb/mongodb-driver-core/4.11.1 and org.mongodb/bson/4.11.1 to the dependencies.

[ulbi]

@J-N-K @lsiepel Thanks for your efforts, but so far no luck. Like I mentioned earlier, the build works, I did fix the type problems. The only build step which fails is "karaf:verify" - I can even find the created jar file.

So this seems to be something I miss in the karaf runtime dependencies, which needs to be added.

The resulting feature.xml looks good to me:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.6.0" name="org.openhab.persistence.mongodb-4.2.0-SNAPSHOT">
    <repository>mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-core/4.2.0-SNAPSHOT/xml/features</repository>
    <feature name="openhab-persistence-mongodb" description="MongoDB Persistence" version="4.2.0.SNAPSHOT">
        <feature>openhab-runtime-base</feature>
        <bundle start-level="80">mvn:org.openhab.addons.bundles/org.openhab.persistence.mongodb/4.2.0-SNAPSHOT</bundle>
    </feature>
    <feature name="org.openhab.persistence.mongodb" description="openHAB Add-ons :: Bundles :: Persistence Service :: MongoDB" version="4.2.0.SNAPSHOT">
        <details>This project contains the official add-ons of openHAB</details>
        <feature prerequisite="true" dependency="false">wrap</feature>
        <bundle start-level="80">mvn:org.mongodb/mongodb-driver-sync/4.11.1</bundle>
        <bundle start-level="80">mvn:org.mongodb/bson/4.11.1</bundle>
        <bundle start-level="80">mvn:org.mongodb/mongodb-driver-core/4.11.1</bundle>
        <bundle start-level="80">mvn:org.mongodb/mongodb-crypt/1.8.0</bundle>
        <bundle start-level="80">mvn:net.java.dev.jna/jna/5.11.0</bundle>
        <bundle start-level="80">mvn:org.slf4j/slf4j-api/2.0.11</bundle>
        <bundle start-level="80">wrap:mvn:org.lastnpe.eea/eea-all/2.2.1</bundle>
    </feature>
</features>

You might want to give it a try with the commit and see whether you receive the same build errors.

Build command I use: mvn clean install -T 1C -pl :org.openhab.persistence.mongodb

[lsiepel]

Tried your commit and play with it for over an hour, but i fail to see what is wrong. I'm no karaf/feature/dependency expert, so hope someone else has time to look into this.

[ulbi]

Got a bit further, now it fails in openhab ;-)

2024-01-24 22:21:06.661 [WARN ] [org.apache.felix.fileinstall        ] - Error while starting bundle: file:/openhab/addons/org.openhab.persistence.mongodb-4.2.0-SNAPSHOT.jar
org.osgi.framework.BundleException: Could not resolve module: org.openhab.persistence.mongodb [259]
  Unresolved requirement: Import-Package: com.mongodb; version="[4.11.0,5.0.0)"

        at org.eclipse.osgi.container.Module.start(Module.java:463) ~[org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:445) ~[org.eclipse.osgi-3.18.0.jar:?]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundle(DirectoryWatcher.java:1260) [bundleFile:3.7.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundles(DirectoryWatcher.java:1233) [bundleFile:3.7.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.doProcess(DirectoryWatcher.java:520) [bundleFile:3.7.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.process(DirectoryWatcher.java:365) [bundleFile:3.7.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:316) [bundleFile:3.7.4]

Used this feature.xml

<?xml version="1.0" encoding="UTF-8"?>
<features name="org.openhab.persistence.mongodb-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.4.0">
	<repository>mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-core/${ohc.version}/xml/features</repository>

	<feature name="openhab-persistence-mongodb" description="MongoDB Persistence" version="${project.version}">
		<feature prerequisite="false" dependency="false">openhab-runtime-base</feature>
		<bundle start-level="80">mvn:org.mongodb/mongodb-driver-sync/4.11.1</bundle>
		<bundle start-level="80">mvn:org.mongodb/bson/4.11.1</bundle>
		<bundle start-level="80">mvn:org.mongodb/mongodb-driver-core/4.11.1</bundle>
		<bundle start-level="80">mvn:org.openhab.addons.bundles/org.openhab.persistence.mongodb/${project.version}</bundle>
	</feature>

</features>

Next step - learning how to either load the driver during runtime or include it in the jar file...

[J-N-K]

You don't need to add anything to the feature.xml. Just add all dependencies to the pom.xml with scope compile. They are embedded automatically. If the feature resolution fails, you didn't add all needed bundles to the POM.

[ulbi]

@J-N-K Thanks for the hint - so far this did not change anything. Got a bit further - now it complains about missing com.amazonaws.auth - and when I drill further even more missing things pop up. So I make progress and will continue later on...

[J-N-K]

In case of com.amazon.auth I believe it might be really necessary, for optional dependencies (i.e. needed for features we don't use), you can add a <bnd.importpackage> property, see https://next.openhab.org/docs/developer/buildsystem.html#external-dependency

[ulbi]

@J-N-K Thanks for the hints - it took me some time to get all dependencies right and it finally works with MongoDB 6.0 ;-)

Before creating the pull request - I will fix the bug here and add some unittests as well.

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/mongo-db-cannot-be-cast-to-class-java-lang-number/153019/2

[lsiepel]

Pr is merged