[ModuleClient] IllegalArgumentException on ModuleClient.createFromEnvironment #281

JMayrbaeurl · 2018-07-18T10:33:23Z

OS and version used: Windows 10 with Docker CE and Linux containers
Java runtime used: OpenJDK 8
SDK version used: 1.13.1

Description of the issue:

On calling ModuleClient.createFromEnvironment() with MQTT as protocol at startup, the method fails with an IllegalArgumentException. com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign() method is using HttpsClient and tries to open a connection to "IOTEDGE_WORKLOADURI=http://10.0.75.1:15581/", that will fail because the URL contains 'http' but not 'https'

Code sample exhibiting the issue:

	private ModuleClient createModuleClientWithDefaultTransport() throws ModuleClientException {
		
		return ModuleClient.createFromEnvironment(
				IotHubClientProtocol.valueOf(this.getTransportType())); 
	}

Console log of the issue:

Caused by: java.lang.IllegalArgumentException: Expected URL that uses protocol HTTPS but received one that uses protocol 'http'.

        at com.microsoft.azure.sdk.iot.device.transport.https.HttpsConnection.<init>(HttpsConnection.java:71) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.transport.https.HttpsRequest.<init>(HttpsRequest.java:41) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign(HttpsHsmClient.java:71) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpHsmSignatureProvider.sign(HttpHsmSignatureProvider.java:84) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.createNewSasToken(IotHubSasTokenHsmAuthenticationProvider.java:80) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.create(IotHubSasTokenHsmAuthenticationProvider.java:48) ~[iot-device-client-1.13.1.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:234) ~[iot-device-client-1.13.1.jar!/:na]
 

             "Env": [
                "IOTEDGE_WORKLOADURI=http://10.0.75.1:15581/",
                "IOTEDGE_AUTHSCHEME=sasToken",
                "IOTEDGE_APIVERSION=2018-06-28",
                "EdgeDeviceHostName=vie-jurgenma",
                "IOTEDGE_MODULEGENERATIONID=636666470330034949",
                "IOTEDGE_IOTHUBHOSTNAME=jmedgetestiothub2.azure-devices.net",
                "RuntimeLogLevel=Information",
                "IOTEDGE_DEVICEID=jurgenmaEdgeP50",
                "IOTEDGE_MODULEID=$edgeHub",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "ASPNETCORE_URLS=http://+:80",
                "DOTNET_RUNNING_IN_CONTAINER=true",
                "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true",
                "DOTNET_VERSION=2.1.0",
                "SSL_CERTIFICATE_PATH=/app/certs",
                "SSL_CERTIFICATE_NAME=mqtt-server.pfx"
            ],

Need Support?

Have a feature request for SDKs? Please post it on User Voice to help us prioritize.
Have a technical question? Ask on Stack Overflow with tag “azure-iot-hub”
Need Support? Every customer with an active Azure subscription has access to support with guaranteed response time. Consider submitting a ticket and get assistance from Microsoft support team
Found a bug? Please help us fix it by thoroughly documenting it and filing an issue on GitHub (C, Java, .NET, Node.js, Python).

The text was updated successfully, but these errors were encountered:

timtay-microsoft · 2018-07-18T17:54:42Z

This is a bug on our side, and I'll get a fix in for this soon. Thanks for opening the issue.

timtay-microsoft · 2018-07-19T22:07:10Z

The fix for this issue has been checked into this branch if you want to try building from source to try it out.

We will be merging this into master and releasing within 2 weeks, but I would appreciate any feedback you have on this fix before then.

JMayrbaeurl · 2018-07-20T17:02:50Z

Sorry. But fix doesn't work. New error message:

Caused by: java.lang.UnsupportedOperationException: unrecognized URI scheme. Only HTTPS and UNIX are supported
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign(HttpsHsmClient.java:115) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpHsmSignatureProvider.sign(HttpHsmSignatureProvider.java:84) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.createNewSasToken(IotHubSasTokenHsmAuthenticationProvider.java:80) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.create(IotHubSasTokenHsmAuthenticationProvider.java:48) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:234) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]

Did you test the fix?

timtay-microsoft · 2018-07-20T18:12:45Z

Sorry about that, there was a bit that I missed during testing. I pushed another commit that should work for you.

JMayrbaeurl · 2018-07-20T20:06:14Z

No problem. This time with NPE

Caused by: java.lang.NullPointerException: null
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign(HttpsHsmClient.java:129) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpHsmSignatureProvider.sign(HttpHsmSignatureProvider.java:84) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.createNewSasToken(IotHubSasTokenHsmAuthenticationProvider.java:80) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.create(IotHubSasTokenHsmAuthenticationProvider.java:48) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:234) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]

Sincerely your test bot

timtay-microsoft · 2018-07-20T21:23:29Z

I fixed a few more issues with this functionality, so now would be a good time to pull the latest and try again. I really appreciate the testing help!

JMayrbaeurl · 2018-07-21T08:16:47Z

Still crashing

Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING at line 1 column 12 path $.digest
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226) ~[gson-2.8.5.jar!/:na]
        at com.google.gson.Gson.fromJson(Gson.java:927) ~[gson-2.8.5.jar!/:na]
        at com.google.gson.Gson.fromJson(Gson.java:892) ~[gson-2.8.5.jar!/:na]
        at com.google.gson.Gson.fromJson(Gson.java:841) ~[gson-2.8.5.jar!/:na]
        at com.google.gson.Gson.fromJson(Gson.java:813) ~[gson-2.8.5.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.parser.SignResponse.fromJson(SignResponse.java:30) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign(HttpsHsmClient.java:110) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.HttpHsmSignatureProvider.sign(HttpHsmSignatureProvider.java:84) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.createNewSasToken(IotHubSasTokenHsmAuthenticationProvider.java:80) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.create(IotHubSasTokenHsmAuthenticationProvider.java:48) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]
        at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:234) ~[iot-device-client-1.13.2-SNAPSHOT.jar!/:na]

timtay-microsoft · 2018-07-23T21:11:58Z

We're just about done fixing this flow. I pushed one more commit that should do it for you. I added a little extra logging via System.out. When you run this again, can report back to me what your logs look like in their entirety (with connection strings censored)?

I added that logging temporarily to understand the form of the data that the hsm is returning. There was an odd requirements mix up between this SDK and the Edge team that this logging should clear up.

JMayrbaeurl · 2018-07-24T14:08:58Z

OK. Next try. Again with NPE. BTW. I had to rewrite my Java sample app to get logging enabled in the SDK. Before, with a Spring Boot app, I could not manage to get logging information from the SDK. This is a known bug, that I've already reported 1.5 years ago, and the reason comes from the direct usage of Log4j in CustomLogger.

log4j: reset attribute= "false".
log4j: Threshold ="null".
log4j: Level value for root is  [DEBUG].
log4j: root level set to DEBUG
log4j: Class name: [org.apache.log4j.ConsoleAppender]
log4j: Parsing layout of class: "org.apache.log4j.PatternLayout"
log4j: Setting property [conversionPattern] to [%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n].
log4j: Adding appender named [console] to category [root].
2018-07-24 14:02:28 INFO  SimpleJavaIoTEdgeApp:21 - Simple Java module app started
Creating sign response from json, json:
{"digest":"+EQWquqJDykfEWKGSP5KZc/MA1mjwIhJ1dSmp6JuncA="}
2018-07-24 14:02:28 INFO  DeviceClientConfig:23 - DeviceClientConfig object is created successfully with IotHubName=jmedgetestiothub2.azure-devices.net, deviceID=jurgenmaEdgeP50 , method name is <init>
2018-07-24 14:02:28 INFO  DeviceIO:23 - DeviceIO object is created successfully, method name is <init>
Exception in thread "main" java.lang.NullPointerException
        at com.microsoft.azure.sdk.iot.device.transport.https.HttpsConnection.writeOutput(HttpsConnection.java:198)
        at com.microsoft.azure.sdk.iot.device.transport.https.HttpsRequest.send(HttpsRequest.java:79)
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sendRequestBasedOnScheme(HttpsHsmClient.java:178)
        at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.getTrustBundle(HttpsHsmClient.java:151)
        at com.microsoft.azure.sdk.iot.device.edge.HttpsHsmTrustBundleProvider.getTrustBundleCerts(HttpsHsmTrustBundleProvider.java:41)
        at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:243)
        at samples.com.microsoft.azure.jm.iot.SimpleJavaIoTEdgeApp.main(SimpleJavaIoTEdgeApp.java:24
```)

timtay-microsoft · 2018-07-24T18:40:11Z

I pushed another commit that should fix that issue for you. Again, I added some extra logging just in case something goes wrong again.

Just for some context, the flow you are using here is communicating with an HSM to create a sas token for authentication and to get the trust bundle. Based on your logs, it looks like you retrieved the sas token just fine, and with this fix, you will get the trust bundle, too. After that, the rest of this flow has been tested and it works. It was just the HSM steps that we don't have a nice automated test on yet.

Thanks again!

JMayrbaeurl · 2018-07-24T20:54:20Z

And now with StackOverflow from gson library. See attached log file
logs2.txt

timtay-microsoft · 2018-07-24T21:36:03Z

Looks like there was an issue with our gson parser class missing a default constructor. I went ahead and committed the fix to a different branch for you to try out.

JMayrbaeurl · 2018-07-24T22:33:33Z

Sorry. But the new branch doesn't compile:

[INFO] 4 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.912 s
[INFO] Finished at: 2018-07-25T00:31:52+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:testCompile (default-testCompile) on project iot-device-client: Compilation failure: Compilation failure:
[ERROR] /C:/Dev/iotedge/javasdk/azure-iot-sdk-java/device/iot-device-client/src/test/java/tests/unit/com/microsoft/azure/sdk/iot/device/hsm/HttpsHsmClientTest.java:[355,17] constructor TrustBundleResponse in class com.microsoft.azure.sdk.iot.device.hsm.parser.TrustBundleResponse cannot be applied to given types;
[ERROR]   required: no arguments
[ERROR]   found: java.lang.String
[ERROR]   reason: actual and formal argument lists differ in length
[ERROR] /C:/Dev/iotedge/javasdk/azure-iot-sdk-java/device/iot-device-client/src/test/java/tests/unit/com/microsoft/azure/sdk/iot/device/hsm/parser/TrustBundleResponseTest.java:[84,40] constructor TrustBundleResponse in class com.microsoft.azure.sdk.iot.device.hsm.parser.TrustBundleResponse cannot be applied to given types;
[ERROR]   required: no arguments
[ERROR]   found: java.lang.String
[ERROR]   reason: actual and formal argument lists differ in length
[ERROR] /C:/Dev/iotedge/javasdk/azure-iot-sdk-java/device/iot-device-client/src/test/java/tests/unit/com/microsoft/azure/sdk/iot/device/hsm/parser/TrustBundleResponseTest.java:[97,40] constructor TrustBundleResponse in class com.microsoft.azure.sdk.iot.device.hsm.parser.TrustBundleResponse cannot be applied to given types;
[ERROR]   required: no arguments
[ERROR]   found: java.lang.String
[ERROR]   reason: actual and formal argument lists differ in length
[ERROR] /C:/Dev/iotedge/javasdk/azure-iot-sdk-java/device/iot-device-client/src/test/java/tests/unit/com/microsoft/azure/sdk/iot/device/hsm/parser/TrustBundleResponseTest.java:[105,40] constructor TrustBundleResponse in class com.microsoft.azure.sdk.iot.device.hsm.parser.TrustBundleResponse cannot be applied to given types;
[ERROR]   required: no arguments
[ERROR]   found: java.lang.String
[ERROR]   reason: actual and formal argument lists differ in length

timtay-microsoft · 2018-07-24T22:42:29Z

Whoops, you are absolutely correct. I forgot to update a few tests. Sorry about that. I just pushed a commit that should allow you to compile now.

JMayrbaeurl · 2018-07-24T23:33:30Z

Heureka! It works now without any exception, even when sending messages. Unfortunately the sent messages are not arriving at the IoT Hub. I've already checked the routes. Anyway. I'll have to investigate further.

timtay-microsoft · 2018-07-25T17:21:05Z

Awesome! I'll keep digging with you on the routes issue.

jasmineymlo · 2018-08-11T01:02:14Z

In https://github.com/Azure/azure-iot-sdk-java/releases/tag/2018-8-10. Thanks.

marcschroeder · 2020-06-04T07:32:16Z

This issue seems to happen again with the current release:

Caused by: java.lang.IllegalArgumentException: Expected URL that uses protocol HTTPS but received one that uses protocol 'http'.

    at com.microsoft.azure.sdk.iot.device.transport.https.HttpsConnection.<init>(HttpsConnection.java:79) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.transport.https.HttpsRequest.send(HttpsRequest.java:102) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sendRequestBasedOnScheme(HttpsHsmClient.java:204) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.hsm.HttpsHsmClient.sign(HttpsHsmClient.java:82) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.hsm.HttpHsmSignatureProvider.sign(HttpHsmSignatureProvider.java:89) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.createNewSasToken(IotHubSasTokenHsmAuthenticationProvider.java:80) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.hsm.IotHubSasTokenHsmAuthenticationProvider.create(IotHubSasTokenHsmAuthenticationProvider.java:48) ~[iot-device-client-1.22.0.jar!/:na]
    at com.microsoft.azure.sdk.iot.device.ModuleClient.createFromEnvironment(ModuleClient.java:263) ~[iot-device-client-1.22.0.jar!/:na]
    at com.deviceinsight.oi4.itempiq.edge.SpringConfiguration.moduleClient(SpringConfiguration.java:30) ~[classes!/:0.0.1-SNAPSHOT]
    at com.deviceinsight.oi4.itempiq.edge.SpringConfiguration$$EnhancerBySpringCGLIB$$752e2d0d.CGLIB$moduleClient$0(<generated>) ~[classes!/:0.0.1-SNAPSHOT]
    at com.deviceinsight.oi4.itempiq.edge.SpringConfiguration$$EnhancerBySpringCGLIB$$752e2d0d$$FastClassBySpringCGLIB$$e4f373e2.invoke(<generated>) ~[classes!/:0.0.1-SNAPSHOT]
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]
    at com.deviceinsight.oi4.itempiq.edge.SpringConfiguration$$EnhancerBySpringCGLIB$$752e2d0d.moduleClient(<generated>) ~[classes!/:0.0.1-SNAPSHOT]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]

Our current environment configuration is:
HOSTNAME=9efe07756bf3 IOTEDGE_MODULEID=XXX IOTEDGE_MODULEGENERATIONID=XXX IOTEDGE_IOTHUBHOSTNAME=XXX.azure-devices.net IOTEDGE_WORKLOADURI=http://172.17.0.4:15581/ IOTEDGE_GATEWAYHOSTNAME=XXX IOTEDGE_DEVICEID=XXX IOTEDGE_AUTHSCHEME=sasToken

Any ideas on how to fix this?

marcschroeder · 2020-06-10T14:27:28Z

This issue seems to occur with the newest version again. Could we reopen the issue?

timtay-microsoft self-assigned this Jul 18, 2018

timtay-microsoft added the bug label Jul 18, 2018

timtay-microsoft added the fix checked in Fix checked into main or preview, but not yet released. label Aug 9, 2018

jasmineymlo closed this as completed Aug 11, 2018

marcschroeder mentioned this issue Jun 10, 2020

ModuleClient.createFromEnvironment() not working as expected Azure/iotedge#3061

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ModuleClient] IllegalArgumentException on ModuleClient.createFromEnvironment #281

[ModuleClient] IllegalArgumentException on ModuleClient.createFromEnvironment #281

JMayrbaeurl commented Jul 18, 2018

timtay-microsoft commented Jul 18, 2018

timtay-microsoft commented Jul 19, 2018 •

edited

Loading

JMayrbaeurl commented Jul 20, 2018

timtay-microsoft commented Jul 20, 2018

JMayrbaeurl commented Jul 20, 2018

timtay-microsoft commented Jul 20, 2018 •

edited

Loading

JMayrbaeurl commented Jul 21, 2018

timtay-microsoft commented Jul 23, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 25, 2018

jasmineymlo commented Aug 11, 2018

marcschroeder commented Jun 4, 2020 •

edited

Loading

marcschroeder commented Jun 10, 2020

[ModuleClient] IllegalArgumentException on ModuleClient.createFromEnvironment #281

[ModuleClient] IllegalArgumentException on ModuleClient.createFromEnvironment #281

Comments

JMayrbaeurl commented Jul 18, 2018

Description of the issue:

Code sample exhibiting the issue:

Console log of the issue:

Need Support?

timtay-microsoft commented Jul 18, 2018

timtay-microsoft commented Jul 19, 2018 • edited Loading

JMayrbaeurl commented Jul 20, 2018

timtay-microsoft commented Jul 20, 2018

JMayrbaeurl commented Jul 20, 2018

timtay-microsoft commented Jul 20, 2018 • edited Loading

JMayrbaeurl commented Jul 21, 2018

timtay-microsoft commented Jul 23, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 24, 2018

JMayrbaeurl commented Jul 24, 2018

timtay-microsoft commented Jul 25, 2018

jasmineymlo commented Aug 11, 2018

marcschroeder commented Jun 4, 2020 • edited Loading

marcschroeder commented Jun 10, 2020

timtay-microsoft commented Jul 19, 2018 •

edited

Loading

timtay-microsoft commented Jul 20, 2018 •

edited

Loading

marcschroeder commented Jun 4, 2020 •

edited

Loading