DTLS with PSK

[ce237]

Hi,

Actually i am having problems with DTLS connection and my aim is to be able to encrypt all data with a Pre Shared Key (PSK) before sending it to the californium server.

Any idea on how i may do that. I am very new in this area.

Best regards,

Cedric

[boaks]

Start reading our issue template?
That should have filled your issue, when you created it. Was that too long to read for you?

Answer the questions there, best you can.

Provide logs and captures, as told in that issue template, maybe using our wiki page.

Actually i am having problems with DTLS connection

Therefore the issue template, to guide you, what we need to help you.

and my aim is to be able to encrypt all data with a Pre Shared Key (PSK) before sending it to the californium server.

PSK is mainly used for authentication (ensure exchanging messages with right peer), the encryption is then done by negotiated keys. You don't need to care about the details, that is done by (proper) implementations.
I guess, once you provide the information you asked by our issue template, we will soon find answers and help.

[ce237]

Thanks for your reply

I am using the development kit nrf9160 of nordic semiconductor. The server hostname in my code is "californium.eclipseprojects.io" und peer "5683".

With the PUT method of CoAP, i want to be able to send encrypted data to the peer.

here is a screenhot of what i get:

[boaks]

And you sure, your read issue template?
And followed the instructions for captures there?
Maybe, this is one of the very rare cases, where a picture doesn't say more the "words" :-).

Anyway, RFC7252 defines two default ports:
5683 for plain udp coap (no encryption),
5684 for dtls coaps (encrypted)

The first thing to change would be either the scheme to coaps and/or the port.
I'm not sure, if the coap/dtls implementation on the nrf9160 used the right default port based on the scheme.

[boaks]

By the way:

please ensure first without encryption, that the coap-request are responded as intended by you. That's easier without encryption.

[ce237]

And you sure, your read issue template?
And followed the instructions for captures there?
Maybe, this is one of the very rare cases, where a picture doesn't say more the "words" :-).

Yeah i read it this time sorry, truely i also having difficulties running californium on Eclipse IDE, that's why i can present some captures from my running Californium.

What i got from wireshark:

issue.zip

[boaks]

truely i also having difficulties running californium on Eclipse IDE

Again read the issue template! There is a lot you easily can provide ahead!

About the capture:
It shows 0.0.0.0 as destination, which I would guess is not intended.
It decodes the message as CoAP, because the destination is (still) 5683 (plain udp coap).
That can't work, use 5684, as I wrote above.

The ClientHello contains:

Cipher Suites (8 suites)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)

With that, PSK is not proposed by the client. You must consider the documentation of the client's dtls library, who to setup the that library to use PSK.

[boaks]

Just to mention:

I run Californium from the Eclipse IDE without trouble.
But I can't guess, what is going wrong on your side ;-).
Therefore again: read, read, read, the issue template!

[ce237]

I run Californium from the Eclipse IDE without trouble.
But I can't guess, what is going wrong on your side ;-).
Therefore again: read, read, read, the issue template!

Yeah i will be able to run it for sure

That can't work, use 5684, as I wrote above.

here is what i got:

issue 2.zip

[boaks]

That contains now the DNS request and response, and stops after that ...

[ce237]

Yeah and here is what i get from my code:

2021-10-22T12_31_06.274Z-log.zip

any idea of what, i can do to reach the server?

[boaks]

2021-10-22T13:19:06.680Z DEBUG modem << IPv4 Address found 35.185.40.182
2021-10-22T13:19:06.689Z DEBUG modem << setsockopt TLS_PEER_VERIFY: error 109
2021-10-22T13:19:06.690Z DEBUG modem << Keine Verbindung zum Server möglich

Maybe mbedtls TLS_PEER_VERIFY helps.
(I would be able to see in the wireshark capture, if the handshakes reaches that state, where the server sent it's certificate chain and the client refuse that by an alert. But that capture is missing ...)

[boaks]

And you still use x509 rather then PSK.

[ce237]

thanks very much for your support.

Now i am trying to open californium with Intellij IDEA, here is what i got.
error by running.zip

any idea on how i can handle this error?

[boaks]

Sorry, I don't use Intellij IDEA, so I can't be of too much help.

It looks like, Intellij uses "mvn" to start the executable, but that starts on it's own without maven.

java -jar <path-to-jar>/cf-plugtest-server-3.0.0-SNAPSHOT.jar

Otherwise, you may consult the doc of the exec-maven-plugin on yourself.

[ce237]

I tried again.

Is it how it suppose to respond when Plugtestserver is started?
plugtestserver.zip

[boaks]

Yes, as far as a picture can show ...

[ce237]

Here is what i get this time from wireshark:
trace-2021-10-25T10-53-56.070Z.zip

And this one from my code:

2021-10-25T10_59_41.978Z-log.zip

Meaning the network is unreachable

[boaks]

Meaning the network is unreachable

Maybe. But that is hardly related to Californium.

That may be an issue of:

• modem and sim card (maybe private network)
• your server's network environment (e.g. behind a dsl-router or companies network)
• firewalls

On all that, I'm not able to help you.

Once you have a IP capture from the same host, you run the Californium server, which then show the dtls messages and the Californium's logs, and you still have an issue, then I may be able to help you.

[ce237]

Once you have a IP capture from the same host, you run the Californium server, which then show the dtls messages and the Californium's logs, and you still have an issue, then I may be able to help you.

Is PlugtestServer Californium server?

[boaks]

yes.

[ce237]

I tried again.

Is it how it suppose to respond when Plugtestserver is started? plugtestserver.zip

Here is all what i get when i run the californium server

[boaks]

As I already wrote, there is a lot of "home keeping" for you, where I can't help.
Ensure, that the messages sent from your modem reaches the host you run Californium.
Therefore run wireshark on the servers host and capture the traffic. Once you see you ClientHellos there, I guess it's working. if not, we will see, what the capture contains.

[ce237]

Thanks very much for your advices

[boaks]

Your welcome.

Just to mention it again:

please ensure first without encryption, that the coap-request are responded as intended by you. That's easier without encryption.

[ce237]

Ensure, that the messages sent from your modem reaches the host you run Californium.

Good Morning @boaks

i know i ask too much questions. But the only way for to learn is by asking questions since i am a newbee in this area and i also want to thank you for your patience, comprehension and advices.

for me to run a host on californium server, does it means i have to add an Endpoint like this:

server.addEndpoint(new CoapEndpoint(new InetSocketAddress("10.200.1.2", 5684)));

Or should i create a new server and then add it to the list List servers = new CopyOnWriteArrayList<>();?

[boaks]

i know i ask too much questions.

Not generally, just too much for a "day before 3.0.0-RC2" :-).

Your questions helps to find, what is required to be documented. But it makes me also feel, that to document things, but there are not read, doesn't help either. That may be also a sign, that the gap in experience and knowledge is larger as assumed.

The host you run Californium is for me mainly defined by it's "embedding" in your network setup.

That means:

• does your host have a (stable?) reachable ip address?
• does your host receive udp traffic for the reachable ip address?

And if you want to reach it from the "public internet", it even means, is your host reachable by such a public ip-address (which is different to the usage of a local one).

For the first:
Where is your host running?

• Business PC in your company network?
• Home PC in your home network?
• Cloud-machine running in which cloud?

For many company networks, UDP is unfortunately disable and so that could not work at all.
Ask your admin, if it's possible on your company's network.

For many home PCs, their network is very often build by a dsl-router. That comes with a "local-network". Your PC reaches the public internet mostly via a NAT (integrated in the router). That enables the PC to initiate a transfer (acting as client) but not to wait/accept an incoming transfer. That would require to configure a port-forwarding on your dsl-router.

If it's a cloud-machine, then it requires a lot from that hoster and the selected product. Some require to configure and enable the udp traffic explicitly, on others that's the default.

On which network interface it's best to start your plugtest-server depends now mainly on your setup. Does one of the above match yours?

[ce237]

• does your host have a (stable?) reachable ip address?

My host gets an IP address automatically assigned by Dynamic Host Configuration Protocol (DHCP). So i think my host gets an IP address in random meaning it is not stable. waht do you think?

• Business PC in your company network?

yeah, it's the company network and here are the characteristics of the ethernet:

Verbindungslokale IPv6-Adresse: fe80::843c:6be9:66ce:ebfe%7
IPv4-Adresse: 10.254.78.247
IPv4-DNS-Server: 10.214.4.61
10.214.4.62
Primäres DNS-Suffix: emea.ksb.intern
Hersteller: Realtek
Beschreibung: Realtek USB GbE Family Controller
Treiberversion: 10.39.518.2020
Physische Adresse (MAC): E4-B9-7A-D4-E5-58

Here is a sample of the traffic in our company network:
test.zip

[boaks]

(DHCP)

That's mostly a local address. You can't reach it from a public internet.
Now I'm not sure, what the rest of your setup is. And which kind of SIM card you are using.
You need to ask the people, who administrate your companies network and sim-card.
There may be an "vpn tunnel from your mobile provider into your companies network", then you need to "listen" on the related interface address.

If there is no tunnel, then you must ask your network admin as well. Coap usually uses port 5683/udp and 5684/udp(dtls) and your network admin must then provide you a public address and must configure a forwarding. Usually, that's not possible because of general security compliance rules.

So, if the host you're using, is not reachable (by tunnel or public address), a different approach uses a "machine" in the cloud. But also that depends on your companies compliance rules. if you do it privately, many cloud providers have offers, which includes a free starter. Even with that approach, you need to check, if your companies network will allow outgoing ssh to that cloud-server (or you need to do it at the weekend from home :-) ).

[ce237]

Now I'm not sure, what the rest of your setup is. And which kind of SIM card you are using.

My setup is a nRF9160 DK from Nordic Semiconductor with which i am tying to connect to the californium server with an eSim card from iBasis and my laptop from my company running the californium server.

if i understand for me to be able to interact (request/response) with californium server, i need a public address instead of a local address.

And what is when i use a Wi-Fi with these characteristics:

SSID: FRITZ!Box 7530 Loft 2.4 GHz
Protokoll: Wi-Fi 4 (802.11n)
Sicherheitstyp: WPA2-Personal
Netzfrequenzbereich: 2,4 GHz
Netzwerkkanal: 11
IPv6-Adresse: 2003:d5:271d:ad00:6dc6:adcb:650d:ba13
Verbindungslokale IPv6-Adresse: fe80::6dc6:adcb:650d:ba13%12
IPv6-DNS-Server: fd00::de39:6fff:feda:1ddf
IPv4-Adresse: 192.168.0.38
IPv4-DNS-Server: 192.168.0.1
Hersteller: Intel Corporation
Beschreibung: Intel(R) Dual Band Wireless-AC 8265
Treiberversion: 20.70.18.2
Physische Adresse (MAC): 18-1D-EA-E2-49-97

[boaks]

If that would work, then everyone in your company may be able to start and operate a server. I guess this is not intended. And you don't need to ask me, it's your network administrator who knows the answer.

[ce237]

Hello @boaks ,

i tried again and i traced the iteractions, here is what i got. What do you think?

trace-2021-10-28T12-32-41.786Z.zip

From my code i get the error 116 meaning connection timeout. Any idea on how i can handle this error:

2021-10-28T12_52_45.746Z-log.zip

Thanks in advance

[boaks]

About the trace (wireshark capture).
Congratulations! Your handshake is close to be successful.
What I can see in the capture's Client Key Exchange is:

PSK Client Params
    Identity Length: 14
    Identity: 436c69656e745f6964656e746974

That is "Client_identit" in UTF-8. I'm not sure, where you have your PSK credentials in order to be used with the Californium's sandbox, but one of the "interoperability test identities" is:

PlugtestServer

	// easier testing with openssl clients
	public static final String OPENSSL_PSK_IDENTITY = "Client_identity";
	public static final SecretKey OPENSSL_PSK_SECRET = SecretUtil.create("secretPSK".getBytes(), "PSK"); 

so, maybe, if you add the missing "y" at the end, you may have a chance.

[boaks]

Just for my information:
Did you edit the identity? Or is that one from a zephyr file?

[ce237]

Did you edit the identity? Or is that one from a zephyr file?

Good morning @boaks

Sorry for the late response

I editted it myself.

Here is what i get this time:

trace-2021-10-28T13-58-22.731Z.zip

Actually my goal is to be able to send periodic payloads to the server, when the connection and DTLS Handshake have been made. And that in PSM, the connection between client and server should still exist.

[boaks]

Congratulations! The DTLS handshake is now successful.

The CoAP request:

10.160.9.214          35.185.40.182         CoAP     182    NON, MID:12501, PUT, TKN:5c bc, /secure
35.185.40.182         10.160.9.214          CoAP     75     NON, MID:30057, 4.04 Not Found, TKN:5c bc, /secure

looks for the sandbox also well, you can't PUT a new resource "secure" on the californium's sandbox.
If you want specific requests, the idea would be, you implement that, maybe based on the plugtest server. You would need to run such a specific server on your own.

Actually my goal is to be able to send periodic payloads to the server, when the connection and DTLS Handshake have been made. And that in PSM, the connection between client and server should still exist.

I know your ticket at Nordic, I'm the Achim kraus, which recommend to be patient and wait, until device support DTLS Connection ID.

So for now

[ce237]

Congratulations! The DTLS handshake is now successful.

Without you, i won't have made it and for that i want to thank you.

10.160.9.214          35.185.40.182         CoAP     182    NON, MID:12501, PUT, TKN:5c bc, /secure
35.185.40.182         10.160.9.214          CoAP     75     NON, MID:30057, 4.04 Not Found, TKN:5c bc, /secure

Just out of curiosity, do you any programm you use to decrypted the responses from server?

looks for the sandbox also well, you can't PUT a new resource "secure" on the californium's sandbox.
If you want specific requests, the idea would be, you implement that, maybe based on the plugtest server. You would need to run such a specific server on your own.

ok i see thanks.

I know your ticket at Nordic, I'm the Achim kraus, which recommend to be patient and wait, until device support DTLS Connection ID.

Waouh great. I read my ticket again, but to sure i understood. DTLS libraries from zephyr already supports that and i just have to wait that californium server that?

In the meantime i will have to do a DTLS session resumption (make a new connection to server) and since I need my clients to be always reachable by the server at all times, I'll need to keep sending packets to prevent the NAT timeout unfortunately. Meaning my device will never sleep?

[boaks]

Just out of curiosity, do you any programm you use to decrypted the responses from server?

You can do that in wireshark, if you know the "secret". I will update the "IP capture wiki" for that :-).

DTLS libraries from zephyr already supports that and i just have to wait that californium server that?

Vice versa :-).

Californium is ready and compliant to the upcoming RFC9146.

The client's libraries are require to be updated.

See/watch zephyr - cid and mbedtls.

What's required as next steps:

• that mbedtls PR gets cleaned and merged
• mbedtls 3.?.? gets releases
• that zephyr PR gets cleaned and merged
• zephyr gets released

[boaks]

In the meantime i will have to do a DTLS session resumption (make a new connection to server) and since I need my clients to be always reachable by the server at all times, I'll need to keep sending packets to prevent the NAT timeout unfortunately. Meaning my device will never sleep?

Even with DTLS CID, if your device must stay reachable by the server, it will never sleep!

DTLS CID only helps to make the device sleeping, if always the device initiates the communication after such a sleep. That's not bound to DTLS (nor CoAP), that's about IP and what your mobile provider is offering you. To save energy, the most is saved with PSM. And with that, always the clients starts to send data. With eDRX, you may save also energy, not that much compared to PSM, but your device is "somehow" reachable. But eDRX may be not overall available.

Similar as with your "companies network issue", where only your network administrator knows, what to do, for "mobile NB-IoT" only your (SIM) provider knows what to do. So you must try to get that information from there.

[ce237]

Hello @boaks

i am using the resource "create" from Plugtestserver to send a JSON payload "{"command":"periodic","type":"8","number":"33"}" using the PUT-method from CoAP, but the server response is "Bad request":

trace-2021-10-29T11-50-24.530Z.zip

Despite the fact that Json is one of the context format from CoAP.

Can you please tell me, what i am doing wrong? May be is my annotation wrong or can i send a JSON Payload in the first place?

Thanks in advance

[boaks]

1. you don't read the tutorial for captures, otherwise you would select the dtls/coap messages and not just send "all".
2. for reasons, unknown to me, the plugtest server requires you to send the "content format" option. That is also indicated by the response error text in the capture: 4.00 BAD REQUEST, "Content-Format not set".
3. you can sure send JSON as payload. If you use also the content-format-option with the value JSON, then the plugtest server would be much happier. Option 12, application/json 50 (see RFC7252, maybe also time to read that).
4. the payload in the message seems to be messed up, at least in the capture. The payload in the capture show 28 'A' at the begin, followed by 71 times a 0x00.

So, time to start to read :-).

[boaks]

@ce237

Before you start to wonder, the sandbox's ban file contains:

coaps Option Content-Format value of 3 bytes must be in range of [0-2] bytes. 

Though it's coaps, it may be your device. If you send malicious messages, Californium will ban your device's ip-address for a while. That's also a good reason to run a coap-server on your own. CoAP options are "well defined", I'm not sure, what your client library offers as API. Some API cares for you to comply to RFC7252 5.10, others offering a raw access and your code must obey the definitions. For content-format, its:

12,  Content-Format, uint, 0-2 bytes => value range is 0..65535

[ce237]

Hello @boaks ,

Thank you very much for the informations.

I sent a Json Payload without having to do a DTLS handshake, meaning i used normal UDP. Here is what i got:

test_non_secure.zip

For me i think the server responded properly. What do you think please?

In the meantime i have the following issues:

1. The time it takes for my device to connect to the network is too long (more than 10mins). And i think that's not normal and i don't know why the connection time is so long.

2. When i connect to the server, i want the connection between my device and the server to stay. Is it possible in the first place? when yes, are there any configurations i have to make?

Thanks in advance

[boaks]

For me i think the server responded properly. What do you think please?

The payload you send is "{\n\t" 3 (bytes), I guess that's wrong

The time it takes for my device to connect to the network is too long (more than 10mins). And i think that's not normal and i don't know why the connection time is so long.

I guess, the device tries out a couple of bands, may be even not only CAT-NB also CAT-M1. The solution should come from the modem manufacturer. (I use also the nrf9160 and with my sim-card and selecting CAT_NB and the proper band of my provider, it takes about 2-3s, in some cases up to 60s.)

When i connect to the server, i want the connection between my device and the server to stay. Is it possible in the first place? when yes, are there any configurations i have to make?

I don't know, what you exactly mean. Usually UDP doesn't "connect", it just send udp-messages.

With dtls, these messages must be encrypted/decrypted using session/association's keys. Though this is normally referred by the ip-address/port, it only works, if your device keeps them. But AFAIK, with iBasis, that is not supported. So, back to my answer in the nordic forum: you have to wait for DTLS CID. With that, the dtls keys are referred by that cid and so static ip-address is not longer required.
But it is still required, that the device initiates the communication. And after a couple of seconds without message exchange (e.g. 30s) it will get unreachable by the server. But that is IP and your mobile provider, not dtls nor coap, and so Californium can't do anything for it. (Except sending a lot of messages and keep the device awake :-) ).

[ce237]

The payload you send is "{\n\t" 3 (bytes), I guess that's wrong

am creating my JSON Payload with the following function:

cJSON * createJsonFromParams(void)
{

cJSON *root = cJSON_CreateObject();
cJSON_AddNumberToObject(root, "JSON", 50);
cJSON_AddNumberToObject(root, "Content-Form", 12);

return root;

}

The main function :

cJSON_Init();
cJSON *item = createJsonFromParams();
char *payload=cJSON_Print(item);

Do you think it is the appropriate way of forming my JSON Payload?or will this be suitable for the server?

And i really want to thank you again for helping me and for you precious time .

[boaks]

I'm not cJSON specialist, but I guess payload point to the right serialized JSON,

{
    "JSON": 50,
    "Content-Form": 12
} 

But, when you set it as payload, the length is accidentally truncated to 3.

[boaks]

Do you think it is the appropriate way of forming my JSON Payload?
or will this be suitable for the server?

Though, it's unclear, what you want to send as JSON, I can't say to much. In the last trace the content-format option seems to be well, but the payload only contains 3 bytes and was no valid JSON. Though Californium doesn't care on it's own, if the payload is valid JSON, it respond with created.

[ce237]

Hello @boaks,

it's me again. Sorry for disturbing.

I can't reach the californium server over NB-IoT. Here is what i got:

log trace_NB_IoT.zip

ERROR: getaddrinfo fehlgeschlagen -11: which stands for No more processes.

what does that means for me? any idea?

NB: with LTE-M everything works perfectly

[boaks]

That's more a question for the modem's manufacturer.
And I see, that you have already asked there.

[boaks]

The only, not that brilliant work-around may be, not to use "californium.eclipseprojects.io" and instead the literal-ip-address "35.185.40.182". Or to wait a little, and retry the getaddrinfo say 5s later.

[ce237]

The only, not that brilliant work-around may be, not to use "californium.eclipseprojects.io" and instead the literal-ip-address "35.185.40.182". Or to wait a little, and retry the getaddrinfo say 5s later.

Ok thanks for your suggestion. It worked

[ce237]

Hi @boaks,

i want to thank you once again for all your help. Because of you i really made progress and i am very grateful for that.

I was able to send a same payload every 30s and the Server responded appropriately. But all of a sudden when i send the server didn't respond. Is there any reason for that or is normal?

Here a a wireshark trace:

test.zip

Thanks in advance.

[boaks]

I can't see any reason in the capture.
So, I can only guess, what the cause may be:

• NON request. UDP doesn't grant the delivery of request. In rare cases, UDP message are lost. Therefore CoAP also support CON requests, which are retransmitted in case where the request or the response is dropped.

• IP-Address changes by NATs on the ip-route to the server. The ip-capture, close to the modem, doesn't show that. For that, DTLS Connection ID will be required. Or a new dtls-handshake.

I would recommend, you start with using CON and see, if that works.

[boaks]

@ce237

Could this issue be closed? If you have further questions, please open a new issue.
It's hard for others to search for solutions, if an issue gets larger and larger and the questions are not longer related to the title.

[ce237]

for sure, sorry for the late response