Manual blockwise transfer

[kiemlicz]

How to manually conduct blockwise transfer (Californium v. 1.0.3) or perform it without loading whole payload to memory?
I have a large payload (with eg. InputStream) that I don't want to load to memory fully.
Instead I read chunk by chunk from InputStream and prepare
org.eclipse.californium.core.coap.Request (I'm setting the Block1 options properly).
Unfortunately receiving response on such request yields NPE from:
org.eclipse.californium.core.network.stack.BlockwiseLayer.receiveResponse(BlockwiseLayer.java:324)
Which is of course due to missing
org.eclipse.californium.core.network.stack.BlockwiseStatus instance in Exchange abstraction.

Is it the missing functionality or is it 'by design'?

[sophokles73]

So far there is no support for transferring streams of data in Californium. You can either read in all data into memory and let Californium do the chunking or you need to implement the chunking yourself, which you seem to have opted for.
However, since I am not very knowledgable about the blockwise code in Californium this should probably be answered by @mkovatsc

[kiemlicz]

Thank you,
I've done chunking by myself but it ends with aforementioned NPE in BlockwiseLayer (on receiving response)

[mkovatsc]

The original idea was to leave out the BlockwiseLayer, when you want to handle Blockwise Transfers manually. However, the stack wasn't tested in such a configuration recently. Maybe you can give it a try and tell us what happens...

I also explained another possible strategy here https://dev.eclipse.org/mhonarc/lists/cf-dev/msg00654.html:
If someone needs to mix resources with "normal blockwise transfers" and streaming resources, one could implement a flag that let's BlockwiseLayer pass all messages untouched, so that the resource handler can implement block handling.

[wiinguyen]

Please advise if this is a way to turn off BlockWiseLayer

$ git diff californium-core/src/main/java/org/eclipse/californium/core/network/stack/CoapStack.java
diff --git a/californium-core/src/main/java/org/eclipse/californium/core/network/stack/CoapStack.java b/californium-core/src/main/java/org/ecli
index 13658e1..6bfab37 100644
--- a/californium-core/src/main/java/org/eclipse/californium/core/network/stack/CoapStack.java
+++ b/californium-core/src/main/java/org/eclipse/californium/core/network/stack/CoapStack.java
@@ -105,7 +105,7 @@ public class CoapStack {
new Layer.TopDownBuilder()
.add(top)
.add(new ObserveLayer(config))

•                           .add(new BlockwiseLayer(config))
  

•                           //.add(new BlockwiseLayer(config))
                          .add(reliabilityLayer)
                          .add(bottom = new StackBottomAdapter())
                          .create();
  

$git diff californium-core/src/main/java/org/eclipse/californium/core/network/Matcher.java
diff --git a/californium-core/src/main/java/org/eclipse/californium/core/network/Matcher.java b/californium-core/src/main/java/org/eclipse/cali
index 44dfe9b..618c091 100644
--- a/californium-core/src/main/java/org/eclipse/californium/core/network/Matcher.java
+++ b/californium-core/src/main/java/org/eclipse/californium/core/network/Matcher.java
@@ -517,7 +517,7 @@ public class Matcher {
}

                            Request request = exchange.getCurrentRequest();

•                           if (request != null && (request.getOptions().hasBlock1() || response.getOptions().hasBlock2()) ) {
  

•                           if (request != null && response != null && (request.getOptions().hasBlock1() || response.getOptions().hasBlock2
                                  KeyUri uriKey = new KeyUri(request.getURI(), request.getSource().getAddress(), request.getSourcePort())
                                  LOGGER.log(Level.FINE, "Remote ongoing completed, cleaning up ", uriKey);
                                  ongoingExchanges.remove(uriKey);
  

BTW, "response != null" check is needed. Otherwise, it'll crash.

[sophokles73]

@kiemlicz,

we are trying to close out outstanding issues for the next release. We have put a lot of work into the BlockwiseLayer on the 2.0.x branch. FMPOV it should be possible to do a manual blockwise transfer based on this work (using either master or 2.0.x). This work will not find its way into the 1.0.x branch though. I strongly advise to update to 2.0.x in order to take advantage of all other improvements we have been working on as well.

Closing this as won't fix.