Test sending big files (>4MB) to Ultimaker

[peteruithoven]

1. Update the box to the latest beta version (also Remove sketches and settings).
2. Through the Doodle3D client settings window change the Log level to Bulk.
3. Let the Doodle3D send more gcode; In Chrome or Firefox, open the Web console. Paste the following and press enter.
   MAX_POINTS_TO_PRINT = Printer.MAX_GCODE_SIZE = 999999999999999999999;
   Keep this Web console open.
4. SSH into the WiFi-Box.
5. Execute the following command while true; do echo "$(cat /proc/meminfo | grep MemFree | sed -E 's/^[^0-9]+//') $(date '+%H:%M:%S')"; sleep 10; done. (This will keep checking the available RAM). Keep this terminal window open while printing. You might want to increase the buffer of your terminal so it can "store" more results.
6. Make a big drawing (lot's of text, extruded high)
7. Open in seperate browser tabs the following api endpoints (urls):
   • [WiFi-Box ip]/d3dapi/info/status
   • [WiFi-Box ip]/d3dapi/printer/progress
8. In the doodle3d client press the Print button.
9. Check the api endpoints during the print by refreshing the browser tabs. During the printing there should be buffer_size and max_buffer_size values. The buffer_size should increase, but stay under max_buffer.
10. When the buffer is full (when the buffer_size is gets very close to max_buffer_size), in the browser tab with the Doodle3D client, in the Web console, there should appear a message containing buffer_full. When the buffer is full, messages containing Printer:waitForBufferSpace should appear. This should appear for about 30 seconds (because it keeps checking the buffer). Then when the buffer is 75% it should continue sending print parts.
11. Make sure the terminal keeps recording the free memory. We're hoping it doesn't drop below 2000 kB.
12. After te print (fails):
    • Create a comment in this issue
    • Include the firmware version.
    • Copy the terminal contents to a file and sort this, include this file.
    • Export the logfiles. You can attach them to a comment by first compressing them to a zip file.
    • Copy the contents of the Web console and include this in a file. (Usually there is a right mouse button option to export to file).
    • When print failed, try to include the sketch (In the Settings window, press Download SVG).

Let's try this 3 times, without restarting the WiFi-Box (or the printer). After a first print without issues we can skip steps 7, 9 and 10.

[olijf]

Larger doodles makes the page crash in Google Chrome.

If I try to create a doodle with total_lines > max_buffer_size the page crashes everytime.

[olijf]

Print 1 (0.10.10-c)

I tested a very large print on the Ultimaker 2, but stopped it halfway because it was to big. The free memory dropped 2 times below the magic barrier. Included are the logs.

Firmware version: 0.10.10-c
ultimaker print 1.zip

Also after stopping the print navigating away from the page will still result in this message being shown:

[peteruithoven]

On the Chrome crash. This didn't happen the second attempt? It crashed at what moment? While drawing, starting sending, during sending etc?

[olijf]

It crashed after i pressed print. Before that drawing was very sluggish. Tested on both my laptop (8gb ram) and the shuttle (4gb)
As it is fairly reproducible ill try to test it in other browsers too

[peteruithoven]

Ah, and as long as the number of lines didn't go above 3145728 (the max buffer (in kb)) it cashes (when pressing print)?

[olijf]

It crashed at around 1200000 lines so if 1 line of code is around 40 bytes (characters) it would be at 48000000 bytes or ~45MB

[olijf]

Print 2 (0.10.10-c)

2nd print on the ultimaker was succesfull it dropped to 1940 somewhere at the start. Dont know why.
ultimaker print 2.zip

[woutgg]

What should be taken into account while looking at free memory is that log rotation may kick in with 2 minute intervals. This leads to a brief time of less free memory (and increased CPU usage) due to copying and compressing the log file.

[olijf]

Print 3 from tablet (0.10.10-c)

During my print from my Nexus 9 the ultimaker started making strange noises and hanged on 1 location so @mesut-pehlivan powered it off. I hope this is a printer issue. Can we check this in the logs?

ultimaker tablet print.zip

[peteruithoven]

@olijf Could we try this once more, since the first print was manually interrupted?

[olijf]

Print 4 from tablet (0.10.10-c)

Again same effect.

ultimaker tablet print 2.zip

[olijf]

It looks like the wifibox disconnects while the tablet is sending gcode en then the wifibox reconnects but the sending of gcode is never resumed, so the wifibox eventually runs out of gcode to print en then just stops.

[peteruithoven]

@olijf is right: Doodle3D/doodle3d-client#304

Let's do a extra test from a computer with a WiFi-Box connected through ethernet cable, to see if there are other issues.

[olijf]

Print 5 from PC (0.10.10-c)

Printer ran out of gcode to print. Does not look like it is the same issue. It is possible that this was an error because I did not refresh my browser correctly.

Message:set:  Sending doodle to printer: 12% notice false true
printer:sendPrintPart response:  Object {data: Object, msg: "could not add gcode (buffer_full)", status: "fail"}data: Objectmsg: "could not add gcode (buffer_full)"status: "fail"__proto__: Object
71

ultimaker print 5.zip

[olijf]

Print 6 from PC using ethernet (0.10.10-c)

Print failed. hanged at 71% I do not understand why as the ultimaker did have enough buffer to continue. This might be a different issue?
ultimaker print 6.zip

[peteruithoven]

We've released a new beta version: 0.10.10-d. (Release notes)
Let's try these tests again. Maybe twice from a desktop and twice from a tablet. I think we noticed that especially on a tablet the wifi connection was problematic.

Let's also check once that:

• seq_number & seq_total are incuded in /d3dapi/info/status.
• seq_number & seq_total are incuded in the /d3dapi/printer/print response. The Developer tools in browsers include a network tab which lists all the requests. Per request the response can be checked.

[olijf]

Test from tablet using 0.10.10-d

I tested the new fix to see if it helps... unfortunately the print stopped halfway but according to @peteruithoven this is not something related to Doodle3D/doodle3d-client#304
it seems to be a buffer issue.
I cleared my cache and restarted the wifibox and repeated the test but it would not help.
0.10.10-d fix.zip

Also after restarting the printer because it hanged I experienced a lot of

usb 1-1: device descriptor read/64, error -145
usb 1-1: new full-speed USB device number 14 using ehci-platform

errors. seems something is wrong while communicating with the ultimaker. Also print3d does not get started. This seems to be something different... I'll look into it tomorrow.

[olijf]

Test from tablet 0.10.10-d

Tested using my tablet.. unsuccessful.
After recieving an Printer:sendPrintPart: failed (AJAX status: 'timeout') AJAX exception (if any): timeout It did resume sending the data to the wifibox but recieved a Printer:sendPrintPart: unexpected failure response for API endpoint printer/print (seq_num_mismatch) error.
After this it never resumed buffering. Printer stopped printing when the buffer was depleted.
tablet print 2 0.10.10-d.zip

[woutgg]

That is strange as the lines printing that error have been modified to include the specific sequence numbers (here). After checking the js in the image, it seems it still contains old javascript.

[peteruithoven]

For some reason 0.10.10-d indeed contained old javascript code. We've released 0.10.10-e and this does contain the latest javascript code. Let's try the tests from #3 (comment) again.
Let's also download the logs after an issue occurs and then reboot the whole WiFi-Box. This way the logs should only contain the issue at hand.

[olijf]

Test from tablet 0.10.10-e

Tested using my tablet.. unsuccessful.
It hanged while waiting for the buffer to drop below 75% which somehow never happened after a certain time.
0.10.10-e test 1.zip

Also it looks as if ram usage is still an issue: the putty log shows that the wifibox drops below 2000KB

1452 kB 08:07:53
1620 kB 08:07:43
1648 kB 08:04:02
1788 kB 08:07:33

[olijf]

Test from tablet 0.10.10-e

Again same buffer issue: hangs while waiting for the buffer to drop below 75%
0.10.10-e test 2.zip

[olijf]

Test from tablet 0.10.10-e 3

Again same buffer issue.
0.10.10-e print 3.zip

[olijf]

test from pc 0.10.10-e 1

Test successful
0.10.10-e test from pc.zip

[olijf]

Test from pc 0.10.10-e 2

Test succesful
0.10.10-e print 2.zip

[olijf]

I looked into why the ultimaker usb did strange things (read error -145 means timeout btw) and according to openwrt this is a known issue with certain usb devices when they draw too much power from the port on atherors hardware. see: https://dev.openwrt.org/ticket/16467

[olijf]

Test from galaxy tab tablet 0.10.10-e 1

This time I used another tablet to see if the buffer issues would arise. but this test was successful. I am unsure that this has anything to do with the tablet at hand. I did completely reset the wifibox and removed all settings just to have a fresh start.

0.10.10-e print 1 galaxy tab.zip

[olijf]

Test from galaxy tab tablet 0.10.10-e 2

I did another test, this time I didn't enable the heated bed. Shape did not stick very well so I stopped the print.
After I pressed print again the wifibox hung in the buffering state much like printing from my nexus 5.
Also print3d did stop running.
0.10.10-e print 2 galaxy tab.zip

[peteruithoven]

@olijf Could you do 2 more of the same tests with your own tablet and adding a usb hub between the WiFi-Box and the 3D printer? (Relevant for Doodle3D/print3d#44)

[olijf]

Test from nexus 9 tablet 0.10.10-e

I tested using a small usb hub and verified that it worked correctly.
Both prints where successful
nexus 9 0.10.10-e test 1.zip

nexus 9 0.10.10-e test 2.zip

It might be possible that the usb -145 issue is solved using a USB hub although it is not observed often.

[olijf]

Test from galaxy tab tablet 0.10.10-e

I did some more tests on the Galaxy tab. I forgot to upload the logs. If I remember correctly both were successful. Might be useful .
0.10.10-e print 3 galaxy tab.zip
0.10.10-e print 4 galaxy tab.zip

[peteruithoven]

We've released 0.10.10, so I'm closing this test.