Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot download any file from local FS but preferences.json #10

Closed
luc-github opened this issue Apr 9, 2023 · 52 comments
Closed

Cannot download any file from local FS but preferences.json #10

luc-github opened this issue Apr 9, 2023 · 52 comments

Comments

@luc-github
Copy link

It seems on ESP32 files can be uploaded on local FS, but no files but preferences.json can be downloaded
Like the webui theme files.

download

I know on other platform the FS is limited and that is why we introduced the HostPath:"/" parameter in [ESP800], should we add a new parameter ReadOnlyFS to let know webui to disable the theme features and any FS upload

In same idea, it seems webui (index.html.gz) can be updated but not downloaded so it is weird - I did not digged grblHAL code to be honest

Original issue was raised here : luc-github/ESP3D-WEBUI#350

@terjeio
Copy link
Contributor

terjeio commented Apr 9, 2023

There are several issues at play here.
First, I do not set HostPath as I should, code is commented out for that - cannot remember why now.

When removing the commenting and fixing some related issues things improves.

Saving files from Flash still does not work, due to HostPath not beeing prepended to the filename:

image

However, saving from the maintenance page works:

image

Also the dark theme does nothing even if it is loaded, due to it beeing gzipped? The purple theme works as does the dark theme if I download it (from the controller to be sure it has not been corrupted), unzip and upload it again.

image

image

@luc-github
Copy link
Author

luc-github commented Apr 9, 2023

if gz theme does not work it is may be due to fact it is gzziped yes,
the webui only request the file name theme-dark and ESP3D web server check if the file exists, or if theme-dark.gz exists, in this case ( theme-dark.gz exists) it add Content-Encoding:gzip header before streaming it, it is same trick like for index.html.gz to save space
Sorry I did not thinked that may not be implemented on your webserver

The dark theme should give this ui as I tested for the issue on test server and fixed the grblHAL logo not visible on it
theme

Saving files from Flash still does not work, due to HostPath not beeing prepended to the filename:

That is weird I can upload files on FS on current FW
theme2

But indeed the HostPath is not used when upload file from Files panel - I will add it

Do you need me to test something ? or let me know when I can update grblHAL FW to test the upload fix

@terjeio terjeio transferred this issue from grblHAL/ESP32 Apr 9, 2023
@terjeio
Copy link
Contributor

terjeio commented Apr 9, 2023

add Content-Encoding:gzip header before streaming it

Thanks, the header was missing - I need to make the header generation more generic... Fixed for plain .gz files.

Do you need me to test something ?

Upload of from files panel after you have added HostPath.

or let me know when I can update grblHAL FW to test the upload fix

Ready now - also from the Web Builder.

@luc-github
Copy link
Author

luc-github commented Apr 10, 2023

Ok I tested :

  • Upload file from FS File Panel (root and sub directory)
  • Download file from FS File Panel (root and sub directory)
  • Delete file from FS File Panel (root and sub directory)
  • Create directory from FS File Panel
  • Theme loading (.gz and plain)

Everything seems Ok now
Please give a try to :
index.html.gz

If ok on your side too I will update repository

@terjeio
Copy link
Contributor

terjeio commented Apr 10, 2023

Downloads of .gz files from the FS File Panel saves the uncompressed data, from the maintenance page the compressed data.
Downloading index.html.gz from the maintenance page results in WebUI beeing opened in a new tab, no file saved (Firefox).

@luc-github
Copy link
Author

it is because you apply header gzip I guess
if file called is called with .gz extension header should not be applied so it will be downloaded instead of being seen as html

@luc-github
Copy link
Author

I did not check maintenance page yet, will do once fs panel is ok

@terjeio
Copy link
Contributor

terjeio commented Apr 10, 2023

Is the behaviour the same with your backend?

@luc-github
Copy link
Author

I am out today, I will confirm you later

@luc-github
Copy link
Author

Yes on esp3d everything is working as expected click on any .gz does not expand them but download them

download2
But I found a typo with latest webUI fix that lead to time out in ESP3D : not cleaning properly some // lead to remove the site address
it is fixed now and path can be seen in console temporary for checking :
index.html.gz

I cannot check your maintenance page/embedded page source because I did not found where it is sorry- but on mine, indeed I do not use the HostPath I will add it today and test it do as expected

@terjeio
Copy link
Contributor

terjeio commented Apr 11, 2023

Yes on esp3d everything is working as expected click on any .gz does not expand them but download them

What are the response headers when you download a .gz file? Here is what I return for index.html.gz:

HTTP/1.1 200 OK
Server: lwIP/2.1.3 (http://savannah.nongnu.org/projects/lwip)
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Connection: keep-alive
Content-Length: 82623

Another odd issue is that downloading preferences.json behaves different as well, from the maintenance page it opens it in a new browser window, in the Files Panel it downloads it. The only difference I can see is that the maintenance page does not add the &t=<n> cachebuster parameter, content and response headers are the same.

Could it be browsers behaving differently?

I cannot check your maintenance page/embedded page source because I did not found where it is sorry

It is here, compressed. I cannot remember changing the source in any way - and it obviously handles the path to the files correctly. If not using HostPath are you using FlashFileSystem instead? BTW HostPath was referenced once in the code - for setting a variable.

image

@luc-github
Copy link
Author

if the called url is with : xxxxx.gz
you must NOT add the header Content-Encoding: gzip

if the called url is without gz : xxxxx
you must check if a file named xxxxx.gz exists and so you can stream it and add the header Content-Encoding: gzip

If not using HostPath are you using FlashFileSystem instead?

No, I just forget to handled it, that is all ..., I focused on webui and forget the embedded, I have just added the support today (luc-github/ESP3D@247e848d2c75bb2a51f773e5a50c343fe61bb7e2)- I was struggling when testing on grblHAL today because I randomly got
the HostPath to "/" from ESP800
image

I though it was linked to version but even using same I got same behavior - to get proper output I first need to
try the ip address only then try the ip?forcefallback=yes I though it was cache issue but I am not sure

I suggest you to copy this https://github.com/luc-github/ESP3D/tree/3.0/embedded to your webUI plugin repository and do the links changes according grblHALs for reference as I still not not have unified version yet

@luc-github
Copy link
Author

But I am little bit confused as upload was working before introducing the fix

@terjeio
Copy link
Contributor

terjeio commented Apr 11, 2023

if the called url is with : xxxxx.gz
you must NOT add the header Content-Encoding: gzip

Is as this stated in the relevant RFC? Prehaps this:

"If the media type includes an inherent encoding, such as a data format that is always compressed, then that encoding would not be restated in Content-Encoding even if it happens to be the same algorithm as one of the content codings."

or is it just something that works with the ESP webserver?

However - it does not explain why the Files Panel and maintenance page behaves differently with the current code. The only difference is the cachebusting parameter, the base request URI is the same and the response headers too...

Confusing...

@luc-github
Copy link
Author

or is it just something that works with the ESP webserver?

Yes it is more like, it was suggested since I started esp8266 dev to save space on FS to serve files (https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WebServer/examples/FSBrowser/extras/index.htm.gz) very old trick now also used on esp32

About different behavior:
embedded page just use basic API : windows.open(XXXX) to download the file and so let the browser to download in new page (https://github.com/luc-github/ESP3D/blob/3.0/embedded/src/index.js#L659)
when webUI use XmlRequest to have a progression bar and I force type to 'application/octet-stream', (https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/src/components/Panels/Files.js#L338)
So they may behave differently depending of file type

@terjeio
Copy link
Contributor

terjeio commented Apr 11, 2023

if the called url is with : xxxxx.gz
you must NOT add the header Content-Encoding: gzip

You were right - when a file is requested without the .gz extension and is only found with it it is the same as compression has been applied and the Content-Encoding header has to be added. This paragraph in the RFC covers that:

"If one or more encodings have been applied to a representation, the sender that applied the encodings MUST generate a Content-Encoding header field that lists the content codings in the order in which they were applied."

I'll soon commit a fix for this.

I was struggling when testing on grblHAL today because I randomly got the HostPath to "/" from ESP800

This is a bug on my side - I set HostPath to the directory where index.html.gz is found but I did not do that when the maintenance page was loaded.

The remaining difference in behaviour between the maintenance page and the Files Panel is IMO minor and can be ignored.

@luc-github
Copy link
Author

luc-github commented Apr 11, 2023

I am always impressed by your ability to find answers in RFC 👍

Not related but is ssdp already working ? I have enabled it but windows 11 neither android sspd scanner found grblHAL ssdp messages

@terjeio
Copy link
Contributor

terjeio commented Apr 11, 2023

I am always impressed by your ability to find answers in RFC

Since I have been involved in standardization work I am keen to follow standards where possible. Not that I always understand the wording used - that can be hard sometimes...

Not related but is ssdp already working ?

Did you enable SSDP in Network Services?

image

terjeio added a commit that referenced this issue Apr 11, 2023
…preferences.json.

Updated to latest maintenance page version.
@luc-github
Copy link
Author

luc-github commented Apr 11, 2023

Yes I do
image
But no device grblHAL in window:
image
Neither with https://play.google.com/store/apps/details?id=com.vgc.ssdpscan
Screenshot_20230412-071053140 460

I can see my esp3d-tft but not grblHAL, I initially though upnp:rootdevice was missing but I saw it in your code

@terjeio
Copy link
Contributor

terjeio commented Apr 12, 2023

Send $I from the Terminal to check if has SSDP been started. It will show up in the NEWOPT element if so:

[VER:1.1f.20230411:]
[OPT:VNM0SL,35,1024,3,0]
[AXS:3:XYZ]
[NEWOPT:ENUMS,RT+,REBOOT,TC,SED,RTC,WIFI,FTP,SSDP,SD]

I am on Win10 and 7 it works with both.

@luc-github
Copy link
Author

luc-github commented Apr 12, 2023

Sorry I was out all day :

$I
[VER:1.1f.20230401:]
[OPT:VNMSL,35,1024,3,0]
[AXS:3:XYZ]
[NEWOPT:ENUMS,RT+,REBOOT,TC,SED,RTC,WIFI,mDNS,SSDP,SD]
[FIRMWARE:grblHAL]
[NVS STORAGE:*FLASH]
[DRIVER:ESP32]
[DRIVER VERSION:230331]
[DRIVER OPTIONS:v4.3.5-42-g0369fe88c3]
[BOARD:BDRING v4]
[WIFI MAC:24:62:ab:f2:a6:c0]
[IP:192.168.2.108]
[PLUGIN:WebUI v0.15]
[PLUGIN:SDCARD v1.08]
ok

Also checked not visible under windows 10 Home edition

@luc-github
Copy link
Author

I have tested latest grbHAL and indeed all works now as expected - thank you

I agree that embedded page opening known content type is a minor issue. I will try to solve it later without increasing the footprint

About ssdp issue I think it is better to track in another ticket for readibility - let me know if you want me to do more test

I close issue as fixed - thank you

@terjeio
Copy link
Contributor

terjeio commented Apr 14, 2023

About ssdp issue I think it is better to track in another ticket for readibility - let me know if you want me to do more test

I'll come back to this - I have to get hold of a Win11 machine for testing.

@troth530
Copy link

troth530 commented Apr 22, 2023

I am using ESP32 Wroom
Now I cannot upload at all to Flash on WebUI3. Using grblHal [VER:1.1f.20230411:]
I can upload to SD, I tried WEBUI_INFLASH=0 and WEBUI_INFLASH=1

WebUI3
WebUI3SD
WebUI3Flash

Thank you!

@terjeio
Copy link
Contributor

terjeio commented Apr 23, 2023

You have a www folder with index.html.gz in it on the SD card, this sets HostPath to /www and (incorrectly?) redirects flash uploads there.
Add ?forcefallback=yes to the IP address to open the maintenance page, there you can upload to flash.
Mixing flash and SD card storage of index.html.gz and related files (preferences.json etc.) is IMO not a good idea though.

There is a minor bug in grblHAL where saving to flash does not work from the maintenance page when index.html.gz is not found in flash file system nor in the /www folder on the SD card and ?forcefallback=yes is not added to the IP address. I have fixed this and will add the fix in the next commit.

@troth530
Copy link

troth530 commented Apr 24, 2023

@terjeio I tried with WEBUI_INFLASH=1 and no www folder in the SD card and the 12kB index.html.gz from your repo goes to the maintenance page and is unable to upload the 84kB index.html.gz WebUI3.0.

Next attempt with WEBUI_INFLASH=1 (and no www folder on SD card). I replaced the12kB index.html.gz in the embedded folder with 84kB index.html.gz WebUI3.0. This enabled WebUI 3.0 but no settings could be saved and it returned a preferences.json file missing error.

Then finally I tried WEBUI_INFLASH=0 and added 84kb index.html.gz WebUI3.0 to the www folder on SD card. I also added a preferences.json file to that directory. Likewise it enabled WebUI 3.0 but this time without the preferences,json error, and yet again no settings could be saved.

For good hygiene I run a clean and erase_flash command from ESP-IDF. It seems the ?forcefallback=yes does enable writing to Flash so the main problem is fixed. I prefer WEBUI_INFLASH=1 in any case and since I can now write to flash there is no need to use the SD card.

I have a couple of other things to report...

  1. If ESTOP and Safety Door are disabled, regardless the Emergency Stop button appears in WebUI 3.0. If the Emergency Stop and button is hit, it will trigger a door status error. In this state Ctrl+X does not clear the error but hitting the reset button (hard reset) or a using the reset button in IOsender is the only way to clear this state. IOsender usually crashes but the soft reset does work after powering the controller off and on.

ESTOPtriggersSafetyDoorError

  1. WebUI 3.0 is now enabling what I would consider as very dependable writes to the SD card which was much less dependable with WebUI 2.0. However the uploads are still very slow. It takes greater than 3 minutes to upload a less than 2MB file. I am using high quality class 10 SD cards and am close to the WiFi router. I likewise found about 4kBps transfer rates with FTP. All networking services are turned on $70=47. Also, downloads are quite fast.

If it is at all helpful, I tried MKS DLC32 on their WebUI (which is a custom version of Luc's with a frame button for laser gCode)
and uploads are very fast, so I don't know if it's some specific setting for the ESP32 they are using, but their code is here for reference (maybe it'll give some insight) https://github.com/makerbase-mks/MKS-DLC32-FIRMWARE .

Thank you.

@terjeio
Copy link
Contributor

terjeio commented Apr 24, 2023

I tried with WEBUI_INFLASH=1 and no www folder in the SD card and the 12kB index.html.gz from your repo goes to the maintenance page and is unable to upload the 84kB index.html.gz WebUI3.0.

I have a fix pending for this bug.

Next attempt with WEBUI_INFLASH=1 (and no www folder on SD card). I replaced the12kB index.html.gz in the embedded folder with 84kB index.html.gz WebUI3.0. This enabled WebUI 3.0 but no settings could be saved and it returned a preferences.json file missing error.

The embedded filing system is read-only so saving is not possible, I could try saving to the root of the SD-card.

If ESTOP and Safety Door are disabled, regardless the Emergency Stop button appears in WebUI 3.0. If the Emergency Stop and button is hit, it will trigger a door status error. In this state Ctrl+X does not clear the error but hitting the reset button (hard reset) or a using the reset button in IOsender is the only way to clear this state.

The Emergency Stop button sends a safety door triggered command (0x84), I'll have to check the handling of this.

IOsender usually crashes but the soft reset does work after powering the controller off and on.

I have a fix pending fo this too.

However the uploads are still very slow. It takes greater than 3 minutes to upload a less than 2MB file.

Uploading a 4.66 MB file takes 30 seconds to a no-brand SD card with my test board. Bad wiring on yours is somehow blocking update to a higher SPI speed from the one used to initialize the card?

I tried MKS DLC32 on their WebUI

Is this the same board you are running grblHAL on?

@troth530
Copy link

@terjeio ?forcefallback=yes enables writing to the Flash filesystem. Is writing to flash this way expected?

I am not running grblHAL on the DLC32. I am using a WROOM ESP32 dev board with a MicroSD wired in. All pins are pulled up to 3.3V with a 10k resistor. I suppose MISO and MOSI pins could be mismatched a little on length.

@terjeio
Copy link
Contributor

terjeio commented Apr 24, 2023

?forcefallback=yes enables writing to the Flash filesystem. Is writing to flash this way expected?

Yes, when index.html.gz is found in the flash file system you have to add it to open the maintenance page.

I suppose MISO and MOSI pins could be mismatched a little on length.

Using long wires? Reducing the SPI clock frequency may help - but better to fix any underlying issues.
Here are the preconfigured options:

#define SDMMC_FREQ_DEFAULT      20000       /*!< SD/MMC Default speed (limited by clock divider) */
#define SDMMC_FREQ_HIGHSPEED    40000       /*!< SD High speed (limited by clock divider) */
#define SDMMC_FREQ_PROBING      400         /*!< SD/MMC probing speed */
#define SDMMC_FREQ_52M          52000       /*!< MMC 52MHz speed */
#define SDMMC_FREQ_26M          26000       /*!< MMC 26MHz speed */

@troth530
Copy link

@terjeio I tried matching MISO/MISO again and got the same result. The wire length is approximately 75mm (less than 100mm). I could try higher valued pullup resistors, though I believe the driver should not have an issue with 10k. Also, the filter caps are placed near the 3.3V bus directly next to the SD card connector. Unless it is some inherent issue with the WROOM Dev board, I'm a bit at a loss. But I will try a few more things since you are getting much better results.
image

Otherwise I'm very happy with WebUI 3.0 as SD writes are dependable (but still slow).

Also I am using 2209s in UART mode and when I add
#define HAS_BOARD_INIT (along with #define UART2_RX_PIN GPIO_NUM_16 and
#define UART2_TX_PIN GPIO_NUM_17) I get a compilation error on driver.c
Is HAS_BOARD_INIT required for TMC UART?

[1319/1320] Linking CXX executable grbl.elf
FAILED: grbl.elf
cmd.exe /C "cd . && C:\Users\troth530\.espressif\tools\xtensa-esp32-elf\esp-2021r1-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-g++.exe  -mlongcalls -Wno-frame-address   @CMakeFiles\grbl.elf.rsp  -o grbl.elf  && cd ."
c:/users/troth530/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: esp-idf/main/libmain.a(driver.c.obj):(.literal.driver_init+0x94): undefined reference to `board_init'
c:/users/troth530/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: esp-idf/main/libmain.a(driver.c.obj): in function `driver_init':
c:\users\troth530\esp\esp-idf\esp32\build/../main/driver.c:2235: undefined reference to `board_init'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

@terjeio
Copy link
Contributor

terjeio commented Apr 24, 2023

Is HAS_BOARD_INIT required for TMC UART?

Yes, and you have to add your board define here as well.

BTW I can clear the door state in the WebUI by clicking the soft reset button.

@troth530
Copy link

@terjeio

BTW I can clear the door state in the WebUI by clicking the soft reset button.

Yes that works!

Yes, and you have to add your board define [here as well]

Yes, I'm aware of needing to add the board to trinamic_if.c and of course 2209 must be turned on in cMakeLists.txt
However when #define HAS_BOARD_INIT is added to the pinpmap file (in addition to UART2 pins) I still get the error

c:\users\troth530\esp\esp-idf\esp32\build/../main/driver.c:2235: undefined reference to `board_init'

One odd thing about this error is that the line number is 2235, but the board_init function is not called until about 39 lines later and it is a linking error.

2273 #ifdef HAS_BOARD_INIT
2274 board_init();
2275 #endif

Using this same setup 2209 had been working in GRBL_BUILD 20221009 but I get this error now in GRBL_BUILD 20230411

@troth530
Copy link

@terjeio please ignore the HAS_BOARD_INIT error. After trying to double check the HAS_BOARD_INIT error on another PC I am no longer having errors during linking. I'm not sure why it was happening.

I am still trying more things to speed up SD card. I will try wiring with coax to debug, but not sure why reads are so fast and writes are so slow.

Thank you!

@terjeio
Copy link
Contributor

terjeio commented Apr 25, 2023

I see that you have a pullup on the CS line and not the CLK line - the opposite is correct? At least that is what I have on my test board and on a separate commercial breakout I use for the RP2040.

@troth530
Copy link

Hi @terjeio from all I've researched pullup on CLK should not be required, but I will definitely try it.

It's really good that WebUI 3 is reading all the grblHAL settings, I sometimes have an issue where I have to power cycle the board because some of the settings are not parsed properly. The error is "parsing data error, data are incorrect"
image

This should be Enable (not Disable) laser during hold
image

Also please note some of the units are annotated in the comments incorrectly in the config.h file. Current is millamps and acceleration is mm/s^2 (not mm/min^2). Previously in the config file units for accel were mm/min^2 and we had to multiply accel6060 in the config file even though $12x has always been reported as mm/s.

Thank you!

@terjeio
Copy link
Contributor

terjeio commented Apr 27, 2023

I sometimes have an issue where I have to power cycle the board because some of the settings are not parsed properly.

Which ones? Note that some settings are dependent on others - if processed out of sequence errors may occur.

This should be Enable (not Disable) laser during hold

The label is correct but not the underlying functionality/default value. Fixed in the latest commit.

Also please note some of the units are annotated in the comments incorrectly in the config.h file.

Fixed in the latest commit.

@troth530 Do you have a Win11 computer and can test SSDP?

@luc-github I have tested the ssdpscan app you linked above and it works for me, both an ESP32 and a RP2040 controller shows up correctly. Odd that the ESP32 controller does not for you.

@luc-github
Copy link
Author

let me know if you want any test/debug from me

@troth530
Copy link

troth530 commented May 1, 2023

Hi @terjeio, I do not know which settings are and are not being parsed correctly. I only get a blank screen when the settings are not parsed correctly (see below). If preferences.json is out of sync due to changes to $$ settings through serial could it be that the data cannot be parsed? Maybe full $$ grbl settings should be read first, then parse the grblHAL specific ones.
ParsingError

There is a typo in grbl/settings.c (double D and needs comma at the end of the line)
SettingsdotcTypo

The behavior of $63 is now correct, it seems you corrected some of the logic as well. However, stopping the job while running does not spin the spindle down, but it will disable the Spindle_Enable pin. Also, after stopping the job, M5 will not spin the spindle down either, it seems a M4 is required then M5 will spin the spindle down once the job is stopped. I've confirmed this in several senders including IOSender. Pause and Resume work as expected though. I am using laser mode by the way.

$14x (motor current) has a typo in the units in comments of config.h, should be milliamps (not Amps).

$73=3 seems a little less useful now. Previously if it did not connect to a 2.4GHz router in station mode it would revert back to AP mode and when you connected to the AP you had the full WebUI (though 192.168.5.1). Now if it does not connect to the router in station mode, connecting to the AP will enable an AP connect page which allows you to only change $74 and $75 and no WebUI. So really $73=3 enables only station mode or station mode through AP connect page, but does not allow AP mode.
Reverting back to $73=2 does allow AP mode, but it must be done through serial if STA fails to connect.
STA+APModeConnected to AP

Yes, I can test SSDP with a Win11 computer, but I am still debugging a few more things. First of all I have tried pullups on CLK but no improvement. Here is the MKS DLC 2.1 schematic as a reference and speed was good, no pullups on CLK, a pulldown on MISO otherwise the same schematic as I am using. Is there a specific way to format the SD cards that may improve. I can always have a prototype board, but not sure how to proceed. So far I continue to find very slow uploads and very fast downloads with the SD card.
DLC32_2pt1_Schematic

@luc-github
Copy link
Author

@troth530 you can do a [ESP400]json=YES in webui terminal and share output to see where in json the problem is, you can also get the output from the web browser dev tool when you do a refresh in setting

image

@troth530
Copy link

troth530 commented May 2, 2023

@troth530 you can do a [ESP400]json=YES in webui terminal and share output to see where in json the problem is, you can also get the output from the web browser dev tool when you do a refresh in setting

For some reason the window is blank.
ESP400Preview

Also I get this result
ESP701

Thank you!

@luc-github
Copy link
Author

luc-github commented May 2, 2023

be sure verbose mode is checked in webui
image

also you did not opened the [ESP800] response but [ESP701] as you see cmd:"701" in preview
Should be like this:
image

@terjeio
Copy link
Contributor

terjeio commented May 2, 2023

I only get a blank screen when the settings are not parsed correctly (see below).

This could be due to the controller crashing and resetting itself? grblHAL supports both WebUI v2 and 3, either at the same time or a specific version. Local builds enable both at the same time and which version to activate is determined from the content of the [ESP800] request sent when the WebUI is loaded. If the controller was reset then a page reload should bring the settings back. Connecting to the controller with a terminal program before the WebUI is opened and monitoring the output will reveal if a crash is happening.
Note that a reason for resetting is that a setting has been changed that requires a reset and you have confirmed the reset in the WebUI - this should lead to an automatic reload/reconnect after a wait. This type of reset should not affect operation.

Previously if it did not connect to a 2.4GHz router in station mode it would revert back to AP mode and when you connected to the AP you had the full WebUI (though 192.168.5.1).

Which previous version?

For the the SD card issue, can you look at the signalling with a scope when writing. Is the clock signal looking good and what is its frequency. Same frequency when reading? The SD card driver code is by Espressif, either check their forums for your problem or raise a new issue with them?

BTW I have fixed the typos and the spindle off bug. However, please open new issues instead of adding new ones to this as you are not the OP and this issue has been closed.

@troth530
Copy link

troth530 commented May 7, 2023

Hi @terjeio, I believe the parsing bug is related to more than 3 axes. If I treat the 4th axis as ganged it will parse correctly (sort of), but it seems to only be able to parse 3 axes and even with ganged, it will read the first 3 axes properly but will not read the A axis or M3. I will open up a new issue. #12 (GrblHAL WebUI 3.0 unable to parse $$ settings from controller)

It looks like you have fixed the bug with the laser (spindle) not spinning down, but the spindle enable pin still remains active even when M5 is issued and even if one issues M5, one must still issue M4, then M5 to turn off the spindle enable pin. At least it is now safe as the laser will turn off when stop command is given or M5 is given. I will open up a new issue for this as well.
(grblHAL/ESP32#73)

Other odd behavior such as $73=3 now reverts to a fully functional AP if STA does not connect to router. So this has definitely been fixed!

SSDP does not seem to work at all, the device will not boot if only that setting is enabled. I'll scope out the SD card clk and TX/RX pair and figure out what's going on with the signals.

Hi @luc-github yes I had verbose mode on, but when it does not parse, there is no debugging data in the preview window or the console window. Using the same settings and 3 axes (4th ganged) looks like this
3AxisParsing

Thank you!

@terjeio
Copy link
Contributor

terjeio commented May 8, 2023

SSDP does not seem to work at all, the device will not boot if only that setting is enabled.

I am not able to replicate that. FYI the SSDP daemon will not be started unless HTTP is also enabled - since clients reads a xml file via http to get details about the service. Win10 info screen:

image

I'll scope out the SD card clk and TX/RX pair and figure out what's going on with the signals.

As a test I wired up a bare board to a commercial breakout with rather long wires, still getting nearly 300Kb/sec write speed:

IMG_20230508_174103

@troth530
Copy link

troth530 commented May 9, 2023

Hi @terjeio you give me hope! Can you give me more details on the SD breakout board such as a link you where you ordered the SD breakout. I may have to strip down to a bare bones setup for debug. Can you also share your cMakeLists.txt and any other non default settings. At the risk of posing a seemingly silly question- it also it looks like you have a wire connected to GPIO13, is that used for anything?

Also please post your cMakeLists.txt file so I can test SSDP. The controller is crashing (or just not booting) when I simply enable SSDP setting with my current working settings.

Thank you!

@terjeio
Copy link
Contributor

terjeio commented May 9, 2023

Can you give me more details on the SD breakout board such as a link you where you ordered

The link is long gone, this is similar.

you have a wire connected to GPIO13, is that used for anything?

It is for measuring the spindle enable signal - without risking shorting pins.

Also please post your cMakeLists.txt file so I can test SSDP.

CMakeLists.txt

$RST=& after flashing to get default values.

@troth530
Copy link

troth530 commented Jul 5, 2023

Hi @terjeio,
I realize this is a closed issue but it has all the history...

I have tried the SD card module wired directly to the ESP32 as you have done and it still reads very slow (4-8kB/sec) uploading through WebUI. I don't know whether your ESP32 WROOM board uses a Silabs or CH340g USB->UART. I wired it as short as possible with similar wire lengths. The breakout board schematic is similar to as I'm using on my board that I posted April 24. It seems you are using a different upload method other than WebUI!? If using WebUI, what version you using? Also, I am using a Class 4 brand name SD card (Sandisk) 4GB formatted at FAT32. Still scratching my head as to why this occurs, but it is definitely slow.

Based on your CMakeList.txt is seems you have SoftAP turned off. As soon as I turn on SoftAP, then SSDP does not work, the device will not boot. Also you have SD streaming off, I typically leave this on.

Thank you!

@terjeio
Copy link
Contributor

terjeio commented Jul 6, 2023

I don't know whether your ESP32 WROOM board uses a Silabs or CH340g USB->UART.

I am using a Silabs CP210x driver, I guess that matches the chip - I can no longer read the markings. However, which chip is used for USB should not matter at all for WebUI performance. But to be sure, do you power the ESP32 via the USB port?

It seems you are using a different upload method other than WebUI!?

I have tested with both WebUI and FTP, both completes a 4.7Mbyte upload in ~30s. I just retested with the latest version:

image

Based on your CMakeList.txt is seems you have SoftAP turned off. As soon as I turn on SoftAP, then SSDP does not work, the device will not boot

I'll look into this again, SSDP should not be started before a client connection is established? Or perhaps not at all in SoftAP mode?

Also you have SD streaming off, I typically leave this on.

Does not matter, it is automatically switched on when the WebUI is enabled.

@terjeio
Copy link
Contributor

terjeio commented Jul 11, 2023

Fix for SSDP comitted.

@troth530
Copy link

Hi @terjeio, great work on SSDP and in general. However, I am still very confused about the SD card. Can you post your CMakesList.txt and a bin file. For some reason I cannot get more than ultra-slow writes to SD card. I have tested both powered from USB (which uses a LM117 5V to 3.3V LDO) and powered from a mainboard that is providing 3.3V directly the WROOM (with the LM117 removed from the WROOM).

Thank you!

IMG_20230705_103240071 (002)

@terjeio
Copy link
Contributor

terjeio commented Jul 13, 2023

Can you post your CMakesList.txt and a bin file.

Last test I did was with Web Builder generated firmware. Generally I test with a CNC BoosterPack that has on-board EEPROM, this will not start if the EEPROM is not present. You can build for this with EEPROM disabled in the Web Builder.

@troth530
Copy link

Hi @terjeio, I am trying to build bare bones and just with the SD card and the WROOM device currently to match our result. I am definitely not changing the SPI pins in any testing I've done. I posted the error with Web Builder in this issue and will wait for a fix to test again.
#12

My goal is to replicate exactly your simplified setup and then systematically determine the source, but currently cannot even replicate your result after ordering the exact same SD card breakout and using what I believe to be the exact same hardware.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants