New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PPPOS reconnection/disconnection faliuer after sometime. (IDFGH-6123) #7803
Comments
@AdiXPS could you tell us which modem are you using? Also, are you seeing the first crash with just the code you've shared or was there anything else running on ESP32 at the same time? As for the first issue, null dereference in the internal UART structures, I couldn't think of anything else but memory corruption. Is this happening randomly or could it be somehow related to the second issue? (like one occurs after another?) As for the second problem, it might be possible that |
Hi @david-cermak,
Its randomly. I notice that its crashing on Core 1? not sure why though.. moreover, based on my observation, its not related as its rare..
Its the same as sim800. i've added 1s delay before initiating ath command (based on last change you've proposed in another issue). But that didn't prevent the issue from happening again.. I notice one thing that modem in exiting PPP is still sometimes sending data by UART to esp even when confirming for +++.. "10/30/2021 11:33:35 PM",CONNECT 150000000 "10/30/2021 11:33:35 PM",~ÿ}#À!}!í} }9}"}&} } } } }#}%Â#}%}%}&Ø}+‘¹}'}"}(}" ˆ "10/30/2021 11:33:36 PM",�� -��ø0 "10/30/2021 11:33:36 PM",D’T��Të�7ƒ�Të9æn�~~€!�� ��� "10/30/2021 11:33:36 PM",D’T��Të�7ƒ�Të9æ_„~ "10/30/2021 11:33:40 PM",OK <=============this is when esp sends +++, module sends OK but esp keeps getting data afterwards.. "10/30/2021 11:33:43 PM",~!E s³ì@ é�+ä6ÌÞO <==== this is the famous ~!E when code crash.. "10/30/2021 11:33:43 PM",D’T�»äê�E3-��_€� @?ë ��� "10/30/2021 11:33:43 PM",@`Åñ�!ÿç��� "ûÞzêFçÈ�°rúß“Ü={k¹Å!Ð�á¤ê�z©]Z¼ëŠí��� �¡#<©�€¯•Y‹›Fèõ8ü%©B‚~~!E s�•@ ê�Ã;6ÌÞO "10/30/2021 11:33:43 PM",D’T�»å�@t „àA�J€� 9põ ��� "10/30/2021 11:33:43 PM",�™�Æ{( £��� "�.Kþ�Z��0^#}]ãÝtPV#HÌ�MÁÜe‡�5šT8�½ê��� �ØwG«+ÙéßÍ6H×@-Ü�DÛóQ~ Its happening randomly |
This is weird, I've never seen anything like that with any device. Is the Anyway, I'll look into it and try to reproduce the issue, at least the first one. For the second one, we'd probably have to purchase your device |
Yup, I totally agree with you.
Yes it is.
I appreciate to get your advice regarding proper way to get the hexdump.
I've double check, mux is not enabled.
Thanks.. from what I can see that there are other issues raised here facing same issue, but no one actually tried to check the data coming from modem. I always notice |
You can use
Are you sure this was the correct version? config.dte_buffer_size = CONFIG_EXAMPLE_MODEM_UART_RX_BUFFER_SIZE / 2; This line from your code suggests it might have been I've been testing the while loop on |
@AdiXPS Thanks for reporting, would you please help share if any updates for the issue? Thanks. |
Still there is no update, same issue is occurring. Actually just today I got new sim7070g, I'll de-solder the one I'm using and replace it with new one just in case. I'll rerun it again with hexdump and I'll post the update within two days God willing. |
@david-cermak @Alvin1Zhang I am still facing the same issue with different SIM7070G module. Different PCB also. @david-cermak Regarding hexdump may I know where exactly I need to place it, to help more in the investigation e.g. at the end of esp_handle_uart_data() function? |
@david-cermak , @Alvin1Zhang We'll, seems that the issue is the module 7070G/7080 have backdoor after full analysis. The modem keeps sending data even after exiting data mode, I have also requested to get the latest firmware from simcom (just to eliminate any M-in-M kind of issues) but its even worse now. Its getting certificates and all kind of alarming stuff: it tries to get fake cert. for *.piojm.tech and this brings me to conclude that all ~!E famous text received is when simcom tries to establish malicious connections..
I am not interested in this aspect (backdoor or not), how can we ignore such coming data? |
Hi @AdiXPS The famous
I think you can try to reset the device, either using a reset command or pulling the reset line directly. Or you can try to use CMUX mode, but it depends on your usecase. What's the reason for switching to command mode? Do you need to use AT interface or close the network connection? |
|
I highly doubt this, as we got confirmation from the module 'OK' (mentioned earlier in the logs) for exiting data and hangup, but the module keeps sending and receiving data. As long as we got ATH => OK this should kill any connection, regardless if its public cert or not. Moreover, even if its public cert, its no where mentioned all of their documentation regarding this behavior/activity. btw the domain cert that we've found in the module is considered as one of the blacklisted DNSs. I am reporting the findings here for anyone will use this module.
I need to close the connection and get the lon,lat by AT command. and do battery readings.. but as long as we keep getting such data uart will be unuasable |
You can use CMUX mode for that and keep the network/PPP still on. |
Hi @AdiXPS Is there any update from your side? This issue seems to be related to specifically your devices of |
Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates. Thanks. |
Environment
Problem Description
I've tried @david-cermak example, basiclly I found two main issues after leave it running for a couble of hours:
1-after awhile, I'am getting the famous "GGuru Mediation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled." One more thing here, I notice that its showing Core 1 while I dont have any task running on it..?
`Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x400d52e5 PS : 0x00050031 A0 : 0x400826f8 A1 : 0x3ffb0dc0
0x400d52e5: uart_ll_get_intsts_mask at C:/esp4/esp-idf/components/hal/esp32/include/hal/uart_ll.h:170
(inlined by) uart_rx_intr_handler_default at C:/esp4/esp-idf/components/driver/uart.c:746
0x400826f8: _xt_lowint1 at C:/esp4/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1105
A2 : 0x3ffb7760 A3 : 0x3ffb0aa0 A4 : 0x00000091 A5 : 0x8ca96391
A6 : 0x00000910 A7 : 0x00000000 A8 : 0x00000001 A9 : 0x3ffb0d50
A10 : 0x00000000 A11 : 0x00000000 A12 : 0x80103e4e A13 : 0x3ffb86c0
A14 : 0x3ffb2f6c A15 : 0x3ffb86e0 SAR : 0x0000001a EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000008 LBEG : 0x00000000 LEND : 0x00000000 LCOUNT : 0x00000000
Backtrace:0x400d52e2:0x3ffb0dc0 0x400826f5:0x3ffb0e10 0x40118563:0x3ffb8710 0x400d845b:0x3ffb8730 0x40088505:0x3ffb8750 0x4008a525:0x3ffb8770
0x400d52e2: uart_ll_get_intsts_mask at C:/esp4/esp-idf/components/hal/esp32/include/hal/uart_ll.h:170
(inlined by) uart_rx_intr_handler_default at C:/esp4/esp-idf/components/driver/uart.c:746
0x400826f5: _xt_lowint1 at C:/esp4/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1105
0x40118563: cpu_ll_waiti at C:/esp4/esp-idf/components/hal/esp32/include/hal/cpu_ll.h:183
(inlined by) esp_pm_impl_waiti at C:/esp4/esp-idf/components/esp_pm/pm_impl.c:827
0x400d845b: esp_vApplicationIdleHook at C:/esp4/esp-idf/components/esp_common/src/freertos_hooks.c:63
0x40088505: prvIdleTask at C:/esp4/esp-idf/components/freertos/tasks.c:3839 (discriminator 1)
0x4008a525: vPortTaskWrapper at C:/esp4/esp-idf/components/freertos/port/xtensa/port.c:168
ELF file SHA256: d843f1542d225f73`
2-Mostly the crash is happening when stopping PPP only if there is existing connection. If for any reason the disconnection happen from host (ISP; no IP EVENT) its very stable.
[0;32mI (511128) esp-modem: entering esp_modem_stop_ppp...[0m [0;31mE (512058) esp-modem: esp_dte_handle_line(124): handle line failed[0m [0;33mW (512058) pppos_example: Unknow line received: ~!E[0m Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Always getting Unknow line received: ~!E.
When I monitor the modem I am getting this transmitted:
"10/30/2021 3:50:38 PM",CONNECT 150000000 "10/30/2021 3:50:38 PM",~ÿ}#À!}!¦} }9}"}&} } } } }#}%Â#}%}%}&Öcº÷}'}"}(}"£�~~ÿ}#À!}"}!} }4}"}&} } } } }%}&M§T}$}'}"}(}"³ø~~ÿ}#À!}!§} }8}"}&} } } } }#}$À#}%}&Öcº÷}'}"}(}"í×~~ÿ}#À!}+¨} }(Öcº÷CC~~À# ý0~~€!< jà~~€! "10/30/2021 3:50:39 PM", -ø0~~€!= ¶º~~€! "10/30/2021 3:50:39 PM",™ m�Të7ƒTë9æÄG~~€! "10/30/2021 3:50:39 PM",™ m�Të7ƒTë9æõÙ~~!E ;å@ UÄ "10/30/2021 3:50:43 PM",E "10/30/2021 3:50:43 PM",™ m»�b
µÇª€ }]y"10/30/2021 3:50:43 PM",¼}^€bV_9 x(~
"10/30/2021 3:50:43 PM",OK
"10/30/2021 3:50:47 PM",OK
"10/30/2021 3:50:59 PM",~!E (®²@ 3Ú¶13‚.
"10/30/2021 3:50:59 PM",™ m»âpK.hì¶Ñ�P s‘
“~~!E ;æ@ Uà "10/30/2021 3:51:23 PM",E "10/30/2021 3:51:23 PM",™ m»�b
µÇª€ áw"10/30/2021 3:51:23 PM",¼�cV_9 µ~~!E 0 @ sIa13‚.`
seems the esp sending +++ and ath but the modem keep sending garbage? or it didnt actually exit data mode?
here is sample from modem when there is no issues..:
`"10/30/2021 5:05:32 PM",CONNECT 150000000
"10/30/2021 5:05:32 PM",
ÿ}#À!}!}1} }9}"}&} } } } }#}%Â#}%}%}&Ö¨N½}'}"}(}"°}#ÿ}#À!}"}!} }4}"}&} } } } }%}&×É£Õ}'}"}(}"_Dÿ}#À!}!}2} }8}"}&} } } } }#}$À#}%}&Ö¨N½}'}"}(}"¡¿ÿ}#À!}+}3} }(Ö¨N½œ°À# ý0€!Ø ªŠ€!"10/30/2021 5:05:34 PM", -ø0
€!Ù vЀ!"10/30/2021 5:05:34 PM",AN�Të7ƒTë9毥~~€!
"10/30/2021 5:05:34 PM",AN�Të7ƒTë9æž;
"10/30/2021 5:05:38 PM",OK
"10/30/2021 5:05:41 PM",OK
"10/30/2021 5:05:52 PM",OK`
Expected Behavior
Should successfully connect and disconnect.
Actual Behavior
Crashes after period of time..
Steps to reproduce
// If possible, attach a picture of your setup/wiring here.
Code to reproduce this issue
// If your code is longer than 30 lines, GIST is preferred.
Debug Logs
Other items if possible
build
folder (note this may contain all the code details and symbols of your project.)The text was updated successfully, but these errors were encountered: