Open Ethernet emulation for esp-idf 3.2 (QEMU-50) #22
Labels
Comments
|
This doesn't look easy to achieve as the Ethernet driver in IDF started supporting multiple MAC implementations only in v4.0. Until then there was only an implementation for the real ESP32 Ethernet controller. I guess you could try to override the entire component from your application. The implementation of openeth MAC driver would indeed need to be changed to match the Ethernet driver in v3.2. |
|
Isn't this now fixed with OpenEth in QEMU? |
igrr
pushed a commit
that referenced
this issue
Aug 2, 2022
Include the qtest reproducer provided by Alexander Bulekov in https://gitlab.com/qemu-project/qemu/-/issues/542. Without the previous commit, we get: $ make check-qtest-i386 ... Running test tests/qtest/intel-hda-test AddressSanitizer:DEADLYSIGNAL ================================================================= ==1580408==ERROR: AddressSanitizer: stack-overflow on address 0x7ffc3d566fe0 #0 0x63d297cf in address_space_translate_internal softmmu/physmem.c:356 #1 0x63d27260 in flatview_do_translate softmmu/physmem.c:499:15 #2 0x63d27af5 in flatview_translate softmmu/physmem.c:565:15 #3 0x63d4ce84 in flatview_write softmmu/physmem.c:2850:10 #4 0x63d4cb18 in address_space_write softmmu/physmem.c:2950:18 #5 0x63d4d387 in address_space_rw softmmu/physmem.c:2960:16 #6 0x62ae12f2 in dma_memory_rw_relaxed include/sysemu/dma.h:89:12 #7 0x62ae104a in dma_memory_rw include/sysemu/dma.h:132:12 #8 0x62ae6157 in dma_memory_write include/sysemu/dma.h:173:12 #9 0x62ae5ec0 in stl_le_dma include/sysemu/dma.h:275:1 #10 0x62ae5ba2 in stl_le_pci_dma include/hw/pci/pci.h:871:1 #11 0x62ad59a6 in intel_hda_response hw/audio/intel-hda.c:372:12 #12 0x62ad2afb in hda_codec_response hw/audio/intel-hda.c:107:5 #13 0x62aec4e1 in hda_audio_command hw/audio/hda-codec.c:655:5 #14 0x62ae05d9 in intel_hda_send_command hw/audio/intel-hda.c:307:5 #15 0x62adff54 in intel_hda_corb_run hw/audio/intel-hda.c:342:9 #16 0x62adc13b in intel_hda_set_corb_wp hw/audio/intel-hda.c:548:5 #17 0x62ae5942 in intel_hda_reg_write hw/audio/intel-hda.c:977:9 #18 0x62ada10a in intel_hda_mmio_write hw/audio/intel-hda.c:1054:5 #19 0x63d8f383 in memory_region_write_accessor softmmu/memory.c:492:5 #20 0x63d8ecc1 in access_with_adjusted_size softmmu/memory.c:554:18 #21 0x63d8d5d6 in memory_region_dispatch_write softmmu/memory.c:1504:16 #22 0x63d5e85e in flatview_write_continue softmmu/physmem.c:2812:23 #23 0x63d4d05b in flatview_write softmmu/physmem.c:2854:12 #24 0x63d4cb18 in address_space_write softmmu/physmem.c:2950:18 #25 0x63d4d387 in address_space_rw softmmu/physmem.c:2960:16 #26 0x62ae12f2 in dma_memory_rw_relaxed include/sysemu/dma.h:89:12 #27 0x62ae104a in dma_memory_rw include/sysemu/dma.h:132:12 #28 0x62ae6157 in dma_memory_write include/sysemu/dma.h:173:12 #29 0x62ae5ec0 in stl_le_dma include/sysemu/dma.h:275:1 #30 0x62ae5ba2 in stl_le_pci_dma include/hw/pci/pci.h:871:1 #31 0x62ad59a6 in intel_hda_response hw/audio/intel-hda.c:372:12 #32 0x62ad2afb in hda_codec_response hw/audio/intel-hda.c:107:5 #33 0x62aec4e1 in hda_audio_command hw/audio/hda-codec.c:655:5 #34 0x62ae05d9 in intel_hda_send_command hw/audio/intel-hda.c:307:5 #35 0x62adff54 in intel_hda_corb_run hw/audio/intel-hda.c:342:9 #36 0x62adc13b in intel_hda_set_corb_wp hw/audio/intel-hda.c:548:5 #37 0x62ae5942 in intel_hda_reg_write hw/audio/intel-hda.c:977:9 #38 0x62ada10a in intel_hda_mmio_write hw/audio/intel-hda.c:1054:5 #39 0x63d8f383 in memory_region_write_accessor softmmu/memory.c:492:5 #40 0x63d8ecc1 in access_with_adjusted_size softmmu/memory.c:554:18 #41 0x63d8d5d6 in memory_region_dispatch_write softmmu/memory.c:1504:16 #42 0x63d5e85e in flatview_write_continue softmmu/physmem.c:2812:23 #43 0x63d4d05b in flatview_write softmmu/physmem.c:2854:12 #44 0x63d4cb18 in address_space_write softmmu/physmem.c:2950:18 #45 0x63d4d387 in address_space_rw softmmu/physmem.c:2960:16 #46 0x62ae12f2 in dma_memory_rw_relaxed include/sysemu/dma.h:89:12 qemu#47 0x62ae104a in dma_memory_rw include/sysemu/dma.h:132:12 #48 0x62ae6157 in dma_memory_write include/sysemu/dma.h:173:12 ... SUMMARY: AddressSanitizer: stack-overflow softmmu/physmem.c:356 in address_space_translate_internal ==1580408==ABORTING Broken pipe Aborted (core dumped) Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211218160912.1591633-4-philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
igrr
pushed a commit
that referenced
this issue
Aug 2, 2022
The issue reported by OSS-Fuzz produces the following backtrace:
==447470==ERROR: AddressSanitizer: heap-buffer-overflow
READ of size 1 at 0x61500002a080 thread T0
#0 0x71766d47 in sdhci_read_dataport hw/sd/sdhci.c:474:18
#1 0x7175f139 in sdhci_read hw/sd/sdhci.c:1022:19
#2 0x721b937b in memory_region_read_accessor softmmu/memory.c:440:11
#3 0x72171e51 in access_with_adjusted_size softmmu/memory.c:554:18
#4 0x7216f47c in memory_region_dispatch_read1 softmmu/memory.c:1424:16
#5 0x7216ebb9 in memory_region_dispatch_read softmmu/memory.c:1452:9
#6 0x7212db5d in flatview_read_continue softmmu/physmem.c:2879:23
#7 0x7212f958 in flatview_read softmmu/physmem.c:2921:12
#8 0x7212f418 in address_space_read_full softmmu/physmem.c:2934:18
#9 0x721305a9 in address_space_rw softmmu/physmem.c:2962:16
#10 0x7175a392 in dma_memory_rw_relaxed include/sysemu/dma.h:89:12
#11 0x7175a0ea in dma_memory_rw include/sysemu/dma.h:132:12
#12 0x71759684 in dma_memory_read include/sysemu/dma.h:152:12
#13 0x7175518c in sdhci_do_adma hw/sd/sdhci.c:823:27
#14 0x7174bf69 in sdhci_data_transfer hw/sd/sdhci.c:935:13
#15 0x7176aaa7 in sdhci_send_command hw/sd/sdhci.c:376:9
#16 0x717629ee in sdhci_write hw/sd/sdhci.c:1212:9
#17 0x72172513 in memory_region_write_accessor softmmu/memory.c:492:5
#18 0x72171e51 in access_with_adjusted_size softmmu/memory.c:554:18
#19 0x72170766 in memory_region_dispatch_write softmmu/memory.c:1504:16
#20 0x721419ee in flatview_write_continue softmmu/physmem.c:2812:23
#21 0x721301eb in flatview_write softmmu/physmem.c:2854:12
#22 0x7212fca8 in address_space_write softmmu/physmem.c:2950:18
#23 0x721d9a53 in qtest_process_command softmmu/qtest.c:727:9
A DMA descriptor is previously filled in RAM. An I/O access to the
device (frames #22 to #16) start the DMA engine (frame #13). The
engine fetch the descriptor and execute the request, which itself
accesses the SDHCI I/O registers (frame #1 and #0), triggering a
re-entrancy issue.
Fix by prohibit transactions from the DMA to devices. The DMA engine
is thus restricted to memories.
Reported-by: OSS-Fuzz (Issue 36391)
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/451
Message-Id: <20211215205656.488940-3-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
igrr
pushed a commit
that referenced
this issue
Aug 2, 2022
Include the qtest reproducer provided by Alexander Bulekov in https://gitlab.com/qemu-project/qemu/-/issues/451. Without the previous commit, we get: $ make check-qtest-i386 ... Running test qtest-i386/fuzz-sdcard-test ==447470==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61500002a080 at pc 0x564c71766d48 bp 0x7ffc126c62b0 sp 0x7ffc126c62a8 READ of size 1 at 0x61500002a080 thread T0 #0 0x564c71766d47 in sdhci_read_dataport hw/sd/sdhci.c:474:18 #1 0x564c7175f139 in sdhci_read hw/sd/sdhci.c:1022:19 #2 0x564c721b937b in memory_region_read_accessor softmmu/memory.c:440:11 #3 0x564c72171e51 in access_with_adjusted_size softmmu/memory.c:554:18 #4 0x564c7216f47c in memory_region_dispatch_read1 softmmu/memory.c:1424:16 #5 0x564c7216ebb9 in memory_region_dispatch_read softmmu/memory.c:1452:9 #6 0x564c7212db5d in flatview_read_continue softmmu/physmem.c:2879:23 #7 0x564c7212f958 in flatview_read softmmu/physmem.c:2921:12 #8 0x564c7212f418 in address_space_read_full softmmu/physmem.c:2934:18 #9 0x564c721305a9 in address_space_rw softmmu/physmem.c:2962:16 #10 0x564c7175a392 in dma_memory_rw_relaxed include/sysemu/dma.h:89:12 #11 0x564c7175a0ea in dma_memory_rw include/sysemu/dma.h:132:12 #12 0x564c71759684 in dma_memory_read include/sysemu/dma.h:152:12 #13 0x564c7175518c in sdhci_do_adma hw/sd/sdhci.c:823:27 #14 0x564c7174bf69 in sdhci_data_transfer hw/sd/sdhci.c:935:13 #15 0x564c7176aaa7 in sdhci_send_command hw/sd/sdhci.c:376:9 #16 0x564c717629ee in sdhci_write hw/sd/sdhci.c:1212:9 #17 0x564c72172513 in memory_region_write_accessor softmmu/memory.c:492:5 #18 0x564c72171e51 in access_with_adjusted_size softmmu/memory.c:554:18 #19 0x564c72170766 in memory_region_dispatch_write softmmu/memory.c:1504:16 #20 0x564c721419ee in flatview_write_continue softmmu/physmem.c:2812:23 #21 0x564c721301eb in flatview_write softmmu/physmem.c:2854:12 #22 0x564c7212fca8 in address_space_write softmmu/physmem.c:2950:18 #23 0x564c721d9a53 in qtest_process_command softmmu/qtest.c:727:9 0x61500002a080 is located 0 bytes to the right of 512-byte region [0x615000029e80,0x61500002a080) allocated by thread T0 here: #0 0x564c708e1737 in __interceptor_calloc (qemu-system-i386+0x1e6a737) #1 0x7ff05567b5e0 in g_malloc0 (/lib64/libglib-2.0.so.0+0x5a5e0) #2 0x564c71774adb in sdhci_pci_realize hw/sd/sdhci-pci.c:36:5 SUMMARY: AddressSanitizer: heap-buffer-overflow hw/sd/sdhci.c:474:18 in sdhci_read_dataport Shadow bytes around the buggy address: 0x0c2a7fffd3c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c2a7fffd3d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c2a7fffd3e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c2a7fffd3f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c2a7fffd400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x0c2a7fffd410:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c2a7fffd420: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c2a7fffd430: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c2a7fffd440: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c2a7fffd450: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c2a7fffd460: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Heap left redzone: fa Freed heap region: fd ==447470==ABORTING Broken pipe ERROR qtest-i386/fuzz-sdcard-test - too few tests run (expected 3, got 2) Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211215205656.488940-4-philmd@redhat.com> [thuth: Replaced "-m 4G" with "-m 512M"] Signed-off-by: Thomas Huth <thuth@redhat.com>
|
I'll close this issue since IDF v3.2 is now EoL. |
github-actions
bot
changed the title
espressif-bot
added
Status: In Progress
mattiantonini
pushed a commit
to mattiantonini/ESP32-QEMU
that referenced
this issue
Feb 23, 2023
Fixes this tsan crash, easy to reproduce with any large enough program:
$ tests/unit/test-qht
1..2
ThreadSanitizer: CHECK failed: sanitizer_deadlock_detector.h:67 "((n_all_locks_)) < (((sizeof(all_locks_with_contexts_)/sizeof((all_locks_with_contexts_)[0]))))" (0x40, 0x40) (tid=1821568)
#0 __tsan::CheckUnwind() ../../../../src/libsanitizer/tsan/tsan_rtl.cpp:353 (libtsan.so.2+0x90034)
#1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) ../../../../src/libsanitizer/sanitizer_common/sanitizer_termination.cpp:86 (libtsan.so.2+0xca555)
espressif#2 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::addLock(unsigned long, unsigned long, unsigned int) ../../../../src/libsanitizer/sanitizer_common/sanitizer_deadlock_detector.h:67 (libtsan.so.2+0xb3616)
espressif#3 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::addLock(unsigned long, unsigned long, unsigned int) ../../../../src/libsanitizer/sanitizer_common/sanitizer_deadlock_detector.h:59 (libtsan.so.2+0xb3616)
espressif#4 __sanitizer::DeadlockDetector<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::onLockAfter(__sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >*, unsigned long, unsigned int) ../../../../src/libsanitizer/sanitizer_common/sanitizer_deadlock_detector.h:216 (libtsan.so.2+0xb3616)
espressif#5 __sanitizer::DD::MutexAfterLock(__sanitizer::DDCallback*, __sanitizer::DDMutex*, bool, bool) ../../../../src/libsanitizer/sanitizer_common/sanitizer_deadlock_detector1.cpp:169 (libtsan.so.2+0xb3616)
espressif#6 __tsan::MutexPostLock(__tsan::ThreadState*, unsigned long, unsigned long, unsigned int, int) ../../../../src/libsanitizer/tsan/tsan_rtl_mutex.cpp:200 (libtsan.so.2+0xa3382)
espressif#7 __tsan_mutex_post_lock ../../../../src/libsanitizer/tsan/tsan_interface_ann.cpp:384 (libtsan.so.2+0x76bc3)
espressif#8 qemu_spin_lock /home/cota/src/qemu/include/qemu/thread.h:259 (test-qht+0x44a97)
espressif#9 qht_map_lock_buckets ../util/qht.c:253 (test-qht+0x44a97)
espressif#10 do_qht_iter ../util/qht.c:809 (test-qht+0x45f33)
espressif#11 qht_iter ../util/qht.c:821 (test-qht+0x45f33)
espressif#12 iter_check ../tests/unit/test-qht.c:121 (test-qht+0xe473)
espressif#13 qht_do_test ../tests/unit/test-qht.c:202 (test-qht+0xe473)
espressif#14 qht_test ../tests/unit/test-qht.c:240 (test-qht+0xe7c1)
espressif#15 test_default ../tests/unit/test-qht.c:246 (test-qht+0xe828)
espressif#16 <null> <null> (libglib-2.0.so.0+0x7daed)
espressif#17 <null> <null> (libglib-2.0.so.0+0x7d80a)
espressif#18 <null> <null> (libglib-2.0.so.0+0x7d80a)
espressif#19 g_test_run_suite <null> (libglib-2.0.so.0+0x7dfe9)
espressif#20 g_test_run <null> (libglib-2.0.so.0+0x7e055)
espressif#21 main ../tests/unit/test-qht.c:259 (test-qht+0xd2c6)
espressif#22 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 (libc.so.6+0x29d8f)
espressif#23 __libc_start_main_impl ../csu/libc-start.c:392 (libc.so.6+0x29e3f)
espressif#24 _start <null> (test-qht+0xdb44)
Signed-off-by: Emilio Cota <cota@braap.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230111151628.320011-5-cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230124180127.1881110-30-alex.bennee@linaro.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Many popular libraries are built on version 1.0.4 of the espressif/arduino-esp32 layer. This layer compiles against esp-idf 3.2, which does not include open ethernet. Unfortunately, this version of esp-idf does not include the open-eth component, so emulation of applications that are built on it can't do networking.
Can the open-eth component be included on the application side? This will allow applications built on esp-idf 3.2 to use the network features of the emulator.
In the newer versions of esp-idf that include open-eth, open-eth refers to some of the other components, so it is unclear if it can be cleanly separated from the esp-idf it is embedded in for use as a standalone component.
The text was updated successfully, but these errors were encountered: