Crash (UAF) on Teardown after Upload #1391
Description
Describe your environment.
PAL implementation: CPP11
SDK version: EVT-3.4.317.0
Steps to reproduce.
We make use of the SDK for a lightweight telemetry client we build which simply sends all events in one shot. I seem to be hitting issues where events are not uploaded and remain "offline" when looking at the mat_debug logs it looks as if the API is not behaving properly is it appears the SDK does not properly shutdown as i do not see the final MATSDK.PAL|Shut down in the mat-debug logs.
The way we use the SDK is LogEvent in batch and then during our app teardown is call UploadNow and FlushAndTeardown.
Ive posted a sample of our destructor which performs Upload once we've finished all of our LogEvent calls.
TelemetryManager::~TelemetryManager() noexcept
{
status_t status = m_logManager->UploadNow();
if (STATUS_SUCCESS != status)
{
OsConfigLogError(m_log, "Telemetry upload during shutdown failed. status=%d", status);
}
std::this_thread::sleep_for(std::chrono::microseconds(10)); // Minimal sleep required for upload to be triggered properly
m_logManager->FlushAndTeardown();
OsConfigLogInfo(m_log, "Telemetry shutdown complete.");
}Without the sleep there it appears to not teardown properly (my log is missing the final Telemetry shutdown complete) but with the sleep there, it works just fine and the mat-debug logs also show the final MATSDK.PAL|Shut down.
mat-debug.log (with sleep)
2025-11-12T22:55:53.317Z|00008792|W|MATSDK|upload = 819 ms
2025-11-12T22:55:53.317Z|00008792|W|MATSDK|abort = 0 ms
2025-11-12T22:55:53.317Z|00008792|W|MATSDK|stop = 1 ms
2025-11-12T22:55:53.317Z|00008792|W|MATSDK|worker = 0 ms
2025-11-12T22:55:53.317Z|00008792|W|MATSDK|storage = 0 ms
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.LogManager|Telemetry system stopped
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.RecordStats|RecordStats destroyed: 13
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.RecordStats|RecordStats destroyed: 14
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.RecordStats|RecordStats destroyed: 15
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.RecordStats|RecordStats destroyed: 16
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.RecordStats|RecordStats destroyed: 17
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.StorageHandler|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorageHandler.cpp:62
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.StorageHandler|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorageHandler.cpp:62
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.LogManager|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/api/LogManagerImpl.cpp:855
2025-11-12T22:55:53.317Z|00008792|D|EventsSDK.LogManager|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/api/LogManagerImpl.cpp:855
2025-11-12T22:55:53.317Z|00008792|D|MATSDK.PAL|Shutting down...
2025-11-12T22:55:53.317Z|00008792|I|MATSDK.PAL|queue item=0x7ffc4600e8e0
2025-11-12T22:55:53.317Z|00008792|D|MATSDK.PAL|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/pal/WorkerThread.cpp:81
2025-11-12T22:55:53.317Z|00008792|D|MATSDK.PAL|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/pal/WorkerThread.cpp:81
2025-11-12T22:55:53.317Z|00008793|D|MATSDK.PAL|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/pal/WorkerThread.cpp:189
2025-11-12T22:55:53.317Z|00008793|D|MATSDK.PAL|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/pal/WorkerThread.cpp:189
2025-11-12T22:55:53.317Z|00008792|I|MATSDK.PAL|Shut down
(END)
mat-debug.log (no sleep)
2025-11-12T23:08:49.395Z|00010035|I|MATSDK|offline records=340, pending uploads=0
2025-11-12T23:08:49.395Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.395Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.395Z|00010035|D|EventsSDK.Storage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
2025-11-12T23:08:49.395Z|00010035|D|EventsSDK.Storage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
2025-11-12T23:08:49.497Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.499Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.499Z|00010035|D|EventsSDK.Storage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
2025-11-12T23:08:49.499Z|00010035|D|EventsSDK.Storage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
2025-11-12T23:08:49.500Z|00010035|I|MATSDK|offline records=340, pending uploads=0
2025-11-12T23:08:49.500Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.500Z|00010035|D|EventsSDK.MemoryStorage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/MemoryStorage.cpp:467
2025-11-12T23:08:49.500Z|00010035|D|EventsSDK.Storage|LOCKGUARD lockin at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
2025-11-12T23:08:49.500Z|00010035|D|EventsSDK.Storage|LOCKGUARD locked at /workspaces/azure-osconfig/build2/adapters/mc/telemetry-external-prefix/src/telemetry-external-build/lib/1dscpplib-prefix/src/1dscpplib/lib/offline/OfflineStorage_SQLite.cpp:925
(END)
Ive also come across the article How to Not Crash on Shutdown in case of static LogManager instance
which sounds eerily familiar to this issue but even after using the suggestion (Flush, PauseActivity, WaitPause and finally FlushAndTeardown), the behavior does not change.
What is the expected behavior?
I expect, that even without the sleep, the application shuts down properly and mat-debug logs contain the MATSDK.PAL|Shut down trace at the end.
What is the actual behavior?
Without the sleep, it appears to crash the SDK
Additional context.
N/A