Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release candidate #244

Merged
merged 45 commits into from
Mar 11, 2024
Merged

release candidate #244

merged 45 commits into from
Mar 11, 2024

Conversation

hassanctech
Copy link
Contributor

@hassanctech hassanctech commented Feb 8, 2024

Issue #, if available:

What was changed?

  • Merging release items into master branch

Why was it changed?

  • To update master with the release related changes

How was it changed?

  • Threadpool
  • Enhancements to the internal state machinery to avoid potential stack overflows
  • Deadlock fixes
  • Stability enhancements

What testing was done for the changes?

  • Verify CI runs clean

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

daveisfera and others added 30 commits September 8, 2023 14:20
* Modify threadpool teardown to avoid unlocking mutexes from a different thread than locked it.
* Define iterator function, add a Stream member var

* Replace recursive stepState calls with iterator

* Replace all Stream stepState calls with iterator

* Fix existing comment typos

* CLANG

* Rename 'keepIterating' member to be more clear

* Add null pointer checks to iterator

* CLANG

* Stylistic change based on precedent

* Remove redundant pStateMachine null check

* Remove another redundant null pointer check

* Fix null checks
* Add timerqueuekick and checkstatemachinetransition

* Increase sleep at end of threadpool test to give spawned threads time to free their memory
Utils - Remove duplicate error code
This reverts commit f2096b8.
* Add gcc M1 test

* testing

* more

* Update gcc path
Fix definition of LP64 for gcc
* fix 32-bit crash in threadpool and possible seg fault if get item was null

* clang format

* remove null item and keep iterating
* remove unnecessary double compiles, separate out test ci run into a separate section, address compile warning

* address multiple compile warnings

* address more compiler warnings for linux, separate out few more tests from build in ci

* move stuff around

* use consistent flags in pic library and test

* clang format

* only run test if flag FIXUP_ANNEX_B_TRAILING_NALU_ZERO is set, previously we re-built pic with this flag for the tests but the test itself segfaults if it's not set

* put tests behind ifdefs which require flags set

* tweak ci yml

* more ifdefing tests

* more ifdefs for tests

* more changes

* correct path for tsan/ubsan suppressions files

* address warning for linux implicit decl of pthread_getname_np

* define only once

* more ifdefs

* more ifdefs for tests

* ifdef more tests

* more ifdefs

* more ifdefs

* more ifdefs

* more ifdefs

* more ifdefs

* if thread count is going to overflow, return 0

* another if/def

* add build and testing with ALIGNED_MEMORY_MODEL and FIXUP_ANNEX_B_TRAILING_NALU_ZERO to test both paths in CI

* more ifdefs for unalign access

* more ifdefs for ubsan

* ifdef more tests for aligned memory model

* more memory aligned ifdefs for tests

* more if defs in tests

* make sure correct number of bytes for snprintf
* Add state machine tag

* Add tag for client state machine

* Use char with fixed size

* Use default tags

* Random string

* Allow tag setting for state machines

* Add unit tests

* Free state machine

* Use createStateMachineWithTag instead

* Rename

* Address comments
disa6302 and others added 9 commits January 4, 2024 08:35
* enable tsan

* lock access to global count to address tsan violation

* remove globals, use struct to thread

* some changes

* lock to protect data race in case of canceled thread still

* lock around terminate in tests for tsan data race

* free mutex in tests
* Switch to ASSERT_EQ from assert to prevent release build optimization

* Fix assert

* Cleanup log statements
* Protect accessing threadpool with threaddata lock

* Fix potential seg fault from attempting to access queue after threadpool has been destroyed by locking the dataMutex around that queue, and having the destructor pass waiting tasks onto the queue when threads are stuck waiting for on said queue

* Fix typo from merge

* Reduce sleeps in threadpool

* Lock test utils mutex, and check for termination in actors before assigning pQueue

* need allocators set check

* wait on teardown of test threads

* removing accessing gTerminateCount in the for loop

* longer sleep to wait for threads to free their heap

* dramatically increase sleep

* Add sleep in the heap check in case it's still waiting for a thread to exit

* Infinite loop if memory isn't clear. Tests will timeout on GHA if the memory leak actually exists. If it's just waiting for threads to clean up, then it will keep waiting.

* Remove sleep from termination task
* Use abort instead

* custom assert

* Fix definition

* Fix fprintf

* Simple assert

* Clang formatted

* Fix build issue
#240)

* safe gmtime, guard with mutex since not thread safe on non windows platforms

* clang format

* add cygwin to keep original gmtime since pthread isn't used there in our thread impl

* address pr comments, fix compile warnings in threadpool

* missed the extern variable name change

* add EXPECT_EQ around thread create and thread join calls in the unit tests, possibly address tsan complain where it it looks like multiple tests executing at the same time
@codecov-commenter
Copy link

codecov-commenter commented Feb 8, 2024

Codecov Report

Attention: Patch coverage is 71.79487% with 66 lines in your changes are missing coverage. Please review.

Project coverage is 80.35%. Comparing base (57637ea) to head (fd7d68c).

❗ Current head fd7d68c differs from pull request most recent head d4768e5. Consider uploading reports for the commit d4768e5 to get more accurate results

Files Patch % Lines
src/utils/src/Threadpool.c 71.42% 28 Missing ⚠️
src/utils/src/TimerQueue.c 0.00% 15 Missing ⚠️
src/client/src/Stream.c 50.00% 4 Missing ⚠️
src/state/src/State.c 90.24% 4 Missing ⚠️
src/utils/src/CustomAssert.c 0.00% 4 Missing ⚠️
src/utils/src/FileIo.c 40.00% 3 Missing ⚠️
src/view/src/ContentView.c 25.00% 3 Missing ⚠️
src/client/src/StreamState.c 87.50% 2 Missing ⚠️
src/client/src/Client.c 0.00% 1 Missing ⚠️
src/heap/src/HybridFileHeap.c 94.11% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #244      +/-   ##
==========================================
+ Coverage   75.18%   80.35%   +5.17%     
==========================================
  Files          52       53       +1     
  Lines       10182    10811     +629     
==========================================
+ Hits         7655     8687    +1032     
+ Misses       2527     2124     -403     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

hassanctech and others added 6 commits February 8, 2024 10:48
using xcode 14.3.1 results in a linker issue with gcc, 15.1 fixes the issue, trying 15.2 since this appears to be the latest available.
ci.yml formatting
Add readFile fix on Windows to release
@hassanctech hassanctech merged commit 65e38da into master Mar 11, 2024
14 checks passed
@hassanctech hassanctech deleted the rc branch March 11, 2024 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants