Add ftl::Fibtex #66

cwfitzgerald · 2018-09-24T17:12:31Z

Here's the implementation of a mutex for fibers. Closes #64.

RichieSams · 2018-09-24T17:14:18Z

include/ftl/atomic_counter.h

+	/**
+	 * A wrapper over std::atomic_uint::compare_exchange_strong()
+	 *
+	 * The compare_exchange_strong *with* be atomic, but this function as a while is *not* atomic


Pedantic, but with should be will

Also while should be whole

Haha good catch, my brain isn't in gear today 😄

RichieSams · 2018-09-24T17:16:16Z

include/ftl/atomic_counter.h

+	 *
+	 * \param expectedValue The value that is expected to be in the atomic counter
+	 * \param newValue      The value that the atomic counter will be set to if comparison succeeds.
+	 * \param memoryOrder   The memory order to use for the compare_exchange_strong


Small thing, but if you're in there fixing the above, can you have 4 spaces after the longest parameter value. Also, can you updated it to @param instead of \param. So the style matches the other comments

👌 That's what my editor generates (as I use that style) but I obviously didn't catch them all.

RichieSams · 2018-09-24T17:18:34Z

include/ftl/fibtex.h

+	/**
+	 * All Fibtex's have to be aware of the task scheduler in order to yield.
+	 *
+	 * @param taskScheduler ftl::TaskScheduler that will be using this mutex.


Same as above, can it be:

* @param taskScheduler ftl::TaskScheduler that will be using this mutex.

cwfitzgerald · 2018-09-24T17:22:33Z

Once you're done with the code review, I'll commit all the changes.

RichieSams · 2018-09-24T17:28:53Z

include/ftl/fibtex.h

+
+namespace ftl {
+/**
+ * A fiber aware mutex. Does not block in the traditional way.


Can you add a note here that the member function names are named to so they can be used with C++ standard stuff?

RichieSams · 2018-09-24T17:47:59Z

Overall, the code changes are great. Just small comments / documentation stuff for fixes. I really like the FTL_PAUSE additions to the rest of the code.

RichieSams · 2018-09-24T17:49:53Z

include/ftl/fibtex.h

+	}
+
+	/**
+	 * Lock mutex using an finite spinlock. Does not spin if there is only one backing thread.


using a finite

martty · 2018-09-24T18:55:07Z

include/ftl/config.h

@@ -53,6 +53,13 @@
 	#endif
 #endif

+#ifdef __SSE2__


https://msdn.microsoft.com/en-us/library/b0084kay.aspx

MSVC does not set this macro, even when compiling with SSE2 instructions.

Oof. I'm basically detecting for <emmintrin.h> and _mm_pause(). I'll see if there's a better way of doing this.

include/ftl/fibtex.h

martty · 2018-09-24T19:14:46Z

Hey, looks great so far! Fibtex is missing thread pinning support, which is unfortunately something that is needed for adoption.

RichieSams · 2018-09-24T19:24:27Z

include/ftl/fibtex.h

+	 */
+	void unlock() {
+		if (!m_atomicCounter.CompareExchange(1, 0)) {
+			printf("Mutex was unlocked by another fiber or was double unlocked.\n");


Thinking about this again, this should probably be more shouty. At least an assert(), and maaaaybe an exception. C++ standard doesn't have any definition for double unlocking. It just says it's undefined behavior.

I would take assert. There is a big programming error like that it really should die on debug mode. I'll add that.

cwfitzgerald · 2018-09-24T19:37:08Z

@martty Thanks! How would you expect this to be implemented? Should the mutex always resume on the same thread, or should it just have an option with the lock functions to be able to do that (defaulted parameter).

include/ftl/fibtex.h

cwfitzgerald · 2018-09-24T19:42:29Z

I'm also going to add some RAII helpers for things std::lock_guard can't deal with.

martty · 2018-09-24T19:44:15Z

@martty Thanks! How would you expect this to be implemented? Should the mutex always resume on the same thread, or should it just have an option with the lock functions to be able to do that (defaulted parameter).

Following the API of WaitForCounter, I think a defaulted ctor argument would be fine (default = unpinned).

martty · 2018-09-24T19:50:22Z

Can you explain why you chose to implement three lock() functions?

cwfitzgerald · 2018-09-24T19:53:52Z

At the bare minimum I wanted to implement a yielding lock and a spin lock. I split up the two spin lock functions because the behavior of .spin_lock(0) is ambiguous (could mean block immediately, could mean never block), and it eliminated extra checks inside the spin_lock function.

martty · 2018-09-24T20:27:59Z

Having multiple lock*() functions makes it more difficult to use, since only lock() is integrated with STL (calling these by hand is sure way to foot shooting IMO :) ). What do you think of specifying the spinning mechanism in FibTex?

cwfitzgerald · 2018-09-24T20:32:05Z

@martty This is why I am currently working on specific lock guards for use with these functions. I'll push them momentarily for comment.

cwfitzgerald · 2018-09-24T20:43:13Z

I want to avoid making the mutex itself spin/yield specific, as you might want to have it spin in some situations but yield in others.

martty · 2018-09-24T20:45:06Z

Sure, I meant specifying the spinning mechanism with a constructor parameter or a template parameter.

cwfitzgerald · 2018-09-24T20:48:58Z

That's what I mean, sorry if I wasn't clear. I don't want to force a particular instance of a lock to be bound to a locking type because you may need to use the same lock in different ways depending on the situation. If fiber 0 is just updating a flag guarded by the lock, but fiber 1 is updating the whole data structure, Fiber 0 will want to do a blocking lock, and fiber 1 will want to do a spin lock.

cwfitzgerald · 2018-09-24T21:14:19Z

I have implemented lock guards and I'll implement unqiue locks in a bit, but I need to go home, so I put these up for review in the mean time 😄

RichieSams · 2018-09-24T21:16:26Z

Nice job!

cwfitzgerald · 2018-09-24T23:45:16Z

Alright, I've added a basic Unique Lock impl. I have followed the standard (i.e. cppreference) to the letter with the exception of the swap function being a friend function to replace the the specialization of std::swap (which isn't possible). If you think we should deviate from the standard at any point lmk. I'll get on documenting and building the 2 other unique locks.

martty · 2018-09-25T21:10:39Z

Can you make FibTex noncopyable and nonmovable?

ChemistAion

Make these: GetThreadCount() and GetFiberCount() as const

cwfitzgerald · 2018-09-27T00:46:22Z

@ChemiaAion I'll do a pass over all of the members and make them as restrictive as possible.

cwfitzgerald · 2018-09-27T01:04:46Z

Still need to do a round of commenting and such, but the functionally should all be there at this point unless someone thinks of something else that should be there.

cwfitzgerald · 2018-10-22T15:57:36Z

This PR should be ready to go in terms of functionality. Github is still a little hiccup-ey and we still have that random segfault bug in the tests. I might also add some tests to make sure it clears all the compilers.

The rationale behind changing the names of the spin locks is that having an infinite spin is almost never the right thing to do. This change makes the right-er thing a shorter name, and makes it clear which will put you in a spin lock infinitely.

cwfitzgerald · 2018-12-16T22:11:26Z

The biggest thing that seems to be blocking ftl::Fibtex from merging is this race condition that is described in #75.

cwfitzgerald · 2019-07-29T14:39:43Z

I am going to close this to prevent it from cluttering things up because I no longer have the time to work on it, and I have basically abandoned c++ for my projects. Keep up the good work y'all.

RichieSams reviewed Sep 24, 2018

View reviewed changes

martty reviewed Sep 24, 2018

View reviewed changes

RichieSams reviewed Sep 24, 2018

View reviewed changes

include/ftl/fibtex.h Outdated Show resolved Hide resolved

RichieSams reviewed Sep 24, 2018

View reviewed changes

cwfitzgerald commented Sep 24, 2018

View reviewed changes

include/ftl/fibtex.h Show resolved Hide resolved

ChemistAion suggested changes Sep 26, 2018

View reviewed changes

cwfitzgerald added 23 commits December 16, 2018 15:29

Fix SSE detection

2be7a88

Fix typos and errors in documentation

571700a

Add missing #pragma once

53a539d

Use assert on double unlock

9e48899

Fix additional documentation typo

195c6c2

Fix typo in documentation.

bcf130b

Add lock guards for ftl::Fibtex.

d0d0faf

Add basic ftl::UniqueLock for comment

541df62

Make ftl::Fibtex non-copyable and non-movable

88a8029

Move Fibtex member functions in order of preference.

e69921e

Allow all types of locking to ftl::Fibtex

d9f99b4

Add ftl::ScopedLock and ftl::lock.

73bf321

Fix memory order

3ce6689

Fix unintuitive initialization order.

1edece4

Finish documentation of ftl::Fibtex

e6eef95

[ci skip] Beginnings of fibtex tests

22c1442

[ci skip] Remove unnecessary code and fix a couple bugs.

2111bf0

Fix store/fetchadd within AddTask

34dc6f1

Fix test issues

78b69c4

Fix fibtex documentation typo

baa9728

[ci skip] Temporary tests

2c200c5

Add generic Werror cmake config

67653c6

cwfitzgerald force-pushed the fibtex branch from 594d05c to 67653c6 Compare December 16, 2018 20:31

cwfitzgerald added 2 commits December 16, 2018 16:20

Run static-analysis over ftl::Fibtex

c6a8d1f

Add cmake support for NUM_WAITING_FIBER_SLOTS

623b8b4

cwfitzgerald closed this Jul 29, 2019

RichieSams mentioned this pull request Feb 17, 2020

Add ftl::fibtex #95

Merged

Add ftl::Fibtex #66

Add ftl::Fibtex #66

Conversation

cwfitzgerald commented Sep 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RichieSams Sep 24, 2018 • edited

Choose a reason for hiding this comment

cwfitzgerald commented Sep 24, 2018

RichieSams Sep 24, 2018 • edited

Choose a reason for hiding this comment

RichieSams commented Sep 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martty commented Sep 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cwfitzgerald commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

martty commented Sep 24, 2018

martty commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

martty commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

martty commented Sep 24, 2018 • edited

cwfitzgerald commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

RichieSams commented Sep 24, 2018

cwfitzgerald commented Sep 24, 2018

martty commented Sep 25, 2018

ChemistAion left a comment • edited

Choose a reason for hiding this comment

cwfitzgerald commented Sep 27, 2018

cwfitzgerald commented Sep 27, 2018

cwfitzgerald commented Oct 22, 2018

cwfitzgerald commented Dec 16, 2018

cwfitzgerald commented Jul 29, 2019

RichieSams Sep 24, 2018 •

edited

RichieSams Sep 24, 2018 •

edited

martty commented Sep 24, 2018 •

edited

ChemistAion left a comment •

edited