feat: Implemented GlobalThreadPool for async functions

[mkopcins]

Description

Added singleton class GlobalThreadPool for single threadpool management so that we don't have to spawn new threads for each async function and instead we can delegate functions to the threadpool.

Also added pthreadpool and cpuinfo binaries for iOS to allow for XNNPack threadpool configuration just like on Android

Introduces a breaking change?

• Yes
• No

Type of change

• Bug fix (change which fixes an issue)
• New feature (change which adds functionality)
• Documentation update (improves or adds clarity to existing documentation)
• Other (chores, tests, code style improvements etc.)

Tested on

• iOS
• Android

Testing instructions

Screenshots

Related issues

Checklist

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation accordingly
• My changes generate no new warnings

Additional notes

[msluszniak]

This is completely up to you, but I would probably define constructor for this struct, it's not classic POD because you have a non-copyable std::unique_ptr here. What are the benefits? More concise initialization, priority and enqueueTime might be marked private.

[jakmro]

Consider removing this class and leaving all functions inside the threads::global_thread_pool namespace

[msluszniak]

HighPerformanceThreadPool should be probably implemented similarly to this one:

class Singleton {
public:
    Singleton(const Singleton&) = delete;
    Singleton& operator=(const Singleton&) = delete;
    Singleton(Singleton&&) = delete;
    Singleton& operator=(Singleton&&) = delete;

    static Singleton& getInstance() {
        static std::once_flag initFlag;
        if (!instance) { 
            throw std::logic_error("Instance not initialized.");
        }
        return *instance;
    }

    static void initialize(int arg) {
        static std::once_flag initFlag;
        std::call_once(initFlag, [arg]() {
            instance = std::make_unique<Singleton>(arg);
        });
    }

private:
    int value;
    Singleton(int val) : value(val) {}

    static std::unique_ptr<Singleton> instance;
};

std::unique_ptr<Singleton> Singleton::instance = nullptr;

[mkopcins]

I think you meant GlobalThreadPool, right? HighPerformanceThreadPool was never supposed to be a singleton

[msluszniak]

Currently, HighPerformanceThreadPool is used only in the context of GlobalThreadPool. I guess Jakub meant that GlobalThreadPool is only a wrapper on a class that might be a singleton itself. If in the future there might be any other way to use HighPerformanceThreadPool the current design will make more sense to me then.

[mkopcins]

That was my idea when creating this, HighPerformanceThreadPool is not meant to be a singleton, but GlobalThreadPool is. Also semantically it makes more sense to me to keep GlobalThreadPool as a class instead of a namespace even though it doesn't have any nonstatic members.

[msluszniak]

Add conditional closing of this issue: #565 in the PR

[msluszniak]

Very small NITs

[msluszniak]

🚀

[msluszniak]

You should wrap the whole function implementation with #ifdef __ANDROID__ to not introduce overhead of translating empty function body on iOS. Check other applicable places

[msluszniak]

Some functions like here or getCurrentThreadId might be marked static or moved outside of the class

[chmjkb]

besides the single comment lgtm

[chmjkb]

Shouldn't this be third-party/include/executorch/...? If this is the case then it is already included in the line above