Add Asynchronous Inference option

[LinusJungemann]

No description provided.

[Copilot]

Pull Request Overview

This PR introduces support for asynchronous inference by adding dynamic queue implementations and callback/drain mechanisms, and updates tests and benchmarks accordingly.

• Added DynamicSPSCQueue and replaced static queue usage in asynchronous buffer wrappers.
• Extended DeviceHandler, Accelerator, and BaseDriver APIs with registerCallback and drain methods.
• Updated integration tests and benchmarks to remove fixed sleeps and rely on blocking or callback-driven result retrieval.

Reviewed Changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.

File	Description
unittests/utils/SPSCQueueTest.cpp	Replaced range-insert with manual push_back loop
src/FINNCppDriver/utils/FPGAReset.hpp	Added unit-test bypass for reset and included chrono literals in header
src/FINNCppDriver/core/BaseDriver.hpp	Exposed registerCallback and drain APIs for async inference
integrationtest/AsyncInference.cpp	Removed sleeps and switched to blocking getResults()

Comments suppressed due to low confidence (1)

unittests/utils/SPSCQueueTest.cpp:514

• [nitpick] Instead of manually looping and calling push_back, you can use all_consumed.insert(all_consumed.end(), results.begin(), results.begin() + dequeued) for clearer intent and potentially better performance.

                for (size_t i = 0; i < dequeued; i++) {

[Copilot]

[nitpick] Avoid using namespace directives in headers as they pollute the global namespace for all translation units. Consider qualifying chrono literals where needed or moving the using directive into the implementation file.

Suggested change

	using namespace std::chrono_literals;
	// Removed `using namespace std::chrono_literals;` to avoid namespace pollution.

[Copilot]

[nitpick] This use of std::enable_if does not actually disable the template when the condition is false. To constrain these methods properly, use std::enable_if_t or C++20 concepts (requires) so the overload is SFINAE-friendly.

Suggested change

	template<typename = std::enable_if<!SynchronousInference>>
	template<typename = std::enable_if_t<!SynchronousInference>>

[Copilot]

The destructor no longer calls queue.shutdown(), so the worker thread may attempt to enqueue after it's signaled to stop, causing potential data races or hangs. Ensure the queue is shut down before thread teardown (e.g., call queue.shutdown() before logging).

Suggested change

	~AsyncDeviceInputBuffer() override { FINN_LOG(loglevel::info) << "Destructing Asynchronous input buffer" << std::endl; };
	~AsyncDeviceInputBuffer() override {
	prepareForShutdown();
	FINN_LOG(loglevel::info) << "Destructing Asynchronous input buffer" << std::endl;
	};

[Copilot]

Removing the sleep assumes getResults() will block until data is ready, but if the async pipeline hasn't produced anything yet, this may hang or fail. Add an explicit wait or callback notification to ensure data availability before calling getResults().