Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use class method for processors #17

Merged
merged 1 commit into from
Oct 25, 2023
Merged

Conversation

magland
Copy link
Collaborator

@magland magland commented Oct 25, 2023

@luiztauffer

Here I have implemented the new class method for protocaas processors.

You can see the examples here:
https://github.com/scratchrealm/pc-spike-sorting/blob/main/mountainsort5/main.py
https://github.com/scratchrealm/pc-spike-sorting/blob/main/kilosort3/main.py
https://github.com/scratchrealm/pc-spike-sorting/blob/main/kilosort2_5/main.py
https://github.com/scratchrealm/pc-spike-sorting/blob/main/dandi_upload/main.py
https://github.com/scratchrealm/pc-spike-sorting/blob/main/spike_sorting_utils/main.py

The tricky part was getting the meta information about the processor context from the type hints, etc. That is done here:

https://github.com/scratchrealm/protocaas/blob/1a5134ebbc1e8673c2de44a3b32740e28ddbb024/python/protocaas/sdk/AppProcessor.py#L169-L268
https://github.com/scratchrealm/protocaas/blob/1a5134ebbc1e8673c2de44a3b32740e28ddbb024/python/protocaas/sdk/_get_context_attributes_for_dataclass.py#L1-L32

Unfortunately, inspecting of type hints did not work with your method of dynamically importing the main.py module, so I had to revert to calling main.py with the SPEC_OUTPUT_FILE env var. However, the CLI works the same as you had it, for example

protocaas make-app-spec-file --app-dir kilosort3 --spec-output-file kilosort3/spec.json

(also, sprinkled into this PR, are some improvements/simplifications to the mock system for testing)

@codecov
Copy link

codecov bot commented Oct 25, 2023

Codecov Report

Attention: 110 lines in your changes are missing coverage. Please review.

Files Coverage Δ
...n/protocaas/api_helpers/clients/MockMongoClient.py 84.52% <100.00%> (+0.37%) ⬆️
...protocaas/api_helpers/clients/_get_mongo_client.py 81.48% <100.00%> (+4.20%) ⬆️
python/protocaas/api_helpers/clients/pubsub.py 43.47% <100.00%> (-6.53%) ⬇️
...aas/api_helpers/routers/compute_resource/router.py 76.13% <100.00%> (-0.54%) ⬇️
python/protocaas/mock.py 100.00% <100.00%> (ø)
python/protocaas/sdk/__init__.py 100.00% <100.00%> (ø)
...api_helpers/routers/gui/compute_resource_routes.py 46.71% <50.00%> (-0.70%) ⬇️
python/protocaas/sdk/ProcessorBase.py 68.42% <68.42%> (ø)
python/protocaas/sdk/_make_spec_file.py 0.00% <0.00%> (ø)
...ocaas/sdk/_get_context_attributes_for_dataclass.py 45.45% <45.45%> (ø)
... and 2 more

📢 Thoughts on this report? Let us know!.

@magland magland merged commit 1a5134e into main Oct 25, 2023
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant