{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":602233296,"defaultBranch":"main","name":"neuropixel-pipeline","ownerLogin":"atlab","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-02-15T19:22:06.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/2198969?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1676488933.438526","currentOid":""},"activityList":{"items":[{"before":"5bc9aec28e9d61ff0309681eb9914c25d406dccc","after":"1bdeb638ca6bb82064fb46167d24d9e123831fb1","ref":"refs/heads/main","pushedAt":"2023-06-27T17:33:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Cpapa97","name":"Christos Papadopoulos","path":"/Cpapa97","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26700068?s=80&v=4"},"commit":{"message":"Ecephys spike sorting (#2)\n\n* fix labview metadata default config data\r\n\r\n* wip\r\n\r\n* important note\r\n\r\n* add session search for atlab\r\n\r\n* reorganize atlab helper to allow it to be run as a module\r\n\r\n* remove old atlab file (using the directory module now)\r\n\r\n* more reorganization and progress for the atlab module\r\n\r\n* more progress on a one-run pipeline\r\n\r\n* add minion cron config and rename helpers folder to modules\r\n\r\nminion not fully supported by the code yet, need to add a populate check for --mode=minion\r\n\r\n* finish renaming helpers to modules in dependant code\r\n\r\n* use task_mode arg\r\n\r\n* formatting\r\n\r\n* linting fixes\r\n\r\n* change name of default clustering method constant\r\n\r\n* rework clustering task runner to only handle triggering\r\n\r\n* add simple logging to stdout (timestamps)\r\n\r\n* round elapsed time to be readable\r\n\r\n* add failed/successful job history limits\r\n\r\n* formatting\r\n\r\n* add pipeline mode enum\r\n\r\n* populate postclustering\r\n\r\n* add curation ability assuming no curation as default\r\n\r\n* enable no curation mode\r\n\r\n* try adding auto_increment to curation\r\n\r\n* fix curation input\r\n\r\n* move atlab main pipeline to AtlabParams\r\n\r\n* formatting\r\n\r\n* a few changes from meeting\r\n\r\n* raw LFP data shouldn't be saved in the database\r\n\r\n* curation task table\r\n\r\n* fix class path\r\n\r\n* fix class path\r\n\r\n* start using pipeline mode\r\n\r\n* validate_call is hanging up on ScanKey when used directly\r\n\r\n* remove validate_call decorations until it's more stable\r\n\r\n* pass probe calibration suffix through\r\n\r\n* fix incorrect type\r\n\r\n* switch atlab params to a BaseModel\r\n\r\n* coerce atlab params to model\r\n\r\n* remove some references to ScanKey that would be overly restrictive or wouldn't coerce without validate_call\r\n\r\n* disable LFP populate for now\r\n\r\n* fix paramset restriction\r\n\r\n* fix task source key construction\r\n\r\n* formatting\r\n\r\n* enum to str\r\n\r\n* fix faulty conditional\r\n\r\n* remove auto_increment from Session so auto_increment can be used later in the pipeline\r\n\r\n* remove auto_increment from Curation\r\n\r\n* formatting\r\n\r\n* fix custom auto increments\r\n\r\n* remove old api classes\r\n\r\n* abstract filepaths to ensure more generality\r\n\r\n* remove pydantic schema api for now\r\n\r\n* remove pydantic schema api for now\r\n\r\n* move import\r\n\r\n* fix restriction type\r\n\r\n* fix property arguments\r\n\r\n* fix restriction\r\n\r\n* fix curation path\r\n\r\n* fix kilosort class path\r\n\r\n* coerce path for Kilosort reader\r\n\r\n* formatting\r\n\r\n* actually populate curated clusterting\r\n\r\n* make atlab populate notebook\r\n\r\n* correctly fetch kilosort run parameters\r\n\r\n* use correct argument name\r\n\r\n* switch back to correct name\r\n\r\n* correct invocation of KilosortRunner\r\n\r\n* yet again fix argument name\r\n\r\n* remove package version\r\n\r\n* add option to check for existing clustering results before re-running\r\n\r\n* print message for check_for_existing_results\r\n\r\n* specify the arguments that go into ecephys_spike_sorting analysis\r\n\r\n* formatting\r\n\r\n* remove old import\r\n\r\n* fix populate for QualityMetrics\r\n\r\n* improve error message\r\n\r\n* get correct session path\r\n\r\n* fix naming conventions\r\n\r\n* account for the sync channel from the session binary file\r\n\r\n* make the same change for quality metrics\r\n\r\n* fix operator precedence\r\n\r\n* correct sync channel masking out\r\n\r\n* fix path search\r\n\r\n* use updated argument returns for ecephys_spike_sorted utils.load_kilosort_data\r\n\r\n* use actual correct return arguments\r\n\r\n* the story on these arguments is more complicated\r\n\r\n* update with new arguments\r\n\r\n* do not include principal components analysis\r\n\r\n* fill out waveformset and qualitymetric parameters\r\n\r\n* set include_pcs to False again\r\n\r\n* change quality metrics to use the main method for constructing metrics.csv\r\n\r\n* remove old parameters that won't be used for QualityMetrics (although might be used for WaveformSet)\r\n\r\n* fix alias typo\r\n\r\n* implement waveformset runner\r\n\r\n* remove file arg\r\n\r\n* fix instance variable path\r\n\r\n* fix typo\r\n\r\n* try original ecephys_spike_sorting instead of the fork (issue with change in function datatype without change in signature)\r\n\r\n* use AllenInstitute ecephys_spike_sorting instead (still need to change WaveformSetRunner)\r\n\r\n* change arg name\r\n\r\n* fix default for pc_metrics\r\n\r\n* fix dictionary assignment\r\n\r\n* additional argument\r\n\r\n* fully switch to AllenInstitute ecephys_spike_sorting\r\n\r\n* cleanup and formatting","shortMessageHtmlLink":"Ecephys spike sorting (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1777437476\" data-permission-text=\"Title is private\" data-url=\"https://github.com/atlab/neuropixel-pipeline/issues/2\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/atlab/neuropixel-pipeline/pull/2/hovercard\" href=\"https://github.com/atlab/neuropixel-pipeline/pull/2\">#2</a>)"}},{"before":"0e2372f75096a7db14ddd6d09998521998d7165d","after":"5bc9aec28e9d61ff0309681eb9914c25d406dccc","ref":"refs/heads/main","pushedAt":"2023-06-19T15:38:12.512Z","pushType":"push","commitsCount":1,"pusher":{"login":"Cpapa97","name":"Christos Papadopoulos","path":"/Cpapa97","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26700068?s=80&v=4"},"commit":{"message":"Update notebook-gpu.yaml","shortMessageHtmlLink":"Update notebook-gpu.yaml"}},{"before":"f9dfb97d0fb778edbc7e391e53094373d2a4774a","after":"0e2372f75096a7db14ddd6d09998521998d7165d","ref":"refs/heads/main","pushedAt":"2023-06-19T15:37:47.991Z","pushType":"push","commitsCount":1,"pusher":{"login":"Cpapa97","name":"Christos Papadopoulos","path":"/Cpapa97","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26700068?s=80&v=4"},"commit":{"message":"Update notebook.yaml","shortMessageHtmlLink":"Update notebook.yaml"}},{"before":"dd2e9eb3465a4ab20aa653bf8d54be3b5a70dc68","after":"f9dfb97d0fb778edbc7e391e53094373d2a4774a","ref":"refs/heads/main","pushedAt":"2023-06-19T15:35:01.099Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Cpapa97","name":"Christos Papadopoulos","path":"/Cpapa97","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26700068?s=80&v=4"},"commit":{"message":"Data interface (#1)\n\n* partial wip for different project structure\r\n\r\n* partial update on moving pipeline logic to pydantic types\r\n\r\n* wip\r\n\r\n* black formatting\r\n\r\n* pull neuropixel metadata into struct\r\n\r\n* formatting\r\n\r\n* use pydantic v2 field_validator instead\r\n\r\n* formatter\r\n\r\n* formatter\r\n\r\n* partially upgraded for new api\r\n\r\n* stores config utility and cleanup after testing\r\n\r\n* note on filepath protocol adapters (using pydantic)\r\n\r\n* adapters initial structure\r\n\r\n* rename for clarity\r\n\r\n* condense utils\r\n\r\n* partial\r\n\r\n* recommended extensions\r\n\r\n* rewrite for functionality\r\n\r\n* reorganize populate api\r\n\r\n* temporarily use setup.py due to pip breaking when creating a wheel for pyproject + docker-compose notebook support\r\n\r\n* access, kilosort triggering, quality control\r\n\r\n* fix starting args\r\n\r\n* bring the non-gpu notebook inline with the gpu version\r\n\r\n* formatting\r\n\r\n* test change\r\n\r\n* add package check notebook\r\n\r\n* fix install into conda env\r\n\r\n* change kernel name\r\n\r\n* remove redunant step\r\n\r\n* test with remote jupyter kernel\r\n\r\n* note on probe setup (which may be subject to change)\r\n\r\n* add post clustering\r\n\r\n* fix pyproject wheel building\r\n\r\n* fix dynamic versioning with pyproject\r\n\r\n* add missing dependency\r\n\r\n* testing fix for circular import\r\n\r\n* remove old function\r\n\r\n* revert change\r\n\r\n* temporarily remove\r\n\r\n* Revert \"temporarily remove\"\r\n\r\nThis reverts commit 5aecb315534df341e9cd6891adc716b3ca36383c.\r\n\r\n* update helper notebook\r\n\r\n* for some reason this fixes a circular import in the notebooks\r\n\r\n* update to new kilosort-runner\r\n\r\n* rename params to reduce potential confusion\r\n\r\n* update curation\r\n\r\n* add current timestamp for session key insertion\r\n\r\n* partial wip\r\n\r\n* add probe setup specific for atlab\r\n\r\n* move filepath to secondary key\r\n\r\n* fix recording rig\r\n\r\n* fix\r\n\r\n* reinstate EphysFile as its own table\r\n\r\n* change task runner to fit better for \"load\" mode\r\n\r\n* missing model_validate\r\n\r\n* fix path to class\r\n\r\n* update kilosort runner to latest kilosort-runner api\r\n\r\n* coerce clusterting_output_dir to Path\r\n\r\n* KSLabel more important?\r\n\r\n* fix curation paths\r\n\r\n* fix erroneous path for labview metadata\r\n\r\n* fix key name\r\n\r\n* use the correct QualityMetricsRunner\r\n\r\n* fix ecephys\r\n\r\n* swap imports\r\n\r\n* formatting\r\n\r\n* add additional data\r\n\r\n* updated populate test\r\n\r\n* formatting","shortMessageHtmlLink":"Data interface (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1763778997\" data-permission-text=\"Title is private\" data-url=\"https://github.com/atlab/neuropixel-pipeline/issues/1\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/atlab/neuropixel-pipeline/pull/1/hovercard\" href=\"https://github.com/atlab/neuropixel-pipeline/pull/1\">#1</a>)"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADSiRaoAA","startCursor":null,"endCursor":null}},"title":"Activity · atlab/neuropixel-pipeline"}