Tests for particle diffusion #2209

jlubo · 2023-08-15T09:07:56Z

Python tests are added to ensure the working of the diffusion of arbitrary particles in setups with 1-3 segments (cf. issues #2084 and #2145).

Besides showing that the diffusion seems to work if segments have the same radius, the tests that are added here also point to the fact that the diffusion across segments of different radius is still erroneous. Therefore, I left the tests for different radii commented until #2145 has been resolved.

thorstenhater · 2023-08-16T09:39:36Z

Hey @jlubo,

thanks for adding this. Could you run black . and push the results? That'll make the Linter
check pass.

thorstenhater · 2023-08-16T12:19:16Z

And flake8 . :D

jlubo · 2023-08-16T12:20:11Z

Okay, black now passes (I had previously used an older version, seems that they changed something). Yeah, now flake8 is complaining... I'll try to fix that, too :)

thorstenhater

Hey,
made some comments for improvements, rest looks great, thanks.

thorstenhater · 2023-08-16T16:30:17Z

.gitignore

@@ -105,3 +105,6 @@ _skbuild

 # generated by test scripts
 results
+
+# environment setting script
+set_arbor_env*


Where does this file come from? If it's common (like editor/build system/... temporaries) I have no problem with adding it, but we had clashes before and so I am rather conservative.

It's a file that I usually use to set the Arbor environment, I guess it can't be considered common :D

test/unit/diffusion/synapse_with_diffusion.mod

test/unit/diffusion/neuron_with_diffusion.mod

thorstenhater · 2023-08-16T16:35:18Z

python/test/readme.md

@@ -39,6 +39,6 @@ In subfolders `unit`/`unit_distributed`:

 ## Naming convention

- modules: `test_xxxs` (ending with `s` since module can consist of multiple classes)
- class(es): `TestXxxs` (ending with `s` since class can consist of multiple test functions)
+- modules: `test_xxxs` (if suitable, ending with `s` since module can consist of multiple classes)


Suggested change

- modules: `test_xxxs` (if suitable, ending with `s` since module can consist of multiple classes)

- modules: `test_xxxs` (if applicable use the plural as modules can comprise multiple tests)

thorstenhater · 2023-08-16T16:35:31Z

python/test/readme.md

- modules: `test_xxxs` (ending with `s` since module can consist of multiple classes)
- class(es): `TestXxxs` (ending with `s` since class can consist of multiple test functions)
+- modules: `test_xxxs` (if suitable, ending with `s` since module can consist of multiple classes)
+- class(es): `TestXxxs` (if suitable, ending with `s` since class can consist of multiple test functions)


Same idea here

python/test/unit/test_diffusion.py

thorstenhater · 2023-08-16T16:38:24Z

python/test/unit/test_diffusion.py

+            cat  # use the provided catalogue of diffusion mechanisms
+        )
+        self.the_props.set_ion("s", 1, 0, 0, 0)  # use diffusive particles "s"
+        self.inc_0 = inc_0  # increase in particle amount at 0.1 s (in 1e-18 mol)


Adding a list of [(t, synapse, amount)] instead of these three hardcoded injections seems like it could make things
more general and even clearer to understand.

python/test/unit/test_diffusion.py

…pointing to the fact that the diffusion across segments of different radius is still erroneous; see arbor-sim#2145)

jlubo · 2023-08-24T14:16:33Z

Hey, made some comments for improvements, rest looks great, thanks.

Thanks @thorstenhater! I've revised the code according to the comments.

thorstenhater · 2023-08-25T07:01:11Z

ext/fmt

I'd recommend git fetch --all && git merge origin/master && git submodule sync ... changing submodules in a feature PR is likely not what you want.

Thanks. I must have missed that they were in.

thorstenhater · 2023-08-25T07:01:20Z

ext/pugixml

Same as above.

thorstenhater

Thank you very much!

thorstenhater · 2023-08-29T07:50:44Z

cscs-ci run daint-gpu

thorstenhater · 2023-08-29T08:56:54Z

Uh-oh

subprocess.CalledProcessError: Command 'make' returned non-zero exit status 2.
 Error:
/tmp/tmpqgxm6fo0/build/generated/diffusion/neuron_with_diffusion_gpu.cu(59): warning #177-D: function "arb::diffusion_catalogue::<unnamed>::multiply" was declared but never referenced
/tmp/tmpqgxm6fo0/build/generated/diffusion/synapse_with_diffusion_gpu.cu(74): error: identifier "n_" is undefined
/tmp/tmpqgxm6fo0/build/generated/diffusion/synapse_with_diffusion_gpu.cu(55): warning #177-D: function "arb::diffusion_catalogue::<unnamed>::multiply" was declared but never referenced
1 error detected in the compilation of "/tmp/tmpqgxm6fo0/build/generated/diffusion/synapse_with_diffusion_gpu.cu".
make[2]: *** [CMakeFiles/diffusion-catalogue.dir/build.make:183: CMakeFiles/diffusion-catalogue.dir/generated/diffusion/synapse_with_diffusion_gpu.cu.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/diffusion-catalogue.dir/all] Error 2

@boeschf could you take a look? I suspect the recent changes to event delivery clash with the diffusion handling:

    if (ii_ < (end_ - begin_)) {
        const auto tid_ = (begin_ + ii_)->mech_index;
        if ((ii_ > 0) && ((begin_ + (ii_ - 1))->mech_index == tid_)) return;
        for (auto i_ = begin_ + ii_; i_ < end_; ++i_) {
            if (i_->mech_index != tid_) break;
            [[maybe_unused]] auto weight = i_->weight;
            unsigned lane_mask_ = arb::gpu::ballot(0xffffffff, tid_<n_);

this was emitted by modcc for the NET_RECV. Note the missing n_, which I assume should be
end_ - begin_ here?

…de readability improved

jlubo · 2023-10-07T19:24:33Z

The different-radii and different-length tests are now uncommented. They should succeed if they are run with this new fix in the diffusion solver: thorstenhater@7577092

thorstenhater · 2023-10-09T13:27:00Z

Hey @jlubo,

could you leave them disabled, I'll add them to the to-be-made PR for the fixes.

…ent_radii' again (for as long as the related fix is not merged)

jlubo · 2023-10-10T09:04:30Z

Done :)

thorstenhater · 2023-10-10T12:02:06Z

I'll fix the GPU problem in my follow-up #2226

python/test/unit/test_diffusion.py

thorstenhater

Nice! See also downstream #2226. Two questions:

It seems like lots of special casing for the different setups (num_sgs=1,2,3). Do you think it would be cleaner to split these into different tests altogether?
Question also inline: Can't we always compute the equilibrium concentration?

…egments

jlubo · 2023-10-14T15:02:21Z

1. It seems like lots of special casing for the different setups (num_sgs=1,2,3). Do you think it would be cleaner to split these into different tests altogether?

I agree, I've now split the function get_morph_and_decor() into into individual functions for 1, 2, and 3 segments.

2. Question also inline: Can't we always compute the equilibrium concentration?

Yes we can! I've introduced a more general computation of CV volumes making this possible now.

boeschf · 2023-10-23T12:02:55Z

cscs-ci run daint-gpu

jlubo mentioned this pull request Aug 15, 2023

Flaws in the diffusion process #2145

Open

jlubo requested a review from thorstenhater August 15, 2023 09:13

thorstenhater requested changes Aug 16, 2023

View reviewed changes

jlubo added 8 commits August 24, 2023 15:55

Tests for particle diffusion added (for different morphologies; also …

85486cf

…pointing to the fact that the diffusion across segments of different radius is still erroneous; see arbor-sim#2145)

Black style formatting applied

6062daf

Black style formatting (v23.7.0) applied

890f315

Cleanup

1bd43cc

Black reformatting

72b0d61

Revision: list of dictionaries for stimulation added; small improvements

5b9ecd3

Split function to reduce complexity (following flake8 complaint)

e6b48b9

Black style applied

2c1e802

jlubo force-pushed the tests_for_particle_diffusion branch from 74a8f99 to 2c1e802 Compare August 24, 2023 14:04

jlubo requested a review from thorstenhater August 24, 2023 16:22

thorstenhater requested changes Aug 25, 2023

View reviewed changes

Revision: discretization policy simplified

7f77d60

jlubo force-pushed the tests_for_particle_diffusion branch from 25fe938 to 7f77d60 Compare August 25, 2023 17:52

jlubo requested a review from thorstenhater August 25, 2023 17:54

thorstenhater previously approved these changes Aug 25, 2023

View reviewed changes

jlubo added 2 commits September 27, 2023 12:20

tests for different segment length added (also fail as of v0.9.0); co…

0bcc339

…de readability improved

Merge branch 'master' into tests_for_particle_diffusion

b78d1cb

jlubo dismissed thorstenhater’s stale review via b78d1cb September 27, 2023 11:55

fix: apply 'estimated' tests only if radius is fixed

8fe2207

jlubo requested review from thorstenhater and boeschf October 7, 2023 19:19

thorstenhater mentioned this pull request Oct 9, 2023

Fix coefficients in the diffusion solver #2226

Draft

disabled 'test_diffusion_different_length' and 'test_diffusion_differ…

51d3990

…ent_radii' again (for as long as the related fix is not merged)

thorstenhater reviewed Oct 10, 2023

View reviewed changes

python/test/unit/test_diffusion.py Outdated Show resolved Hide resolved

thorstenhater requested changes Oct 10, 2023

View reviewed changes

jlubo added 4 commits October 12, 2023 12:56

generalized tests for 'estimated' equilibrium values; annotations added

c3b5bf7

Merge branch 'master' into tests_for_particle_diffusion

b334f29

fixed estimated values in two segment-case

da87dea

split function 'get_morph_and_decor()' into special cases for 1,2,3 s…

e72bace

…egments

jlubo requested a review from thorstenhater October 14, 2023 15:04

This was referenced Jan 9, 2024

Fix/diffusive concentrations #2244

Draft

Tests for particle diffusion II #2245

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests for particle diffusion #2209

Tests for particle diffusion #2209

jlubo commented Aug 15, 2023

thorstenhater commented Aug 16, 2023

thorstenhater commented Aug 16, 2023

jlubo commented Aug 16, 2023

thorstenhater left a comment

thorstenhater Aug 16, 2023

jlubo Aug 24, 2023

thorstenhater Aug 16, 2023

thorstenhater Aug 16, 2023

thorstenhater Aug 16, 2023

jlubo commented Aug 24, 2023

thorstenhater Aug 25, 2023

jlubo Aug 25, 2023

thorstenhater Aug 25, 2023

thorstenhater left a comment •

edited

thorstenhater commented Aug 29, 2023

thorstenhater commented Aug 29, 2023 •

edited

jlubo commented Oct 7, 2023

thorstenhater commented Oct 9, 2023

jlubo commented Oct 10, 2023

thorstenhater commented Oct 10, 2023

thorstenhater left a comment

jlubo commented Oct 14, 2023

boeschf commented Oct 23, 2023

	- modules: `test_xxxs` (if suitable, ending with `s` since module can consist of multiple classes)
	- modules: `test_xxxs` (if applicable use the plural as modules can comprise multiple tests)

Tests for particle diffusion #2209

Are you sure you want to change the base?

Tests for particle diffusion #2209

Conversation

jlubo commented Aug 15, 2023

thorstenhater commented Aug 16, 2023

thorstenhater commented Aug 16, 2023

jlubo commented Aug 16, 2023

thorstenhater left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlubo commented Aug 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thorstenhater left a comment • edited

Choose a reason for hiding this comment

thorstenhater commented Aug 29, 2023

thorstenhater commented Aug 29, 2023 • edited

jlubo commented Oct 7, 2023

thorstenhater commented Oct 9, 2023

jlubo commented Oct 10, 2023

thorstenhater commented Oct 10, 2023

thorstenhater left a comment

Choose a reason for hiding this comment

jlubo commented Oct 14, 2023

boeschf commented Oct 23, 2023

thorstenhater left a comment •

edited

thorstenhater commented Aug 29, 2023 •

edited