Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Single-file models #2119

Open
dweindl opened this issue Jun 6, 2023 · 0 comments
Open

Single-file models #2119

dweindl opened this issue Jun 6, 2023 · 0 comments
Labels
enhancement performance python

Comments

@dweindl
Copy link
Member

dweindl commented Jun 6, 2023

Refactor model code generation to allow switching between one-file-per-model-quantity and everything-inside-one source file. This should speed up import of small models significantly. (#1453 (comment))
dweindl added a commit that referenced this issue Aug 18, 2023
@dweindl
Combine code for sparse model functions and their index files
b7e9ed6 
Combine code for sparse model functions and their index files, i.e. generate
only a single file instead of 3 individual files for content, rowvals, and
colptrs, respectively.

Advantage: Faster import of smaller models and fewer files. For a toy model,
this reduced the build steps from 44 to 28, and reduces build time by >20%.

Disadvantage: None found, so I don't think it worth adding an option
for (not) combining those files. For larger models, there shouldn't be any impact.
The extra time for compiling the index arrays should be negligible compared
to computing the contents.

Related to #2119
@dweindl dweindl mentioned this issue Aug 18, 2023
Merged
dweindl added a commit that referenced this issue Aug 25, 2023
@dweindl
Combine code for sparse model functions and their index files (#2159)
98c782d 
Combine code for sparse model functions and their index files, i.e. generate only a single file instead of 3 individual files for content, rowvals, and colptrs, respectively.

Advantage: Faster import of smaller models and fewer files. For a toy model, this reduced the build steps from 44 to 28, and reduced build time by >20% on my computer.

Disadvantage: None found, so I don't think it worth adding an option for (not) combining those files. For larger models, there shouldn't be any impact. The extra time for compiling the index arrays should be negligible compared to computing the contents.

Related to #2119


Here a test for a large model (N=1):

| File         | Size    | Compilation time (s) |
|--------------|--------:|---------------------:|
| dwdx         | 22.4MiB |              3413.64 |
| dwdx_colptrs |  2.0KiB |                 2.79 |
| dwdx_rowvals | 65.6KiB |                 2.66 |
| *combined*   |         |              3416.79 |

I'd consider this time increase negligible.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement performance python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dweindl