Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compound models do not scale well with complexity #5985

Closed
perrygreenfield opened this issue Apr 26, 2017 · 1 comment
Closed

compound models do not scale well with complexity #5985

perrygreenfield opened this issue Apr 26, 2017 · 1 comment

Comments

@perrygreenfield
Copy link
Member

The current implementation of compound models essentially replicate all parameter instances at every level of the expression tree. This effectively results in the number of parameter instances increasing roughly with the square of the number of class instances times (assuming roughly even distribution of parameters with model instances). This is not an idle worry. Some JWST WCS models involve nearly 100 model components and almost 300 parameters. This leads to over 10 thousand parameter instances and huge memory usage, and consequent time instantiating the model (significant fractions of an hour for MOS cases). This basically renders the current implementation impractical for such complex models.

One could try to optimize this with the existing implementation, but there is no simple way to make the user interface transparent if python expressions are to be used to create compound models. I have tried an alternate implementation of compound models that I will submit a PR soon for comments.

@nden
Copy link
Contributor

nden commented Dec 4, 2019

CompoundModel was reworked in v 4.0 essentially fixing this issue.

@nden nden closed this as completed Dec 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants