Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory issues on CRAN #165

Closed
osorensen opened this issue Oct 11, 2023 · 6 comments
Closed

Memory issues on CRAN #165

osorensen opened this issue Oct 11, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@osorensen
Copy link
Collaborator

Seem to be caused by this in the case where newind = -2

galamm/src/update_funs.h

Lines 24 to 46 in 7cd848a

template <typename T>
void update_X(Mdual<T>& X, const Vdual<T>& lambda,
const std::vector<std::vector<int>>& lambda_mapping_X){
if(lambda_mapping_X.size() == 0) return;
for(int i = 0; i < X.size(); i++){
std::vector<int> newinds = lambda_mapping_X[i];
T loading{0};
bool update{false};
int j{0};
for(int newind : newinds){
if(newind != -1){
loading += lambda(newind);
update = true;
}
j++;
}
if(update){
*(X.data() + i) *= loading;
}
}
};

I was able to reproduce a problem using R -d lldb. There may be more problems, of course. This change fixed it:

galamm/src/update_funs.h

Lines 24 to 53 in d4e2ab5

template <typename T>
void update_X(Mdual<T>& X, const Vdual<T>& lambda,
const std::vector<std::vector<int>>& lambda_mapping_X){
if(lambda_mapping_X.size() == 0) return;
if(lambda_mapping_X.size() != X.size()) Rcpp::stop("Mismatch in lambda_mapping_X size.");
for(size_t i{}; i < lambda_mapping_X.size(); i++){
std::vector<int> newinds = lambda_mapping_X[i];
T loading{0};
bool update{false};
int j{0};
for(int newind : newinds){
if(newind == -2){
loading = 0;
update = true;
} else if(newind != -1){
loading += lambda(newind);
update = true;
}
j++;
}
if(update){
*(X.data() + i) *= loading;
}
}
};

@osorensen osorensen added the bug Something isn't working label Oct 11, 2023
@osorensen osorensen mentioned this issue Oct 11, 2023
@osorensen
Copy link
Collaborator Author

On rocker/r-devel-san, R CMD check now returns 1 NOTE, 0 WARNINGs, 0 ERRORs

* checking installed package size ... NOTE
  installed size is 35.1Mb
  sub-directories of 1Mb or more:
    doc    1.9Mb
    libs  32.0Mb

@osorensen
Copy link
Collaborator Author

I have also reproduced the CRAN issue on rocker/r-devel-san, using the current CRAN version. The same issue goes away when using the fix described above.

@osorensen
Copy link
Collaborator Author

osorensen commented Oct 13, 2023

Furthermore, on rocker/r-devel-san with the new update (the current CRAN version crashed it):

> devtools::test()
ℹ Testing galamm| F W  S  OK | Context|         12 | galamm-glmm [125.8s]                                                                                                         
✔ |         10 | galamm-heteroscedastic [28.0s]                                                                                               
✔ |          8 | galamm-latent-covariates-interaction [690.7s]                                                                                
✔ |         36 | galamm-lmm [505.8s]                                                                                                          
✔ |         13 | galamm-mixed-resp [708.5s]                                                                                                   
✔ |         68 | galamm-semiparametric [832.4s]                                                                                               
✔ |         40 | galamm-setup [485.7s]                                                                                                        

══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 3377.1 s

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 187 ]

devtools::run_examples() also succeeded.

@osorensen
Copy link
Collaborator Author

A note here. As an additional safety measure, I now explicitly make sure the program never asks for vector elements at negative indices:

galamm/src/update_funs.h

Lines 30 to 50 in 724f799

for(size_t i{}; i < lambda_mapping_X.size(); i++){
std::vector<int> newinds = lambda_mapping_X[i];
T loading{0};
bool update{false};
int j{0};
for(int newind : newinds){
if(newind == -2){
loading = 0;
update = true;
} else if(newind >= 0){
loading += lambda(newind);
update = true;
}
j++;
}
if(update){
*(X.data() + i) *= loading;
}

@osorensen
Copy link
Collaborator Author

Keeping the issue open until the CRAN checks are ok.

@osorensen
Copy link
Collaborator Author

CRAN checks ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant