Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory optimization #41

Closed
wants to merge 3 commits into from
Closed

Memory optimization #41

wants to merge 3 commits into from

Conversation

Ziqi-Li
Copy link
Member

@Ziqi-Li Ziqi-Li commented Oct 15, 2018

This PR is aim to reduce memory footprint of the base GWR model from method in FastGWR. Two approaches are 1) not to store N by N weight matrix 2) not to store N by N hat matrix.

For weight matrix, at every location, a row vector of weights is calculated but not stored. This may add some computation but very trivial. The entire N by N W matrix can be computed on demand as a method to GWRResults class.

For hat matrix, only tr_S and tr_STS are needed and they are calculated inside of GWR.fit() fitting loop additively without storing S. A boolean parameter hat_matrix is offered for whether to store the hat matrix. By default, hat_matrix is set to False for GWR. However, in MGWR, hat matrix of GWR is used to compute projection matrices of MGWR, so hat_matrix is switched to True behind the scene for calibrating MGWR.

Overall speed performance is similar, but memory footprint is much lower. Based on a dataset with 10k locations, peak memory drops from 4487MB to 155MB on my laptop.

@Ziqi-Li Ziqi-Li closed this Feb 21, 2019
@Ziqi-Li Ziqi-Li deleted the mem_opt branch December 31, 2019 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant