Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring fill xarray function #9

Merged
merged 5 commits into from
Apr 12, 2024

Conversation

huche6
Copy link
Contributor

@huche6 huche6 commented Apr 11, 2024

The idea here is to drastically improve the efficiency of the function fill_xarray_from_input_parameters
Previously, it iterated through all indexes of an initialized xarray DataArray called array.

What I propose here is to keep the same iterative process but instead of allocating values directly to a DataArray, I append dictionnaries into a list. This list of dictionnary is then used to create a pandas DataFrame.
For each column of interest (powertrain, size, value, year, parameter), I flatten (through explode function) the lists contained within each row of the dataframe. I use a dummy variable "g" which count elements in each lists for each row of the DataFrame.
The final DataFrame is built using the flatten dataframe and the corresponding parameters data. NaN are automatically used when a DataFrame is broadcasted. Then I use the ffill method from DataFrame.fillna to replace the NaN by the expected values.
The columns size, powertrain, parameter, year and value are then passed as index of the DataFrame.
The resulting DataArray is created from the final pandas Series obtained.

It was not easy to detailled the process here, do not hesite to ask further question if needed.

All tests in carculator_utils, carculator and carculator_trucks seem to work properly.

@huche6 huche6 changed the title Fill xarray function Refactoring fill xarray function Apr 11, 2024
@romainsacchi romainsacchi merged commit 57fd32e into romainsacchi:master Apr 12, 2024
3 of 4 checks passed
@romainsacchi
Copy link
Owner

Awesome, thanks @huche6

@huche6 huche6 deleted the fill_xarray branch April 12, 2024 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants