Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a function to export the whole solution to file (e.g., CSV, parquet, JSON, etc.) #295

Closed
Tracked by #97
abelsiqueira opened this issue Nov 22, 2023 · 7 comments · Fixed by #491
Closed
Tracked by #97
Assignees
Labels
Type: improvement Better way of doing something Zone: export & visualisation How data leaves model & is visualised
Milestone

Comments

@abelsiqueira
Copy link
Member

abelsiqueira commented Nov 22, 2023

Currently, we only export the assets investment.

@abelsiqueira abelsiqueira mentioned this issue Nov 22, 2023
11 tasks
@abelsiqueira
Copy link
Member Author

Related to #115

@abelsiqueira
Copy link
Member Author

abelsiqueira commented Nov 22, 2023

How should we write the flow output? It is indexed in (u,v), rp, and B, where B is the time block.

  • Do we write the whole time block variable as a column?
  • Or do we divide the flow value between all time steps?

I.e., if flow[("a","b"), 1, 1:3] = 30.0, should we write

from,to,rp,B,value
a,b,1,1:3,30.0

or

from,to,rp,time_step,value
a,b,1,1,30.0
a,b,1,2,30.0
a,b,1,3,30.0

@abelsiqueira abelsiqueira added Type: improvement Better way of doing something Zone: data & import Interfacing between database and Julia labels Nov 23, 2023
@clizbe
Copy link
Member

clizbe commented Dec 12, 2023

Personally I prefer the second, but I guess it depends how you want to analyse the data.
@gnawin What do you think?

@gnawin
Copy link
Member

gnawin commented Dec 12, 2023

@clizbe For me as well the second is more useful, because it makes it easier to manipulate the results further.

I have another conceptual question though, if flow[("a","b"), 1, 1:3] = 30, does it mean flow[("a","b"), 1, 1] = flow[("a","b"), 1, 2] = flow[("a","b"), 1, 3] = 30 or flow[("a","b"), 1, 1] = flow[("a","b"), 1, 2] = flow[("a","b"), 1, 3] = 10?
Or more generally, when we define any variable over a time block, is the value the sum over the time block, or of every time step in the time block? @datejada @abelsiqueira

@clizbe
Copy link
Member

clizbe commented Dec 13, 2023

For storage it's energy, for flows it's power. We should add this explanation of exactly what the output means to the documentation.

@clizbe clizbe added Zone: export & visualisation How data leaves model & is visualised and removed Zone: data & import Interfacing between database and Julia labels Jan 4, 2024
@clizbe clizbe added the good first issue Good for newcomers label Jan 29, 2024
@datejada datejada removed the good first issue Good for newcomers label Jan 31, 2024
@datejada
Copy link
Member

datejada commented Feb 5, 2024

With @suvayu, we discussed the new duckDB workflow, which will cover this part as well as exporting the outputs to other types of files. So, we added it to the epic issue #289

@datejada datejada changed the title Create a function to export the whole solution to CSV Create a function to export the whole solution to file (e.g., CSV, parquet, JSON) Feb 5, 2024
@datejada datejada changed the title Create a function to export the whole solution to file (e.g., CSV, parquet, JSON) Create a function to export the whole solution to file (e.g., CSV, parquet, JSON, etc.) Feb 5, 2024
@clizbe clizbe added this to the M2 - End June milestone Feb 5, 2024
@abelsiqueira
Copy link
Member Author

I will create a simple version to be used right now, independent of DuckDB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: improvement Better way of doing something Zone: export & visualisation How data leaves model & is visualised
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants