Power Systems Economics PUC - RIO 2019.2 #19

andrewrosemberg · 2019-10-11T17:03:29Z

This issue is for questions about studies using the Package

Veronicafeijao · 2019-10-12T00:54:17Z

Which version of Julia should I use?

gcmgabrielmartinez · 2019-10-12T19:47:08Z

So I am trying to install the SDDP package in JuliaPro, but turns out that the installation does not conclude due to a packages called MathOptFormat and HTTP. Apparently, I succeded in installing the SDDP in Julia by the way, but it seems that I have to install for juliaPro as well, because when I call "using SDDP", it can not find it. Also, as I checked the packages files, I found SDDP packages both in julia and JuliaPro (1.0 and 1.2), so I believe that SDDP in JuliaPro may be corrupted at some point. Can you figure it out if it is possible to compile Julia from JuliaPro?

gcmgabrielmartinez · 2019-10-12T20:44:00Z

Also, I have another issue, this time with HydroPowerModels in Julia itself. It seems the file called Cairo was not able to be downloaded as installing proccess was running. Now, when I call using HydroPowerModels I have the warnning below:

Veronicafeijao · 2019-10-12T22:03:57Z

When I use ATOM, I also have problem with MathOptFormat. Now, I'm not using ATOM and the problem is with package HTTP (to install SDDP.jl) @andrewrosemberg , what's the problem?

gcmgabrielmartinez · 2019-10-12T22:07:44Z

@Veronicafeijao which IDE are you using instead of ATOM?

Veronicafeijao · 2019-10-12T22:12:47Z

I went in ATOM>Julia>Open REPL. There I put: (julia> ] add https://github.com/odow/SDDP.jl/#master )

gcmgabrielmartinez · 2019-10-12T22:31:51Z

Well, I took a chance on your suggestion, now have problem with both MathOptFormat and HTTP. I checked my first print and actually I already have had the same issue

andrewrosemberg · 2019-10-13T15:27:01Z

Which version of Julia should I use?

I have been using Julia 1.0 up to 1.05 to test the code, so those versions are preferred, but I believe it should work with later versions of Julia.

andrewrosemberg · 2019-10-13T15:34:41Z

@Veronicafeijao and @GabrielCalvo2812, sometimes there are issues installing SDDP. A solution provided in one of the package's issue (odow/SDDP.jl#182 (comment)) is to first add MathOptFormat.jl:

] add https://github.com/odow/MathOptFormat.jl

] add https://github.com/odow/SDDP.jl

andrewrosemberg · 2019-10-13T15:38:19Z

Also, I have another issue, this time with HydroPowerModels in Julia itself. It seems the file called Cairo was not able to be downloaded as installing proccess was running. Now, when I call using HydroPowerModels I have the warnning below:

@GabrielCalvo2812, I believe once you have SDDP properly installed, HydroPowerModels will successfully build and then using it will not produce these errors. If it persists, let me know.

Veronicafeijao · 2019-10-13T16:13:22Z

But SDDP already was installed with success. And even so Hydropowermodels has problem with Cairo. Get Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Andrew David Werner Rosemberg <notifications@github.com> Sent: Sunday, October 13, 2019 12:38:20 PM To: andrewrosemberg/HydroPowerModels.jl <HydroPowerModels.jl@noreply.github.com> Cc: Veronicafeijao <vr.eng@outlook.com>; Mention <mention@noreply.github.com> Subject: Re: [andrewrosemberg/HydroPowerModels.jl] Power Systems Economics PUC - RIO 2019.2 (#19) Also, I have another issue, this time with HydroPowerModels in Julia itself. It seems the file called Cairo was not able to be downloaded as installing proccess was running. Now, when I call using HydroPowerModels I have the warnning below: @GabrielCalvo2812<https://github.com/GabrielCalvo2812>, I believe once you have SDDP properly installed, HydroPowerModels will successfully build and then using it will not produce these errors. If it persists, let me know. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#19?email_source=notifications&email_token=ANOXOYW6VS6YNF5NKGMLRCDQOM6GZA5CNFSM4I74GF52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBCY45A#issuecomment-541429364>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANOXOYXHQISEKBZEJHN2TJTQOM6GZANCNFSM4I74GF5Q>.

odow · 2019-10-13T16:15:03Z

I'm trying to get SDDP.jl registered (JuliaRegistries/General#4247), so once it is, this should resolve the version issues.

I believe that the issue is that JuliaPro includes a version of HTTP.jl that is missing a required bug-fix for MathOptFormat (we need 0.8.2+). So I would try

] add HTTP

Then run ] st and check that HTTP is at least version 0.8.2. Then run

] add MathOptFormat
] add https://github.com/odow/SDDP.jl

Once that's done, you can add HydroPowerModels.

If you get the libcairo error, try ] add Cairo or ] build Cairo.

djiegow · 2019-10-13T17:06:40Z

Which version of Julia should I use?

I have been using Julia 1.0 up to 1.05 to test the code, so those versions are preferred, but I believe it should work with later versions of Julia.

@andrewrosemberg, I was using Julia 1.2. I added the MathOptFormat, SDDP and HydroPowerModels and I had this problem below:

I also failed in building "Fontconfig".

After this, I tried Julia 1.0.5. And I had another problem:

Am I forgetting anything?

Thanks!

andrewrosemberg · 2019-10-13T18:11:34Z

@djiegow, could you send me the output of your project status :
] st

gcmgabrielmartinez · 2019-10-13T23:21:46Z

@odow I tried your suggestion, I used the command add HTTP and it updated some files:

but then using ] st it did not show which version was on the files:

Actually I looked for in the files explorer for the HTTP package in juliaPro1.2 and wasn't there ( don't mind about HydroPowerModels, it is corrupted anyway):

andrewrosemberg · 2019-10-14T01:27:28Z

@GabrielCalvo2812 I would start a new project from scratch to test adding the necessary packages

gcmgabrielmartinez · 2019-10-14T01:34:38Z

@andrewrosemberg what you mean with a new project? I am just installing the packages

djiegow · 2019-10-14T15:25:32Z

@djiegow, could you send me the output of your project status :
] st

Sure!

andrewrosemberg · 2019-10-14T18:21:42Z

teste_hydropowermodels.zip

Hi Guys,

This is a small project with a basic script to test the package. I would advise using this as a start.
The above folder contains a test script, a case folder, and a Julia environment (Project.toml and Manifest.toml).

The lines

using Pkg
Pkg.activate(".")
Pkg.instantiate()

will activate the environment and download all the necessary packages at their proper version (SDDP, HydroPowerModels, GLPK).

Please let me know if this doesn't run as intended.

andrewrosemberg · 2019-10-14T18:25:09Z

I also just fixed the plotting error

ERROR: Cannot convert Tuple{Array{Float64,2},Array{Float64,2}} to series data for plotting

(caused by an unexpected bug at the new version of Plots.jl).

ncihangir · 2019-10-14T20:01:48Z

Hello, I am using JuliaPro 1.2.0. Just tried to run the test script, but still getting the same error. Please see below. Thank you!

andrewrosemberg · 2019-10-14T20:05:57Z

@ncihangir could you send the output from the activate+instantiate call?

ncihangir · 2019-10-14T21:41:56Z

Hello Andrew, please see below. It gives "resolving package versions" for quite a long time and seems like not able to complete:

andrewrosemberg · 2019-10-14T22:16:39Z

@ncihangir the output from the activate function shows you are not running this section on the folder of test_hydropowermodels.

You have to run those commands in the same folder of the Project.toml and Manifest.tml.

thamires-bap · 2019-10-15T00:58:39Z

Hey, @andrewrosemberg ! I tried to run the test, but when I tried it I got this message:

And then I tried adding the package:

but it didn't work as well...

Could you help me please?

I think I might be doing something wrong. Do I have to do something specific to install HydroPowerModels?

Thanks a lot!

odow · 2019-10-15T02:05:49Z

@thamires-bap it looks like you're not running the command from the right folder.

To update Andrew's comment:

Download https://github.com/andrewrosemberg/HydroPowerModels.jl/files/3725797/teste_hydropowermodels.zip and unzip it somewhere nice (like ~/Documents/HydroPowerModels/).
Open Julia (or Atom) (or Juno)
Run cd("~/Documents/HydroPowerModels/") (or wherever you saved it.) The path should be such that "~/Documents/HydroPowerModels/Manifest.toml" points to the one you extracted from the zip file
Run using Pkg; Pkg.activate("."); Pkg.instantiate()

JJhern · 2019-10-15T10:20:19Z

@andrewrosemberg I am getting this error: #19 (comment)

How did you fix it ?

thamires-bap · 2019-10-15T10:21:20Z

Hey, @odow ! I did just as you said, but the result was this:

mflb · 2019-10-15T11:16:25Z

Hey, @andrewrosemberg ! I tried to run the test script, but I am getting some errors.
Please see below.

Could you help me, please?
Thank you!

ncihangir · 2019-10-15T12:29:09Z

Hello, I also get similar error as Thamires when running Pkg.instantiate( ).

andrewrosemberg · 2019-10-15T13:04:32Z

@thamires-bap and @ncihangir searching about your problem in google I found this: https://discourse.julialang.org/t/pkg-instantiate-fails/11709/7

Maybe try the fix at the end of this issue. It seems like Windows is having a protocol problem unrelated to the package.

andrewrosemberg · 2019-10-15T13:18:05Z

@mflb, your issue is from TimeZones.jl. It doesn't recognize 'America/Sao_Paulo' as a valid time zone. A quick fix as proposed here is to change the timezone of your system.

Veronicafeijao · 2019-10-15T13:18:19Z

@mflb , I had a similar problem, but with other different files. I resolved by opening the entire test_hydropowermodels folder with the atom in File> Open folder. Then File> open file> (teste.jl). When I ran, these errors no appeared.
Try it

andrewrosemberg · 2019-10-15T13:20:18Z

@andrewrosemberg I am getting this error: #19 (comment)

How did you fix it ?

@JJhern Just update our package and it will be solved. I have downgraded Plots.jl to version "0.26.0", since the new version "0.27.0" had an issue.

Veronicafeijao · 2019-10-15T13:21:51Z

@mflb, Andrew helped you in the first problem(America/Sao Paulo), and I was talking about the next problems.

lopeesfelipe · 2019-10-16T00:13:13Z

I guess the Plots.jl issue was not solved with it's downgrade.. I already updated the package and it is using the version "0.26.3" of Plots.jl. I also tried to downgrade to version "0.26.0", but the issue persists.

hernanijj · 2019-10-16T00:48:07Z

@andrewrosemberg I am getting this error: #19 (comment)
How did you fix it ?

@JJhern Just update our package and it will be solved. I have downgraded Plots.jl to version "0.26.0", since the new version "0.27.0" had an issue.

it is still not working @andrewrosemberg

andrewrosemberg · 2019-10-16T02:19:06Z

@lopeesfelipe @hernanijj the examples are being executed with plots 0.26.0 in the docs. Please verify you are not using old precompilations of HydroPowerModels.

lopeesfelipe · 2019-10-16T02:54:00Z

You're right, but when I update the HydroPowerModels package (with the command ] add https://github.com/andrewrosemberg/HydroPowerModels.jl.git) it's used Plots.jl in 0.26.3 version.

andrewrosemberg · 2019-10-16T15:01:50Z

You're right, but when I update the HydroPowerModels package (with the command ] add https://github.com/andrewrosemberg/HydroPowerModels.jl.git) it's used Plots.jl in 0.26.3 version.

@lopeesfelipe I don't believe this is a problem, but, all the same, instantiate should be installing 0.26.0.

andrewrosemberg · 2019-10-16T16:44:06Z

@lopeesfelipe what might happen is that, if you start adding or updating packages after the "instantiate" command, Plots can update to v0.26.3 (where indeed the bug already exists). If you don't feel like instantiating again the original environment, do the following:

] add Plots#v0.26.0
] pin Plots

andrewrosemberg · 2019-10-18T00:50:22Z

Hi Guys,

Some have asked me about LMPs (Nodal price or PLDs in Portuguese). Following the PowerModels output, they are called 'lam_kcl_r' and are inside the bus solution dictionary (PS.: they are acctualy the negative of the LMPs). So, to access the different scenarious, do the following:

# circuit MVA
baseMVA = results[:data][1]["powersystem"]["baseMVA"]
# LMPs
nbus = length(results[:data][1]["powersystem"]["bus"])
idxbus = collect(1:nbus)
scen_lmp = convert(Array{Array{Float64,2},1},[[-results[:simulations][i][j][:powersystem]["solution"]["bus"]["$bus"]["lam_kcl_r"] for i=1:nsim, j=1:nstages]' for bus =1:nbus])/baseMVA

I devide by baseMVA to find the LMPS in ($/MW).

andrewrosemberg · 2019-10-18T01:12:31Z

In order to set different demands for each stage manually, one may use the 'set_active_demand!' function:

HydroPowerModels.set_active_demand!(data,demand)

where demand should be an Array{Float64,2}. Lines represent stage and columns bus.
PS.: The matrix 'demand' should have columns for all busses, even if no load is set to some of the busses. Also, this only works if you have at max 1 load per bus.

andrewrosemberg · 2019-10-18T04:03:32Z

For those trying to plot the future cost function, there isn't a function for this yet (SDDP.jl's issue).

I have implemented a simple function to help until a better solution appears:

using Plots

function plot_cuts(model::SDDP.PolicyGraph{T};node_idx::Int=1) where {T}
   node_name_parser = SDDP._node_name_parser
   cuts = Dict{String, Any}[]
   node= model.nodes[1];
   states = vcat([[values(stat.state)...] for stat in model[1].bellman_function.global_theta.cut_oracle.states]...)
   
   ncuts = length(node.bellman_function.global_theta.cut_oracle.cuts)
   plt = plot() 
   for i = 1:ncuts
       cut = node.bellman_function.global_theta.cut_oracle.cuts[i]
       cut_coef_aux = [values(copy(cut.coefficients))...][:,:]
       vals = cut.intercept .- states*cut_coef_aux

       plot!(plt,states,vals)
   end
   
   return plot(plt, legend=false)

end

PS.: This might be wrong, critical view advised.

To use it:

plot_cuts(hydromodel.policygraph)

felipefmds · 2019-10-29T03:04:41Z

Hi @andrewrosemberg . How can I do to get reservoir volume for each simulation (from 1 to 100) and scenario (1 to 12) ?
Thank you.

andrewrosemberg · 2019-10-29T20:40:06Z

Hi Guys,

If you are trying to get the sampled inflows:

scen_inflows = convert(Array{Array{Float64,2},1},[[results[:data][1]["hydro"]["Hydrogenerators"][res]["inflow"][HydroPowerModels.cidx(j,results[:data][1]["hydro"]["size_inflow"][1]),results[:simulations][i][j][:noise_term]] for i=1:nsim, j=1:nstages]' for res = 1:results[:data][1]["hydro"]["nHyd"]])

If you are trying to get the reservoir volume (@felipefmds):

scen_voume = convert(Array{Array{Float64,2},1},[[results[:simulations][i][j][:reservoirs][:reservoir][res].out for i=1:nsim, j=1:nstages]' for res = 1:results[:data][1]["hydro"]["nHyd"]])

andrewrosemberg · 2019-10-30T19:57:22Z

FAQ: Dual and primal variables are changing if more SDDP iterations are executed even though it already has statistically converged. Is this expected?

Answer: Yes, a more conservative notion of convergence would require both primal variable stalling and dual variable stalling. For case 3, this happens around the 1000th iteration.

odow · 2019-10-30T21:34:58Z

FAQ: Dual and primal variables are changing if more SDDP iterations are executed even though it already has statistically converged. Is this expected?

Answer: Yes, a more conservative notion of convergence would require both primal variable stalling and dual variable stalling. For case 3, this happens around the 1000th iteration.

I'm so glad this came up. For those interested, see odow/SDDP.jl#178. I recommend plotting the solution every X iterations until you "see" it stabilize.

Repository owner deleted a comment from ncihangir Oct 16, 2019

andrewrosemberg added the discussion label Jun 10, 2020

andrewrosemberg closed this as completed Oct 27, 2020

andrewrosemberg mentioned this issue Oct 27, 2020

Power Systems Economics PUC - RIO 2020.2 #44

Closed

Power Systems Economics PUC - RIO 2019.2 #19

Power Systems Economics PUC - RIO 2019.2 #19

Comments

andrewrosemberg commented Oct 11, 2019

Veronicafeijao commented Oct 12, 2019

gcmgabrielmartinez commented Oct 12, 2019 • edited

gcmgabrielmartinez commented Oct 12, 2019 • edited

Veronicafeijao commented Oct 12, 2019

gcmgabrielmartinez commented Oct 12, 2019 • edited

Veronicafeijao commented Oct 12, 2019

gcmgabrielmartinez commented Oct 12, 2019 • edited

andrewrosemberg commented Oct 13, 2019

andrewrosemberg commented Oct 13, 2019

andrewrosemberg commented Oct 13, 2019

Veronicafeijao commented Oct 13, 2019 via email

odow commented Oct 13, 2019

djiegow commented Oct 13, 2019

andrewrosemberg commented Oct 13, 2019

gcmgabrielmartinez commented Oct 13, 2019 • edited

andrewrosemberg commented Oct 14, 2019

gcmgabrielmartinez commented Oct 14, 2019

djiegow commented Oct 14, 2019

andrewrosemberg commented Oct 14, 2019

andrewrosemberg commented Oct 14, 2019

ncihangir commented Oct 14, 2019

andrewrosemberg commented Oct 14, 2019

ncihangir commented Oct 14, 2019

andrewrosemberg commented Oct 14, 2019

thamires-bap commented Oct 15, 2019

odow commented Oct 15, 2019

JJhern commented Oct 15, 2019 • edited

thamires-bap commented Oct 15, 2019

mflb commented Oct 15, 2019

ncihangir commented Oct 15, 2019

andrewrosemberg commented Oct 15, 2019

andrewrosemberg commented Oct 15, 2019

Veronicafeijao commented Oct 15, 2019

andrewrosemberg commented Oct 15, 2019

Veronicafeijao commented Oct 15, 2019

lopeesfelipe commented Oct 16, 2019

hernanijj commented Oct 16, 2019

andrewrosemberg commented Oct 16, 2019

lopeesfelipe commented Oct 16, 2019

andrewrosemberg commented Oct 16, 2019

andrewrosemberg commented Oct 16, 2019

andrewrosemberg commented Oct 18, 2019 • edited

andrewrosemberg commented Oct 18, 2019 • edited

andrewrosemberg commented Oct 18, 2019 • edited

felipefmds commented Oct 29, 2019

andrewrosemberg commented Oct 29, 2019 • edited

andrewrosemberg commented Oct 30, 2019

odow commented Oct 30, 2019

gcmgabrielmartinez commented Oct 12, 2019 •

edited

gcmgabrielmartinez commented Oct 12, 2019 •

edited

gcmgabrielmartinez commented Oct 12, 2019 •

edited

gcmgabrielmartinez commented Oct 12, 2019 •

edited

gcmgabrielmartinez commented Oct 13, 2019 •

edited

JJhern commented Oct 15, 2019 •

edited

andrewrosemberg commented Oct 18, 2019 •

edited

andrewrosemberg commented Oct 18, 2019 •

edited

andrewrosemberg commented Oct 18, 2019 •

edited

andrewrosemberg commented Oct 29, 2019 •

edited