Scale on load

[MRossol]

Add ability to scale datasets during rechunk

[MRossol]

@grantbuster If you don't have time to review np, I tested this on the Solar Vietnam data for Paul. That said I did add a really cool Retry Decorator from my hsds_etl package that I had forgotten about that you should check out!

[grantbuster]

Yeah the rechunk utils are your domain, go for it!

[grantbuster]

Send me a link to the retry decorator?

[ryancoe]

@grantbuster - When I do the following, do I need to apply a scale factor, or is this applied automatically? (From what I can see, I would guess that I do need to scale it manually.)

from rex import WindX

wtk_file = '/nrel/wtk/hawaii/Hawaii_2010.h5'
lat_lon = (19.3131, -163) # xy[0,:]
with WindX(wtk_file, hsds=True) as f:
    wspd = f.get_lat_lon_df('windspeed_100m', lat_lon)

[MRossol]

@ryancoe you do not need to do any manual scalig. rex scales the resource data back to native units automatically. That's one of its primary functions.

[ryancoe]

Thanks @MRossol! I think I must be messing something up, because to me it looks like at 100m up at an offshore location near Hawaii, the average wind speed in 2010 was 130 m/s?

from rex import WindX

wtk_file = '/nrel/wtk/hawaii/Hawaii_2010.h5'
lat_lon = (19.3131, -163) # xy[0,:]
with WindX(wtk_file, hsds=True) as f:
    wspd = f.get_lat_lon_df('windspeed_100m', lat_lon)

fig, ax = plt.subplots()
wspd.plot(ax=ax)
ax.axhline(nwtc_wspd.mean().values,color='r')

https://energy.hawaii.gov/wp-content/uploads/2011/10/Hawaii_FINAL_SPD100m_26July04.pdf

[MRossol]

Hmm, that does look suspect. Can you try:

1. a different location?
2. a different year?

[ryancoe]

Thanks again for your help @MRossol.

Looks similarly fishy for 22°00'00.0"N 159°00'00.0"W in 2012 (which shows up on the AWS sheet above as 7.5-8.0 m/s).

[MRossol]

Fishy indeed! @grantbuster Do you know if the Hawaii data has been updated recently? Might be worth checking with Jay or John to see if the scaling is messed up in HSDS.

[grantbuster]

on Eagle, the hawaii wind data hasnt been updated since 2020. I'll forward this chain to Jay and see if he can check the hsds data.

[ryancoe]

hi guys - any updates on this? Should we create a separate issue?

[grantbuster]

This isnt a software issue as far as i can tell. Pretty sure you just need to get Jay to reupload the data to hsds. I asked but he's busy.

[MRossol]

@ryancoe try submitting an issue here:
https://github.com/openEDI/documentation