-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nighttime flag should account for time within an interval #567
Comments
The day/night labeler in pvanalytics assumes that you don't yet trust the location or timestamps. We haven't really discussed beyond that inference algorithm. Here, the challenge is to label day/night for time averages where the location and timestamps are trusted. What about an approach that identifies the periods when day/night change (sunrise/sunset) and computes the daylight fraction of those periods? Maybe more complicated but perhaps faster than computing solar position for of all the downscaled time series. |
That's an interesting alternative. sun_rise_set_transit_spa (or ephem/geometric variants) give us the zenith = 90 points, so we'd need to let go of consistency with the results of 1 minute observations resampled to longer intervals. calc_time would let us find when zenith = 87, but I suspect it's much slower. Or were you suggesting that we find sunrise/sunset and then compute the solar position of the downscaled time series for just the hours around those times? Perhaps an opportunity to add some more performance tests to pvlib. |
sun_rise_set_transit_ephem accepts a 'horizon' kwarg that defines the angle for sunrise/sunset. I don't know if SPA could be extended with a similar kwarg.
I was thinking the above, but, if we know exactly when sunrise occurs, then it should be easy to compute the minutes between sunrise and interval start, and end, without having to downscale the interval. |
Thanks for reminding me of that feature. pvlib/pvlib-python#1059 adds
It takes several seconds to compile the numba code, so it's only useful for repeated calculations. My understanding of the workflow is that we would not benefit unless using precompiled numba code, which I don't have experience with. @alorenzo175 can you comment on the applicability of numba in the workers? Also worth pointing out that any database IO that involves 100 days of 1 minute resolution data is going to take a bit of time - seemed like a couple of seconds going through csv on the dashboard. I'd rather not add another second of latency but that's not the end of the world and hundreds of ms is not a big deal. |
I think the workers spawn process to perform the validation. So the numba codes is precompiled at worker startup and fork is used to spawn the process, it might not need to be recompiled for each validation. Hard to say if the time to test and implement that is worth it over going without numba. |
Similar issue in solarforecastarbiter-core/solarforecastarbiter/validation/validator.py Lines 487 to 507 in 4ee531a
Perhaps we should change |
Strikes me as an improvement. I haven't looked at what it would change in |
The NIGHTTIME flag is set based on the solar zenith at the interval label timestamp. For example, for an observation with interval label = beginning, for an interval
[start, end)
the flag is computed based on onlysolar_zenith(start) < 87
. And for an observation with interval label = ending, for an interval(start, end]
the flag is computed based on onlysolar_zenith(end) < 87
. This is fine for 1 minute observations, but becomes problematic for longer intervals. We instead should label intervals by the fraction of time within an interval that is nighttime.Here are the relevant sections of code:
solarforecastarbiter-core/solarforecastarbiter/validation/validator.py
Lines 559 to 575 in 58944fe
solarforecastarbiter-core/solarforecastarbiter/validation/tasks.py
Lines 31 to 37 in ea5d701
solarforecastarbiter-core/solarforecastarbiter/pvmodel.py
Lines 23 to 46 in 886897e
My proposal for the arbiter is the following:
tasks._solpos_night
computes solar position for sub-intervals, similar topersistence_scalar_index
. We'd need to choose an appropriate sub-interval length. 1 minute? 5 minutes like inpersistence_scalar_index
?validator.check_irradiance_day_night
computes the flag on the higher resolution solar position data.tasks._solpos_night
applies boolean resampling logic, similar to accounting for marginal nighttime and clearsky conditions when resampling observations #556, to create a flag for the times in the original index.A. Resample the solar position data to match the observation rules, and return.
B. Subset and continue to return the instantaneous values at the labels.
C. Return the full high resolution solar position dataframe and modify the downstream functions to account for averaging/subsetting. Perhaps my favorite approach, but also the most work.
solar_position, night_flag
tuple.@cwhanse have you looked into this for pvanalytics? I didn't see any code or issues.
The text was updated successfully, but these errors were encountered: