Inaccurate calculation of sunrise for Boston area (perhaps others) [rt.cpan.org #132336]

[jforget]

Migrated from rt.cpan.org#132336 (status was 'patched')

Requestors:

• timezones@kosowsky.org

From timezones@kosowsky.org on 2020-04-13 22:52:21
:

Hi,
I was just testing DateTime::Event::Sunrise for Newton Center,
Massachuseets for the month of April 2020 against several other
calculators, including:
1. NOAA (https://www.esrl.noaa.gov/gmd/grad/solcalc/table.php?lat=42.3358&lon=-71.2044&year=2020)
2. timeanddate.com (https://www.timeanddate.com/sun/@4945289)
3. https://keisan.casio.com/exec/system/1224686065

They all were consistent within about 1 minute.

On the other hand, DateTime::Event::Sunrise for *sunrise* seemed to be
about 2 minutes earlier on most days in my tested month of April.
Yet, for *sunset* the times were accurate for the month.

Any ideas on what might be going on here?

Note I used the following code:

#!/usr/bin/perl

use strict;
use warnings;
use DateTime;
use DateTime::Event::Sunrise;

my $sunriseset = DateTime::Event::Sunrise->new( #Note this is a different object type (used with sunrise_datetime and sunset_datetime)
	             longitude => 42.3358,
	             latitude => -71.2044,
		     altitude => '-0.833',
                     precise => 1);

my $dt = DateTime->new(
	        year => 2020,
	        month => 4, 
    	        day => 12,
	        hour => 12,
                time_zone => 'America/New_York');

for(my $i=1; $i<=30; $i++) {
    $dt->set(day => $i);
    print $dt->mdy, "\t", $sunriseset->sunrise_datetime($dt)->hms, "\t", $sunriseset->sunset_datetime($dt)->hms, "\n";
};

From j2n-forget@orange.fr on 2020-04-14 18:27:14
:

Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
[ cut ]
Thank you for reporting this bug.

There is a error in your code example: your location
is 42°E, 71°S while I think you meant 71°W 42°N.
I think you live near Boston, rather than on the
Antarctic coast. But it is not important.

You wrote: Any ideas on what might be going there?
Here are some ideas.

The sunrise module was created as Astro::Sunrise.
It did not integrate with DateTime, because DateTime
was first released in 2003 and Astro::Sunrise was first
released in 2000. In 2003, the original author duplicated
his module Astro::Sunrise into DateTime::Event::Sunrise.
For a while, he maintained both modules. But
very soon, he stopped being active in the Perl world.

After submitting a bug report in 2008, I took over
the maintenance of both modules in 2013. I fixed the
simple errors, and I postponed fixing the difficult bugs.
Among these difficult bugs, there is the "precise" method
which, as written, does not look good to me.

In 2017, I decided to tackle the issue of the precise
algorithm. The problem is, I decided to work on
Astro::Sunrise and only when Astro::Sunrise would be completely
debugged, I would port the fixes to DT::E::Sunrise.

So the situation now is:
-- Astro::Sunrise, last released in January 2019, contains a precise
algorithm which seems correct to me,
-- DT::E::Sunrise, last released in December 2015, contains another
"precise" algorithm which I do not understand and which seems wrong
to me.

Up to this morning when I read your bug report, I had the intention
to work again on Astro::Sunrise and find a fix to
https://rt.cpan.org/Public/Bug/Display.html?id=110016
https://rt.cpan.org/Public/Bug/Display.html?id=109992

But because of your bug report, I will do what I should have done
before, that is, port the Astro::Sunrise fixes to DT::E::Sunrise.
And maybe find a fix to ticket 110016 at the same time.
So DT::E::Sunrise would be ahead of Astro::Sunrise this time.
But you will have to wait several weeks before a new module release.
I am not paid to maintain Perl modules, I do this in my evening free time.

Meanwhile, if you want, you can do a few experiments, such as
comparing the Astro::Sunrise precise algorithm with the DT::E::Sunrise
"precise" algorithm. and comparing them with the simpler algorithm.
Actually, I would not be surprised if the DT::E::Sunrise "precise"
algorithm is worse than the simpler one.

Also, you can read:
https://github.com/jforget/Astro-Sunrise/blob/master/doc/astronomical-notes.pod
Note: this includes a paragraph about the choice between the simple
and the precise algorithms. Actually, for Boston, you have no real need
for the precise algorithm, the simple one would be sufficient. On the other
hand, I still need to do the A::Sunrise � DT::E::Sunrise port.

Thank you for your interest in my module(s).

From timezones@kosowsky.org on 2020-04-14 22:54:46
:

Thanks Jean for the helpful and thoughtful response... and thanks for
your work as maintainer and advancer of the module.
It's such a shame that the audience for PERL is declining - it's a
wonderfully powerful language.

And yes, I do live near Boston -- though would love to visit
Antarctica one day...

Jeff

Jean Forget via RT wrote at about 14:27:15 -0400 on Tuesday, April 14, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
 > [ cut ]
 > Thank you for reporting this bug.
 > 
 > There is a error in your code example: your location
 > is 42°E, 71°S while I think you meant 71°W 42°N.
 > I think you live near Boston, rather than on the
 > Antarctic coast. But it is not important.
 > 
 > You wrote: Any ideas on what might be going there?
 > Here are some ideas.
 > 
 > The sunrise module was created as Astro::Sunrise.
 > It did not integrate with DateTime, because DateTime
 > was first released in 2003 and Astro::Sunrise was first
 > released in 2000. In 2003, the original author duplicated
 > his module Astro::Sunrise into DateTime::Event::Sunrise.
 > For a while, he maintained both modules. But
 > very soon, he stopped being active in the Perl world.
 > 
 > After submitting a bug report in 2008, I took over
 > the maintenance of both modules in 2013. I fixed the
 > simple errors, and I postponed fixing the difficult bugs.
 > Among these difficult bugs, there is the "precise" method
 > which, as written, does not look good to me.
 > 
 > In 2017, I decided to tackle the issue of the precise
 > algorithm. The problem is, I decided to work on
 > Astro::Sunrise and only when Astro::Sunrise would be completely
 > debugged, I would port the fixes to DT::E::Sunrise.
 > 
 > So the situation now is:
 > -- Astro::Sunrise, last released in January 2019, contains a precise
 > algorithm which seems correct to me,
 > -- DT::E::Sunrise, last released in December 2015, contains another
 > "precise" algorithm which I do not understand and which seems wrong
 > to me.
 > 
 > Up to this morning when I read your bug report, I had the intention
 > to work again on Astro::Sunrise and find a fix to
 > https://rt.cpan.org/Public/Bug/Display.html?id=110016
 > https://rt.cpan.org/Public/Bug/Display.html?id=109992
 > 
 > But because of your bug report, I will do what I should have done
 > before, that is, port the Astro::Sunrise fixes to DT::E::Sunrise.
 > And maybe find a fix to ticket 110016 at the same time.
 > So DT::E::Sunrise would be ahead of Astro::Sunrise this time.
 > But you will have to wait several weeks before a new module release.
 > I am not paid to maintain Perl modules, I do this in my evening free time.
 > 
 > Meanwhile, if you want, you can do a few experiments, such as
 > comparing the Astro::Sunrise precise algorithm with the DT::E::Sunrise
 > "precise" algorithm. and comparing them with the simpler algorithm.
 > Actually, I would not be surprised if the DT::E::Sunrise "precise"
 > algorithm is worse than the simpler one.
 > 
 > Also, you can read:
 > https://github.com/jforget/Astro-Sunrise/blob/master/doc/astronomical-notes.pod
 > Note: this includes a paragraph about the choice between the simple
 > and the precise algorithms. Actually, for Boston, you have no real need
 > for the precise algorithm, the simple one would be sufficient. On the other
 > hand, I still need to do the A::Sunrise � DT::E::Sunrise port.
 > 
 > Thank you for your interest in my module(s).

From timezones@kosowsky.org on 2020-04-23 03:35:35
:

Jean Forget via RT wrote at about 14:27:15 -0400 on Tuesday, April 14, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
 > [ cut ]
 > Thank you for reporting this bug.
 > 
 > 
 > Meanwhile, if you want, you can do a few experiments, such as
 > comparing the Astro::Sunrise precise algorithm with the DT::E::Sunrise
 > "precise" algorithm. and comparing them with the simpler algorithm.
 > Actually, I would not be surprised if the DT::E::Sunrise "precise"
 > algorithm is worse than the simpler one.

I did some experimenting and it appears that at least for Boston, the
"simple" algorithm aligns with other sources whereas the "precise" is
off by 2-3 minutes.

Note for some reason, at least at this time of the year, the error is
larger for sunrise than for sunset

Please let me know when you have uploaded the fixed code and I will
test.
Thanks!

> Also, you can read:
 > https://github.com/jforget/Astro-Sunrise/blob/master/doc/astronomical-notes.pod
 > Note: this includes a paragraph about the choice between the simple
 > and the precise algorithms. Actually, for Boston, you have no real need
 > for the precise algorithm, the simple one would be sufficient. On the other
 > hand, I still need to do the A::Sunrise � DT::E::Sunrise port.
 > 
 > Thank you for your interest in my module(s).

From j2n-forget@orange.fr on 2020-04-23 05:07:58
:

Le Jeu 23 Avr 2020 05:35:35, timezones@kosowsky.org a écrit :
> Jean Forget via RT wrote at about 14:27:15 -0400 on Tuesday, April 14,
> 2020:
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
> >
> > Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
> > [ cut ]
> > Thank you for reporting this bug.
> >
> >
> > Meanwhile, if you want, you can do a few experiments, such as
> > comparing the Astro::Sunrise precise algorithm with the
> > DT::E::Sunrise
> > "precise" algorithm. and comparing them with the simpler algorithm.
> > Actually, I would not be surprised if the DT::E::Sunrise "precise"
> > algorithm is worse than the simpler one.
> 
> I did some experimenting and it appears that at least for Boston, the
> "simple" algorithm aligns with other sources whereas the "precise" is
> off by 2-3 minutes.
> 
> Note for some reason, at least at this time of the year, the error is
> larger for sunrise than for sunset
> 
> Please let me know when you have uploaded the fixed code and I will
> test.
> Thanks!
> 
The updated code is not yet uplooaded to CPAN. You can use my Github repo at
https://github.com/jforget/DateTime-Event-Sunrise
It is still a work in progress, but the situation has improved.

By the way, I told you that the precise algorithm was already implemented
in Astro::Sunrise. While working on DT::E::Sunrise, I think I have found a
bug in Astro::Sunrise. I will fix it after I have released DT::E::Sunrise
to CPAN.

Jean Forget

From timezones@kosowsky.org on 2020-04-23 13:09:58
:

Jean Forget via RT wrote at about 01:07:59 -0400 on Thursday, April 23, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Jeu 23 Avr 2020 05:35:35, timezones@kosowsky.org a écrit :
 > > Please let me know when you have uploaded the fixed code and I will
 > > test.
 > > Thanks!
 > > 
 > The updated code is not yet uplooaded to CPAN. You can use my Github repo at
 > https://github.com/jforget/DateTime-Event-Sunrise
 > It is still a work in progress, but the situation has improved.
 > 
 > By the way, I told you that the precise algorithm was already implemented
 > in Astro::Sunrise. While working on DT::E::Sunrise, I think I have found a
 > bug in Astro::Sunrise. I will fix it after I have released DT::E::Sunrise
 > to CPAN.
 > 
 > Jean Forget

Thanks for the update!
Amazing how bugs can persist in even the most stable code... :)

Good luck on finishing the work in progress...

From j2n-forget@orange.fr on 2020-06-18 05:42:56
:

Le Jeu 23 Avr 2020 15:09:58, timezones@kosowsky.org a écrit :
[ cut ]
After I have implemented the proper precise algorithm (already
implemented in Astro::Sunrise), there are still differences between
the values obtained by Stellarium and the NOAA on one side and by
DT::E::S on the other side. I have found that these differences are
the caused by the "magic" values 15 and 15.04107 used in the module.

15.04107 is the angular speed of Earth's spin, while 15 is the combination
of Earth's spin with the orbital speed of the Sun-Earth system.
According to Paul Schlyter 's description of the precise algorithm, we
should use 15.04107, but I have found that using 15 instead gives results
closer to Stellarium.

More abstractly, the problem is: when you are an low-level amateur astronomer,
who is an authoritative source for you? Whom do you trust? Paul Schlyter or
the Stellarium team or the NOAA?

I have decided to defer this question to the module user. There is a new
parameter, "algo", which lets the user choose between trusting Paul Schlyter
more than Stellarium or trusting Stellarium more than Paul Schlyter.

So you should use:

  my $sunriseset = DateTime::Event::Sunrise->new(
                     latitude  => 42.3358,
                     longitude => -71.2044,
                     altitude  => '-0.833',
                     precise   => 1,
		     algo      => 'Stellarium');

I will release the new version of DT::E::S to CPAN next week.
If you have a better idea for the name of the new parameter, please
tell me before this release.

In the meantime, the updated code is available on Github.

From timezones@kosowsky.org on 2020-06-18 20:02:30
:

Sounds great! and thanks for investing in the updates.

An alternative variable approach would be to call it "spin" and let it
be either the actual numerical value for the spin or the text value
'Stellarium' or 'Schlyter', in which case the values they advocate for
would be substituted for the text.

Jean Forget via RT wrote at about 01:42:57 -0400 on Thursday, June 18, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Jeu 23 Avr 2020 15:09:58, timezones@kosowsky.org a écrit :
 > [ cut ]
 > After I have implemented the proper precise algorithm (already
 > implemented in Astro::Sunrise), there are still differences between
 > the values obtained by Stellarium and the NOAA on one side and by
 > DT::E::S on the other side. I have found that these differences are
 > the caused by the "magic" values 15 and 15.04107 used in the module.
 > 
 > 15.04107 is the angular speed of Earth's spin, while 15 is the combination
 > of Earth's spin with the orbital speed of the Sun-Earth system.
 > According to Paul Schlyter 's description of the precise algorithm, we
 > should use 15.04107, but I have found that using 15 instead gives results
 > closer to Stellarium.
 > 
 > More abstractly, the problem is: when you are an low-level amateur astronomer,
 > who is an authoritative source for you? Whom do you trust? Paul Schlyter or
 > the Stellarium team or the NOAA?
 > 
 > I have decided to defer this question to the module user. There is a new
 > parameter, "algo", which lets the user choose between trusting Paul Schlyter
 > more than Stellarium or trusting Stellarium more than Paul Schlyter.
 > 
 > So you should use:
 > 
 >   my $sunriseset = DateTime::Event::Sunrise->new(
 >                      latitude  => 42.3358,
 >                      longitude => -71.2044,
 >                      altitude  => '-0.833',
 >                      precise   => 1,
 > 		     algo      => 'Stellarium');
 > 
 > I will release the new version of DT::E::S to CPAN next week.
 > If you have a better idea for the name of the new parameter, please
 > tell me before this release.
 > 
 > In the meantime, the updated code is available on Github.

From timezones@kosowsky.org on 2020-07-06 03:35:51
:

Hi Jean,
Can you let me know when the official update is posted to cpan?
Thanks again for all your contributions! Amazing stuff!
Jeff

Jean Forget via RT wrote at about 01:07:59 -0400 on Thursday, April 23, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Jeu 23 Avr 2020 05:35:35, timezones@kosowsky.org a écrit :
 > > Jean Forget via RT wrote at about 14:27:15 -0400 on Tuesday, April 14,
 > > 2020:
 > > > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > > >
 > > > Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
 > > > [ cut ]
 > > > Thank you for reporting this bug.
 > > >
 > > >
 > > > Meanwhile, if you want, you can do a few experiments, such as
 > > > comparing the Astro::Sunrise precise algorithm with the
 > > > DT::E::Sunrise
 > > > "precise" algorithm. and comparing them with the simpler algorithm.
 > > > Actually, I would not be surprised if the DT::E::Sunrise "precise"
 > > > algorithm is worse than the simpler one.
 > > 
 > > I did some experimenting and it appears that at least for Boston, the
 > > "simple" algorithm aligns with other sources whereas the "precise" is
 > > off by 2-3 minutes.
 > > 
 > > Note for some reason, at least at this time of the year, the error is
 > > larger for sunrise than for sunset
 > > 
 > > Please let me know when you have uploaded the fixed code and I will
 > > test.
 > > Thanks!
 > > 
 > The updated code is not yet uplooaded to CPAN. You can use my Github repo at
 > https://github.com/jforget/DateTime-Event-Sunrise
 > It is still a work in progress, but the situation has improved.
 > 
 > By the way, I told you that the precise algorithm was already implemented
 > in Astro::Sunrise. While working on DT::E::Sunrise, I think I have found a
 > bug in Astro::Sunrise. I will fix it after I have released DT::E::Sunrise
 > to CPAN.
 > 
 > Jean Forget

From j2n-forget@orange.fr on 2020-07-06 18:33:09
:

Le Lun 06 Jui 2020 05:35:51, timezones@kosowsky.org a écrit :
> Hi Jean,
> Can you let me know when the official update is posted to cpan?
> Thanks again for all your contributions! Amazing stuff!
> Jeff
> 
I have promised a release for "next week", 3 weeks ago.
In the meantime, I have received a few advices, one of them
being that it was a bad idea to allow the users choosing a
angular speed value which would give an erroneous result.
So I have decided to work a bit more on the module and
to fix it and refactor it. 

Sorry for the delay. Maybe sometime during July.

Jean

From timezones@kosowsky.org on 2020-07-07 02:34:01
:

All good. No rush. Thanks!
Jean Forget via RT wrote at about 14:33:10 -0400 on Monday, July 6, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Lun 06 Jui 2020 05:35:51, timezones@kosowsky.org a écrit :
 > > Hi Jean,
 > > Can you let me know when the official update is posted to cpan?
 > > Thanks again for all your contributions! Amazing stuff!
 > > Jeff
 > > 
 > I have promised a release for "next week", 3 weeks ago.
 > In the meantime, I have received a few advices, one of them
 > being that it was a bad idea to allow the users choosing a
 > angular speed value which would give an erroneous result.
 > So I have decided to work a bit more on the module and
 > to fix it and refactor it. 
 > 
 > Sorry for the delay. Maybe sometime during July.
 > 
 > Jean

From timezones@kosowsky.org on 2020-07-19 04:49:31
:

Hi Jean,
I noticed that you posted an updated version to CPANN.
Can I assume this is the latest fixed version?

Thanks for all the help!
Jeff

Jean Forget via RT wrote at about 01:07:59 -0400 on Thursday, April 23, 2020:
 > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > 
 > Le Jeu 23 Avr 2020 05:35:35, timezones@kosowsky.org a écrit :
 > > Jean Forget via RT wrote at about 14:27:15 -0400 on Tuesday, April 14,
 > > 2020:
 > > > <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
 > > >
 > > > Le Mar 14 Avr 2020 00:52:21, timezones@kosowsky.org a écrit :
 > > > [ cut ]
 > > > Thank you for reporting this bug.
 > > >
 > > >
 > > > Meanwhile, if you want, you can do a few experiments, such as
 > > > comparing the Astro::Sunrise precise algorithm with the
 > > > DT::E::Sunrise
 > > > "precise" algorithm. and comparing them with the simpler algorithm.
 > > > Actually, I would not be surprised if the DT::E::Sunrise "precise"
 > > > algorithm is worse than the simpler one.
 > > 
 > > I did some experimenting and it appears that at least for Boston, the
 > > "simple" algorithm aligns with other sources whereas the "precise" is
 > > off by 2-3 minutes.
 > > 
 > > Note for some reason, at least at this time of the year, the error is
 > > larger for sunrise than for sunset
 > > 
 > > Please let me know when you have uploaded the fixed code and I will
 > > test.
 > > Thanks!
 > > 
 > The updated code is not yet uplooaded to CPAN. You can use my Github repo at
 > https://github.com/jforget/DateTime-Event-Sunrise
 > It is still a work in progress, but the situation has improved.
 > 
 > By the way, I told you that the precise algorithm was already implemented
 > in Astro::Sunrise. While working on DT::E::Sunrise, I think I have found a
 > bug in Astro::Sunrise. I will fix it after I have released DT::E::Sunrise
 > to CPAN.
 > 
 > Jean Forget

From j2n-forget@orange.fr on 2020-07-26 19:51:56
:

Le 19/07/2020 à 06:49, timezones@kosowsky.org via RT a écrit :
>         Queue: DateTime-Event-Sunrise
>   Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132336 >
> 
> Hi Jean,
> I noticed that you posted an updated version to CPANN.
> Can I assume this is the latest fixed version?
> 
> Thanks for all the help!
> Jeff
> 

Yes this is the latest fixed version. I still need to comment
a few things, but this will take place in the "doc" subdirectory
of the Astro::Sunrise distribution. And I still need to test
what happens beyong the polar circles. At least, with the current
version, sunrises and sunsets are correct for Boston Newton Center
and the few other locations I have tested.

Sorry for the late answer, I was in vacations.

Thank you for your interest and your trust.

Jean Forget