Consider unshipping `islamic` and `islamic-rgsa` calendars

[dminor]

Although the list of calendars is implementation defined, historically implementers have shipped what is supported by ICU4C / CLDR. While investigating differences between ICU4C and ICU4X, we've come across some questions about the value of shipping the islamic and islamic-rgsa calendars to the web.

In particular, the islamic uses an astronomical calculation that requires an observation point to be specified, but there's no ECMA-402 API to specify that location. More research is needed, but this would suggest that the result of using this calendar is not well-defined.

The islamic-rgsa calendar is not implemented in either ICU4C or ICU4X. It is observational based, impossible to predict in the future, and would require lookup tables to implement for the past.

Although support is implementation defined, we do have at least some test cases that assume the presence of these calendars in implementations.

Although what calendars are shipped is implementation defined, having agreement between implementations on the value of these calendars would be beneficial for the web. One possible result here would be a note saying these calendars are not recommended for use on the web.

[justingrant]

FYI @dminor - I edited your OP to correct islamic-rgba to the correct spelling.

[hsivonen]

In particular, the islamic uses an astronomical calculation that requires an observation point to be specified, but there's no ECMA-402 API to specify that location.

I think "API to specify that location" is getting way ahead of things. It would be good to understand what use case this kind of simulation that's neither actual sighting nor a specific known-outside-software Hijri calendar variant could address to user satisfaction.

[dminor]

I agree, I'm not suggesting that we need to add such an API, it's that what we have now probably isn't useful without one.

[Manishearth]

It would be good to understand what use case this kind of simulation that's neither actual sighting nor a specific known-outside-software Hijri calendar variant could address to user satisfaction.

100% this.

Almost nobody follows an observational calendar based on their current location. Rather, it is based on a location chosen by your sect/subsect/country. If we wanted to make this configurable, this should be some choice from an enum, not configurable.

The question is: is the astronomical point used by ICU4C (and ICU4X) one that is meaningfully used by many people? If so, can islamic be explicitly defined as that one?

[sffc]

@younies told me: "In countries with established local calendars, like Egypt, people tend to use those. Conversely, in countries without such calendars, such as Norway, the observational calendar from Saudi Arabia is commonly followed."

[Manishearth]

So some background on the types of calendar here:

Tabular calendars follow a simplified mathematical set of rules, typically some form of Metonic cycle.

Astronomical calendars (like Umm al-Qura) follow moon math based on a particular fixed observational point.

Observational calendars have observations as their source of truth: some religious or government official is in charge of looking at the moon and reporting whether it is seen. This may or may not be affected by cloud cover. In Egypt, apparently they use a helicopter to avoid this problem, however even then the observations may disagree from astronomical calculations: the ultimate source of truth is the observation.

Implementation-wise, parametrizing these is potentially useful, but from a user POV what you want is actual calendars used by groups of people, whether they be civil calendars or commonly-followed religious calendars. And you want the calendars to be accurate.

So we may end up having multiple astronomical calendars, parametrized internally by observation location or thursday-vs-friday epoch, but having different names externally. We may also have multiple observational calendars, parametrized internally and externally by observing body.

We should probably focus on observational calendars that can actually be predicted: if a calendar has a 99% chance of being properly predicted by astronomical calculations, that seems fine, but if a calendar is affected by cloud cover it is probably not good to include.

(In theory, with frequently-updated data, ICU4X could support purely observational calendars that include cloud cover. We don't right now, and neither does ICU4C.)

[justingrant]

The question is: is the astronomical point used by ICU4C (and ICU4X) one that is meaningfully used by many people? If so, can islamic be explicitly defined as that one?

👍 If this can be solved via documentation, that sounds like a much better outcome than removing calendars because many apps likely use a calendar without knowing (or perhaps caring?) that the results are not always accurate. Breaking those would be bad.

Also, once there's a resolution here, I should update my answer to this Stack Overflow question: https://stackoverflow.com/questions/70996839/discrepancy-in-javascript-intl-datetimeformat-outputs-for-the-islamic-hijri

[sffc]

Needs more research, but a potential solution could be:

1. Define in CLDR the Islamic observational calendars that in modern times are reliably approximated by astronomical calculations (not adversely impacted by uncertainties such as cloud cover). It seems that the Egyptian and Saudi observations adhere to this, so we might get islamic-rgsa and islamic-rgeg, for example.
2. If the specified calendar is islamic, we use the region subtag (or -u-rg or -u-sd) to check if we have a calendar with that same observation point. If we do, we use it. Otherwise, we fall back to an implementation-defined default, which will probably be islamic-rgsa.

For example:

Locale	Resolved Calendar	Explanation
und-u-ca-islamic-rgsa	islamic-rgsa	Explicit calendar variant
und-EG-u-ca-islamic	islamic-rgeg	Infer variant from region subtag
und-SA-u-ca-islamic-rg-egzzzz	islamic-rgeg	Infer variant from region extension
und-u-ca-islamic	islamic-rgsa	Use the Saudi Arabia observation as the default

[sffc]

Another thought, which should be discussed with CLDR, is whether the formatting of the calendars should also specify the observation point for additional clarity. For example, instead of saying "1446 AH", maybe it should say "1446 AH (Egypt)". Likewise, islamic-tbla could render as "1446 AH (Tabular)" or something. Whether or not the "(Egypt)" or "(Tabular)" is displayed can be controlled by e.g. the Era Display proposal.

[sffc]

TG2 discussion: https://github.com/tc39/ecma402/blob/main/meetings/notes-2025-03-06.md#consider-unshipping-islamic-and-islamic-rgsa-calendars-971

[hsivonen]

Needs more research, but a potential solution

This gets ahead of things as it assumes two things: 1) that a simulation that won't 100% match sighting addresses use cases satisfactorily (and more satisfactorily than islamic-civil, which is well-defined and can also be expected not to be an exact match for sighting or Umm al-Qura, whose simulation reference location won't match when the sighting location to be matched isn't Mecca) and 2) that the use cases for simulation-based approximation of sighting are in scope for the Web Platform to address.

Regarding the first point about accuracy: Umm al-Qura is an astronomical simulation that has official status and is meant to be suitable for civil use. However, looking at how well the simulation has matched sighting-based important dates in the last 23 Hijri years (I'm ignoring the incomplete last row for 1446 AH), given 5 important dates to predict, three of them were mispredicted 5 times, one was mispredicted 3 times, and one was mispredicted 4 times with insufficient data for the period. It seems safe to say that misprediction for a given date appears to be on the level of 5 out of 23 times.

That's about 78% accuracy, which is way lower than what @sffc suggested for the accuracy of Cairo-reference-point simulation vs. actual sighting in Egypt.

Umm al-Qura is meant for civil use and still (after the adoption of Gregorian calendar to at least some official extent in 2016) appears to have some kind official status unlike some other simulation parameters that CLDR/ICU/TC39 could define, so it makes sense to include Umm al-Qura regardless of how well it has matched sighting. However, simulations defined by CLDR/ICU/TC39 wouldn't have official status to motivate inclusion. They'd need to have better predictive accuracy than islamic-civil and Umm al-Qura (relative to non-Saudi Arabia sightings) to make sense and even then it wouldn't be clear that better would be good enough to satisfy use cases. Would they have accuracy on the level to satisfy use cases?

As for the second point about inclusion criteria:

In countries with established local calendars, like Egypt, people tend to use those.

For what use cases other than determining when holidays take place?

whether they be civil calendars or commonly-followed religious calendars

Does CLDR have explicitly-stated calendar inclusion criteria? At least some non-criteria can be inferred from the exclusion of the Julian calendar, which is exceptionally well known and of exceptional historical interest but is no longer in civil use despite remaining in religious use.

Also, once there's a resolution here, I should update my answer to this Stack Overflow question: https://stackoverflow.com/questions/70996839/discrepancy-in-javascript-intl-datetimeformat-outputs-for-the-islamic-hijri

I observe that this answer gives enough information that if you know that there are multiple Hijri calendar variants, you can figure out which variants islamic-umalqura, islamic-tbla, and islamic-civil are. However, it does not give enough information to figure out what islamic is. It doesn't even say that it's an astronomical simulation.

Another answer there has interesting data about the percentage of days within a 50-Gregorian-year period differing across the ICU4C Hijri calendar variants.

[sffc]

About Umm al-Qura: this is good information; I'd be interested in a similar analysis for Reingold Cairo observations versus whatever the official Egyptian almanac says. (CC @younies: could you link to a place where the official Egyptian observations get published?)

[younies]

Most countries, including Egypt, publish a calendar for each Gregorian year (a couple of months before the end of the year). They use the latest astronomical data to create these calendars.