Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add activeOrDeletedSince parameter to RefreshDnsForAllDomainsAction #2548

Closed
wants to merge 2,483 commits into from

Conversation

jicelhay
Copy link
Collaborator

@jicelhay jicelhay commented Sep 9, 2024

This parameter will be used to enqueue refresh DNS tasks for domains with deletion time >= of its value. This to fix state of deleted domains that have failed calls in the past and are not being picked up by the system.

See b/364282239.


This change is Reviewable

weiminyu and others added 30 commits January 23, 2024 17:07
Already renamed to BsaUnlockableDomain table.
…config presubmits and syncing (google#2292)

* Add some updates to UpdateReservedListCommand to facilitate internal config presubmits and syncing

Added a dry-run tag for presubmit tests

Added early exit behavior when there are no new changes to the list

Added a new --build_environment tag to be used to indicate command runs from build tools. This tag was also added to UpdatePremiumListCommand. Once this new tag is deployed, and break glass behavior is added, these commands will be modified to prevent runs on the command line in the production environment unless the --build_environment or --break_glass flag is used.

* Fix capitalization

* Added in commented out production environment check for buildEnv flag
…gle#2275)

Failures to initialize the tool transaction manager seem to often be caused by
stale local credentials.
TESTED=submitted a GCB job locally and it ran successfully.
TESTED=ran gradle jIFA locally after intentionally mis-formatting a Java
file.
We should not assume org.gradle.java.home to exist on kokoro or GCB.
* Stop saving BSA empty refresh changes

We thought that as a way to verify the refresh job to be running, browsing
the GCS bucket with empty files is easier than quering the DB or go to GCP
logging dashboard, but there are too many of them to be useful.
Also fix a linter warning.
… email (google#2300)

The fallback should only apply on creates, not on updates, otherwise it can
override an existing value for the email address when only the referral email
should be what's updated.

This fixes a bug introduced back in commit in 0ead4f8.

BUG= http://b/322026165
Query size is borderline too-large for the replica.

At 50000, about 2 out of 30 took more than 30 seconds and were retried.
Lower to 40000 and we will keep monitoring the executions.
* Change tld-update to db-object-updater

* rename sync_tlds.sh to sync_db_objects.sh

* Change to configured command name

* Change environment to sandbox explicitly for testing on alpha

* Add remaining object steps and change cloudbuild-tld-sync to cloudbuild-sync-db-objects

* Add build_environment flag

* Change order of command and directory

* Uncomment out reserved list part
… updatePremiumListCommand (google#2317)

* Add check for build_environment flag in updateReservedListCommand

* Do the same for premium list
It looks like the command was renamed at some point to be shorter but then the test class itself was forgotten.
* Add ALLOW_BSA allocation type

Add a new type to allow creation of domains blocked by BSA.
Except for the BSA semantics, the new type behaves exactly
like SINGLE_USE.

* Addressing reviews

* Addressing review
* Add java changes for createBillingCostTransitions

* Add negative cost test

* Remove default value

* remove unused variable

* Add check that create cost and trnasitions map are the same

* inject clock, only use key set when checking for missing fields

* Add test for removing map
This removes a dependency on the App Engine SDK. It also looks like
(from the logs at least) that shutdown hooks registered the old  way stopped
working after the runtime is upgraded to Java 17.

Also removed some random leftover dependencies on the App Engine SKD
that are not needed any more.
Also uses the new Optional.stream() in one class.

Thank you Java 17!
This is the start of a long and low priority migration, but for now I wanted to do a couple of them just to see what it looks like.

This also demonstrates the pattern for use of an @autobuilder to replace an @AutoValue.Builder. See https://github.com/google/auto/blob/main/value/userguide/records.md#builders for full details on that.
)

The Distinct transform removes duplicates based on the serialized format
of the elements. By providing a deterministic coder, we can guarantee
that no duplicates exist.
…ogle#2329)

This makes the callsites look neater, as the work to execute itself is often a
many line lambda, whereas the transaction isolation level is not more than a
couple dozen characters.
Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @jicelhay)


core/src/main/java/google/registry/tools/server/ToolsServerModule.java line 86 at r1 (raw file):

  @Parameter("deletionTime")
  static Optional<DateTime> provideDeletionTime(HttpServletRequest req) {
    DateTimeFormatter formatter = DateTimeFormat.forPattern("MM-dd-yyyy");

We use ISO 8601-formatted date strings globally throughout the codebase, definitely never MM-dd-yyyy. Just use the same mechanism we use for other DateTime querystring paramters: https://cs.opensource.google/nomulus/nomulus/+/master:core/src/main/java/google/registry/request/RequestParameters.java;l=273-290;drc=49d2e34e12eed5ff52671a8dd747fb9c64522405 (and see usages of that method)

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 3 unresolved discussions (waiting on @jicelhay)


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 55 at r1 (raw file):

 * recommended to be somewhere between 200 and 500. The default value is 250.
 *
 * <p>If {@code deletionTime} is passed in the request, this action will enqueue DNS publish tasks

&deletionTime= as a parameter name doesn't quite get at the intent of what this new parameter does (and a naive reading of it would assume that it's only processing deleted domains, which is the opposite of what it's actually doing). I might suggest instead using &activeSince=, as that most properly describes the intent of using this parameter, as in, "I want to refresh the DNS of all domains that were active since January 2023."


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 57 at r1 (raw file):

 * <p>If {@code deletionTime} is passed in the request, this action will enqueue DNS publish tasks
 * on all domains with a deletion time equal or greater than the value provided and there's no
 * guaranteed that they will be active. {@code deletionTime} will be set at 0:00:00 UTC zone for the

s/and there's no guaranteed that they will be active/including domains that have since been deleted/

Code quote:

 * on all domains with a deletion time equal or greater than the value provided and there's no
 * guaranteed that they will be active. {@code deletionTime} will be set at 0:00:00 UTC zone for the

Several jars in our dependencies are now multi-release, including
dnsjava and snakeyaml, and a few more. Such jars include
jvm-version-specific classes that will only be loaded by the vm that can
handle them. All it takes is a new manifest attribute.

This change allows us to upgrade to dnsjava3.6+: the base (java 8) version of
this jar breaks java21. The correct manifest allows java21 to find the
classes it needs.
Copy link
Collaborator Author

@jicelhay jicelhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 3 unresolved discussions (waiting on @CydeWeys)


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 55 at r1 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

&deletionTime= as a parameter name doesn't quite get at the intent of what this new parameter does (and a naive reading of it would assume that it's only processing deleted domains, which is the opposite of what it's actually doing). I might suggest instead using &activeSince=, as that most properly describes the intent of using this parameter, as in, "I want to refresh the DNS of all domains that were active since January 2023."

Fair, but I also think that having a param named activeSince, with the caveat that "there's no guarantee that they will be active" is equally if not more confusing for readers.

What do you think of activeAt? I think that conveys that the domain was active at param value, but might have change status since.


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 57 at r1 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

s/and there's no guaranteed that they will be active/including domains that have since been deleted/

Done.


core/src/main/java/google/registry/tools/server/ToolsServerModule.java line 86 at r1 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

We use ISO 8601-formatted date strings globally throughout the codebase, definitely never MM-dd-yyyy. Just use the same mechanism we use for other DateTime querystring paramters: https://cs.opensource.google/nomulus/nomulus/+/master:core/src/main/java/google/registry/request/RequestParameters.java;l=273-290;drc=49d2e34e12eed5ff52671a8dd747fb9c64522405 (and see usages of that method)

Done.

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @jicelhay)


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 55 at r1 (raw file):

Previously, jicelhay (Juan Celhay) wrote…

Fair, but I also think that having a param named activeSince, with the caveat that "there's no guarantee that they will be active" is equally if not more confusing for readers.

What do you think of activeAt? I think that conveys that the domain was active at param value, but might have change status since.

So I see where you're coming from, but activeAt is also not quite right, as it seems to imply that a domain was active at a given date in order to be refreshed, but this action will be refreshing domains that were created after that time (and thus were not active then). What we really want is wording that captures the meaning of "active at any time since X". Or alternatively, notDeletedBefore? That's accurate, right? A little strange to phrase the parameter negatively, but if that's the most accurate wording possible, then ...

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @jicelhay)


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 55 at r1 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

So I see where you're coming from, but activeAt is also not quite right, as it seems to imply that a domain was active at a given date in order to be refreshed, but this action will be refreshing domains that were created after that time (and thus were not active then). What we really want is wording that captures the meaning of "active at any time since X". Or alternatively, notDeletedBefore? That's accurate, right? A little strange to phrase the parameter negatively, but if that's the most accurate wording possible, then ...

activeOrDeletedSince is accurate too, but wordier.

Copy link
Collaborator Author

@jicelhay jicelhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 4 files reviewed, 1 unresolved discussion (waiting on @CydeWeys)


core/src/main/java/google/registry/tools/server/RefreshDnsForAllDomainsAction.java line 55 at r1 (raw file):

Previously, CydeWeys (Ben McIlwain) wrote…

activeOrDeletedSince is accurate too, but wordier.

I prefer activeOrDeletedSince. notDeletedBefore might be confusing as well because readers might assume that it's deleted after that, which we know is not true.

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need to update the commit description now with the new param name.

Reviewable status: 0 of 4 files reviewed, all discussions resolved

@jicelhay jicelhay changed the title Add deletionTime parameter to RefreshDnsForAllDomainsAction Add activeOrDeletedSince parameter to RefreshDnsForAllDomainsAction Sep 12, 2024
Copy link
Collaborator Author

@jicelhay jicelhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Reviewable status: 0 of 4 files reviewed, all discussions resolved (waiting on @CydeWeys)

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 0 of 4 files reviewed, all discussions resolved

Copy link
Member

@CydeWeys CydeWeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r3, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @jicelhay)

@jicelhay jicelhay closed this Sep 13, 2024
auto-merge was automatically disabled September 13, 2024 17:28

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants