Prober circular list (#218)

* Fixed changes suggested by CydeWeys * Fixed Javadoc and comments for ActionHandler * Fixed comments and JavaDoc on other files * EOL added * Removed Unnecessary Files * fixed .gradle files styles * Removed outbound message from ActionHandler's fields and renamed Marker Interfaces * Fixed javadoc for Marker Interfaced * Modified Comments on ActionHandler * Removed LocalAddress from Protocol * Fixed Travis Build Issues * Rebased to Master and added in modified Handlers and ProbingAction * Added missing license headers and JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor Style Fix * Full WebWhoIs Sequence Added * fixed build issues * Refactored by responses suggested by jianglai. * Minor Style Fixes * Updated build.gradle file * Modified license header dates * Updated WebWhois tests. * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * SpotlessApply run to fix style issues * Added license header and newline where appropriate. * Javadoc style fix in tests and removed unused methods * Refactored ProbingAction to minimize number of unnecessary methods * Initial Commit. * Initial Commit. * Deleted unfinished features. Added ActionHandler and its Unit Tests. * Included prober subproject in settings.gradle * Added Protocol Class and its Basic Unit Tests * Added Protocol Class and its Basic Unit Tests * Added Changes Suggested by jianglai * Fixed Gitignore to take out AutoValue generated code * Fixed Gitignore to take out AutoValue generated code * Removed AutoValue java files * Added gitignore within prober * Removed all generated java * Final Changes in .gitignore * Final Changes in .gitignore * Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class * Fixed build.gradle changes requested * Removed Files irrelevant to current pull request * Fixed changes suggested by CydeWeys * Fixed changes suggested by CydeWeys * Fixed changes suggested by CydeWeys * Fixed changes suggested by CydeWeys * Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle * Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle * Fully Updated ActionHandler (missing updated JavaDoc) * Added changed Protocol and both Inbound and Outbound Markers * Removed AutoVaue ignore clause from .gitignore * Removed AutoVaue ignore clause from .gitignore * removed unneccessary dependencies in build.gradle * Fixed Javadoc and comments for ActionHandler * Fixed comments and JavaDoc on other files * EOL added * Removed Unnecessary Files * fixed .gradle files styles * Removed outbound message from ActionHandler's fields and renamed Marker Interfaces * Fixed javadoc for Marker Interfaced * Modified Comments on ActionHandler * Removed LocalAddress from Protocol * Fixed Travis Build Issues * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor Style Fix * Minor Style Fix * Minor Style Fix * Minor Style Fix * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * fixed build issues * fixed build issues * fixed build issues * fixed build issues * fixed build issues * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Minor Style Fixes * Minor Style Fixes * Minor Style Fixes * Minor Style Fixes * Minor Style Fixes * Updated build.gradle file * Updated build.gradle file * Updated build.gradle file * Updated build.gradle file * Modified license header dates * Modified license header dates * Modified license header dates * Modified license header dates * Modified license header dates * Updated WebWhois tests. * Updated WebWhois tests. * Updated WebWhois tests. * Updated WebWhois tests. * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * SpotlessApply run to fix style issues * SpotlessApply run to fix style issues * SpotlessApply run to fix style issues * SpotlessApply run to fix style issues * SpotlessApply run to fix style issues * Added license header and newline where appropriate. * Added license header and newline where appropriate. * Added license header and newline where appropriate. * Added license header and newline where appropriate. * Added license header and newline where appropriate. * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Refactored ProbingAction to minimize number of unnecessary methods * Refactored ProbingAction to minimize number of unnecessary methods * Refactored ProbingAction to minimize number of unnecessary methods * Refactored ProbingAction to minimize number of unnecessary methods * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Removed TestProvider from TestUtils. * Removed TestProvider from TestUtils. * Removed TestProvider from TestUtils. * Removed TestProvider from TestUtils. * Removed TestProvider from TestUtils. * Rebased to master * Rebased to master * Updated issues in rebasing * Updated issues in rebasing * Minor style change on prober/build.gradle * Minor style change on prober/build.gradle * Fixed warnings for java compilation * Fixed warnings for java compilation * Fixed files to pass all style tests * Fixed files to pass all style tests * Fixed files to pass all style tests * Minor syle fixes after succesful rebase onto master * Fixed changes suggested by CydeWeys * Fixed changes suggested by CydeWeys * Fixed changes suggested by CydeWeys * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Rebased to Master and added in modified Handlers and ProbingAction * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Added missing license headers and JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor fix in NewChannelAction JavaDoc * Minor Style Fix * Minor Style Fix * Minor Style Fix * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * Full WebWhoIs Sequence Added * fixed build issues * fixed build issues * fixed build issues * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Refactored by responses suggested by jianglai. * Minor Style Fixes * Minor Style Fixes * Updated build.gradle file * Updated build.gradle file * Updated build.gradle file * Modified license header dates * Modified license header dates * Modified license header dates * Updated WebWhois tests. * Updated WebWhois tests. * Updated WebWhois tests. * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * Refactored WebWhois to accomodate jianglai's suggested changes and modified tests to reflect this refactoring * SpotlessApply run to fix style issues * Added circular linked list to utils * Added circular linked list to utils * Added circular linked list to utils * Added circular linked list to utils * Added circular linked list to utils * License Header added * License Header added * License Header added * License Header added * License Header added * Added license header and newline where appropriate. * Added license header and newline where appropriate. * Refactored probing sequence to be circular linked list iterator * Refactored probing sequence to be circular linked list iterator * Refactored probing sequence to be circular linked list iterator * Refactored probing sequence to be circular linked list iterator * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Javadoc style fix in tests and removed unused methods * Modified ProbingStep tests to reflect new ProbingStep structure. * Modified ProbingStep tests to reflect new ProbingStep structure. * Refactored ProbingAction to minimize number of unnecessary methods * Refactored ProbingAction to minimize number of unnecessary methods * Refactored ProbingAction to minimize number of unnecessary methods * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Modified tests for WebWhois according to changes suggested by laijiang. * Removed TestProvider from TestUtils. * Removed TestProvider from TestUtils. * Rebased to master * ProbingStepTest modified to have fewer unnecessary helper methods * ProbingStepTest modified to have fewer unnecessary helper methods * Updated issues in rebasing * Updated issues in rebasing * Added missing license header to DefaultCircularLinkedListIterator * Fixed max column length to be 100 * Fixed max column length to be 100 * Minor changes to pass style tests * Successful rebase onto finished web-whois branch * Removed need for TestTokens with Mockito mocks of Tokens * Fixed style issues in DefaultCircularLinkedListIterator and AbstractCircularLinkedListIterator * Modified CircularList according to changes suggested by jianglai. * Merge branch 'master' into prober-circular-list * Modified ProbingSequenceTest to not expect unnecessary NullPointerException * ProbingSequence and tests modified to reflect addition of UnrecoverableStateException and restarts on failures * Modified ProbingSequence and its tests to reflect action generation and calling being put in the same try catch block
google · Aug 12, 2019 · df5f450 · df5f450
1 parent 89a44f1
commit df5f450
Show file tree

Hide file tree

Showing 10 changed files with 683 additions and 267 deletions.
diff --git a/prober/src/main/java/google/registry/monitoring/blackbox/ProbingAction.java b/prober/src/main/java/google/registry/monitoring/blackbox/ProbingAction.java
@@ -135,12 +135,12 @@ private static void addHandlers(
    * when the connection future is successful.</p>
    *
    * <p>Once the connection is successful, we establish which of the handlers in the pipeline is
-   * the {@link ActionHandler}.From that, we can obtain a future that is marked as a success when
+   * the {@link ActionHandler}.From that, we can obtain a future that is marked as a success when we
-   * we receive an expected response from the server.</p>
+   * receive an expected response from the server.</p>
    *
    * <p>Next, we set a timer set to a specified delay. After the delay has passed, we send the
-   * {@code outboundMessage} down the channel pipeline, and when we observe a success or failure,
+   * {@code outboundMessage} down the channel pipeline, and when we observe a success or failure, we
-   * we inform the {@link ProbingStep} of this.</p>
+   * inform the {@link ProbingStep} of this.</p>
    *
    * @return {@link ChannelFuture} that denotes when the action has been successfully performed.
    */

diff --git a/prober/src/main/java/google/registry/monitoring/blackbox/ProbingSequence.java b/prober/src/main/java/google/registry/monitoring/blackbox/ProbingSequence.java
@@ -14,14 +14,25 @@
 
 package google.registry.monitoring.blackbox;
 
+import com.google.common.flogger.FluentLogger;
+import google.registry.monitoring.blackbox.exceptions.UnrecoverableStateException;
 import google.registry.monitoring.blackbox.tokens.Token;
+import google.registry.util.CircularList;
 import io.netty.bootstrap.Bootstrap;
 import io.netty.channel.AbstractChannel;
+import io.netty.channel.ChannelFuture;
 import io.netty.channel.EventLoopGroup;
 
 /**
  * Represents Sequence of {@link ProbingStep}s that the Prober performs in order.
  *
+ * <p>Inherits from {@link CircularList}, with element type of
+ * {@link ProbingStep} as the manner in which the sequence is carried out is similar to the {@link
+ * CircularList}. However, the {@link Builder} of {@link ProbingSequence} override
+ * {@link CircularList.AbstractBuilder} allowing for more customized flows, that are looped, but
+ * not necessarily entirely circular.
+ * Example: first -> second -> third -> fourth -> second -> third -> fourth -> second -> ...
+ * </p>
  *
  * <p>Created with {@link Builder} where we specify {@link EventLoopGroup}, {@link AbstractChannel}
  * class type, then sequentially add in the {@link ProbingStep.Builder}s in order and mark which one
@@ -31,78 +42,202 @@
  * the first one is activated with the requisite {@link Token}, the {@link ProbingStep}s do the rest
  * of the work.</p>
  */
-public class ProbingSequence {
+public class ProbingSequence extends CircularList<ProbingStep> {
 
-  private ProbingStep firstStep;
+  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
 
   /**
    * Each {@link ProbingSequence} requires a start token to begin running.
    */
   private Token startToken;
 
-  private ProbingSequence(ProbingStep firstStep, Token startToken) {
+  /**
-    this.firstStep = firstStep;
+   * Each {@link ProbingSequence} is considered to not be the last step unless specified by the
+   * {@link Builder}.
+   */
+  private boolean lastStep = false;
+
+  /**
+   * {@link ProbingSequence} object that represents first step in the sequence.
+   */
+  private ProbingSequence first;
+
+
+  /**
+   * Standard constructor for {@link ProbingSequence} in the list that assigns value and token.
+   */
+  private ProbingSequence(ProbingStep value, Token startToken) {
+    super(value);
     this.startToken = startToken;
   }
 
+  /**
+   * Method used in {@link Builder} to mark the last step in the sequence.
+   */
+  private void markLast() {
+    lastStep = true;
+  }
+
+  /**
+   * Obtains next {@link ProbingSequence} in sequence instead of next {@link CircularList}.
+   */
+  @Override
+  public ProbingSequence next() {
+    return (ProbingSequence) super.next();
+  }
+
+  /**
+   * Starts ProbingSequence by calling first {@code runStep} with {@code startToken}.
+   */
   public void start() {
-    // calls the first step with startToken;
+    runStep(startToken);
-    firstStep.accept(startToken);
+  }
+
+  /**
+   * Generates new {@link ProbingAction} from {@link ProbingStep}, calls the action, then retrieves
+   * the result of the action.
+   *
+   * @param token - used to generate the {@link ProbingAction} by calling {@code
+   * get().generateAction}.
+   *
+   * <p>Calls {@code runNextStep} to have next {@link ProbingSequence} call {@code runStep}
+   * with next token depending on if the current step is the last one in the sequence.
+   *
+   * <p>If unable to generate the action, or the calling the action results in an immediate error,
+   * we note an error. Otherwise, if the future marked as finished when the action is completed is
+   * marked as a success, we note a success. Otherwise, if the cause of failure will either be a
+   * failure or error. </p>
+   */
+  private void runStep(Token token) {
+    ProbingAction currentAction;
+    ChannelFuture future;
+
+    try {
+      // Attempt to generate new action. On error, move on to next step.
+      currentAction = get().generateAction(token);
+
+      // Call the generated action.
+      future = currentAction.call();
+
+    } catch (UnrecoverableStateException e) {
+      // On an UnrecoverableStateException, terminate the sequence.
+      logger.atSevere().withCause(e).log(
+          "Unrecoverable error in generating or calling action.");
+      return;
+
+    } catch (Exception e) {
+      // On any other type of error, restart the sequence at the very first step.
+      logger.atWarning().withCause(e).log("Error in generating or calling action.");
+
+      // Restart the sequence at the very first step.
+      restartSequence();
+      return;
+    }
+
+    future.addListener(f -> {
+      if (f.isSuccess()) {
+        // On a successful result, we log as a successful step, and note a success.
+        logger.atInfo().log(String.format("Successfully completed Probing Step: %s", this));
+
+      } else {
+        // On a failed result, we log the failure and note either a failure or error.
+        logger.atSevere().withCause(f.cause()).log("Did not result in future success");
+
+        // If not unrecoverable, we restart the sequence.
+        if (!(f.cause() instanceof UnrecoverableStateException)) {
+          restartSequence();
+        }
+        // Otherwise, we just terminate the full sequence.
+        return;
+      }
+
+      if (get().protocol().persistentConnection()) {
+        // If the connection is persistent, we store the channel in the token.
+        token.setChannel(currentAction.channel());
+      }
+
+      //Calls next runStep
+      runNextStep(token);
+
+    });
+  }
+
+  /**
+   * Helper method to first generate the next token, then call runStep on the next {@link
+   * ProbingSequence}.
+   */
+  private void runNextStep(Token token) {
+    token = lastStep ? token.next() : token;
+    next().runStep(token);
+  }
+
+  /**
+   * Helper method to restart the sequence at the very first step, with a channel-less {@link
+   * Token}.
+   */
+  private void restartSequence() {
+    // Gets next possible token to insure no replicated domains used.
+    Token restartToken = startToken.next();
+
+    // Makes sure channel from original token isn't passed down.
+    restartToken.setChannel(null);
+
+    // Runs the very first step with starting token.
+    first.runStep(restartToken);
   }
 
   /**
    * Turns {@link ProbingStep.Builder}s into fully self-dependent sequence with supplied {@link
    * Bootstrap}.
    */
-  public static class Builder {
+  public static class Builder extends CircularList.AbstractBuilder<ProbingStep, ProbingSequence> {
 
-    private ProbingStep currentStep;
+    private ProbingSequence firstRepeatedSequenceStep;
-    private ProbingStep firstStep;
-    private ProbingStep firstRepeatedStep;
 
     private Token startToken;
 
+    /**
+     * This Builder must also be supplied with a {@link Token} to construct a {@link
+     * ProbingSequence}.
+     */
     public Builder(Token startToken) {
       this.startToken = startToken;
     }
 
     /**
-     * Adds {@link ProbingStep}, which is supplied with {@link Bootstrap}, built, and pointed to by
+     * We take special note of the first repeated step.
-     * the previous {@link ProbingStep} added.
      */
-    public Builder addStep(ProbingStep step) {
+    public Builder markFirstRepeated() {
+      firstRepeatedSequenceStep = current;
+      return this;
+    }
 
-      if (currentStep == null) {
+    @Override
-        firstStep = step;
+    public Builder add(ProbingStep value) {
-      } else {
+      super.add(value);
-        currentStep.nextStep(step);
+      current.first = first;
-      }
 
-      currentStep = step;
       return this;
     }
 
-    /**
+    @Override
-     * We take special note of the first repeated step.
+    protected ProbingSequence create(ProbingStep value) {
-     */
+      return new ProbingSequence(value, startToken);
-    public Builder markFirstRepeated() {
-      firstRepeatedStep = currentStep;
-      return this;
     }
 
     /**
-     * Points last {@link ProbingStep} to the {@code firstRepeatedStep} and calls private
+     * Points last {@link ProbingStep} to the {@code firstRepeatedSequenceStep} and calls private
      * constructor to create {@link ProbingSequence}.
      */
+    @Override
     public ProbingSequence build() {
-      if (firstRepeatedStep == null) {
+      if (firstRepeatedSequenceStep == null) {
-        firstRepeatedStep = firstStep;
+        firstRepeatedSequenceStep = first;
       }
 
-      currentStep.nextStep(firstRepeatedStep);
+      current.markLast();
-      currentStep.lastStep();
+      current.setNext(firstRepeatedSequenceStep);
-      return new ProbingSequence(this.firstStep, this.startToken);
+      return first;
     }
   }
 }
-