Skip to content

Test Plan

Jump to bottom
Elliotte Rusty Harold edited this page Nov 19, 2019 · 65 revisions

What follows are the steps we follow to qualify a build for release. Generally these will be performed after the plugin has been signed and uploaded to the version specific bucket in GCS; but before the Lorry entry has been approved.

Ideally, these steps should be performed by a different person than the one who cut the release, but that is not required.

If no problems are encountered when following these steps, approve the Lorry entry. (If you also cut the release, you have to ask a different team member to approve the Lorry entry.)

If regressions are encountered, file and fix the relevant bugs. Then cut a new release with a new version. If you discover existing unknown issues (not regressions) these are usually not release blockers, but that needs to be evaluated on a case-by-case basis.

Particular problems to look for are noted below. However, this is far from an exhaustive list. If you find anything suspicious, file a bug and alert the team. Feel free to look for additional problems not covered here, especially around any features that have been added or refactored since the last release.

Prerequisites

  1. Create a GCP project in the Cloud Console. Note the account you signed in with and the project ID. You'll need both of these later. TBD: check with IntelliJ team to see if they have a standard sample project we can use for testing. If not, figure out if we can create one.

  2. If necessary, create an App Engine App in this project. This link takes you to the application creation flow. TBD: link to instructions

Install

  1. Open a clean instance of Eclipse; it's recommended to download a new version from download.eclipse.org. This should not be the same instance you use for day-to-day development.

  2. Switch to a new empty workspace (File > Switch Workspace > Other...), unless you have started with a new, empty workspace in step 1.

  3. Install the plugin from the public versioned API such as "https://storage.googleapis.com/cloud-tools-for-eclipse/version/".

  4. Verify If a warning about unsigned content pops up, make sure none of the jars it's warning about are our first party jars. For example, it's OK if Eclipse warns you that com.google.guava is unsigned. However it's not OK if Eclipse warns you that com.google.cloud.tools.eclipse.util or com.google.cloud.tools.appengine is unsigned.

    Note: this error is a problem with the build process, not the code. Thus it does not require you to file a bug. However it must be fixed before the release can proceed.

  5. Restart Eclipse.

Branding

  1. The Eclipse Welcome view should open to the Overview page.
    1. Click on the Cloud Tools for Eclipse entry in the Overview page. Verify it takes you to the CT4E Product Overview webpage.
    2. Click on the Home button in the Welcome toolbar and select Tutorials. Ensure Cloud Tools for Eclipse section is shown with examples for App Engine Standard Environment, App Engine Flexible Environment, and Cloud Dataflow. Verify each entry takes you to the corresponding CT4E Creating an XXX webpage.
    3. Click on the Home button in the Welcome toolbar and select What's New. Click on the Cloud Tools for Eclipse entry. Verify it takes you to the CT4E Release Notes webpage.
    4. Click on the Home button in the Welcome toolbar and select Migrate. (If there does not exist Migrate, click on the Customize toolbar button to add the Migrate root page.) Click on the Migration from the Google Plugin for Eclipse (GPE) entry. Verify it takes you to the CT4E Migrating from the Google Plugin for Eclipse webpage.
  2. Branding: open the About dialog and ensure the GCP feature is shown (see #1231 for an example).

Login

  1. Verify You should see a small rainbow colored G in the lower right hand side of the Eclipse window. If not, the plugin may not be installed properly.

  2. Click the G.

  3. A pop-up should appear in the center of the screen. If any accounts are listed, click the "Sign Out..." button.

  4. Click the G again. Your web browser should open showing a list of accounts. Login with whichever one you want to test deployment later. Follow the flow and you should see "Received verification code. You may now close this window...". Close the window.

Create a New Project and Run it Locally

  1. Back in Eclipse, use the wizard to create a new Google App Engine Standard Java project.

  2. On the second screen of the wizard add the App Engine API library.

  3. Verify there are no build errors in the project, and no warnings except possibly a couple about serialVersionUID.

  4. In HelloAppEngine.java change

    response.getWriter().print("Hello App Engine!\r\n");

    to

    LogServiceException ex = new LogServiceException(" _Today's date_");
response.getWriter().print("Hello App Engine on " + ex.getMessage() + "\r\n");

    Use the quickfix to import LogServiceException and save the file.

  5. Right click the project in the Package Explorer and select Run As > App Engine from the context menu.

  6. A web browser (possibly internal to Eclipse) should open up and show you the Hello World HTML page. Click the link to "The servlet".

  7. Verify that you see "Hello Today's date!"

  8. Open the Servers pane in Eclipse.

  9. Select the running server and click the red square (Stop).

  10. Reload the servlet in the browser and verify that you can no longer connect to the server.

  11. Set a breakpoint in the servlet.

  12. Right click the project in the Package Explorer and select Debug As > App Engine from the context menu.

  13. A web browser (possibly internal to Eclipse) should open up and show you the Hello World HTML page.

  14. Click the link to "The servlet".

  15. Verify that the debug perspective opens and stops at the breakpoint.

  16. Single step through to the end of the servlet.

  17. Verify that you see "Hello Today's date!"

  18. Open the Servers pane in Eclipse.

  19. Select the running server and click the red square (Stop).

Deploy

Sometimes deployment can fail for reasons that have nothing to do with our code. For example, if you log in with a different account than the one that owns the test GCP project, or you enter an incorrect project ID. If you need to correct these things to complete a deployment, that's fine. However, do not qualify the release until you have successfully deployed a project.

  1. In the Eclipse Package Explorer, select your project and activate the context menu.

  2. Select Deploy to App Engine Standard...

  3. Verify that the account you logged in with earlier is already selected.

  4. Select the project ID you previously registered in the Cloud Console from the list.

  5. Click Deploy.

  6. Verify Watch the output in the Eclipse Console view. Make sure there are no obvious problems. You should see a line like "Deployed service [default] to [https://projectname.appspot.com]" near the end of the output.

  7. Verify https://projectname.appspot.com should open in a browser, possibly the Eclipse internal browser. Check that you see a link to the servlet.

  8. Verify Click the link to the servlet. Make sure you see today's date. (i.e. that you're not just looking at an older version of the app.) On occasion, you may need to Shift-F5 to reload the page and/or wait a few minutes (< 10) for App Engine to start serving the new version of the page.)

Maven

Go back to Create a New Project and Run it Locally and repeat the steps from that point forward, only this time checking the "Create as Maven project" in the wizard.

Servlet 2.5

Go back to Create a New Project and Run it Locally and repeat the steps from that point forward, only this time using servlet 2.5.

App Engine Flexible

  1. Back in Eclipse, use the wizard to create a new Google App Engine Flexible Java project.

  2. Verify there are no build errors in the project, and no warnings except possibly a couple about serialVersionUID.

  3. In HelloAppEngine.java change

    response.getWriter().print("Hello App Engine!\r\n");

    to

    response.getWriter().print("Hello _Today's date_!\r\n");

    and save the file.

  4. In the Eclipse Package Explorer, select your project and activate the context menu.

  5. Select Deploy to App Engine Flexible...

  6. Verify that the account you logged in with earlier is already selected.

  7. Select the project ID you previously registered in the Cloud Console from the list.

  8. Click Deploy.

  9. Verify Watch the output in the Eclipse Console view. Make sure there are no obvious problems. You should see a line like "Deployed service [default] to [https://projectname.appspot.com]" near the end of the output.

  10. Verify https://projectname.appspot.com should open in a browser, possibly the Eclipse internal browser. Check that you see a link to the servlet.

  11. Verify Click the link to the servlet. Make sure you see today's date. (i.e. that you're not just looking at an older version of the app.) On occasion, you may need to Shift-F5 to reload the page and/or wait a few minutes (< 10) for App Engine to start serving the new version of the page.)

  12. Important Disable the flex application in the cloud console to avoid continuing billing charges which can add up to a few hundred dollars a month for a hello world app.

Dataflow

  1. File > New > Other... > Google Cloud Platform > Google Cloud Dataflow Java Program

  2. Enter com.example for Group ID: and some value for Artifact ID:. Verify Finish is enabled.

  3. Click Next >.

  4. Select an Account:.

  5. Select a valid, Dataflow-enabled project ID. Click on the small down arrow of the staging location field to verify existing buckets have been populated into the combo.

  6. Enter or select a valid staging location. Verify the info message "Verified bucket ... is accessible." and Finish is enabled.

  7. Click Finish.

  8. Run > Run Configurations.... Select Dataflow Pipeline and click the new icon to create a new run configuration.

  9. In the Main tab, select the project you created. Select or enter com.example.StarterPipeline in Main class:.

  10. Click Run. Verify the Console window shows INFO: WORLD and INFO: HELLO.

  11. Run > Run Configurations.... Select the configuration you created earlier, and select the Pipeline Arguments tab.

  12. Select DataflowRunner. Wait for a second, and verify Pipeline Options change to list many more options.

  13. Uncheck Use Default Dataflow options. This should enable all the fields under the checkbox. (FYI, all the fields under the checkbox are supposed to be disabled when the checkbox is ticked, but due to a bug, you might see some of them are enabled beforehand.)

  14. Select an account and a project ID.

  15. Enter <a unique, non-existing bucket name to create>/subfolder in the staging location. You'll see an error message "Cloud not fetch bucket ...". Verify Create Bucket is enabled.

  16. Click Create Bucket. The dialog should show a message that the bucket is created.

  17. Click Apply and then Run.

  18. Verify the Console window shows logs about staging/uploading files and the Developers Console URL.

  19. Cleanup: Delete the bucket you created through the Developers Console after finishing the test plan.

GCP Status

  1. Verify the GCP status widget setting matches that in the GCP menu item.
  2. Click on the GCP status menu item to open the status.cloud.google.com page. Verify the status matches.
  3. Disconnect from the network and select the GCP status menu item.
    • The status menu item should show a gray sign.
    • There should be a status page load failed error dialog.
    • The status hover / GCP menu item should show Cannot connect to Google Cloud Platform
  4. Reconnect and wait approximately 30 seconds and the status widget should reflect the GCP status again.

Error Log

Verify Open the Error log (not the same thing as the problems view) and verify that no exceptions have been thrown at any point during this process.

End User Documentation

Contributor Docs

Clone this wiki locally