Skip to content

Troubleshooting synthetics scripts

Jump to bottom
rpatlte edited this page Dec 12, 2019 · 2 revisions

Recording works correctly, but playback hangs and times out at some point. Why?

By default, recorded output uses selenium’s “implicit waits”. This is setup in the configureDriver function that’s output from the recorder, and can be tweaked after recording.

For example, if I configure an implicit wait of 5 seconds, and the element is not found, webdriver will sit at that step for 5 seconds before considering the test failed.

Mouseover events

Configured timeout is too low

  • Problem: the element you’re trying to select takes 10 seconds to appear, but you’ve configured an implicit wait of 5 seconds.
  • To fix:
    • Option A - Increase the implicit wait timeout
    • Option B - Use an explicit wait with a longer timeout

Randomized id/attributes

  • Problem: sometimes the CSS output in the script will depend on data that is randomized on each page load.
  • Example: if the ID of my element is random on each page load, the findElement command output from the recorder that depends on this ID will fail on subsequent runs.
  • To fix:
    • Modify the selector output from the recorder so that it will select based on something that’s consistent between page loads.

Playback Speed

  • Problem: the playback does not (currently) match the speed you recorded at, some things can be time sensitive.
  • Example: say I have an application where clicking a button doesn’t do anything until some backend data is fetched. Usually this data is fetched in under 200ms, so in practice the data is almost always loaded by the time a human user would be able to click it. However, when you play in the recorder without any waits, it will attempt to go as fast as possible, so will likely attempt to click prior to the button actually working.
  • To fix:
    • Option A - Configure implicit waits with enough time (done by default by recorded output)
    • Option B - Use explicit wait commands to wait for the data to be loaded
    • Option C - Use a sleep command with a specific duration, e.g., await driver.sleep(5 * 1000); // waits for 5 seconds

Animation Classes

  • Problem: sometimes the CSS output in the script will depend on animation classes, which can be time sensitive
  • Example: I click a button in the middle of it fading in. The recorder outputs some click(By.css(‘.fade-in’)). When I playback the click is not attempted until after the button is faded in, so that class does not exist.
  • To fix:
    • Inspect the element using chrome devtools and find a unique CSS identifier you can use in place of the animation class.

Element Obscured/Clicking Element Center

  • Problem: the click command does not work if the spot to be clicked is covered by another element. The click command also will always attempt to click the target element at its center point.
  • Example: when recording there’s a button that’s mostly covered, but whose corner is visible. I click on the corner of the button. When playing back the script, even though part of the button is visible, the click attempts to click the center and fails, due to it being covered.
  • To fix:
    • Option A - Usually it’s possible to work around this by trying to record again and clicking a different element (for example, if there is a button composed of multiple elements, try to click in a different area).
    • Option B - Try to close/move the obscuring element prior to clicking. For example, if there’s a modal, dismiss this prior to proceeding with the click.
    • Option C - It’s possible to write a custom command to “click” in a specific spot on the button (https://github.com/thousandeyes/transaction-scripting-examples/blob/master/examples/clickSpecificPosition.js)

Global/Localization problems

  • Problem: sometimes the content of an application will change based on locale, which can cause certain selectors to fail
  • Example: my script selects a link based on its text. This works fine in the US. However, an agent running in Japan gets routed to the Japanese localization of the website, so the text no longer matches.
  • To fix:
    • Modify the script so that it will select based on something that’s consistent between different locales. Usually non-user visible attributes, such as class names, will be consistent.

My script works in the recorder, but fails in the app. Why?

Timeout Exceeded The test settings defaults to a 30 second test timeout. For larger transactions, you will usually need to increase this

User Agent The default curl user agent we use for HTTP tests is rejected by many larger sites. You should supply a chrome user agent or similar so that all layers use the same UA.

Agent Location Scripts that work fine locally may fail on agents in other regions, as they may be routed differently, land on different sites etc. Make sure you rule out any location specific issues.

Agent Network Conditions

  • The time that network requests take can have an impact on whether your script works or not. For example, if you have a driver.sleep command that waits for 5 seconds, that may work for you but not be sufficient for an agent in some far off rural area.
  • It’s best to not use hard coded sleep values, but instead use waits with sufficient timeouts

Window size The agent may run the script at a different resolution than when you tested it. It’s a good practice to always set an explicit resolution for your script (the recorder will output this automatically).

My script used to work, but now it doesn’t. Why?

Time Sensitive/Stateful applications

  • Problem: sometimes the content of an application will change over time, so a script that worked last week may fail today.
  • Examples
    • Delta airlines has seats available for a flight next week. Those seats sell out after a few days, so my script no longer works.
    • Amazon has a new sale going on, so the front page contains different items. Now my script that depended on the old item ids is no longer working.
  • To fix:
    • Modify the script so that it will select based on something that’s consistent between page loads.
    • Use conditional logic to ensure edge cases are handled in the event an item is sold out, etc.

Application code is changed

  • Problem: sometimes a development change to your application may cause your selectors to stop working.
  • Examples
    • My company just went through a redesign of our corporate page, and now the corporate page transaction script no longer works.
    • I have a selector .someClass that used to match
      . A developer made a code change to that element so it now shows as
      .
    • I have a selector span > div that used to match the first div nested in a span element. The developer later changes the span to a div. Now I need to use div > div to get the same effect
  • To fix:
    • Update the broken selector to a new, working one. Follow best practices so that this new selector is less likely to break upon further development changes.
Clone this wiki locally