Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cy.route() unable to mock same url multiple times if requests happen quickly #4460

Closed
bahalperin opened this issue Jun 13, 2019 · 67 comments · Fixed by #14513
Closed

cy.route() unable to mock same url multiple times if requests happen quickly #4460

bahalperin opened this issue Jun 13, 2019 · 67 comments · Fixed by #14513

Comments

@bahalperin
Copy link

bahalperin commented Jun 13, 2019

Current behavior:

If you make multiple requests to the same endpoint and want to mock each response differently based on the order that they occur, cypress appears to be unable to do it unless you wait X milliseconds between requests. If there is no pause between requests, cypress will never catch the first cy.wait(...) and will mock every request to that endpoint with the same response.

Desired behavior:

cy.server()
cy.route(url, res1).as(get)
cy.wait(@get)
cy.route(url, res2).as(get)

Should result in the first GET request to url responding with res1 and the second GET request to url responding with res2, regardless of the amount of time between requests.

Steps to reproduce: (app code and test code)

Run tests here

@jennifer-shehane
Copy link
Member

jennifer-shehane commented Jul 18, 2019

I can recreate this behavior in Cypress 3.4.0 with the tests here.

In the example provided, the application does a request every 200ms, and if you want the response to change for each response - like id: 1, id: 2, id: 3, it'll always respond with the first cy.route() response definition, even though you want it to respond with the 2nd at that point. It's almost as if we want to have a waitOnce.

Screen Shot 2019-07-18 at 4 13 14 PM

@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Jul 18, 2019
@mpvosseller
Copy link

Same. Need a way to specify the response of the Nth request.

@nijk

This comment has been minimized.

@ohelixa

This comment has been minimized.

@avilaj

This comment has been minimized.

@bautistaaa
Copy link

bautistaaa commented Aug 8, 2019

I came up with a really odd way thats solved a similar problem but not quite the same.

In the context of cypress we append an intent queryString so cypress will deem it as different. Note: Depending on how you are triggering these calls will also determine if you can use this.

export function createCypressUrlOrDefault(url, intent) {
  if (!url) {
    throw Error('Url must be supplied!');
  }
  if (!intent) {
    throw Error('A value must be supplied for intent for cypress stubbing!');
  }

  if (root.Cypress) {
    return `${url}?${intent}`;
  }

  return url;
}

@mpvosseller
Copy link

@bautistaaa @avilaj I considered that too but didn't really want to have the client do different things in test mode. Another option would be to add an artificial delay in the client code (between the requests). Again though that requires special code to run in the client for test mode only which is less than ideal.

@lifeiscontent
Copy link

@jennifer-shehane any followup on this? Mocking GraphQL calls is near impossible without this.

@btferus5
Copy link

btferus5 commented Oct 3, 2019

@lifeiscontent
one way to solve this using Apollo is to change your URL to include the queryname as a param. It allowed me to track requests in Cypress like

cy.route('POST', '**?q=getThing**)

@aaron-peloquin
Copy link

@lifeiscontent if you can add a dummy ?q=${operationName} to all of the calls made by Apollo, you could alias that in Apollo

@lifeiscontent
Copy link

@aaron-peloquin @btferus5 sure, but that's clearly a hack, I'd expect cypress to have support for this, not having to hack the library, then later once they've figured it out, undo all the hacks.

@btferus5
Copy link

btferus5 commented Oct 3, 2019

@lifeiscontent no need to hack the library, it's a built in option using context.

It also gives you better visibility in and outside of Cypress just debugging issues and tracking network requests.

I also don't see how cypress would implement an alternative way to track requests that all share the same URL

@nvkhanh08t4

This comment has been minimized.

@RaulEscobarRivas

This comment has been minimized.

@sybrendotinga
Copy link

sybrendotinga commented Oct 23, 2019

I have the same kind of question.

it('calls the route twice', () => {
      // Assign: stub first request
      cy.route('POST', `**/employees/options`, 'fixture:employees/options/post-200-unfiltered.json').as('firstRequest')

      // Act
      cy.openDetail(customerId)

      // Assign: Override the route to now only return the filtered options
      cy.wait('@firstRequest').then(() => {
        // This second request has some filters in the request body
        cy.route('POST', `**/employees/options`, 'fixture:employees/options/post-200-filtered-byid.json').as('secondRequest')
      })

      // Assert
      cy.wait('@secondRequest')
})

In this case second request is never asserted. Also without the then-callback it is not fired.

Schermafbeelding 2019-10-23 om 09 15 08

@stephane-dereppe
Copy link

Should be possible to discrimate request with body's content?

@Petercopter
Copy link

Apollo/GraphQL user here too 👋. I was really surprised this didn't work, because based on the documenation, it should...
https://docs.cypress.io/api/commands/wait.html#Wait-automatically-increments-responses

But in the documentation it's actually make new requests, it's not making multiple XHR calls on page load.

@stearm

This comment has been minimized.

@bonnett89

This comment has been minimized.

@arkhamRejek

This comment has been minimized.

@santieee

This comment has been minimized.

@lifeiscontent

This comment has been minimized.

@tommy-anderson

This comment has been minimized.

@mj-meyer
Copy link

Jup. Also have this issue. Have a polling service I'm trying to test. Want it to poll a few times, then change the response, and then it should trigger a call to a new endpoint. Doesn't matter what I do, it just returns the original response.

@matthewdaniel
Copy link

I've run into the same issue and have created a command that supports 2 different methods which should be a good jumping off point for those that need it.

@roddc
Copy link

roddc commented Sep 18, 2020

I want to make multiple requests to the same endpoint but having different response:

cy.route("POST", "/api/v1/resource_package_deploy", {
  code: 1,
  msg: "Fail",
 }).as("stubbedError");

cy.route("POST", "/api/v1/resource_package_deploy", {
  code: 0,
  msg: "Success",
}).as("stubbedDeploy");

cy.visit("/maintenance/resourceManagement");
cy.get("[data-cy-deployBtn=pkg02]").click();
cy.get("[data-cy=robot-cb]")
  .first()
  .find("input.mat-checkbox-input")
  .check({ force: true });
cy.get("[data-cy=robot-cb]")
  .eq(2)
  .find("input.mat-checkbox-input")
  .check({ force: true });
cy.get("[data-cy=deploy-submit]").click();
cy.wait("@stubbedError");
cy.wait("@stubbedDeploy");

But I got a timeout on waiting for the first request. How do I fix this?

@jrosspaperless
Copy link

I used the suggestions in this thread to write a blog post with something I hope is reusable across projects. Please take a look:

https://medium.com/life-at-paperless/cypress-graphql-response-mocking-7d49517f7754

@williamfenu
Copy link

This works for me:

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearchErr.json').as('error')
    cy.get('#searchInput')
    .type('1234567890 {enter}')
    .wait(['@error'])

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearch.json').as('success')
cy.get('#searchInput')
    .type('1762336105094 {enter}')
    cy.wait(['@success'])

U saved my life!! It worked for me too

@lpanger
Copy link

lpanger commented Dec 10, 2020

This works for me:

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearchErr.json').as('error')
    cy.get('#searchInput')
    .type('1234567890 {enter}')
    .wait(['@error'])

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearch.json').as('success')
cy.get('#searchInput')
    .type('1762336105094 {enter}')
    cy.wait(['@success'])

Unfortunately this pattern does not seem to work with cy.intercept

@bahunov
Copy link

bahunov commented Dec 10, 2020

This works for me:

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearchErr.json').as('error')
    cy.get('#searchInput')
    .type('1234567890 {enter}')
    .wait(['@error'])

cy.route('POST', '**/app/globalSearch', 'fixture:globalSearch.json').as('success')
cy.get('#searchInput')
    .type('1762336105094 {enter}')
    cy.wait(['@success'])

Unfortunately this pattern does not seem to work with cy.intercept

You could always revert back to version 5 where server/route is still available

@lpanger
Copy link

lpanger commented Dec 10, 2020

Yes that's exactly what I'm doing. Server/route are still available on v6

@cmraible
Copy link

cmraible commented Dec 16, 2020

I was having this issue as well, and came up with a not so terrible workaround without reverting to cy.server or cy.route.

var i = 0
cy.intercept('GET', url, (req) => {
       if (i === 0) {
           req.reply({
               statusCode: 200,
               fixture: 'firstResponse.json'
           })
       } else {
           req.reply({
               statusCode: 200,
               fixture: 'secondResponse.json'
           })
       }
       i++
});

@jennifer-shehane
Copy link
Member

I would like to see the resolution of this issue #9302, so that afterwards hopefully this can be resolved by using the new cy.intercept().

@Alpine78
Copy link

let useDefaultFixture = true;

describe('Use multiple fixture with single intercept', function () {
    beforeEach(() => {
        cy.intercept('GET', 'api/endpoint*', req => {
            if (useDefaultFixture) {
                req.reply(res => {
                    res.send({ fixture: 'defaultFixture.json' })
                })
            } else {
                req.reply(res => {
                    res.send({ fixture: 'secondFixture.json' })
                })
            }
        }).as('getSomething')
        cy.visit('/')
    })

    it('using defaultFixture', () => {
        // Uses defaultFixture.json when fetching api/endpoint*
    })

    it('needs secondFixture', () => {
        useDefaultFixture = false;
        // Using secondFixture.json when fetching api/endpoint*
    })

@craig-dae
Copy link

This has just become a problem for us. We were advised to start writing tests like this: https://docs.cypress.io/guides/references/best-practices.html#Having-tests-rely-on-the-state-of-previous-tests

This means that separate it blocks should have no dependencies on each other. So I've been combinging it-blocks. I have one test that creates a hierarchy of items. This means 4 posts against the same endpoint in a row. I also check that the status of each call comes back with a 201.

However, I'm not able to intercept the same endpoint more than once in a single it block. I have no idea what the correct way of writing this test is, if I can't intercept more than once, and I'm not supposed to use multiple it blocks.

@bahmutov
Copy link
Contributor

@craig-dae in your test you do something like this?

cy.intercept('POST', '/somewhere').as('posted')
cy.click()
cy.wait('@posted') // 1
cy.click()
cy.wait('@posted') // 2
cy.click()
cy.wait('@posted') // 3
cy.click()
cy.wait('@posted') // 4

@craig-dae
Copy link

Similar.

cy.intercept('POST', '/somewhere').as('posted')
cy.click()
cy.waitWithStatus('@posted', 201) // 1
cy.click()
cy.waitWithStatus('@posted', 201) // 2
cy.click()
cy.waitWithStatus('@posted', 201) // 3
cy.click()
cy.waitWithStatus('@posted', 201) // 4

Where waitWithStatus is:

Cypress.Commands.add(
  'withStatusCode',
  {
    prevSubject: true,
  },
  (subject: Interception, status: number) => {
    if (!subject.response) {
      cy.log('No statuscode. Was this endpoint called more than once?');
    } else if (subject.response?.statusCode !== status) {
      cy.log('http response', subject.response?.body);
    }
    return cy.wrap(subject).its('response.statusCode').should('eq', status);
  },
);

Cypress.Commands.add('waitWithStatus', (name: string, status: number) => {
  return cy.wait(`${name}`).withStatusCode(status);
});

The problem is that only the first wait gives me a statusCode. Every one after the first is undefined. I always check for statuscodes, because wait does not throw an error if my backend returns with a 500.

We use Cypress as regression tests, and we catch a LOT of backend problems that they don't catch with their unit tests. Something as simple as verifying that we got a 200 or 201 catches a LOT of bugs.

@bahmutov
Copy link
Contributor

@craig-dae I hear you, just confirming the response is a good strategy.

So I have tried to recreate your problem in our cypress-example-recipes example https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/stubbing-spying__intercept

What happens if you do not use a custom command and instead have the test like I have written it?

it('spies on multiple requests', () => {
  cy.intercept({
    method: 'POST',
    pathname: '/users',
  }).as('postUser')

  cy.get('#post-user').click()
  cy.wait('@postUser').its('response.statusCode').should('equal', 201)

  // post 2nd time
  cy.get('#post-user').click()
  cy.wait('@postUser').its('response.statusCode').should('equal', 201)

  // post 3rd time
  cy.get('#post-user').click()
  cy.wait('@postUser').its('response.statusCode').should('equal', 201)

  // post 4th time
  cy.get('#post-user').click()
  cy.wait('@postUser').its('response.statusCode').should('equal', 201)
})

You could shorten this of course without using a custom command, something like

const waitForStatus = (alias, code) =>
  cy.wait(alias).its('response.statusCode').should('equal', code)

it('spies on multiple requests', () => {
  cy.intercept({
    method: 'POST',
    pathname: '/users',
  }).as('postUser')

  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

  // post 2nd time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

  // post 3rd time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

  // post 4th time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)
})

@gk-patel
Copy link

Hi @bahmutov,

The example you gave is working, but can you confirm that what you show above works for cypress 6.0+, but if the scenario is tweaked a bit, then we I see some problems,

const waitForStatus = (alias, code) =>
  cy.wait(alias).its('response.statusCode').should('equal', code)

it('spies on multiple requests', () => {
  cy.intercept({
    method: 'POST',
    pathname: '/users',
  }).as('postUser')

  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

// ##### DO LOTS OF CY.GET and similar tests #####

  // post 2nd time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

// ##### DO LOTS OF CY.GET and similar tests #####

  // post 3rd time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)

// ##### DO LOTS OF CY.GET and similar tests #####

  // post 4th time
  cy.get('#post-user').click()
  waitForStatus('@postUser', 201)
})

I actaully perform my "DO LOTS OF CY.GET" in a for-loop and when I do the tests on the first few conditions/elements, the test gets completed successfully. But, when I increase the number of cy.get test cases to +50 then, I see the XHR getting sent, it is also coming to the backend API but the cy.wait on the intercept fails.

Should this be the case ?

@bahmutov
Copy link
Contributor

Can you clone the repo, put the code with lots of gets and give us the url to see? Because maybe my idea of lots is different and I don't see what you see.

@craig-dae
Copy link

craig-dae commented Feb 17, 2021

@bahmutov That works. You were right. For some reason, my custom command is the issue. My guess is it's a race-condition with the way cypress uses Promises?

I REALLY want better support for:

  1. Checking the status code of an endpoint call. I do this every time, and .its('response.statusCode').should('equal', 201) is a lot to type.
  2. Showing the full response (because it includes a backend stack trace), especially on the Cypress dashboard, when the status code is not what is expected.

That's why I wrote the custom command. On our project, we've got a frontend and backend team. Whenever a test fails, and it's because some DOM element doesn't exist, they (the backend team) throw the bug to us. We look into it and find that the DOM element doesn't exist because an endpoint call failed, and have to throw it back to them. It's much easier to diagnose status code was 400. expected 200.

Is there any way to fix my custom command while I continue to try to convince you guys to add this feature? I can use the arrow function in the meantime.

Thanks for taking the time to figure this out btw.

@craig-dae
Copy link

craig-dae commented Feb 17, 2021

@bahmutov if I simplify my command to this, it works:

Cypress.Commands.add('waitForStatus', (alias, code) =>
  cy.wait(alias).its('response.statusCode').should('equal', code),
);

It looks like maybe all this stuff with prevSubject was the problem. When I originally wrote it, you did it like this: cy.wait('@load').withStatus(200). That's why I was using prevSubject.

I think you have me on the right path. And I think this conversation is out of scope for the original purpose of this issue posting. Thanks for your help!

Edit: Final iteration back to what I originally had:

Cypress.Commands.add('waitWithStatus', (name: string, status: number) => {
  return cy.wait(`${name}`).then((subject) => {
    if (!subject.response) {
      cy.log('No statuscode. Was this endpoint called more than once?');
    } else if (subject.response?.statusCode !== status) {
      cy.log('http response', subject.response?.body);
    }
    return cy.wrap(subject).its('response.statusCode').should('eq', status);
  });
});

@gk-patel
Copy link

Hi @bahmutov

Thanks for the quick reply. I setup something for you to check out.

https://github.com/gk-patel/cypress-example-recipes/blob/master/examples/stubbing-spying__intercept/cypress/integration/spy-on-fetch-spec.js#L58

On my development VM (ram 2GB, just informing as its not that powerful), if I set DEFINE_LOT to 10 then all tests passes where us at 100 test fail.

FYI, I have randomly placed the function call lotOfGets() either before, after or in between the click-action and the wait - so there is no particular reason why it is called how it is.

The main point is, by chaning 10 to 100 the tests fail, which should not be the case. Also, in case of 100, the call sometimes fail on the third and sometimes on the fourth request.

image

@gk-patel
Copy link

Hi @bahmutov

Can you confirm if this problem is reproducible on your side ?

Thanks for your support.

@jennifer-shehane
Copy link
Member

I've been waiting for the ability to override cy.intercept() to retest the reproducible case in this issue. This issue is still reproducible from https://github.com/bahalperin/cypress-route-bug-repro when replacing cy.route() with cy.intercept().

@cypress-bot cypress-bot bot added stage: pending release There is a closed PR for this issue and removed stage: needs investigating Someone from Cypress needs to look at this labels May 10, 2021
@cypress-bot
Copy link
Contributor

cypress-bot bot commented May 10, 2021

The code for this is done in cypress-io/cypress#14513, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@jennifer-shehane
Copy link
Member

This has been released in 7.3.0 as part of cy.intercept(). cy.route() is deprecated and no new fixes of features will be released for it.

In order to mock different responses for requests that happen quickly, back to back - that is, there is no action between the requests, we recommend using the new times option in the RouteMatcher of cy.intercept(). To rewrite the original issue with times: https://github.com/jennifer-shehane/cypress-intercept-times-flake

it(`test ${i}`, () => {
  cy.intercept('https://jsonplaceholder.typicode.com/todos/1', { times: 1 }, { title: 'baz' }).as('getTodo')
  cy.intercept('https://jsonplaceholder.typicode.com/todos/1', { times: 1 }, { title: 'bar' }).as('getTodo')
  cy.intercept('https://jsonplaceholder.typicode.com/todos/1', { times: 1 }, { title: 'foo' }).as('getTodo')
  cy.visit('/')
  cy.wait('@getTodo')
  cy.wait('@getTodo')
  cy.wait('@getTodo')

  cy.get('#api-response-list li')
    .first().should('have.text', 'foo')
    .next().should('have.text', 'bar')
    .next().should('have.text', 'baz')
})

There's currently another issue with the example above that I've found and already has a fix ready for it here: #16457 So after that is resolved, I think everyone should have a solution.

This comment thread has been locked.

If you are still experiencing this issue after upgrading to Cypress v7.3.0, please open a new issue with a reproducible example.

@cypress-io cypress-io locked as resolved and limited conversation to collaborators May 11, 2021
@jennifer-shehane jennifer-shehane removed the stage: pending release There is a closed PR for this issue label May 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.