type / submit in Cypress not triggering HTML validation on elements #6678

brettz9 · 2020-03-09T04:34:03Z

Current behavior:

Not finding invalid elements, whether by:

cy.get('<selector>:invalid')

cy.get('[data-name="name"]:invalid').should('have.length', 1);

validity property

cy.get('[data-name="name"]').then(($input) => {
  return expect($input[0].validity.tooShort).to.be.true;
});

Desired behavior:

Correctly indicate that the element is indeed invalid (so that assertions can be based on this).

Test code to reproduce

These tests both fail in showing any problems with invalidity, despite the HTML element being queried having minlength attribute of 3 and our test typing only a single letter ("a"):

it('should catch invalid element by pseudo-selector', () => {
    cy.visit('http://localhost:8000/');
    const tooShortOfAName = 'a';
    cy.get('[data-name="name"]').type(tooShortOfAName);

    // Fails with or without a form click
    // cy.get('[data-name="action2"]').click();

    // Fails here
    cy.get('[data-name="name"]:invalid').should('have.length', 1);
  })

  it('should catch invalid element by JS API', function () {
    cy.visit('http://localhost:8000/');
    const tooShortOfAName = 'a';
    cy.get('[data-name="name"]').type(tooShortOfAName);

    // Fails with or without a form click
    // cy.get('[data-name="action2"]').click();

    // Fails here
    cy.get('[data-name="name"]').then(($input) => {
      return expect($input[0].validity.tooShort).to.be.true;
    });
  });

Here's the minimal repo to reproduce: (note that I've added a cypress install as I think you may want to do in your boilerplate also since your docs refer to installing cypress locally)

https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity

(i.e., on the not-getting-invalidity branch)

Outdated info

I've also tacked on an environmental variable test (same as #2605 ?)

it('should obtain environmental variables', function () {
  cy.log(Cypress.env('secret'))
  expect(Cypress.env('secret')).to.equal('Be good');
});

...which is not consistently getting the secret despite it being set in plugins/index.js

module.exports = (on, config) => {
  config.env = config.env || {};
  config.env.secret = 'Be good';
  return config;
);

...consistent with the approach described for dynamically setting the environmental variables at https://docs.cypress.io/api/plugins/configuration-api.html#Switch-between-multiple-configuration-files or https://docs.cypress.io/api/cypress-api/env.html#From-a-plugin.

Versions

4.1.0, Mac OSX 10.15.2, Chrome 80.0.3987.132 (Official Build) (64-bit)

The text was updated successfully, but these errors were encountered:

brettz9 · 2020-03-09T04:48:32Z

I might add my validity checks are akin to that added in cypress-io/cypress-documentation#1919 (at https://docs.cypress.io/faq/questions/using-cypress-faq.html#Can-I-check-that-a-form%E2%80%99s-HTML-form-validation-is-shown-when-an-input-is-invalid )

jennifer-shehane · 2020-03-10T07:38:45Z

So the problem here is not that it's not finding elements with a psuedo-selector of invalid, it's that the actions being performed through Cypress are not triggering the validation checks on the form for some reason.

You can use jQuery to query for input:invalid on the application under test after Cypress performs the type and even the click to submit the form and see that it finds no invalid elements.

Reproducible Example

spec.js

it('does not find invalid input', () => {
  cy.visit('index.html')
  cy.get('input').type('a')
  // cy.contains('Update').click()
  cy.get('input:invalid').should('have.length', 1)
})

index.html

<!DOCTYPE html>
<body>
    <input minlength="3">
    <button type="submit">Update</button>
</body>
</html>

Events triggered during type.

If you type manually into the form while the test is running, it will all of a sudden pass.

shamgang · 2020-06-14T08:28:48Z

In case it helps: I have 2 forms, one with a text and password input, and one with a text, password, and numerical input. In both cases only the text input exhibits this bug. The password and numerical inputs behave as expected - form validation popups are appearing. This remains true if I reorder the inputs on the page and change the order I type into them in Cypress.
Node 12.13.0
Cypress 4.8.0
Chrome 83.0.4103.97
Windows 10 Home v1903 build 18362.900

zedrdave · 2020-07-10T10:41:06Z

Just to add my own experience: getting this bug on a minlength validation on a password input:
when using cy.get().type() to type text that should be too short, I am able to submit the form both manually and through Cypress.

In essence, it looks as if Cypress is not sending the necessary key events for the form to perform validation.

shin10 · 2020-08-18T11:46:54Z

I came across this issue as well. The minlength attribute will be ignored. Validation of the pattern
attribute pattern=".{6}" however will be triggered. Unfortunately the resulting message ("Please match the requested format") is very vague and rather confusing for the user. Further adding setCustomValidity manually wouldn't be my favourite option.

sainthkh · 2020-09-07T07:18:25Z

I tried form.reportValidity(). But somehow, it doesn't work. It always return true when there's an error in the form.

yanfengliu · 2020-12-29T23:09:36Z

According to MDN, minLength is only triggered if value is changed by user.

brettz9 · 2021-02-02T03:59:34Z

Is this planned, e.g., on a Roadmap, or given a priority? It is a pretty fundamental need to be able to check forms, testing for validity (for those using the standard mechanisms for doing so). Thanks in advance...

sainthkh · 2021-02-02T06:49:24Z

@Brett9 Unfortunately, it's hard to find browser API that triggers form validation. Maybe, this can be solved with native events.

brettz9 · 2021-02-04T00:19:05Z

@sainthkh : Thanks for the info and idea. If this would be best addressed by Native Browser Events, I would hope its consideration might be added to the list on #311 , @brian-mann .

bahmutov · 2021-02-04T03:17:22Z

An update: for me the validations are all working, you can see in https://glebbahmutov.com/cypress-examples/6.4.0/recipes/form-validation.html

Note: cy.submit does not check the validity of the form before submitting #14911 while clicking on the submit button does check

cypress-bot · 2021-02-25T21:49:32Z

~~The code for this is done in cypress-io/cypress#14965, but has yet to be released.~~
~~We'll update this issue and reference the changelog when it's released.~~

This change was reverted in #15302

brettz9 · 2021-02-26T09:07:40Z

Sorry for the delay in replying...

As one example, I have:

<input type="password" autocomplete="new-password" class="form-control" required="" name="pass" id="pass-tf" data-name="reset-pass" minlength="6">

but this code fails:

      const tooShortPassword = 'a';
      cy.get('[data-name="reset-pass"]').type(tooShortPassword);

      cy.get('[data-name="reset-pass"]:invalid').should('have.length', 1); // Should be 1, but gives 0

The minimal test repo in the original post covers this and the other cases I believe are still occurring.

I.e., it's not just about the submission.

brettz9 · 2021-02-28T13:55:56Z

#14965 seems not to actually be covering this issue. My concern is before submit, so I think this issue should still be open.

jennifer-shehane · 2021-02-28T19:23:50Z

@brettz9 Yeah, you're right. I'm actually not able to get the original example to work with the new implementation. https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity?rgh-link-date=2020-03-09T04%3A34%3A03Z cc @sainthkh

I don't see .submit() failing the test as the PR described it should.
I don't see a way to check the conditions of the validity after submitting the form. How do you verify that the correct message is displayed for an invalid field in the example?

sainthkh · 2021-03-01T07:19:47Z

Confirmed that #14965 didn't fix the issue.

But the problem is that it cannot be fixed now. Because all validations are not created equal in browsers. As for the validations like required, mix, min, they always work whether it's a user input or not. But for those like minlength, maxlength, they are only triggered when users input the value.

If you want to test it yourself, sample code is below:

<input id="x" minlength="3" value="a" />
<input id="y" required />

cy.get('#x').then($x => expect($x[0].checkValidity()).to.be.false) // => fail
cy.get('#y').then($y => expect($y[0].checkValidity()).to.be.false) // => succeed

Native events might solve the problem. But I'm not sure about that.

tusharAdvanced · 2022-09-23T10:32:05Z

Is there any solution on this ? I am facing the same.

a-bashkatov · 2023-02-23T10:37:10Z

I could solve this problem using "realType" from cypress-real-events, then the input field is validated as with real text input. However, this solution is not suitable, for example, for Firefox.

This comment has been minimized.

Sign in to view

jennifer-shehane added pkg/driver This is due to an issue in the packages/driver directory type: bug labels Mar 10, 2020

cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Mar 10, 2020

jennifer-shehane added the topic: cy.type ⌨️ label Mar 10, 2020

jennifer-shehane changed the title ~~Not finding invalid elements~~ type / submit in Cypress not triggering HTML validation on elements Mar 10, 2020

sainthkh mentioned this issue Feb 8, 2021

feat: cy.submit validates inputs before sending data. #14965

Merged

2 tasks

cypress-bot bot added stage: work in progress and removed stage: needs investigating Someone from Cypress needs to look at this labels Feb 8, 2021

cypress-bot bot added stage: pending release and removed stage: work in progress labels Feb 25, 2021

jennifer-shehane closed this as completed Feb 28, 2021

jennifer-shehane reopened this Feb 28, 2021

sainthkh removed the stage: pending release label Mar 8, 2021

cypress-bot bot added the stage: icebox label Sep 9, 2021

nagash77 added the prevent-stale mark an issue so it is ignored by stale[bot] label Apr 3, 2023

mschile mentioned this issue May 5, 2023

Form is submitted even with invalid password input #26643

Closed

jennifer-shehane removed the stage: icebox label Nov 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

type / submit in Cypress not triggering HTML validation on elements #6678

type / submit in Cypress not triggering HTML validation on elements #6678

brettz9 commented Mar 9, 2020 •

edited

brettz9 commented Mar 9, 2020

This comment has been minimized.

jennifer-shehane commented Mar 10, 2020

shamgang commented Jun 14, 2020 •

edited

zedrdave commented Jul 10, 2020

shin10 commented Aug 18, 2020

sainthkh commented Sep 7, 2020 •

edited

yanfengliu commented Dec 29, 2020

brettz9 commented Feb 2, 2021

sainthkh commented Feb 2, 2021

brettz9 commented Feb 4, 2021

bahmutov commented Feb 4, 2021

cypress-bot bot commented Feb 25, 2021 •

edited by mschile

brettz9 commented Feb 26, 2021

brettz9 commented Feb 28, 2021

jennifer-shehane commented Feb 28, 2021

sainthkh commented Mar 1, 2021

tusharAdvanced commented Sep 23, 2022

a-bashkatov commented Feb 23, 2023

type / submit in Cypress not triggering HTML validation on elements #6678

type / submit in Cypress not triggering HTML validation on elements #6678

Comments

brettz9 commented Mar 9, 2020 • edited

Current behavior:

Desired behavior:

Test code to reproduce

Versions

brettz9 commented Mar 9, 2020

This comment has been minimized.

jennifer-shehane commented Mar 10, 2020

Reproducible Example

shamgang commented Jun 14, 2020 • edited

zedrdave commented Jul 10, 2020

shin10 commented Aug 18, 2020

sainthkh commented Sep 7, 2020 • edited

yanfengliu commented Dec 29, 2020

brettz9 commented Feb 2, 2021

sainthkh commented Feb 2, 2021

brettz9 commented Feb 4, 2021

bahmutov commented Feb 4, 2021

cypress-bot bot commented Feb 25, 2021 • edited by mschile

brettz9 commented Feb 26, 2021

brettz9 commented Feb 28, 2021

jennifer-shehane commented Feb 28, 2021

sainthkh commented Mar 1, 2021

tusharAdvanced commented Sep 23, 2022

a-bashkatov commented Feb 23, 2023

brettz9 commented Mar 9, 2020 •

edited

shamgang commented Jun 14, 2020 •

edited

sainthkh commented Sep 7, 2020 •

edited

cypress-bot bot commented Feb 25, 2021 •

edited by mschile