Intermittent failures #80

Closed
cpytel opened this Issue Jun 16, 2011 · 9 comments

Comments

Projects
None yet
8 participants
@cpytel
Owner

cpytel commented Jun 16, 2011

I mention internal thoughtbot code in this issue, but hey.

The following feature in Trajectory features/users/estimate_stories.feature:56 in the commit 76948caff4fd711324b5e87afc9d2a4c9b80ab61 fails intermittently with the error:

@javascript
Scenario: Estimate multiple stories that affect the number of iterations when the inline story form is visible # features/users/estimate_stories.feature:56
Given today is "August 2, 2010" # features/step_definitions/time_steps.rb:1
And I am signed in as a member of the "Snacks" project # features/step_definitions/user_steps.rb:26
Validation failed: Account can't be blank (ActiveRecord::RecordInvalid)
./features/step_definitions/user_steps.rb:29:in /^I (?:am signed in|sign in) as a member of the "([^"]+)" project$/' features/users/estimate_stories.feature:58:inAnd I am signed in as a member of the "Snacks" project'
And the "Snacks" project has a default velocity of 20 points #

@bricestacey

This comment has been minimized.

Show comment Hide comment
@bricestacey

bricestacey Oct 27, 2011

I am not sure if I should open another ticket, but I have experienced intermittent failures too from FactoryGirl when using capybara-webkit. I am getting the following:

Background:                                     # features/books.feature:5
Given I am not authenticated                  # features/step_definitions/devise_steps.rb:5
And I am a new, authenticated user            # features/step_definitions/devise_steps.rb:13
And A donor exists with a name of "Anonymous" # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And A bin exists with a name of "Top Shelf"   # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And the following books exists:               # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:100
  | title   | author  | published | publisher | recommendation | donor           | bin             |
  | Walden  | Thoreau | 1854      | foo       | undecided      | name: Anonymous | name: Top Shelf |
  | AWOCAMR | Thoreau | 1849      | bar       | keep           | name: Anonymous | name: Top Shelf |
  NoMethodError: undefined method `fields' for nil:NilClass:           SELECT DISTINCT(attr.attname)
            FROM pg_attribute attr
            INNER JOIN pg_depend dep ON attr.attrelid = dep.refobjid AND attr.attnum = dep.refobjsubid
            INNER JOIN pg_constraint cons ON attr.attrelid = cons.conrelid AND attr.attnum = cons.conkey[1]
            WHERE cons.contype = 'p'
              AND dep.refobjid = $1::regclass
   (ActiveRecord::StatementInvalid)
  features/books.feature:10:in `And the following books exists:'
And I am on the home page                     # features/step_definitions/web_steps.rb:44
And I follow "Books"                          # features/step_definitions/web_steps.rb:56

When I run it again, all is well.

This same block of code, I've gotten this error for a different step:

Background:                                     # features/books.feature:5
Given I am not authenticated                  # features/step_definitions/devise_steps.rb:5
And I am a new, authenticated user            # features/step_definitions/devise_steps.rb:13
And A donor exists with a name of "Anonymous" # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And A bin exists with a name of "Top Shelf"   # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
  NoMethodError: undefined method `fields' for nil:NilClass: INSERT INTO "bins" ("active", "created_at", "name", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" (ActiveRecord::StatementInvalid)
  features/books.feature:9:in `And A bin exists with a name of "Top Shelf"'
And the following books exists:               # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:100
  | title   | author  | published | publisher | recommendation | donor           | bin             |
  | Walden  | Thoreau | 1854      | foo       | undecided      | name: Anonymous | name: Top Shelf |
  | AWOCAMR | Thoreau | 1849      | bar       | keep           | name: Anonymous | name: Top Shelf |
And I am on the home page                     # features/step_definitions/web_steps.rb:44
And I follow "Books"                          # features/step_definitions/web_steps.rb:56

I am not sure if I should open another ticket, but I have experienced intermittent failures too from FactoryGirl when using capybara-webkit. I am getting the following:

Background:                                     # features/books.feature:5
Given I am not authenticated                  # features/step_definitions/devise_steps.rb:5
And I am a new, authenticated user            # features/step_definitions/devise_steps.rb:13
And A donor exists with a name of "Anonymous" # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And A bin exists with a name of "Top Shelf"   # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And the following books exists:               # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:100
  | title   | author  | published | publisher | recommendation | donor           | bin             |
  | Walden  | Thoreau | 1854      | foo       | undecided      | name: Anonymous | name: Top Shelf |
  | AWOCAMR | Thoreau | 1849      | bar       | keep           | name: Anonymous | name: Top Shelf |
  NoMethodError: undefined method `fields' for nil:NilClass:           SELECT DISTINCT(attr.attname)
            FROM pg_attribute attr
            INNER JOIN pg_depend dep ON attr.attrelid = dep.refobjid AND attr.attnum = dep.refobjsubid
            INNER JOIN pg_constraint cons ON attr.attrelid = cons.conrelid AND attr.attnum = cons.conkey[1]
            WHERE cons.contype = 'p'
              AND dep.refobjid = $1::regclass
   (ActiveRecord::StatementInvalid)
  features/books.feature:10:in `And the following books exists:'
And I am on the home page                     # features/step_definitions/web_steps.rb:44
And I follow "Books"                          # features/step_definitions/web_steps.rb:56

When I run it again, all is well.

This same block of code, I've gotten this error for a different step:

Background:                                     # features/books.feature:5
Given I am not authenticated                  # features/step_definitions/devise_steps.rb:5
And I am a new, authenticated user            # features/step_definitions/devise_steps.rb:13
And A donor exists with a name of "Anonymous" # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
And A bin exists with a name of "Top Shelf"   # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:118
  NoMethodError: undefined method `fields' for nil:NilClass: INSERT INTO "bins" ("active", "created_at", "name", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" (ActiveRecord::StatementInvalid)
  features/books.feature:9:in `And A bin exists with a name of "Top Shelf"'
And the following books exists:               # factory_girl-2.2.0/lib/factory_girl/step_definitions.rb:100
  | title   | author  | published | publisher | recommendation | donor           | bin             |
  | Walden  | Thoreau | 1854      | foo       | undecided      | name: Anonymous | name: Top Shelf |
  | AWOCAMR | Thoreau | 1849      | bar       | keep           | name: Anonymous | name: Top Shelf |
And I am on the home page                     # features/step_definitions/web_steps.rb:44
And I follow "Books"                          # features/step_definitions/web_steps.rb:56
@sbleon

This comment has been minimized.

Show comment Hide comment
@sbleon

sbleon Jan 3, 2012

I also have a test that fails intermittently about 20% of the time. I'm using capybara-webkit in a Test::Unit integration test.

@cpytel, this is a pretty old issue. Has capybara-webkit been working reliably for you guys, or are you still experiencing these kinds of intermittent issues?

sbleon commented Jan 3, 2012

I also have a test that fails intermittently about 20% of the time. I'm using capybara-webkit in a Test::Unit integration test.

@cpytel, this is a pretty old issue. Has capybara-webkit been working reliably for you guys, or are you still experiencing these kinds of intermittent issues?

@cpytel

This comment has been minimized.

Show comment Hide comment
@cpytel

cpytel Jan 4, 2012

Owner

We've gotten a lot of the problems fixed but I understand that there may still be intermitted failures.

On Jan 3, 2012, at 10:11 AM, sbleon wrote:

I also have a test that fails intermittently about 20% of the time. I'm using capybara-webkit in a Test::Unit integration test.

@cpytel, this is a pretty old issue. Has capybara-webkit been working reliably for you guys, or are you still experiencing these kinds of intermittent issues?


Reply to this email directly or view it on GitHub:
#80 (comment)


Chad Pytel, Founder and CEO
thoughtbot, inc.
t: 617-482-1300 x113
f: 866-217-5992
http://www.thoughtbot.com
http://www.twitter.com/thoughtbot

Owner

cpytel commented Jan 4, 2012

We've gotten a lot of the problems fixed but I understand that there may still be intermitted failures.

On Jan 3, 2012, at 10:11 AM, sbleon wrote:

I also have a test that fails intermittently about 20% of the time. I'm using capybara-webkit in a Test::Unit integration test.

@cpytel, this is a pretty old issue. Has capybara-webkit been working reliably for you guys, or are you still experiencing these kinds of intermittent issues?


Reply to this email directly or view it on GitHub:
#80 (comment)


Chad Pytel, Founder and CEO
thoughtbot, inc.
t: 617-482-1300 x113
f: 866-217-5992
http://www.thoughtbot.com
http://www.twitter.com/thoughtbot

@rodrigomanhaes

This comment has been minimized.

Show comment Hide comment
@rodrigomanhaes

rodrigomanhaes Apr 17, 2012

Same kind of error here, when combining capybara-webkit + factory_girl:

Failure/Error: livro = Factory.create(:livro, titulo: 'Quantum Mechanics for Dummies')
     ActiveRecord::StatementInvalid:
       NoMethodError: undefined method `fields' for nil:NilClass:             SELECT COUNT(*)
                   FROM pg_class c
                   LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
                   WHERE c.relkind in ('v','r')
                   AND c.relname = $1
                   AND n.nspname = ANY (current_schemas(false))```

Same kind of error here, when combining capybara-webkit + factory_girl:

Failure/Error: livro = Factory.create(:livro, titulo: 'Quantum Mechanics for Dummies')
     ActiveRecord::StatementInvalid:
       NoMethodError: undefined method `fields' for nil:NilClass:             SELECT COUNT(*)
                   FROM pg_class c
                   LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
                   WHERE c.relkind in ('v','r')
                   AND c.relname = $1
                   AND n.nspname = ANY (current_schemas(false))```
@Capncavedan

This comment has been minimized.

Show comment Hide comment
@Capncavedan

Capncavedan Jun 4, 2012

I'm experiencing this error intermittently as well, with latest FactoryGirl / FactoryGirl Rails.

A little more detail:

What appears to be happening is that association objects created in a factory are sometimes instantiated with an ID of 0, even though the records have an ID of (for example) 1 in the database.

Given a Bond :belongs_to an Agency

FactoryGirl.define do
factory :bond do
status 'new'
association(:agency)
end
end

@bond = FactoryGirl.create(:bond)

sometimes the :agency_id field on the @bond object will be 0, and the @agency object will similarly also report having an ID of 0.

However, using ActiveRecord, with agency = Agency.last, we get the correct agency object, with an expected ID of (say) 1.

Since an agency does not exist in the database with ID of 0, at some point in a Cucumber scenario we get a nil object failure when the bond's association is loaded.

We just recently switched to capybara-webkit and began running into this. Weird that it is intermittent. Also weird to try to imagine how capybara-webkit and FactoryGirl might be interacting to produce this bug.

I'm experiencing this error intermittently as well, with latest FactoryGirl / FactoryGirl Rails.

A little more detail:

What appears to be happening is that association objects created in a factory are sometimes instantiated with an ID of 0, even though the records have an ID of (for example) 1 in the database.

Given a Bond :belongs_to an Agency

FactoryGirl.define do
factory :bond do
status 'new'
association(:agency)
end
end

@bond = FactoryGirl.create(:bond)

sometimes the :agency_id field on the @bond object will be 0, and the @agency object will similarly also report having an ID of 0.

However, using ActiveRecord, with agency = Agency.last, we get the correct agency object, with an expected ID of (say) 1.

Since an agency does not exist in the database with ID of 0, at some point in a Cucumber scenario we get a nil object failure when the bond's association is loaded.

We just recently switched to capybara-webkit and began running into this. Weird that it is intermittent. Also weird to try to imagine how capybara-webkit and FactoryGirl might be interacting to produce this bug.

@jferris

This comment has been minimized.

Show comment Hide comment
@jferris

jferris Jul 11, 2012

Owner

Can you guys try again with the latest master? We've fixed a number of synchronization and load failure issues.

Owner

jferris commented Jul 11, 2012

Can you guys try again with the latest master? We've fixed a number of synchronization and load failure issues.

@johncant

This comment has been minimized.

Show comment Hide comment
@johncant

johncant Sep 18, 2012

I've been getting errors like this, but I'm convinced that they are DB synchronization problems in your tests. I'm guessing you're sharing an ActiveRecord connection across threads (Cucumber does this by default), so you have to make sure that your app completes a request before your test tries to interact with the DB. A possible reason why you might not have had these errors before is that your other driver waited for a response after it visited a page.

I've been getting errors like this, but I'm convinced that they are DB synchronization problems in your tests. I'm guessing you're sharing an ActiveRecord connection across threads (Cucumber does this by default), so you have to make sure that your app completes a request before your test tries to interact with the DB. A possible reason why you might not have had these errors before is that your other driver waited for a response after it visited a page.

@jopotts

This comment has been minimized.

Show comment Hide comment
@jopotts

jopotts Feb 16, 2013

I get intermittent errors too. Before I dig deeper, I was wondering if this is still a known issue? I'm not using any factories or fixtures. Thanks.

jopotts commented Feb 16, 2013

I get intermittent errors too. Before I dig deeper, I was wondering if this is still a known issue? I'm not using any factories or fixtures. Thanks.

@jferris

This comment has been minimized.

Show comment Hide comment
@jferris

jferris Feb 18, 2013

Owner

At this point, there are no known synchronization errors specific to capybara-webkit. Using any asynchronous driver with Capybara can lead to intermittent errors.

You can eliminate most async issues by performing assertions against the UI using Capybara's helpers (has_text?/have_text, etc).

I'm going to close this. If you're still running into async issues while using Capybara's helpers, please open a new issue.

Owner

jferris commented Feb 18, 2013

At this point, there are no known synchronization errors specific to capybara-webkit. Using any asynchronous driver with Capybara can lead to intermittent errors.

You can eliminate most async issues by performing assertions against the UI using Capybara's helpers (has_text?/have_text, etc).

I'm going to close this. If you're still running into async issues while using Capybara's helpers, please open a new issue.

@jferris jferris closed this Feb 18, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment