Skip to content

Stateful test failure using randomly seed 1697538841 looks like a faker issue #17734

Closed
@cofiem

Description

@cofiem

Describe the bug

When running warehouse tests, this randomly seed fails (seed is from #15398 ):

 make tests TESTARGS="--randomly-seed=1697538841 -vvv" T=tests/unit/admin/views/test_emails.py::TestEmailList

with the two errors

FAILED tests/unit/admin/views/test_emails.py::TestEmailList::test_wildcard_query - AssertionError: assert equals failed
  -{'emails': <paginate.Page: Page 1/1>, 'query': 'ubrown@example.or%'}                              +{
                                                                                                    +  'emails': [
                                                                                                    +    EmailMessage(created=datetime.datetime(2025, 3, 7, 14, 35, 9, 91506), status=<EmailStatuses.
                                                                                                    +Accepted: 'Accepted'>, message_id='IqdyfEuXOtQR', from_='andrew89@example.com', to='ubrown@examp
                                                                                                    +le.org', subject='Before send window.', missing=False, id=UUID('836af322-8b4a-49d1-af21-978e19f5
                                                                                                    +26a0')),
                                                                                                    +  ],
                                                                                                    +  'query': 'ubrown@example.or%',
                                                                                                    +}
FAILED tests/unit/admin/views/test_emails.py::TestEmailList::test_basic_query - AssertionError: assert equals failed
  -{'emails': <paginate.Page: Page 1/1>, 'query': 'ubrown@example.org'}                              +{
                                                                                                    +  'emails': [
                                                                                                    +    EmailMessage(created=datetime.datetime(2025, 3, 7, 14, 35, 9, 91506), status=<EmailStatuses.
                                                                                                    +Accepted: 'Accepted'>, message_id='IqdyfEuXOtQR', from_='andrew89@example.com', to='ubrown@examp
                                                                                                    +le.org', subject='Before send window.', missing=False, id=UUID('fc1ba556-b2ed-4321-b993-1db542de
                                                                                                    +2486')),
                                                                                                    +  ],
                                                                                                    +  'query': 'ubrown@example.org',
                                                                                                    +}

Results (21.19s):
       7 passed
       2 failed
         - tests/unit/admin/views/test_emails.py:104 TestEmailList.test_wildcard_query
         - tests/unit/admin/views/test_emails.py:57 TestEmailList.test_basic_query

where <paginate.Page: Page 1/1> is

[EmailMessage(
created=datetime.datetime(2025, 3, 7, 14, 11, 25, 124019), 
status=<EmailStatuses.Accepted: 'Accepted'>,
message_id='IqdyfEuXOtQR', 
from_='andrew89@example.com',
to='ubrown@example.org', 
subject='Before send window.',
missing=False,
id=UUID('971b4791-4ca6-4e4c-b550-a8cc2a830422')), 
EmailMessage(created=datetime.datetime(2025, 3, 6, 8, 10, 10, 717537), status=<EmailStatuses.Accepted: 'Accepted'>,
message_id='NNidOFjCuWWu', 
from_='smithmorgan@example.org', 
to='ubrown@example.org', 
subject='Trip enter office near include language provide.', 
missing=False, 
id=UUID('ae299de1-9a51-4516-aa20-304748f22e47'))
]

I looks like this is an instance of FactoryBoy/factory_boy#820 where the same value is produced more than once.

Others have fixed this by not using faker and generating strings with incrementing numbers instead, e.g.

     message_id = factory.Faker("pystr", max_chars=12)
-    from_ = factory.Faker("safe_email")
-    to = factory.Faker("safe_email")
+    from_ = factory.Sequence(lambda n: f'person_from{n}@example.com')
+    to = factory.Sequence(lambda n: f'person_to{n}@example.com')
     subject = factory.Faker("sentence")

The tests seem to pass with the above change.

Expected behavior
The tests should pass.

To Reproduce
See above.

My Platform
main branch at 6fefaa1

Metadata

Metadata

Assignees

No one assigned

    Labels

    developer experienceAnything that improves the experience for Warehouse devstestingTest infrastructure and individual tests

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions