Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit tests fail on fresh dev install on Ubuntu #483

Closed
DanBeard opened this issue Jun 14, 2022 · 16 comments
Closed

Unit tests fail on fresh dev install on Ubuntu #483

DanBeard opened this issue Jun 14, 2022 · 16 comments
Labels
bug Reports of unexpected problems or errors i18n/L10n Issues relating to internationalization and/or localization
Milestone

Comments

@DanBeard
Copy link
Contributor

DanBeard commented Jun 14, 2022

Unit tests fail on fresh development install. This originally happened on my box with Ubuntu 20.04.4. I figured it might have been something weird in my local python dev environment. I wanted to start fresh and try out PopOS, so I reformatted my machine and saw the same failures on a brand new fresh OS install.

OS: PopOS 22.04 (based on Ubuntu 22.04) and Ubuntu 20.04.4
Steps to reproduce:

  1. clone master on fresh install of ubuntu
  2. follow instructions on https://docs.baby-buddy.net/contributing/development-environment/#installation
  3. run gulp test

Expected result:
All unit tests pass.

Actual result:
failures.log

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Huh... those failures are bizarre. I'll give this a try in a fresh environment.

@DanBeard
Copy link
Contributor Author

DanBeard commented Jun 14, 2022

Looking at TemperatureFormsTestCase, The refresh still shows the old value. Maybe a timing issue?

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Test working for me testing on Digital Ocean droplet with Ubuntu 20.04.4.

root@ubuntu-s-1vcpu-1gb-sfo3-01:~/babybuddy# cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

ubuntu-20.04.4-lts.log

Will try 22.04 next. Anything else about your setup that may have an effect here?

The steps to get set up do seem a bit dated, I had to fiddle with pipenv and stuff to get it installed and happy. Maybe some difference about that process creates an issue?

@DanBeard
Copy link
Contributor Author

DanBeard commented Jun 14, 2022

On my old OS maybe. Even though pipenv should prevent it, I did had done lot of development so there might have been some weird dependency or env config that creeped in.

After starting from scratch with PopOS, this is literally the first project I cloned and installed. Only other thing installed beyond stock is Pycharm.

Some more env info:
python --version : Python 3.10.4
nvm list: default -> 16 (-> v16.15.1)
pipenv graph: pipenv.log

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Tests also working on Digital Ocean droplet with Ubuntu 22.04.

babybuddy@ubuntu-22-04:~/babybuddy$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

ubuntu-22.04-lts.log

Note in the above log I include my entire session from a fresh Ubuntu 22.04 install. Maybe you can spot some difference in Python or pipenv setup steps?

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Some more env info:
python --version : Python 3.10.4
nvm list: default -> 16 (-> v16.15.1
pipenv graph: pipenv.log
)

Looks to be almost exactly the same in Ubuntu 22.04 test I just did --

babybuddy@ubuntu-22-04:~/babybuddy$ pipenv run python --version
Python 3.10.4
babybuddy@ubuntu-22-04:~/babybuddy$ nvm list
->     v16.15.1
default -> 16 (-> v16.15.1)
babybuddy@ubuntu-22-04:~/babybuddy$ pipenv graph

pipenv.log

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Is it always the same tests that fail? It's weird that it is just a subset of those form tests while there are others with almost exactly the same test that do not fail. I'm not seeing any significant differences...

Here's one of the example failing ones --

def test_edit(self):
params = {
"child": self.change.child.id,
"time": self.localtime_string(),
"wet": self.change.wet,
"solid": self.change.solid,
"color": self.change.color,
"amount": 1.23,
}
page = self.c.post("/changes/{}/".format(self.change.id), params, follow=True)
self.assertEqual(page.status_code, 200)
self.change.refresh_from_db()
self.assertEqual(self.change.amount, params["amount"])
self.assertContains(
page, "Diaper Change entry for {} updated".format(str(self.change.child))
)

Fails at line 274 after attempting the POST. I wonder if you could drop some debugging in there to see what the state of the page is after the POST? I don't recall the best way to do this in Python hah.

@DanBeard
Copy link
Contributor Author

DanBeard commented Jun 14, 2022

yeah i did a GET and it still showed the old temp value. something really fishy is up.

Now that you mention it it's usually those tests, but after running it a few times sometimes I get these additional failures and errors:
failures2.log

this definitely feels like a timing or caching or env issue. The only differences I saw from your log were using pipx to install pipenv, so I uninstalled pipenv and used pipx to reinstall it, and it still showed the same thing. Also I don't see where you did "export DJANGO_SETTINGS_MODULE=babybuddy.settings.development" or "pipenv shell" before running gulp. Could it be using the wrong settings in test on my env?

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Also I don't see where you did "export DJANGO_SETTINGS_MODULE=babybuddy.settings.development" or "pipenv shell" before running gulp.

No, sorry for the confusion -- I did do that before running the gulp commands but I had to hack together my log a little bit to remove some mistypes haha.

For trying to track down exactly what is going on... I think there is some way to just spew out the whole page content after the POST. I'm curious to know if the page has some error on it that could give us more useful information...

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Oh hey! I can repro the error when running a single test case...

~/PycharmProjects/babybuddy master *4 > pipenv run python manage.py test --settings=babybuddy.settings.test core.tests.tests_forms.DiaperChangeFormsTestCase
Loading .env environment variables...
Found 3 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
..F
======================================================================
FAIL: test_edit (core.tests.tests_forms.DiaperChangeFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 274, in test_edit
    self.assertEqual(self.change.amount, params["amount"])
AssertionError: 0.45 != 1.23

----------------------------------------------------------------------
Ran 3 tests in 0.357s

FAILED (failures=1)
Destroying test database for alias 'default'...

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Ok yeah this was my guess... when I print(page.content) after running that POST there is an error on the datetime field of "Enter a valid date/time."

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Not seeing any obvious reason for this... running all the form tests at once also passes.

~/PycharmProjects/babybuddy master *4 !1 > pipenv run python manage.py test --settings=babybuddy.settings.test core.tests.tests_forms
Loading .env environment variables...
Found 53 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.....................................................
----------------------------------------------------------------------
Ran 53 tests in 4.129s

OK
Destroying test database for alias 'default'...

@DanBeard could you test that and see if you still get fails?

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

More oddness... if I remove the first three test cases specifically (InitialValuesTestCase, BMIFormsTest, and ChildFormsTestCase) I can (consistently) reproduce more of the random test fails you get even when running the full group.

Removing any of those cases individually or any other combination of cases appears to have no effect at all (i.e., running as a group passes and running just DiaperChangeFormsTestCase fails). 🤯

~/PycharmProjects/babybuddy master *4 !1 > pipenv run python manage.py test --settings=babybuddy.settings.test core.tests.tests_forms
Loading .env environment variables...
Found 42 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
..F..F...........F..F......F..F...F.EEF...
======================================================================
ERROR: test_validate_duration (core.tests.tests_forms.ValidationsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 642, in test_validate_duration
    page = self.c.post("/tummy-time/add/", params, follow=True)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 852, in post
    response = super().post(
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 441, in post
    return self.generic(
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 541, in generic
    return self.request(**r)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 810, in request
    self.check_exception(response)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 663, in check_exception
    raise exc_value
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/base.py", line 84, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/base.py", line 119, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/edit.py", line 184, in post
    return super().post(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/edit.py", line 152, in post
    if form.is_valid():
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 200, in errors
    self.full_clean()
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 435, in full_clean
    self._post_clean()
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/models.py", line 492, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/db/models/base.py", line 1390, in full_clean
    self.clean()
  File "/Users/wellc/PycharmProjects/babybuddy/core/models.py", line 662, in clean
    validate_unique_period(TummyTime.objects.filter(child=self.child), self)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 215, in __get__
    raise self.RelatedObjectDoesNotExist(
core.models.TummyTime.child.RelatedObjectDoesNotExist: TummyTime has no child.

======================================================================
ERROR: test_validate_time (core.tests.tests_forms.ValidationsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 663, in test_validate_time
    page = self.c.post("/tummy-time/add/", params, follow=True)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 852, in post
    response = super().post(
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 441, in post
    return self.generic(
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 541, in generic
    return self.request(**r)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 810, in request
    self.check_exception(response)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/client.py", line 663, in check_exception
    raise exc_value
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/base.py", line 84, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/base.py", line 119, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/edit.py", line 184, in post
    return super().post(request, *args, **kwargs)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/views/generic/edit.py", line 152, in post
    if form.is_valid():
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 200, in errors
    self.full_clean()
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/forms.py", line 435, in full_clean
    self._post_clean()
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/forms/models.py", line 492, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/db/models/base.py", line 1390, in full_clean
    self.clean()
  File "/Users/wellc/PycharmProjects/babybuddy/core/models.py", line 662, in clean
    validate_unique_period(TummyTime.objects.filter(child=self.child), self)
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 215, in __get__
    raise self.RelatedObjectDoesNotExist(
core.models.TummyTime.child.RelatedObjectDoesNotExist: TummyTime has no child.

======================================================================
FAIL: test_edit (core.tests.tests_forms.DiaperChangeFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 86, in test_edit
    self.assertEqual(self.change.amount, params["amount"])
AssertionError: 0.45 != 1.23

======================================================================
FAIL: test_edit (core.tests.tests_forms.FeedingFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 139, in test_edit
    self.assertEqual(self.feeding.amount, params["amount"])
AssertionError: 2.5 != 100

======================================================================
FAIL: test_edit (core.tests.tests_forms.PumpingFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 317, in test_edit
    self.assertEqual(self.bp.amount, params["amount"])
AssertionError: 50.0 != 52.0

======================================================================
FAIL: test_edit (core.tests.tests_forms.SleepFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 365, in test_edit
    self.assertEqual(self.localtime_string(self.sleep.end), params["end"])
AssertionError: '06/14/2022 04:57:22 PM' != '06/14/2022 08:57:22 PM'
- 06/14/2022 04:57:22 PM
?             ^
+ 06/14/2022 08:57:22 PM
?             ^


======================================================================
FAIL: test_edit (core.tests.tests_forms.TemperatureFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 506, in test_edit
    self.assertEqual(self.temp.temperature, params["temperature"])
AssertionError: 98.6 != 100.6

======================================================================
FAIL: test_edit (core.tests.tests_forms.TimerFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 592, in test_edit
    self.assertEqual(self.localtime_string(self.timer.start), params["start"])
AssertionError: '06/14/2022 08:57:23 PM' != '06/14/2022 07:57:23 PM'
- 06/14/2022 08:57:23 PM
?             ^
+ 06/14/2022 07:57:23 PM
?             ^


======================================================================
FAIL: test_edit (core.tests.tests_forms.TummyTimeFormsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 555, in test_edit
    self.assertEqual(self.tt.milestone, params["milestone"])
AssertionError: '' != 'Moved head!'
+ Moved head!

======================================================================
FAIL: test_validate_unique_period (core.tests.tests_forms.ValidationsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/wellc/PycharmProjects/babybuddy/core/tests/tests_forms.py", line 685, in test_validate_unique_period
    self.assertFormError(
  File "/Users/wellc/.local/share/virtualenvs/babybuddy-76MCX6jD/lib/python3.9/site-packages/django/test/testcases.py", line 613, in assertFormError
    self.assertTrue(
AssertionError: False is not true : The form 'form' in context 0 does not contain the non-field error 'Another entry intersects the specified time period.' (actual errors: none)

----------------------------------------------------------------------
Ran 42 tests in 3.350s

FAILED (failures=8, errors=2)
Destroying test database for alias 'default'...
~/PycharmProjects/babybuddy master *4 !1 >                                                                           8s node 16.13.0 13:57:24

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Ok so it looks like this relates in some manner to #335 and #337, unfortunately. These issues had to deal with customizing how Django handles datetime formats in order to be better support 24H overrides and datetime handling in the frontend with moment.js.

If I comment out the special handling added for those issues (below) tests pass no matter how they are run.

if language == "en-US":
update_en_us_date_formats()
elif language == "en-GB":
update_en_gb_date_formats()

Interestingly even the formats test cases that are supposed to be testing that feature pass so either these tests are not testing what I intended or the changes they are testing are not actually necessary.

~/PycharmProjects/babybuddy master *4 !1 > pipenv run python manage.py test --settings=babybuddy.settings.test babybuddy.tests.formats
Loading .env environment variables...
Found 5 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.....
----------------------------------------------------------------------
Ran 5 tests in 0.032s

OK
Destroying test database for alias 'default'...

Either way this seems like it's going to be complicated to unravel... :feelsgood:

@cdubz cdubz added bug Reports of unexpected problems or errors i18n/L10n Issues relating to internationalization and/or localization labels Jun 14, 2022
@DanBeard
Copy link
Contributor Author

Nice! Can confirm that commenting out those lines solves it for me locally.

Not sure how you figured that out but at least I can run the tests now locally for another feature that my wife request 😅

@cdubz
Copy link
Member

cdubz commented Jun 14, 2022

Not sure how you figured that out [...]

Haha, far too much time spent working on this app! (: Date time issues were my first guess when this came up and that (relatively) recent change to the format handling was a pretty clear potential culprit.

I wonder if in the short term I can come up with some way to short-circuit that for tests... 🤔 At the moment I have no idea how to solve for this. I'd bet there are actual bugs hiding away somewhere beyond just inconsistent test results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Reports of unexpected problems or errors i18n/L10n Issues relating to internationalization and/or localization
Projects
None yet
Development

No branches or pull requests

2 participants