How to run functional tests locally?

[lmilbaum]

OS: macOS 12.6

I've pocked into the code to figure this out. The Makefile in #2311 is a draft of what I figured out. It doesn't work for me yet.
Would appreaciate some insights.

[nejch]

Hey @lmilbaum thanks for looking into things. Generally, you only need tox and docker installed, as documented here:
https://github.com/python-gitlab/python-gitlab/blob/main/CONTRIBUTING.rst#running-unit-tests
https://github.com/python-gitlab/python-gitlab/blob/main/CONTRIBUTING.rst#running-integration-tests

What are the errors you are getting?

Note that the functional tests take quite a while to run, hence the --keep-containers flag, which should help with reusing a running gitlab container. It's quite a chunky container that takes around 2 minutes to spin up.

[nejch]

@lmilbaum the docker setup is handled by the pytest-docker plugin (https://github.com/avast/pytest-docker) so it is implemented as pytest fixtures.

We basically just add a docker_compose_file fixture and the plugin then calls docker compose up with that config file:

python-gitlab/tests/functional/conftest.py

Lines 158 to 160 in 9a6d197

	@pytest.fixture(scope="session")
	def docker_compose_file(fixture_dir):
	return fixture_dir / "docker-compose.yml"

The plugin then provides a few more fixtures for us to easily connect to the service via its container IP etc.

It looks to me like pytest can't connect to your GitLab container, so could also be a networking issue there. If you try http://127.0.0.1:8080, are you able to reach this as the container is spinning up or when running docker-compose up manually as shown above? It will initially show 502 and then you should see the login screen after a few minutes.

[lmilbaum]

Thanks for the spinning up clarification. Got it.
The root cause of my issue is the timeout. It started to work as soon as the timeout value was increased.

[nejch]

Ah great, glad you got it figured out! I wonder if increasing the docker resources even more might help. That does look like the container was struggling a bit. I'll add some docs on that as it's the second time this has happened on mac.

[lmilbaum]

Documenting this issue is a good idea.
We could also consider waiting with running the functional tests until the gitlab service is healthy. And, if it doesn't become healthy in a specific time frame then issuing an error and exit.
That could help the developer in the mitigation process. WDYT?

[nejch]

I think we wanted to have some sensible timeout so that if the startup does actually fail for whatever reason (maybe even just port already in use), then the developer is not left waiting in the dark for too long. 300 seconds is already quite long IMO :)

I think it's just very unclear right now what happens at the moment. Instead, I think we could call pytest.exit() to prevent starting all the tests. We can probably do this here:

python-gitlab/tests/functional/conftest.py

Lines 262 to 270 in 8dbaa5c

	docker_services.wait_until_responsive(
	timeout=300,
	pause=10,
	check=lambda: check_is_alive(
	container=gitlab_container_name,
	start_time=start_time,
	gitlab_url=gitlab_url,
	),
	)

I've opened avast/pytest-docker#88 so we can later narrow down the exception there.

[JohnVillalovos]

One thing that could be done is enable the verbose mode by default. So that it outputs the status of what is happening like is currently being done in the CI.

For me personally I enable the verbose mode when running locally.

[lmilbaum]

Could you please provide the steps for enabling the verbose mode?

[JohnVillalovos]

Sure. It is done here in the CI:

python-gitlab/.github/workflows/test.yml

Line 76 in 8dbaa5c

run: tox -- --override-ini='log_cli=True'

So I use this locally, to run the CLI functional tests. Would use similar arguments for the API functional tests:
$ tox -e cli_func_v4 -- --override-ini='log_cli=True'

[lmilbaum]

Thanks a lot