Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify lock acquire #5695

Merged
merged 1 commit into from May 16, 2019
Merged

Simplify lock acquire #5695

merged 1 commit into from May 16, 2019

Conversation

@stsewd
Copy link
Member

@stsewd stsewd commented May 14, 2019

Currently we are acquiring 3 locks.
We should only have 2 (we can't have only one because of #5672)

Basically I'm removing the locks from setup_python_env and build_docs to only acquire the lock in a central place (run_build)

Currently we are acquiring 3 locks.
We should only have 2 (we can't have only one because of readthedocs#5672)
@stsewd
Copy link
Member Author

@stsewd stsewd commented May 14, 2019

I still want to test this more locally before merging :)

@stsewd stsewd mentioned this pull request May 14, 2019
@stsewd
Copy link
Member Author

@stsewd stsewd commented May 14, 2019

Locking is working, this is ready for review.

@stsewd stsewd requested a review from May 14, 2019
@humitos
Copy link
Member

@humitos humitos commented May 15, 2019

Why not to acquire the lock immediately inside the run method of the build task so the whole build process (API query, setup, build docs, etc) is inside the lock?

Otherwise, the clean_build_task from #5680 could clean these files while we are performing the run_setup step.

@stsewd
Copy link
Member Author

@stsewd stsewd commented May 15, 2019

That's the trick, the lock needs to be acquired inside the with self.build_env context manager. Because build_env catch the VersionLocked error, otherwise the task will not shown as failed and would be retying without showing anything to the user.

@humitos
Copy link
Member

@humitos humitos commented May 16, 2019

Yeah, but we can create the self.build_env (or self.setup_env, it's the same for this) immediately after getting the project, version, build and env vars from the API. That's all what we need to create it and we can use the context manager after that wrapping the rest of the execution. Don't you think?

@stsewd
Copy link
Member Author

@stsewd stsewd commented May 16, 2019

There are two build environments, that's why we need two locks.

build_env_1 -> lock
build_env_2 -> lock

Nesting isn't possible due how build_env works and expects lock to be inside the build env context manager.

Also, the second build_env depends on the first, which gets the config file to initialize the next build_env with the correct docker image.

Copy link
Member

@ericholscher ericholscher left a comment

This makes a lot more sense 💯

@stsewd stsewd merged commit 616a303 into readthedocs:master May 16, 2019
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants