Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pantsd] Remove dead code around prefork graph warming #8002

Closed
blorente opened this issue Jul 3, 2019 · 1 comment
Closed

[pantsd] Remove dead code around prefork graph warming #8002

blorente opened this issue Jul 3, 2019 · 1 comment
Assignees

Comments

@blorente
Copy link
Contributor

blorente commented Jul 3, 2019

Before removing forking from pantsd (#7596 ), we needed to warm up the v2 product graph of the daemon before forking into a pantsd-runner process (relevant code here).

Now, however, this warming is not necessary (because we don't fork a process per run of pants anymore), so we can do some cleanup to that code:

  • We can remove the concept of "warming" (here), which is invoked from scheduler_service.prefork (here). This probably means that we can remove the entire warm_product_graph function.
  • We can reword the code around scheduler_service.prefork so that it doesn't mention forking at all. It was called prefork because that's what we did before Pantsd without forking #7596, but it can be called something different now.
  • We can merge the try:except block in DaemonPantsRunner.create() (here) into the try:except block in DaemonPantsRunner.run() (here). The run() function aims to centralize and correctly handle all the exceptions that happen on a pants run, so the "prefork" logic should be integrated there as well.

This will help a lot in correctly diagnosing pantsd problems, as this split and the wrangled exception handling that happen as a result have caused numerous issues.

@blorente blorente added the pantsd label Jul 3, 2019
@blorente blorente added this to TODO in Pants Daemon via automation Jul 3, 2019
@wisechengyi wisechengyi self-assigned this Jul 18, 2019
blorente pushed a commit that referenced this issue Jul 24, 2019
**Implementation for #8002**
Before removing forking from pantsd (#7596 ), we needed to warm up the v2 product graph of the daemon before forking into a pantsd-runner process (relevant code here).

Now, however, this warming is not necessary (because we don't fork a process per run of pants anymore), so we can do some cleanup to that code:

We can remove the concept of "warming" (here), which is invoked from scheduler_service.prefork (here). This probably means that we can remove the entire warm_product_graph function.
We can reword the code around scheduler_service.prefork so that it doesn't mention forking at all. It was called prefork because that's what we did before #7596, but it can be called something different now.
We can merge the try:except block in DaemonPantsRunner.create() (here) into the try:except block in DaemonPantsRunner.run() (here). The run() function aims to centralize and correctly handle all the exceptions that happen on a pants run, so the "prefork" logic should be integrated there as well.
This will help a lot in correctly diagnosing pantsd problems, as this split and the wrangled exception handling that happen as a result have caused numerous issues.
@blorente
Copy link
Contributor Author

Fixed by #8095

Pants Daemon automation moved this from TODO to Done Jul 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Pants Daemon
  
Done
Development

No branches or pull requests

2 participants