Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a command to the sccache client to query distributed status #390

Closed
luser opened this issue Mar 4, 2019 · 0 comments
Closed

Add a command to the sccache client to query distributed status #390

luser opened this issue Mar 4, 2019 · 0 comments
Assignees

Comments

@luser
Copy link
Contributor

luser commented Mar 4, 2019

With an eye towards making the Firefox build system able to use distributed builds in a seamless way it would be nice if sccache exposed a command to query the status of its distributed compilation configuration. I'm imagining something like sccache dist-status which could return information about possible states:

  1. Distributed compilation is disabled, either because sccache wasn't built with the dist-client feature or because it's not configured in the client's config file. (It might be worthwhile to report these as separate states to help users troubleshoot.)
  2. Distributed compilation is enabled but we don't have a working connection to the scheduler. We should report the reason as well. This could be because the scheduler is not reachable or because authentication isn't configured properly.
  3. Distributed compilation is enabled, configured, and working. Ideally the output here would also include the number of CPU cores available to the scheduler from all build servers. The scheduler doesn't currently expose that info, although it does have a reported CPU count for each server in its list of servers internally. This would be useful so the build could set the right amount of parallelism automatically without requiring developers to manually pass -j100 or other arbitrary values. We could change the scheduler's /status API to include a num_cores alongside the num_servers value it currently reports. The SchedulerStatusResult struct is defined here.

In terms of implementation, the sccache server (that runs client-side) has a dist_client field that contains information about the current state of distributed compilation. Right now that struct only stores enough information to distinguish between "disabled" , "configured but not currently working", and "working". It ought to be straightforward to extend things to provide finer-grained information about the current state. The "not compiled with distributed support enabled" case is gated on #[cfg(feature = "dist-client")] cfg blocks, and all "configured but not working" situations will result in using the DistClientState::RetryCreateAt variant of that enum which doesn't currently capture any reason about why things aren't working, but that could be added. Additionally I don't think the server will actually contact the scheduler until a compilation request is made, so we would likely need to make an actual request to the scheduler to verify that. Hitting the /status API would accomplish that as well as provide a way to expose the number of available cores which would be useful. The client->scheduler API is invoked by trait methods on the dist::Client trait which is implemented in the dist::http::Client struct. Extending that to expose the status API ought to be straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants