You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
buck2 log is pretty cool, since it lets you look at what build commands a user ran and what their output is, and you can do things like log cmd or log replay to watch the build. Is it possible or recommended to 'share' these log files? What would some infrastructure for doing that look like? Maybe something like:
Users run buck2 build ... a bunch
Their logs can get synchronized to something like an HTTP endpoint (maybe a proxy that just fronts an S3 bucket)
Something goes wrong, a user asks for help, posts their Build ID.
You could run buck2 log cmd --trace-id $BUILD_ID to find out what happened.
This would (transparently?) download the log files from some endpoint (somehow?)
There could be other useful things to derive from this possibly, assuming you had these logs. For example, you could use these logs as a direct source of build analytic information from users to derive information about build times, etc.
Is there any kind of "thing" like this inside Meta? Does this seem interesting? To be truly useful it would need some support in the core executable, I think. It seems like something kind of like what buck2 rage does, right? Except I'd want it in all cases, not just failures.
I imagine this could look like the following for OSS users:
A .buckconfig key like buck2_logs.upload_address is set to https://buck2-logs.aseipp.dev/
All .pb.zst files are POST'd to $UPLOAD_ADDRESS/upload with their trace ID as a primary key, asynchronously, by the daemon
When a user says buck2 log cmd --trace-id $TRACE_ID, then:
HTTP GET $UPLOAD_ADDRESS/trace/$TRACE_ID
This should return a json object containing metadata and a new address to find the log content at
That new address is the public URL to download the .pb.zst file
This allows users to materialize logs on demand
Something something something authorization with a bearer token
Ideally the HTTP protocol is simple enough to implement "by hand" in a short afternoon
The text was updated successfully, but these errors were encountered:
Actually, if --trace-id could take an https:// URL and download a file, you could probably do the first part completely outside of the core executable with a file watching tool? Just watch buck-out/v2/log and upload every file that gets written? Is that feasible and perhaps less invasive?
This has been incredibly useful for understanding and providing support for user builds.
I think it would be great if we could figure out the appropriate API for supporting this for open source users as well. I think it probably should be considered together with #226. The answer there may be that it's actually best to have separate apis for them (you'll note that internally we are doing them basically entirely separately).
One important thing we've found is that we want to upload the log incrementally as the build progresses to ensure that we get logs even on failed things (particularly our CI may aggressively kill off timing out or ooming things that makes it hard to reliably capture on failures).
buck2 log
is pretty cool, since it lets you look at what build commands a user ran and what their output is, and you can do things likelog cmd
orlog replay
to watch the build. Is it possible or recommended to 'share' these log files? What would some infrastructure for doing that look like? Maybe something like:buck2 build ...
a bunchbuck2 log cmd --trace-id $BUILD_ID
to find out what happened.There could be other useful things to derive from this possibly, assuming you had these logs. For example, you could use these logs as a direct source of build analytic information from users to derive information about build times, etc.
Is there any kind of "thing" like this inside Meta? Does this seem interesting? To be truly useful it would need some support in the core executable, I think. It seems like something kind of like what
buck2 rage
does, right? Except I'd want it in all cases, not just failures.I imagine this could look like the following for OSS users:
.buckconfig
key likebuck2_logs.upload_address
is set tohttps://buck2-logs.aseipp.dev/
.pb.zst
files are POST'd to$UPLOAD_ADDRESS/upload
with their trace ID as a primary key, asynchronously, by the daemonbuck2 log cmd --trace-id $TRACE_ID
, then:$UPLOAD_ADDRESS/trace/$TRACE_ID
.pb.zst
fileThe text was updated successfully, but these errors were encountered: