-
Notifications
You must be signed in to change notification settings - Fork 19
Description
In order to annotate how many bytes were transferred in a given download request, the upstream log needs to emit bytes that would be parsed here.
It appears that with parse_all=True
, the parser format would need to be updated to allow for the existence of the new field. Does that mean duplicating MESSAGE_V3
to a MESSAGE_V4
with the added param at the end?
linehaul-cloud-function/linehaul/events/parser.py
Lines 121 to 124 in 8b8ed9d
V3_HEADER = L("download") | |
MESSAGE_v3 = ( | |
V3_HEADER + PIPE + REQUEST + PIPE + TLS + PIPE + PROJECT + PIPE + USER_AGENT | |
) |
Will need to add resp.body_bytes_written
to https://github.com/pypi/infra/blob/d6702898cbed406cc431a12b80270831ef292557/terraform/file-hosting/vcl/files.vcl#L589 to emit the value when this library can parse for it.
Will likely need coordination with BigQuery team to add column when the data is correctly parsed.