read and write trace data from cloud providers

[drmorr0]

• I certify that this PR does not contain any code that has been generated with GitHub Copilot or any other AI-based code generation tool, in accordance with this project's policies.

Description

Update sk-tracer to import and export data from cloud providers instead of just local storage.

Once apache/arrow-rs#5912 is resolved and a new version released, we can delete some of the code in here.

Docs have also been updated with the new trace information, and also some hooks info.

Testing done

• manual testing
• new tests added for objectstore, tracer export

Additional info

Resolves #61

[codecov]

Codecov Report

Attention: Patch coverage is 61.18421% with 59 lines in your changes missing coverage. Please review.

Project coverage is 73.14%. Comparing base (5dd669c) to head (44963de).

❗ Current head 44963de differs from pull request most recent head d4747b6

Please upload reports for the commit d4747b6 to get more accurate results.

Files	Patch %	Lines
tracer/errors.rs	0.00%	14 Missing ⚠️
lib/store/external_storage.rs	52.00%	12 Missing ⚠️
cli/export.rs	0.00%	11 Missing ⚠️
tracer/main.rs	50.00%	11 Missing ⚠️
ctrl/objects.rs	88.31%	9 Missing ⚠️
ctrl/main.rs	0.00%	1 Missing ⚠️
driver/main.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #120      +/-   ##
==========================================
- Coverage   73.71%   73.14%   -0.58%     
==========================================
  Files          39       39              
  Lines        2184     2245      +61     
==========================================
+ Hits         1610     1642      +32     
- Misses        574      603      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[drmorr0]

Kubernetes Object DAG

%%{init: {'themeVariables': {'mainBkg': '#ddd'}}}%%
graph LR

classDef default color:#000
subgraph global
  direction LR
  global/simkube[<b>Namespace</b><br>simkube]
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

subgraph sk-tracer
  direction LR
  simkube/sk-tracer-svc[<b>Service</b><br>sk-tracer-svc]
  simkube/sk-tracer-depl[<b>Deployment</b><br>sk-tracer-depl]
  simkube/sk-tracer-sa[<b>ServiceAccount</b><br>sk-tracer-sa]
  sk-tracer/sk-tracer-crb[<b>ClusterRoleBinding</b><br>sk-tracer-crb]
  simkube/sk-tracer-tracer-config[<b>ConfigMap</b><br>sk-tracer-tracer-config]
  simkube/sk-tracer-sa--->simkube/sk-tracer-depl
  sk-tracer/sk-tracer-crb--->simkube/sk-tracer-depl
  simkube/sk-tracer-tracer-config--->simkube/sk-tracer-depl
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

subgraph sk-ctrl
  direction LR
  simkube/sk-ctrl-depl[<b>Deployment</b><br>sk-ctrl-depl]
  simkube/sk-ctrl-sa[<b>ServiceAccount</b><br>sk-ctrl-sa]
  sk-ctrl/sk-ctrl-crb[<b>ClusterRoleBinding</b><br>sk-ctrl-crb]
  simkube/sk-ctrl-sa--->simkube/sk-ctrl-depl
  sk-ctrl/sk-ctrl-crb--->simkube/sk-ctrl-depl
%% DELETED OBJECTS START
%% DELETED OBJECTS END
end

global--->sk-tracer
global--->sk-ctrl

%% STYLE DEFINITIONS START
  style simkube/sk-ctrl-depl fill:#cb4
%% STYLE DEFINITIONS END

Loading

New object
Deleted object
Updated object
Updated object (causes pod recreation)

Detailed Diff

simkube/sk-ctrl-depl: ChangedWithPodRecreate

root['spec']['template']['spec']['containers'][0]['args'][1]:
not present --> "--driver-secrets"

root['spec']['template']['spec']['containers'][0]['args'][2]:
not present --> "simkube"