Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload the colmap rrd file to gcloud #1666

Merged
merged 3 commits into from
Mar 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 49 additions & 2 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -238,9 +238,10 @@ jobs:
shell: bash
# If you change the line below you should almost definitely change the `key:` line above by giving it a new, unique name
run: |
mkdir rrd
pip install -r examples/python/colmap/requirements.txt
python3 examples/python/colmap/main.py --dataset colmap_fiat --resize 800x600 --save colmap.rrd
cp colmap.rrd unpack-dist/${{ env.pkg_folder }}/rerun_sdk/rerun_demo/colmap.rrd
python3 examples/python/colmap/main.py --dataset colmap_fiat --resize 800x600 --save rrd/colmap_fiat.rrd
cp rrd/colmap_fiat.rrd unpack-dist/${{ env.pkg_folder }}/rerun_sdk/rerun_demo/colmap_fiat.rrd

- name: Repack the wheel
shell: bash
Expand All @@ -254,8 +255,54 @@ jobs:
name: wheels
path: dist

# All platforms are currently creating the same rrd file, upload one of them
- name: Save RRD artifact
if: matrix.platform == 'linux'
uses: actions/upload-artifact@v3
with:
name: rrd
path: rrd

# ---------------------------------------------------------------------------

upload_rrd:
name: Upload RRD to GCloud
permissions:
contents: "read"
id-token: "write"
needs: [wheels]
runs-on: "ubuntu-latest"
steps:
- name: Download Artifact
uses: actions/download-artifact@v3
with:
name: rrd
path: rrd

- id: "auth"
uses: google-github-actions/auth@v1
with:
workload_identity_provider: ${{ secrets.GOOGLE_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GOOGLE_SERVICE_ACCOUNT }}

- name: Add SHORT_SHA env property with commit short sha
run: echo "SHORT_SHA=`echo ${{github.sha}} | cut -c1-8`" >> $GITHUB_ENV

- name: "Upload RRD"
uses: google-github-actions/upload-cloud-storage@v1
with:
path: "rrd"
destination: "rerun-example-rrd/commit/${{env.SHORT_SHA}}"
parent: false

- name: "Upload RRD (tagged)"
if: startsWith(github.ref, 'refs/tags/v')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should do if: startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/tags/latest' here and elsewhere to always get a latest web viewer automatically

Copy link
Member Author

@jleibs jleibs Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally agree but I'm going to handle the latest / prerelease logic a bit differently in another PR. There's a bit of a mess around what does / doesn't trigger a workflow. github.ref doesn't mean "this commit happens to be tagged with latest" but rather "pushing the tag latest is what triggered this workflow." But running the whole pipeline a second time (including wheel building on all platforms) seems heavy-handed to effectively update a pointer.

uses: google-github-actions/upload-cloud-storage@v1
with:
path: "rrd"
destination: "rerun-example-rrd/version/${{github.ref_name}}"
parent: false

# See https://github.com/ncipollo/release-action
pre-release:
name: Pre Release
Expand Down
5 changes: 4 additions & 1 deletion crates/re_web_viewer_server/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,10 @@ impl Service<Request<Body>> for Svc {
let response = Response::builder();

let (mime, bytes) = match req.uri().path() {
"/" | "/index.html" => ("text/html", &include_bytes!("../web_viewer/index.html")[..]),
"/" | "/index.html" => (
"text/html",
&include_bytes!("../web_viewer/index_bundled.html")[..],
),
"/favicon.ico" => (
"image/vnd.microsoft.icon",
&include_bytes!("../web_viewer/favicon.ico")[..],
Expand Down
2 changes: 1 addition & 1 deletion rerun_py/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,6 @@ ban-relative-imports = "all"
# with the other `rerun` pypi package. The rerun_sdk.pth adds this to the pythonpath
# which then allows `import rerun` to work as expected.
# See https://github.com/rerun-io/rerun/pull/1085 for more details
include = ["rerun_sdk.pth", "rerun_sdk/rerun_demo/colmap.rrd"]
include = ["rerun_sdk.pth", "rerun_sdk/rerun_demo/colmap_fiat.rrd"]
locked = true
python-packages = ["rerun_sdk/rerun", "rerun_sdk/rerun_demo"]
2 changes: 1 addition & 1 deletion rerun_py/rerun_sdk/rerun_demo/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def run_colmap():
# We don't need to call shutdown in this case. Rust should be handling everything
unregister_shutdown()

rrd_file = pathlib.Path(__file__).parent.joinpath("colmap.rrd").resolve()
rrd_file = pathlib.Path(__file__).parent.joinpath("colmap_fiat.rrd").resolve()
if not rrd_file.exists():
print("No demo file found at {}. Package was built without demo support".format(rrd_file), file=sys.stderr)
exit(1)
Expand Down
13 changes: 12 additions & 1 deletion web_viewer/index.html
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<!DOCTYPE html>
<!-- The version of index.html that will be served by app.rerun.io -->
<html>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Expand Down Expand Up @@ -133,8 +134,18 @@
function on_wasm_loaded() {
console.debug("wasm loaded. starting app…");

// The expected data path is the current pathname relocated to inside of "/data" with the
// index.html stripped off if it's present.
// exa: 'https://app.rerun.io/version/v4.0.0/index.html' -> '/data/version/v4.0.0/'
let data_path = "/data" + window.location.pathname.replace(/index\.html$/, '');

const url_params = new URLSearchParams(window.location.search);
const rrd_file = url_params.get('file') || 'colmap_fiat.rrd';

// By default load an rrd file over http
let url = data_path + rrd_file; // you can use this to point to an .rrd file over http, or a WebSocket Rerun server.

// This call installs a bunch of callbacks and then returns:
let url = null; // you can use this to point to an .rrd file over http, or a WebSocket Rerun server.
wasm_bindgen.start("the_canvas_id", url);

console.debug("app started.");
Expand Down
163 changes: 163 additions & 0 deletions web_viewer/index_bundled.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
<!DOCTYPE html>
<!-- The version of index.html bundled with the Rerun SDK. -->
<html>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

<!-- Disable zooming: -->
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">

<head>
<title>rerun viewer</title>
<style>
html {
/* Remove touch delay: */
touch-action: manipulation;
}

body {
/* Light mode background color for what is not covered by the egui canvas,
or where the egui canvas is translucent. */
background: #909090;
}

@media (prefers-color-scheme: dark) {
body {
/* Dark mode background color for what is not covered by the egui canvas,
or where the egui canvas is translucent. */
background: #404040;
}
}

/* Allow canvas to fill entire web page: */
html,
body {
overflow: hidden;
margin: 0 !important;
padding: 0 !important;
height: 100%;
width: 100%;
}

/* Position canvas in center-top: */
canvas {
margin-right: auto;
margin-left: auto;
display: block;
position: absolute;
top: 0%;
left: 50%;
transform: translate(-50%, 0%);
}

.centered {
margin-right: auto;
margin-left: auto;
display: block;
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
color: #f0f0f0;
font-size: 24px;
font-family: Ubuntu-Light, Helvetica, sans-serif;
text-align: center;
}

/* ---------------------------------------------- */
/* Loading animation from https://loading.io/css/ */
.lds-dual-ring {
display: inline-block;
width: 24px;
height: 24px;
}

.lds-dual-ring:after {
content: " ";
display: block;
width: 24px;
height: 24px;
margin: 0px;
border-radius: 50%;
border: 3px solid #fff;
border-color: #fff transparent #fff transparent;
animation: lds-dual-ring 1.2s linear infinite;
}

@keyframes lds-dual-ring {
0% {
transform: rotate(0deg);
}

100% {
transform: rotate(360deg);
}
}

</style>
</head>

<body>
<!-- The Wasm code will resize the canvas dynamically -->
<canvas id="the_canvas_id"></canvas>
<div class="centered" id="center_text">
<p style="font-size:16px">
Loading…
</p>
<div class="lds-dual-ring"></div>
</div>

<script>
// The `--no-modules`-generated JS from `wasm-bindgen` attempts to use
// `WebAssembly.instantiateStreaming` to instantiate the wasm module,
// but this doesn't work with `file://` urls. This example is frequently
// viewed by simply opening `index.html` in a browser (with a `file://`
// url), so it would fail if we were to call this function!
//
// Work around this for now by deleting the function to ensure that the
// `no_modules.js` script doesn't have access to it. You won't need this
// hack when deploying over HTTP.
delete WebAssembly.instantiateStreaming;
</script>

<!-- this is the JS generated by the `wasm-bindgen` CLI tool -->
<script src="re_viewer.js"></script>

<script>
// We'll defer our execution until the wasm is ready to go.
// Here we tell bindgen the path to the wasm file so it can start
// initialization and return to us a promise when it's done.
console.debug("loading wasm…");
wasm_bindgen("./re_viewer_bg.wasm")
.then(on_wasm_loaded)
.catch(on_wasm_error);

function on_wasm_loaded() {
console.debug("wasm loaded. starting app…");

// This call installs a bunch of callbacks and then returns:
let url = null; // you can use this to point to an .rrd file over http, or a WebSocket Rerun server.
wasm_bindgen.start("the_canvas_id", url);

console.debug("app started.");
document.getElementById("center_text").remove();
}

function on_wasm_error(error) {
console.error("Failed to start: " + error);
document.getElementById("center_text").innerHTML = `
<p>
An error occurred during loading:
</p>
<p style="font-family:Courier New">
${error}
</p>
<p style="font-size:14px">
Make sure you use a modern browser with WebGL and Wasm enabled.
</p>`;
}
</script>
</body>

</html>

<!-- Powered by egui: https://github.com/emilk/egui/ -->