-
Notifications
You must be signed in to change notification settings - Fork 10
fluxorama with accounting: Interactive Docker Container Setup Run Instructions
Christopher Moussa edited this page Jan 8, 2021
·
2 revisions
From your terminal, check your Docker version:
moussa1@trinity1 ~$ docker --version
Docker version 19.03.8, build afacb8bPull down the fluxorama-with-accounting image from Docker Hub:
moussa1@trinity1 ~$ docker pull moussa1/fluxorama-with-accounting:latest
Using default tag: latest
latest: Pulling from moussa1/fluxorama-with-accounting
.
.
.
Digest: sha256:a5ea6d8b185521a5277f2efaacb811ef69dedd41c99f918a1a791a15a3d1eab8
Status: Downloaded newer image for moussa1/fluxorama-with-accounting:latest
docker.io/moussa1/fluxorama-with-accounting:latestYou can run the Docker container with the following command:
moussa1@trinity1 ~$ FLUXUSER_PASSWORD=foo docker run -e FLUXUSER_PASSWORD --tmpfs=/run -ti -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p127.0.0.1:8000:7681 moussa1/fluxorama-with-accountingThen, from your web browser, go to localhost:8000. This will bring up an interactive terminal in your browser from which you can run flux commands:
Welcome to Flux-o-rama! A Flux system instance test environment and playground.
[fluxuser@1ed8a64247a0 ~]$ flux --version
commands: 0.22.0-75-gd35f64dbc
libflux-core: 0.22.0-75-gd35f64dbc
libflux-security: 0.4.0
build-options: +caliper+hwloc==1.11.6Load the job-archive module:
[fluxuser@617c5132623c ~]$ sudo flux module load job-archive dbpath=$(flux getattr rundir)/foo.dbYou can submit jobs as fluxuser:
[fluxuser@1880c353bc24 ~]$ flux mini submit -n 3 hostname
ƒhkqosyy
[fluxuser@1880c353bc24 ~]$Or as another user:
[user2@1ed8a64247a0 ~]$ flux mini run -n 1 hostname
1ed8a64247a0
[user2@1ed8a64247a0 ~]$You can query and look at past jobs using Flux's job-archive module, which stores inactive jobs in a SQLite database. The DB is located at /run/flux/jobs.db.
[fluxuser@1ed8a64247a0 ~]$ flux account -p /run/flux/foo.db view-job-records
UserID Username JobID T_Submit T_Run T_Inactive Nodes R
0 2202 user2 2587097038848 1610131263.48355 1610131263.50468 1610131263.56226 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expir...
1 2202 user2 2570588258304 1610131262.49958 1610131262.52066 1610131262.57514 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expir...
2 2202 user2 2452946419712 1610131255.48759 1610131255.50676 1610131255.56094 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expir...
3 2202 user2 2347166072832 1610131249.18232 1610131249.20115 1610131249.25639 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expir...
4 1002 fluxuser 1880927240192 1610131221.39146 1610131221.41160 1610131221.48441 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0-2"}}],"starttime":0.0,"exp...
5 1002 fluxuser 1551523381248 1610131201.75744 1610131201.77599 1610131201.85066 1 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0-2"}}],"starttime":0.0,"exp...You can also open up a SQLite shell to run direct queries on the job-archive DB:
[fluxuser@1ed8a64247a0 ~]$ sqlite3 /run/flux/foo.db Let's add some formatting to our queries to make output look nicer:
sqlite> .mode columns
sqlite> .headers on
Here are the columns of the jobs table in the job-archive DB:
sqlite> PRAGMA table_info(jobs);
cid name type notnull dflt_value pk
---------- ---------- ---------- ---------- ---------- ----------
0 id CHAR(16) 0 1
1 userid INT 0 0
2 ranks TEXT 0 0
3 t_submit REAL 0 0
4 t_run REAL 0 0
5 t_cleanup REAL 0 0
6 t_inactive REAL 0 0
7 eventlog TEXT 0 0
8 jobspec TEXT 0 0
9 R TEXT 0 0 And we can write queries to get job info:
sqlite> SELECT userid,id,t_submit,t_run,t_inactive,R FROM jobs;
userid id t_submit t_run t_inactive R
---------- ------------- ---------------- ---------------- ---------------- --------------------------------------------------------------------------------------------------------------------
2202 2587097038848 1610131263.48355 1610131263.50468 1610131263.56226 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"]}}
2202 2570588258304 1610131262.49958 1610131262.52066 1610131262.57514 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"]}}
2202 2452946419712 1610131255.48759 1610131255.50676 1610131255.56094 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"]}}
2202 2347166072832 1610131249.18232 1610131249.20115 1610131249.25639 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"]}}
1002 1880927240192 1610131221.39146 1610131221.4116 1610131221.48441 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0-2"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"]
1002 1551523381248 1610131201.75744 1610131201.77599 1610131201.85066 {"version":1,"execution":{"R_lite":[{"rank":"0","children":{"core":"0-2"}}],"starttime":0.0,"expiration":0.0,"nodelist":["1ed8a64247a0"] flux-accounting is installed to /usr/src/.
You can create the flux-accounting database (FluxAccounting.db). For the purposes of this walkthrough, I will make one in /home/fluxuser/accounting-DB:
[fluxuser@f5c590dd2704 ~]$ mkdir accounting-DB
[fluxuser@f5c590dd2704 ~]$ cd accounting-DB/
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account create-db FluxAccounting.dbYou should be able to add banks, users, and view the hierarchy tree:
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-bank A 1
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-bank --parent-bank=A B 1
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-bank --parent-bank=A C 1
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-user --username=user1 --bank=B
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-user --username=user2 --bank=B
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-user --username=user3 --bank=B
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db add-user --username=user4 --bank=C
[fluxuser@1ed8a64247a0 accounting-DB]$ flux account -p FluxAccounting.db print-hierarchy
Bank|User|RawShares
A||1
B||1
B|user1|1
B|user2|1
B|user3|1
C||1
C|user4|1The user fluxuser has permissions to write/edit values in the database. You can run any of the flux account commands; just remember to either specify the path to the database when running any of the commands; not specifying a path will default to /usr/var/FluxAccounting.db:
[fluxuser@30451d797d82 accounting-DB]$ flux account -h
usage: flux-account.py [-h] [-p PATH] [-o OUTPUT_FILE]
{view-user,add-user,delete-user,edit-user,view-job-records,create-db,add-bank,view-bank,delete-bank,edit-bank,print-hierarchy}
...
Description: Translate command line arguments into SQLite instructions for the
Flux Accounting Database.
positional arguments:
{view-user,add-user,delete-user,edit-user,view-job-records,create-db,add-bank,view-bank,delete-bank,edit-bank,print-hierarchy}
sub-command help
view-user view a user's information in the accounting database
add-user add a user to the accounting database
delete-user remove a user from the accounting database
edit-user edit a user's value
view-job-records view job records
create-db create the flux-accounting database
add-bank add a new bank
view-bank view bank information
delete-bank remove a bank
edit-bank edit a bank's allocation
print-hierarchy print accounting database
optional arguments:
-h, --help show this help message and exit
-p PATH, --path PATH specify location of database file
-o OUTPUT_FILE, --output-file OUTPUT_FILE
specify location of output fileTo add a bank:
[fluxuser@30451d797d82 accounting-DB]$ flux account -p FluxAccounting.db add-bank A 1To add a user:
[fluxuser@30451d797d82 accounting-DB]$ flux account -p FluxAccounting.db add-user --username=user1 --bank=ATo view a user's information:
[fluxuser@30451d797d82 accounting-DB]$ flux account -p /var/lib/flux-ba/FluxAccounting.db view-user user1
creation_time mod_time deleted username admin_level bank shares max_jobs max_wall_pj
0 1610140992 1610140992 0 user1 1 B 1 1 60- The
delete-usersubcommand has a bug that is being tracked (#95)