Write the 113 ALTO Hackathon Story

[giralt]

No description provided.

[yryang]

Task 1 (base/one-shot) [Target]

---

Goal. A user (scientist) downloads a dataset from the highest performant RSE. In the general case, it demonstrates ALTO as an ability to provide visibility to the network infrastructure.

Network Setting. We setup network topology using g2-mininet.
The topology should satisfy:

• There is a bottleneck link
• Network layer computes routes to N (e.g., 10) RSEs, with majority of RSEs use the bottleneck link and only one RSE has a high-throughput path.

Application Setting 1 Scientist downloads a file using rucio

• Run the command:
  rucio download --pfn=<filename> // default, which is random
• After entering the command, at the terminal, rucio displays the key steps of the working progress:
  • Display a list of candidates
  • Picks one randomly
  • System records downloading rate from the chosen RSE to redis database

Application Setting 2 Scientist downloads a file using rucio + ALTO:

• Run the command:
  rucio download -v --pfn=<filename> --replica-selection=alto:<serverIP>:bw-available
• After entering the command, at the terminal, rucio displays the key steps of the working progress:
  • Display a list of candidates
  • Display fetched ALTO (bw-available) cost map from RSEs to the terminal node
  • Pick the RSE with the best metric
  • System records downloading rate from the chosen RSE to redis database

Workflow:
Step 1. Scientist logins to a container running rucio.

Step 2: Scientist runs Application Setting 1 m times, and then Application Setting 2 one time. Each experiment is started at the same host. Plot application throughput at normalized starting time.

Task 2 (shared bottleneck estimation) [Stretch]

---

Goal. Scientist estimates downloading rates when concurrently fetching multiple datasets from multiple RSEs. In the general case, it demonstrates ALTO as an ability to provide intelligence derived from visibility to the network infrastructure.

Workflow:
Step 1. Scientist specifies the concurrent downloading flows in a query file:
RSE-1 -> {terminal-11, terminal-12, ...}
...
RSE-K -> {terminal-K11, ...}

Step 2: Scientist runs downloading rate estimator by querying ALTO tput metric:
alto-estimator --alto-server=<serverIP> --metric=tput-g2 --flows=<filename>
alto-estimator queries ALTO endpointcost service with the flows and the metric tput -g2 for the network.

Task 3 (RSE selection for concurrent downloads with shared bottleneck) [Stretch^2]

---

Goal. Scientist picks RSEs with concurrent downloading rate targets.

Workflow:
Step 1. Scientist specifies the concurrent datasets to download and their target rates in a file:
terminal-1 pfn-1 min-rate-1
...
terminal-M pfn-M min-rate-M

Step 2: Scientist conducts scheduling:
rucio-sched-alto --alto-server=<serverIP> --input=<filename>
rucio-sched-alto executes the following:
foreach pfn-i, computes its set of replicas RSEs[pfn-i]
foreach combination (r1, r2, ..., rK) in RSEs[pfn-1] x .. x RSEs[pfn-K]
Invoke task 2 to check returned rate
If found a satisfying assignment, uses the assignment and starts rucio download with given RSE