Data_Traversal

Overview

This section entails:

1. Using gds-ls
   • for traversing the gds file system
2. Using gds-find
   • for finding files or directories recursively and through * globs i.e *.json
3. Using gds-view
   • for observing files without downloading them
4. Using gds-task-view
   • for viewing stderr / stdout of TES tasks without downloading them

Before continuing please make sure you have run ica-context-switcher as described in Context Handling in your current terminal

gds-ls

auto-completion ✅

Run ls on a GDS file system as if it were your local system.

Options:

• folder-path: Single positional argument.

Requirements:

• curl
• jq
• python3

Environment vars:

• ICA_BASE_URL
• ICA_ACCESS_TOKEN
  • You will need to first run ica-context-switcher to get this variable into your environment

gds-find

auto-completion ✅

Run ls on a GDS file system as if it were your local system.

Options:

• --gds-path: Path to gds directory
• --type: Either "directory" or "file"
• --mindepth: Minimum depth to search for directory or file
• --maxdepth: Maximum depth to search for directory or file
• --name: Name of the directory or file, '*' glob is allowed

Requirements:

• curl
• jq
• python3

Environment vars:

• ICA_BASE_URL
• ICA_ACCESS_TOKEN
  • You will need to first run ica-context-switcher to get this variable into your environment

gds-view

auto-completion ✅

View a gds file without first needing to download it.
Works for gzipped files too.
Uses the links program (through docker) to visualise the file if --to-stdout or --browser is not set

Options:

• --gds-path: Path to the gds file you wish to view.
• --to-stdout: Print file to stdout
• --browser: Open up the file in a browser

Requirements:

• curl
• jq
• python3
• docker
• wget

Environment vars:

• ICA_BASE_URL
• ICA_ACCESS_TOKEN
  • You will need to first run ica-context-switcher to get this variable into your environment
• BROWSER
  • Path to your firefox executable

gds-task-view

auto-completion ✅

Given a task run id and --stdout OR --stderr parameter, view the log file of said ICA task run. Calls gds-view to view the file.

Options:

• --task-run-id: Path to gds file
• --stdout: Shows the task stdout
• --stderr: Shows the task stderr
• --to-stdout: Print file to stdout
• --browser: Open up the file in a browser

Requirements

• curl
• jq
• python3
• docker
• wget

Environment vars:

• ICA_BASE_URL
• ICA_ACCESS_TOKEN
  • You will need to first run ica-context-switcher to get this variable into your environment
• BROWSER
  • Path to your firefox executable

🚧 Command example GIF coming soon

Next steps

Head to the Syncing Directories page to start uploading and downloading directories through gds-sync-upload and gds-sync-download commands.