Skip to content

Latest commit

 

History

History
113 lines (88 loc) · 4.44 KB

data_storage.md

File metadata and controls

113 lines (88 loc) · 4.44 KB

Storing data

Data is stored, in college on our personal NAS. Please request access from the administrator. This is a short description of users, directory structure and sync possibilities.

Users

  • Long term lab members have their own user name and belong to lab_members, which grant them:
    • read-only access to "/auto_generated_data"
    • read-only to other members' homes
    • read-write to their own home
    • read-write to "lab_shared"
    • read-write to "Public"
  • Short term lab members/guests share the same user name:
    • login = labguest
    • read-only access to "auto_generated_data"
    • read-only to "lab_shared"
  • Nodes use:
    • login = node
    • RW access to "auto_generated_data"

Directories organisation:

In addition to individual users' home, their are three special directories. Here is a short description of each of them:

  • auto_generated_data: contains private data generated by our acquisition systems. It is read-only for lab members and lab guests.
  • shared: contains private data shared in the lab. It is read-write for lab members and read-only for lab guests.
  • Public: contains public data meant to be shared with in-college collaborators. It is read-write for lab members and read-only for lab guests.
  • homes: every user has a personal home which is read-write for him/herself. In addition, it is read-only for other lab members and lab guests.

Access data

Windows

TODO @esteban

Linux

In linux, the simplest is to mount the nas shared and to use rsync on cronjobs to get new data.

In this example, you want to have a local sync of the auto_generated_data directory on the nas.

  • Make a dir to mount the share to:
# mkdir /mnt/nas/auto_generated_data -p
  • Make a dir on a local drive to put the data in:
# mkdir /data/ethoscope_results -p
  • Locate your fstab file:
# gedit /etc/fstab
  • Add a line in your fstab to mount the share (see here, uncomment the third line).
  • Create a /root/.nas_creds files with your credentials (the login credentials of the nas website)
  • Try mounting the share:
# mount -a
# ls /mnt/nas/auto_generated_data
  • Or specifically the nas:
# mount /mnt/nas/auto_generated_data
  • Run the fetching script
  • You can add this script to a cronjob like in this one (obviously just take the relevant line).

Should you need it, use the same principle to sync ethoscope videos and other resources. Have a look at our workstation scripts page for other jobs you may want to do.

You can also mount your home (to which you have read-write access) by adding this file to your fstab. E.g. :

# //nas.lab.gilest.ro/home /mnt/my_home cifs defaults,uid=1000,gid=1000,credentials=/etc/.nas_creds 0 0

Adding a new lab member (admin)

Through the NAS web interface (http://nas.lab.gilest.ro/:8080/cgi-bin/). Go to Control Panel/Privilege Settings/Users/Create Then simply add user to the lab_members group

Auto-generated data sync

Data generated by our aquisition system (e.g. ethoscope) is synced on auto_generated_data. Typically, this will happen twice a day for ethoscope db files (10:00 and 22:00 GMT), and every 30 minutes for video data.

Remote Backups

This system is a storage platform, not a backup. Every week, on saturday, ethoscope video files are archived are archives an external storage (e.g. box.com). Every week, on sunday, ethoscope db files are archived are archives an external storage (e.g. box.com). This tasks are perforemd by the workstation, and email is sent to the administrator upon completion.

TODO send email to a list like "ethoscope-users@lab.gilest.ro" instead, or publish a backup status page on some html server.

Technical detail

  • Primary MAC address: 00:08:9b:f8:9e:17
  • url (suggested): nas.lab.gilest.ro
  • hostname: la-gilestronas1.bc.ic.ac.uk
  • NAS Name: nas-gilestro
  • One can ssh in the nas as admin only