Upload fastq folders

[yihming]

Add support on uploading fastq folders

Require Stratocumulus at least v0.1.6 to work with AWS backend.

Steps:

• Track column of samples: if Library presents, use it; otherwise, look for Sample.
• If encountering a local folder, justify if it's a fastq folder by folder_is_fastq function.
  • This function returns None if it's not a fastq folder.
  • Otherwise, it returns the file pattern: either <folder-name>/<sample-name>, or <folder-name>/<sample-name>_*.fastq.gz, depending on the actual folder structure.
• Determine the source and dest based on if the folder is a fastq folder or a BCL folder.
• If a fastq folder:
  • Set <folder-name>/<sample-name> as the key in input_file_to_output_url dictionary, with <bucket-folder>/<sample-name> being the value.
  • Use transfer_fastq function to upload fastq files/subfolders to <bucket-folder>.

transfer_fastq function is used within transfer_data function, which is called by transfer_sample_sheet.

Early termination if no access

Raise PermissionError if:

• Input sample sheet is not readable.
• Fastq folders are not executable.
• Fastq files are not readable.