File not found errors with aspera option

[froggleston]

I'm trying to use the enaBrowserTools scripts to download data via aspera, but I am getting file not found errors.

So, this call:

davey@n80295:~/git/enaBrowserTools/python$ ./enaGroupGet -f fastq -as ../aspera_settings.ini SRP006773
Fetching SRR203489
Downloading file with md5 check:fasp.sra.ebi.ac.uk:/vol1/fastq/SRR203/SRR203489/SRR203489.fastq.gz
Error with Aspera transfer: [Errno 2] No such file or directory

But the file exists on the FTP site:

ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR203/SRR203489/SRR203489.fastq.gz

Why are the aspera links non-functional?

Cheers

Rob

[nicsilvester]

Thanks for the problem report. I have contacted our FTP/Aspera team to look into this. I suspect there is something wrong with the servers. Sorry for the inconvenience.

[froggleston]

Thanks for the quick reply! I shall await instructions :)

[nicsilvester]

We've been advised that the server problems are now fixed. Please let me know if you are still experiencing any problems.

[froggleston]

Sorry I'm still seeing the exact same behaviour today.

[nicsilvester]

Thanks for the heads up. I'll look into it

[nicsilvester]

We've heard back from our FTP/Aspera team today: all should now be working and fairly stable again. Please let me know if this is not the case and you still experience problems. Sorry for the inconvenience.

[froggleston]

Sorry to be the bearer of bad news again! Still not working. I raised a ticket by mailing datasubs - should this issue be moved there or are you happy for it to remain here?

davey:~/git/enaBrowserTools/python$ ./enaGroupGet -f fastq -as ../aspera_settings.ini SRP006754
Fetching SRR203225
Downloading file with md5 check:fasp.sra.ebi.ac.uk:/vol1/fastq/SRR203/SRR203225/SRR203225.fastq.gz
Error with Aspera transfer: [Errno 2] No such file or directory
Downloading file with md5 check:fasp.sra.ebi.ac.uk:/vol1/fastq/SRR203/SRR203225/SRR203225.fastq.gz
Error with Aspera transfer: [Errno 2] No such file or directory
Failed to download file after two attempts
Deleting directory SRR203225
ERROR: Something unexpected went wrong please try again.
If problem persists, please contact datasubs@ebi.ac.uk for assistance.
davey:~/git/enaBrowserTools/python$ ./enaGroupGet -f fastq -as ../aspera_settings.ini SRP006773
Fetching SRR203489
Downloading file with md5 check:fasp.sra.ebi.ac.uk:/vol1/fastq/SRR203/SRR203489/SRR203489.fastq.gz
Error with Aspera transfer: [Errno 2] No such file or directory
Downloading file with md5 check:fasp.sra.ebi.ac.uk:/vol1/fastq/SRR203/SRR203489/SRR203489.fastq.gz
Error with Aspera transfer: [Errno 2] No such file or directory
Failed to download file after two attempts
Deleting directory SRR203489
ERROR: Something unexpected went wrong please try again.
If problem persists, please contact datasubs@ebi.ac.uk for assistance.

[nicsilvester]

Thanks for letting me know. I have Helpdesk ticket number 217568, is this the one? I'm happy to pursue this either here or via the ticket.

For information regarding the ongoing issue:
There are some problems with the underlying file system of the FTP/Aspera servers at the moment. This is causing some files to not be visible to the servers and is why this can appear fixed, with all servers seeing the files, only to have the problem reoccur later and inconsistently with some servers serving the files and others with the lost mount points. The FTP/Aspera team are working to improve the stability of the service and I will notify them that the problem is ongoing.

[froggleston]

I'm happy to leave this here, if that's OK? Thanks for the info and following up with this!

[nicsilvester]

No worries with leaving it here. It also has the bonus of advising any other users that are experiencing the same problems.

Could you please try the download once more so that you can give me the following information to pass along to our FTP/Aspera team?
We need the IP from which the attempted download was performed and time of the failure.
This should help diagnose which server is having problems as we've been able to get a successful download through Aspera obviously hitting different servers.
Thanks.

[froggleston]

OK, I'll keep updating this ticket. Unfortunately we're behind a NAT so I won't be able to give you specific IP from behind our firewall. However, all our IPs are prefixed with 149.155.x.x. I tried at 10:34am today.

I hope this helps.

[froggleston]

Any updates on this? I'm still seeing the same errors when trying to download data (I can't get any successful download for the aspera service, but I did download an assembly successfully last week presumably via FTP).

Cheers

Rob

[nicsilvester]

The problem appears to be affecting only certain files and not every access of the file. I will chase up with our FTP/Aspera team again to see if they have made any headway on this. I'm very sorry for the delay.

Is it just the files for this run that you are having problems downloading, or are there other run accessions who's data is proving problematic to retrieve? I was able to download the file successfully when I attempted (highlighting the difficulty of this problem) so if it is limited to specific files, I can look into making these temporarily available to you via other means. This is obviously not the fix, but could at least enable you to continue your work while a long term fix is found.

Assemblies, sequences, WGS sets: these are all obtained from different file systems and servers to the run/analysis data files. It is therefore only the run/analysis data that has this problem.

[nicsilvester]

Hi Rob, Can you please let me know if you are still getting this error? I've chased up our FTP/Aspera team but they are able to download the file when tested against all servers.