2.10.3 brings vdb-config --interactive nightmare

[antonkulaga]

As soon as I migrated from 2.10.0 to 2.10.3 all my bioinformatic workflows failed with

Before continuing, please run: vdb-config --interactive

error. It looks that you broke the way how config if resolved because when it runs prefetch inside bash script I get this "vdb-config --interactive" nighmare
My docker container is https://github.com/antonkulaga/biocontainers/tree/master/downloaders/sra

[seandavi]

The only way I have found to get around this is to run vdb-config --interactive once by hand, store the resulting config file into the docker build directory, and then copy the resulting config into the right location in the container at build time.

I noted the same problem in #282.

[kwrodarmer]

You ran into exactly the problem that we are trying to avoid, which is that had it not been for that failure, you would have deployed a bunch of toolkits without initial configuration. With the 2.10.x release, a lot of things changed, and the SRA Toolkit has been adapting along with the changes.

For example, before we send information about your billing credentials or computing environment, we are asking for user acknowledgement/agreement. Had it not been for giving you the chance to reject it, we could have just started sending it without your knowledge or consent.

For this reason, and at least for the 2.10.3 release, we are asking users to manually configure their system. It's only required once, and it doesn't ask again. And by the way, we have the exact same issue for the Docker images we produce, and so you can expect this to be smoothed out very soon.

[ababaian]

To not have to copy config files in/out of a container you can run vdb-config -i during container build and then kill the process prior to authenticating.

vdb-config -i & read -t 3 ; kill $!

Edit: This does not work if you run it in the Dockerfile as RUN ..., debugging now. See below.

[probalica19]

Hi all, thanks for the helpful notes!

@ababaian, did you manage to find a command like vdb-config -i & read -t 3 ; kill $! which can be run from Dockerfile?

@seandavi , could you please give more information on how to store the config file or where it can be found, and how it can later be read from Dockerfile?

@kwrodarmer , any news on when this will be fixed/enabled?

Thanks a lot!

[ababaian]

@probalica19 negative. See Dockerfile we just copy over the config file from the github repo. Feel free to grab that one if you don't have an obvious place to host it will remain in place for the foreseeable future.

...
COPY docker/serratus-dl/VDB_user-settings.mkfg /root/.ncbi/user-settings.mkfg
RUN mkdir -p /root/.ncbi; vdb-config --report-cloud-identity yes

[kwrodarmer]

We have a ticket to address this case, and I still owe a wiki page to describe the issue.

[kwrodarmer]

@ababaian - you should not share configuration files and we would ask that you not place them in GitHub.

[ababaian]

This is a fully dummy config file, made on a blank container with no personal information inside it. We need a way for production to bypass the forced interaction.

Edit: Unless I'm misunderstanding something here in which case please let me know what it is.

[kwrodarmer]

For a fully blank config file, it certainly has a lot of non-blank entries!

[akaviaLab]

Hi.
I ran into the same problem.
Is there a way of setting
/config/default = "false"
manually?

Thank you,

Uri David

[OrielResearchCure]

Hello,
Spent a week on that. any solution?
$HOME/.ncbi/user-settings.mkfg us available on the worker machine. However, prefetch is called, the same error is fire: Before continuing, please run: vdb-config --interactive error. version is: sratoolkit.2.10.5-centos_linux64
Any advice on that?

[abhinavgarg89]

any update on this guys..even after copying user-settings.mkfg, it gives me same error.

[OrielResearchCure]

Open the config file on a local machine to generate a GUID at the top of the config file. Once you have the GUID, copy it to the other machine Good luck! — Eila www.orielesearch.com https://www.meetup.com/Deep-Learning-In-Production

…

Sent from my iPhone

On May 3, 2020, at 2:38 AM, Abhinav Garg ***@***.***> wrote:  any update on this guys..even after copying user-settings.mkfg, it gives me same error. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[probalica19]

Hi all,

Am I supposed to experience the same issue with the new version (2.10.6)?

I have tried the approach of @ababaian and when I build the docker image as instructed, container started from that image on my local machine seems to be configured, but once this image gets pushed to remote, and docker container run on cloud, the configuration is required again.

Any help is much appreciated!

Thanks, probalica

[kwrodarmer]

Am I supposed to experience the same issue with the new version (2.10.6)?

Yes, yes you are. That is, unless you use the enclosed Docker file.

The behavior you see is something we were forced to implement in order to avoid a race condition that most people who are not running interactively would run into. There are many different requirements - some of them conflicting - from many different users, and we're trying to not kill one while supporting the other.

2.10.6 has a docker file in it, and we will be posting to docker hub within the near future. You should use that.

[s-andrews]

Given that when I run vdb-config -i I can immediately exit without having interacted with the program in any way, surely we can have an option to create the same default profile non interactively? Could you not make vdb-config --restore-defaults create the default profile? If you run it now it gives no errors if you have no profile, but programs like fasterq-dump won't work.

[chenrui333]

yeah, I am having the same problem of upgrading the formula (Homebrew/homebrew-core#55471).

If you run it now it gives no errors if you have no profile, but programs like fasterq-dump won't work.

Can we have some default profile for this matter?

[amizeranschi]

@s-andrews

Given that when I run vdb-config -i I can immediately exit without having interacted with the program in any way, surely we can have an option to create the same default profile non interactively? Could you not make vdb-config --restore-defaults create the default profile? If you run it now it gives no errors if you have no profile, but programs like fasterq-dump won't work.

They probably could, but for some reason, they just don't want to. It's a bit hard to believe that enforcing users to run vdb-config -i and then to press X is the only possible solution for setting up this software, especially when this "race condition" that they mention hasn't been there a couple of months ago, in version 2.10.3.

See also: #77 (comment) and #77 (comment).

Do you know of any alternatives to sra-tools that provides similar functionality and is still able to run non-interactively? It looks like, for now, NCBI are set on preventing any unattended setup and use of their tool.

[kwrodarmer]

@s-andrews , @chenrui333 , @amizeranschi , and anyone else who wants to express an opinion here, PLEASE NOTE!

THERE WOULD BE A RACE CONDITION IF YOUR PROPOSED SOLUTIONS WERE IMPLEMENTED. Please read the previous posts in this issue.

This behavior we introduced - where you have to install and run a tool manually ONCE and thereafter it runs as it always has - to avoid a configuration race condition that would have caused another flood of complaints. The behavior is not permanent, and can be addressed by moving to formal installers. But this will also introduce another flood of complaints from users who do not have admin privileges.

The SRA Toolkit supports a very large number of users across every time zone of the world, and have a lot of demands. We do our best to give people what they need. This issue is understood, your concerns have been heard, I have given you the answers:

1. use the docker file
2. wait until we switch to installer packages
3. work with what we can give you today

[amizeranschi]

@kwrodarmer

Wouldn't it be possible to implement vdb-config --restore-defaults to do THE SAME THING as when running vdb-config -i and then immediately pressing X?

[kwrodarmer]

Wouldn't it be possible to implement vdb-config --restore-defaults to do THE SAME THING as when running vdb-config -i and then immediately pressing X?

Not without encountering a race condition, no.

We are updating a file in some arbitrary file system that we have little control over and we have no effective way of locking the file for modification across multiple hosts and/or multiple processes. Your proposal removes the semaphore behavior of human interaction and that's why there is a race condition.

[s-andrews]

@kwrodarmer thanks for replying to this - I appreciate that you're getting a lot of requests about this and I'm sure you're under a lot of pressure from users reporting issues, but please appreciate that many of us are also trying to placate users of our software and pipelines, and are trying to find workrounds for code which used to work and now doesn't.

Your pointer to the docker script, now that we know where to look provides a completely functional workround for this. For others who haven't seen it, the relevant part is simply:

printf '/LIBS/GUID = "%s"\n' `uuidgen` > /root/.ncbi/user-settings.mkfg

Which does exactly what people have been asking for, so thankyou for that. People here are generally fairly used to finding solutions for these types of issues, but this was not an easy solution to find. Specifically:

1. Your Installation and Configuration page doesn't mention docker anywhere.

2. Your Releases page has no notes on any release.

3. Your Changes page doesn't mention docker for any release.

4. The bug report specifically asking for documentation on how to solve this problem doesn't contain a pointer to the solution.

All people needed was a way to generate the GUUID without having to run a tool which has no user interaction without having to create an interactive environment - which could be non-trivial depending on the use case. I completely understand the need for interaction where the user must agree to conditions for controlled access, but most of us don't need that.

[kwrodarmer]

The Docker release is not complete. We are new to the ecosystem and just moving in. We use Docker ourselves but have not yet used it as a medium for publishing solutions. This is why the Docker file is not being announced - yet.

The importance of the line from Docker is that it is run at a time when there are no race conditions. If this line is extracted and run from a HPC farm, it will exactly cause the problem we took pains to avoid.

[skripche]

Link to the NCBI supported docker documentation can be found here:
https://github.com/ncbi/sra-tools/wiki/SRA-tools-docker

[blajoie]

Is there a solution to building sratoolkit within a docker and avoiding the vdb-config -i flow? I fail to recognize how running vdb-config -i and then exiting does anything of value to then allow the other tools (fastq-dump) to proceed? Is there an official dockerfile yet?

Before continuing, please run: vdb-config --interactive

[golharam]

Interesting, if I run:

docker run --rm -i -v $PWD:$PWD -w $PWD ncbi/sra-tools:2.11.0 fasterq-dump SRR15351398 -p

I don't get the vdb-config issue. However if I run the SAME command using cwl-runner, I do get the error. The CWL command is:

INFO [job fasterq-dump.cwl] /tmp/2qrnlryi$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/2qrnlryi,target=/uaJujC \
    --mount=type=bind,source=/tmp/8w0ew0o0,target=/tmp \
    --mount=type=bind,source=/scratch/CWL,target=/uaJujC/CWL,readonly \
    --workdir=/uaJujC \
    --read-only=true \
    --user=1000:1000 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/uaJujC \
    --cidfile=/tmp/bqpq2mic/20210811024445-621269.cid \
    ncbi/sra-tools:2.11.0 \
    fasterq-dump \
    -p \
    SRR15351398

Is the contributing difference, --env=HOME=/uaJujC?
EDIT: Adding '--user=1000:1000' to the working docker command causes this to fail.

[durbrow]

The toolkit looks in $HOME for the configuration. If $HOME is changed from what it was at the time the image was built, then the configuration file needs to be copied to the new $HOME.

# run inside the container
mkdir -p $HOME/.ncbi
cp /root/.ncbi/user-settings.mkfg $HOME/.ncbi

The only thing I can see that might go wrong is --mount=type=bind,source=/tmp/2qrnlryi,target=/uaJujC seems to imply that the directory is transitory. If that is the case, the copy may be wiped out every time your pipeline runs. (But I am not familiar with CWL.)

[golharam]

Yes, I discovered the image has /root/.ncbi/user-settings.mkfg that is read when you using the image directly. When run through cwl-runner, the image run as a different user hence this file isn't present. I took the file from the image and create it at runtime. Here is my workaround using CWL in the requirements section:

requirements:
  # This is required because fasterq-dump looks for ~/.ncbi/user-settings.mkfg.
  # When this container is run as root, the directory is in the image's /root/
  # folder.  When this image is run via cwl-runner, the HOME directory is
  # specified to be something different hence this file doesn't exist, hence
  # fasterq-dump throws an error about vdg-config needing to be called.
  InitialWorkDirRequirement:
    listing:
      - entryname: .ncbi/user-settings.mkfg
        entry: |-
          /LIBS/IMAGE_GUID = "0df5ebb2-dcc0-4a83-b71f-b5d31e5bda3e"
          /libs/cloud/report_instance_identity = "true"

EDIT: This issue is more than just v2.10.3. I'm using 2.11.0, referencing #489