Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GenomicsDB errors discovered in the wild #4753

Open
lbergelson opened this issue May 9, 2018 · 7 comments

Comments

@lbergelson
Copy link
Collaborator

commented May 9, 2018

@kgururaj @francesperry There's a thread on the GATK forum where people are reporting a number of issues running GenomicsDB. There are a few different issues but they all seem to be edge cases with the file system.

  1. Report of the following error when trying to read from a GenomicsDB that is marked as read only. Is there a reason that the workspace must be writeable in order to read it? Can we avoid that requirement?
terminate called after throwing an instance of 'VariantQueryProcessorException'
2018-01-10T12:15:04.154547266Z   what():  VariantQueryProcessorException : Could not open array genomicsdb_array at workspace: /keep/d22f668d4f44631d98bc650d582975ca+1399/chr22_db
  1. Could not open array genomicsdb_array at workspace when working with a small disk. Changing to a larger disk fixed the problem. Possibly we need a better error message for the case where we are out of disk space?

  2. Reports of similar errors using a Lustre filesystem with file locking disabled. Can GenomicsDB run without file locking? If not, can we emit a clear error message when we hit that problem?

@kgururaj

This comment has been minimized.

Copy link
Collaborator

commented May 10, 2018

Yep, the new jar fetches and prints the TileDB error message

@kgururaj

This comment has been minimized.

Copy link
Collaborator

commented May 10, 2018

Consolidation in TileDB requires locking currently - hence, importing data on Lustre filesystems mounted with locking disabled will fail. See this page for enabling locks on Lustre filesystems (which is what we have been using internally).

@cristinaluengoagullo

This comment has been minimized.

Copy link

commented May 16, 2018

Hi @kgururaj ,

Isn't there any way to disable locking in TileDB then?
Thanks!

Cristina.

@kgururaj

This comment has been minimized.

Copy link
Collaborator

commented Jun 14, 2018

Probably, I'll try to see if I can disable it with a runtime controllable flag

@cristinaluengoagullo

This comment has been minimized.

Copy link

commented Jun 14, 2018

Ok, thank you so much!

@kgururaj

This comment has been minimized.

Copy link
Collaborator

commented Nov 8, 2018

FYI, if you set the environment variable TILEDB_DISABLE_FILE_LOCKING=1 before running any GenomicsDB tool, it doesn't try to lock files on POSIX filesystems (Lustre, NFS, xfs, ext4 etc)

@lbergelson

This comment has been minimized.

Copy link
Collaborator Author

commented Nov 19, 2018

@kgururaj Is it possible to include information about the lock disabling in the lock error messages?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.