/qubes-issues

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qrexec deadlock on large transfers in both directions simultaneusly #1347

Closed
rootkovska opened this Issue Oct 19, 2015 · 10 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
  Release 3.0 updates
2 participants
@rootkovska @marmarek
@rootkovska
Member

rootkovska commented Oct 19, 2015

On a my mutt + Split GPG setup I can't seem to be able to send an encrypted email with any larger attachment (say ~2MB). I suspect some race condition problem in SplitGPG (or, less likely in qrexec).

To reproduce:

  1. Create an uncompressable large file: dd if=/dev/urandom of=blob.bin bs=1k count=2k
  2. Compose an email with the above file as an attachment.
  3. Actually try to send the email. The send operation will not succeed.

FWIW, this is the relevant fragment of my mutt's SplitGPG config:

set pgp_encrypt_sign_command="pgpewrap qubes-gpg-client-wrapper --batch --textmode --armor --always-trust %?a?--encrypt-to %a? --encrypt --sign %?a?-u %a? -- -r %r -- %f | sed -e '2iCharset: UTF-8'"

@rootkovska rootkovska added bug C: other P: critical labels Oct 19, 2015

@rootkovska rootkovska added this to the Release 3.0 updates milestone Oct 19, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Oct 20, 2015

Member

I guess it's qrexec fault:
https://github.com/QubesOS/qubes-core-agent-linux/blob/master/qrexec/qrexec-agent-data.c#L224:

 /* FIXME: use buffered write here to prevent deadlock */

I can reproduce it using Split GPG, but failed to write artificial test for it (at least for now).
Member

marmarek commented Oct 20, 2015

I guess it's qrexec fault:
https://github.com/QubesOS/qubes-core-agent-linux/blob/master/qrexec/qrexec-agent-data.c#L224:

 /* FIXME: use buffered write here to prevent deadlock */

I can reproduce it using Split GPG, but failed to write artificial test for it (at least for now).
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Oct 20, 2015

Member

but failed to write artificial test for it (at least for now).

Never mind, it was a typo...
Member

marmarek commented Oct 20, 2015

but failed to write artificial test for it (at least for now).

Never mind, it was a typo...

marmarek added a commit to marmarek/old-qubes-core-admin that referenced this issue Oct 25, 2015

@marmarek
tests: more qrexec tests, this time for deadlocks 
Tests for recently discovered deadlocks on write(2).

QubesOS/qubes-issues#1347
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
e475e29

@marmarek marmarek changed the title from SplitGPG fails to handle large files to Qrexec deadlock on large transfers in both directions simultaneusly Oct 30, 2015

marmarek added a commit to QubesOS/qubes-core-admin-linux that referenced this issue Oct 30, 2015

@marmarek
qrexec: implement buffered write to child stdin to prevent deadlock 
Otherwise if the child process isn't reading its stdin at that time, it
would deadlock the whole qrexec connection (for example preventing
reading the data from the child, which may be a cause of that deadlock).

QubesOS/qubes-issues#1347
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
0c288aa

@marmarek marmarek closed this in QubesOS/qubes-core-agent-linux@97a3793 Oct 30, 2015

marmarek added a commit to QubesOS/qubes-linux-utils that referenced this issue Oct 30, 2015

@marmarek
libqrexec-utils: bring back buffered write helpers 
It is required to prevent deadlocks in single-threaded select-based IO
programs (namely: qrexec). POSIX API doesn't support checking how much
can be written to pipe/socket without blocking, so to prevent blocking
application must use O_NONBLOCK mode, and somehow deal with non-written
data (buffer it).

QubesOS/qubes-issues#1347
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
6a44eae

marmarek added a commit to QubesOS/qubes-linux-utils that referenced this issue Oct 30, 2015

@marmarek
libqrexec-utils: bring back buffered write helpers 
It is required to prevent deadlocks in single-threaded select-based IO
programs (namely: qrexec). POSIX API doesn't support checking how much
can be written to pipe/socket without blocking, so to prevent blocking
application must use O_NONBLOCK mode, and somehow deal with non-written
data (buffer it).

QubesOS/qubes-issues#1347

(cherry picked from commit 6a44eae)
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
61c3357

marmarek added a commit to QubesOS/qubes-core-admin-linux that referenced this issue Oct 30, 2015

@marmarek
qrexec: implement buffered write to child stdin to prevent deadlock 
Otherwise if the child process isn't reading its stdin at that time, it
would deadlock the whole qrexec connection (for example preventing
reading the data from the child, which may be a cause of that deadlock).

QubesOS/qubes-issues#1347

(cherry picked from commit 0c288aa)
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
819204b

marmarek added a commit to QubesOS/qubes-core-agent-linux that referenced this issue Oct 30, 2015

@marmarek
qrexec: implement buffered write to a child stdin 
Implement one of TODOs left in the code. Without this buffering, it may
happen that qrexec-agent will hang waiting on write(2) to the child
process, while that child will do the same (try to write something to
the qrexec-agent), without reading its stdin. This would end up in a
deadlock.

Fixes QubesOS/qubes-issues#1347

(cherry picked from commit 97a3793)
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
c771381
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Oct 30, 2015

Member

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc20 has been pushed to the r3.0 testing repository for the Fedora fc20 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.0-current-testing
Member

marmarek commented Oct 30, 2015

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc20 has been pushed to the r3.0 testing repository for the Fedora fc20 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.0-current-testing

@marmarek marmarek added the r3.0-fc20-testing label Oct 30, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Oct 30, 2015

Member

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc21 has been pushed to the r3.0 testing repository for the Fedora fc21 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.0-current-testing
Member

marmarek commented Oct 30, 2015

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc21 has been pushed to the r3.0 testing repository for the Fedora fc21 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.0-current-testing

@marmarek marmarek added the r3.0-fc21-testing label Oct 30, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 1, 2015

Member

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb8u1 has been pushed to the r3.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade
Member

marmarek commented Nov 1, 2015

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb8u1 has been pushed to the r3.0 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

@marmarek marmarek added the r3.0-jessie-testing label Nov 1, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 1, 2015

Member

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb7u1 has been pushed to the r3.0 testing repository for the Debian wheezy template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing wheezy-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade
Member

marmarek commented Nov 1, 2015

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb7u1 has been pushed to the r3.0 testing repository for the Debian wheezy template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing wheezy-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

@marmarek marmarek added the r3.0-wheezy-testing label Nov 1, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 12, 2015

Member

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc20 has been pushed to the r3.0 stable repository for the Fedora fc20 template.
To install this update, please use the standard update command:

sudo yum update
Member

marmarek commented Nov 12, 2015

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc20 has been pushed to the r3.0 stable repository for the Fedora fc20 template.
To install this update, please use the standard update command:

sudo yum update

@marmarek marmarek added r3.0-fc20-stable and removed r3.0-fc20-testing labels Nov 12, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 12, 2015

Member

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc21 has been pushed to the r3.0 stable repository for the Fedora fc21 template.
To install this update, please use the standard update command:

sudo yum update
Member

marmarek commented Nov 12, 2015

Automated announcement from builder-github

The package qubes-core-vm-3.0.19-1.fc21 has been pushed to the r3.0 stable repository for the Fedora fc21 template.
To install this update, please use the standard update command:

sudo yum update

@marmarek marmarek added r3.0-fc21-stable and removed r3.0-fc21-testing labels Nov 12, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 12, 2015

Member

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb8u1 has been pushed to the r3.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update
Member

marmarek commented Nov 12, 2015

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb8u1 has been pushed to the r3.0 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

@marmarek marmarek added r3.0-jessie-stable and removed r3.0-jessie-testing labels Nov 12, 2015

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Nov 12, 2015

Member

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb7u1 has been pushed to the r3.0 stable repository for the Debian wheezy template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update
Member

marmarek commented Nov 12, 2015

Automated announcement from builder-github

The package qubes-core-agent_3.0.19-1+deb7u1 has been pushed to the r3.0 stable repository for the Debian wheezy template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

@marmarek marmarek added r3.0-wheezy-stable and removed r3.0-wheezy-testing labels Nov 12, 2015

marmarek added a commit to QubesOS/qubes-core-admin that referenced this issue Nov 15, 2015

@marmarek
tests: more qrexec tests, this time for deadlocks 
Tests for recently discovered deadlocks on write(2).

QubesOS/qubes-issues#1347

(cherry picked from commit e475e29)
Verified
This commit was signed with a verified signature.
@marmarek marmarek Marek Marczykowski-Górecki
GPG key ID: 063938BA42CFA724 Learn about signing commits
c2c6a38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment