Skip to content
This repository has been archived by the owner on Oct 16, 2020. It is now read-only.

ssl issue systemd-journal-remote and libmicrohttpd #927

Closed
kayrus opened this issue Oct 15, 2015 · 4 comments · Fixed by coreos/coreos-overlay#1692
Closed

ssl issue systemd-journal-remote and libmicrohttpd #927

kayrus opened this issue Oct 15, 2015 · 4 comments · Fixed by coreos/coreos-overlay#1692
Labels
component/systemd kind/bug priority/P1 team/os
Milestone
CoreOS 899.1.0

Comments

@kayrus
Copy link

kayrus commented Oct 15, 2015

libmicrohttpd 0.9.44 (yes, I've compiled latest lib with debug messages just to be sure it is not fixed in latest release) fails on huge amount of data when HTTPS is enabled. I've already filed a bug at libmicrohttpd bugtracker: https://gnunet.org/bugs/view.php?id=4007

Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Opened output file /var/log/journal/remote/remote-coreos1.journal
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Creating source for passive fd:9 (coreos1)
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Added RemoteSource as connection metadata 0x56278f394b20
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Queing response 202: OK.
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: microhttpd: Not enough memory for write!
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: microhttpd: Closing connection (failed to create response header)
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Cleaning up connection metadata 0x56278f394b20
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Writer ref count 1
Oct 15 15:25:24 coreos1 systemd-journal-remote[13614]: Closing journal file /var/log/journal/remote/remote-coreos1.journal.
This was referenced Oct 15, 2015
Merged
Closed
@kayrus
Copy link
Author

kayrus commented Oct 18, 2015

libmicrohttpd developers has answered that it is not lib issue. have to debug systemd-journal-remote here.

btw, there are other issues with systemd-journal-remote systemd/systemd#1387 systemd/systemd#1386

dongsupark pushed a commit to dongsupark/systemd that referenced this issue Oct 21, 2015
@dongsupark
journal-remote: introduce new option mhd-conn-memory-limit
6219583 
Introduce a new option --mhd-conn-memory-limit of systemd-journal-remote
to allow users to set maximum amount of memory per HTTP(S) connection.
e.g.:

  # systemd-journal-remote --mhd-conn-memory-limit 65536 ...

Its equivalent option can be also set in /etc/systemd/journal-remote.conf,
as MhdConnMemoryLimit, which defaults to 65536.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
@dongsupark dongsupark mentioned this issue Oct 21, 2015
Closed
dongsupark pushed a commit to endocode/systemd that referenced this issue Oct 28, 2015
journal-remote: increase memory limit per connection to avoid errors …
37a71b1 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
@dongsupark dongsupark mentioned this issue Oct 28, 2015
Closed
@dongsupark dongsupark mentioned this issue Oct 30, 2015
Merged
@dongsupark
Copy link

Hopefully it's fixed by systemd/systemd#1729, merged into systemd:master.

ssahani pushed a commit to ssahani/systemd that referenced this issue Nov 3, 2015
@ssahani
journal-remote: increase memory limit per connection to avoid errors …
0a035f4 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
ssahani pushed a commit to ssahani/systemd that referenced this issue Nov 3, 2015
@ssahani
journal-remote: increase memory limit per connection to avoid errors …
d1d1145 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
ssahani pushed a commit to ssahani/systemd that referenced this issue Nov 3, 2015
@ssahani
journal-remote: increase memory limit per connection to avoid errors …
6326195 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
ssahani pushed a commit to ssahani/systemd that referenced this issue Nov 3, 2015
@ssahani
journal-remote: increase memory limit per connection to avoid errors …
d758b39 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
ssahani pushed a commit to ssahani/systemd that referenced this issue Nov 3, 2015
@ssahani
journal-remote: increase memory limit per connection to avoid errors …
7dff702 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
@crawford crawford reopened this Dec 4, 2015
@mischief mischief self-assigned this Dec 14, 2015
mischief pushed a commit to mischief/systemd that referenced this issue Dec 14, 2015
journal-remote: increase memory limit per connection to avoid errors …
97b0e8d 
…with HTTPS

Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.

This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.

Fixes: coreos/bugs#927
@mischief mischief mentioned this issue Dec 14, 2015
Merged
@mischief mischief added this to the CoreOS 898.0.0 milestone Dec 14, 2015
@mischief mischief mentioned this issue Dec 14, 2015
Merged
@mischief
Copy link

@dongsupark backported your fix into coreos. @kayrus this should be solved now.

@mischief mischief removed their assignment Dec 15, 2015
@kayrus
Copy link
Author

kayrus commented Dec 15, 2015

There is another ticket: systemd/systemd#1766 + #962

Just for the history.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
component/systemd kind/bug priority/P1 team/os
Projects
None yet
Milestone
CoreOS 899.1.0
Development

Successfully merging a pull request may close this issue.

sys-apps/systemd: include systemd-journal-remote memory limit fix mischief/coreos-overlay
4 participants
@crawford @mischief @kayrus @dongsupark