Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZFS L2ARC compression buffer leak #1688

Closed
nielsdec opened this issue Aug 31, 2013 · 3 comments
Closed

ZFS L2ARC compression buffer leak #1688

nielsdec opened this issue Aug 31, 2013 · 3 comments
Milestone

Comments

@nielsdec
Copy link

There seems to be a slight memory leak when using l2ARC compression, the temporary data buffer which holds the compressed data is not always freed.

This was tested by adding an extra entry in arcstats, which keeps track of the data allocated for temporary buffers. This slowly keeps increasing (besides the alloc/dealloc fluctuations), which implies a memory leak.

I think the path is:

  1. buffer allocated in l2arc_compress_buf
  2. arc entry removed/being removed before l2arc_write_done can free the buffer.

In any case arc_hdr_destroy will free the l2arc_buf_hdr without checking if the temp buffer is used/freeing the temp buffer. So if it's allocated at that point it will never be released.

I see if I can make a patch.

@behlendorf
Copy link
Contributor

Add reference to upstream issue:

https://www.illumos.org/issues/3995

@ryao
Copy link
Contributor

ryao commented Oct 17, 2013

danmcd has merged Saso's patch into illumos-gate in illumos/illumos-gate@e4be62a. I will pull it into my pull request #1775 shortly.

ryao pushed a commit to ryao/zfs that referenced this issue Oct 18, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
ryao pushed a commit to ryao/zfs that referenced this issue Oct 18, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
@ryao
Copy link
Contributor

ryao commented Oct 18, 2013

I seem to have made a mistake. The fix for this issue was Illumos 3995. What was merged into illumos-gate was Illumos 3964. I mislabeled Illumos 3995 as Illumos 3964 and included this in my Gentoo patches (and had planned to refresh #1775 with this). I will fix my mistake when I fresh #1775, but please disregard my previous comment.

ryao pushed a commit to ryao/zfs that referenced this issue Oct 24, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
ryao pushed a commit to ryao/zfs that referenced this issue Oct 24, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
ryao pushed a commit to ryao/zfs that referenced this issue Oct 25, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
ryao pushed a commit to ryao/zfs that referenced this issue Oct 25, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
ryao pushed a commit to ryao/zfs that referenced this issue Oct 30, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
behlendorf pushed a commit to behlendorf/zfs that referenced this issue Nov 5, 2013
3964 L2ARC should always compress metadata buffers
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Closes openzfs#1688
FransUrbo pushed a commit to FransUrbo/zfs that referenced this issue Nov 13, 2013
3995 Memory leak of compressed buffers in l2arc_write_done

References:
  https://illumos.org/issues/3995

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1688
Issue openzfs#1775
ryao pushed a commit to ryao/zfs that referenced this issue Nov 21, 2013
3995 Memory leak of compressed buffers in l2arc_write_done

References:
  https://illumos.org/issues/3995

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1688
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this issue Dec 13, 2013
3995 Memory leak of compressed buffers in l2arc_write_done

References:
  https://illumos.org/issues/3995

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1688
Issue openzfs#1775
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants