diff --git a/doc/source/whatsnew/v0.23.4.txt b/doc/source/whatsnew/v0.23.4.txt
index f1cedf139f7d6..69525aaea1d62 100644
--- a/doc/source/whatsnew/v0.23.4.txt
+++ b/doc/source/whatsnew/v0.23.4.txt
@@ -31,6 +31,7 @@ Bug Fixes
 **Groupby/Resample/Rolling**
 
 - Bug where calling :func:`DataFrameGroupBy.agg` with a list of functions including ``ohlc`` as the non-initial element would raise a ``ValueError`` (:issue:`21716`)
+- Bug in ``roll_quantile`` caused a memory leak when calling ``.rolling(...).quantile(q)`` with ``q`` in (0,1) (:issue:`21965`)
 -
 
 **Conversion**
diff --git a/pandas/_libs/window.pyx b/pandas/_libs/window.pyx
index 9e704a9bd8d3f..cea77e2c88b1b 100644
--- a/pandas/_libs/window.pyx
+++ b/pandas/_libs/window.pyx
@@ -1481,6 +1481,8 @@ def roll_quantile(ndarray[float64_t, cast=True] input, int64_t win,
             else:
                 output[i] = NaN
 
+    skiplist_destroy(skiplist)
+
     return output