Skip to content

Commit

Permalink
leaky seq detection, safer/lazier/uniform core fns
Browse files Browse the repository at this point in the history
  • Loading branch information
frenchy64 committed Apr 13, 2024
1 parent 3f53026 commit 1b2832a
Show file tree
Hide file tree
Showing 14 changed files with 2,122 additions and 3 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/build.yml
Expand Up @@ -7,10 +7,11 @@ on:
pull_request:

env:
BABASHKA_VERSION: "0.6.1"
BABASHKA_VERSION: "0.10.163"

jobs:
test:
timeout-minutes: 5
strategy:
matrix:
jdk: ['8', '11', '17', '21', '22']
Expand Down
4 changes: 4 additions & 0 deletions README.md
Expand Up @@ -17,6 +17,10 @@ Utilities for Clojure.
- [latest protocol ops](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.latest-protocol.html) -- implementations of `satisfies?`, `find-protocol-impl`, `find-protocol-method`, `extends?`, `extenders` that look up the latest version of the protocol [such that they have the same behavior with partial](https://clojure.atlassian.net/browse/CLJ-2094).
- non-leaky [clojure.core](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.non-leaky-macros.clojure.core.html), [clojure.test](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.non-leaky-macros.clojure.test.html) macros -- versions of the following macros that don't leak implementation details (eg., recur targets, pre/post, try/catch syntax): `locking` ([upstream report](https://clojure.atlassian.net/browse/CLJ-2573)), `binding`, `with-bindings`, `sync`, `with-local-vars`, `with-in-str`, `dosync`, `with-precision`, `with-loading-context`, `with-redefs`, `delay`, `vswap!`, `lazy-seq`, `lazy-cat`, `future`, `pvalues`, `clojure.test/{deftest,deftest-,testing,with-test,with-test-out}`, `clojure.java.shell/with-sh-{dir,env}`, `clojure.test.tap/with-tap-output`, `clojure.pprint/with-pprint-dispatch`, `clojure.core.async/thread`, `clojure.core.logic.pldb/with-{db,dbs}`, `clojure.tools.trace/dotrace`, `clojure.test.check.properties/for-all`, `clojure.test.check.generators/let`, `clojure.java.jmx/with-connection`, `clojure.core.match.debug/with-recur`
- [`::reify-args`](https://github.com/frenchy64/fully-satisfies/blob/main/src/io/github/frenchy64/fully_satisfies/reify_spec.clj), a [spec for clojure.core/reify](https://clojure.atlassian.net/browse/CLJ-2661). Use [register-reify-spec](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.reify-spec.html#var-register-reify-spec) to add to spec's registry.
- [Leaky-seq detection testing framework](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.leaky-seq-detection.html) for detecting memory leaks caused by holding onto the head of sequences. An [example test suite](https://github.com/frenchy64/fully-satisfies/blob/main/src/io/github/frenchy64/fully_satisfies/leaky_seq_detection_test.clj) is provided exposing issues with clojure.core functions and verifying fixes to them.
- [Safer](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.safer.html) variants of clojure.core functions that improve thread-safety and general robustness when passed mutating collections. Includes: `butlast`, `drop-last`, `every?`, `last`, `not-every?`, `nthrest`, `partitionv-all`, `sort`, `sort-by`, `split-at`, `split-with`, `splitv-at`, `take-last`.
- [Lazier](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.lazier.html) variants of clojure.core functions that are slightly lazier when processing and/or returning lazy seqs. Includes: `bounded-count`, `cycle`, `dedupe`, `iterator-seq`, `sequence`.
- [Uniformly generalized](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.uniform.html) variants of clojure.core functions such that they work consistently for all values. Includes: `partition-by`, `halt-when`.

[Latest API documentation](https://frenchy64.github.io/fully-satisfies/latest)

Expand Down
4 changes: 4 additions & 0 deletions dev/resources/root-templates/README.md
Expand Up @@ -17,6 +17,10 @@ Utilities for Clojure.
- [latest protocol ops](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.latest-protocol.html) -- implementations of `satisfies?`, `find-protocol-impl`, `find-protocol-method`, `extends?`, `extenders` that look up the latest version of the protocol [such that they have the same behavior with partial](https://clojure.atlassian.net/browse/CLJ-2094).
- non-leaky [clojure.core](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.non-leaky-macros.clojure.core.html), [clojure.test](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.non-leaky-macros.clojure.test.html) macros -- versions of the following macros that don't leak implementation details (eg., recur targets, pre/post, try/catch syntax): `locking` ([upstream report](https://clojure.atlassian.net/browse/CLJ-2573)), `binding`, `with-bindings`, `sync`, `with-local-vars`, `with-in-str`, `dosync`, `with-precision`, `with-loading-context`, `with-redefs`, `delay`, `vswap!`, `lazy-seq`, `lazy-cat`, `future`, `pvalues`, `clojure.test/{deftest,deftest-,testing,with-test,with-test-out}`, `clojure.java.shell/with-sh-{dir,env}`, `clojure.test.tap/with-tap-output`, `clojure.pprint/with-pprint-dispatch`, `clojure.core.async/thread`, `clojure.core.logic.pldb/with-{db,dbs}`, `clojure.tools.trace/dotrace`, `clojure.test.check.properties/for-all`, `clojure.test.check.generators/let`, `clojure.java.jmx/with-connection`, `clojure.core.match.debug/with-recur`
- [`::reify-args`](https://github.com/frenchy64/fully-satisfies/blob/main/src/io/github/frenchy64/fully_satisfies/reify_spec.clj), a [spec for clojure.core/reify](https://clojure.atlassian.net/browse/CLJ-2661). Use [register-reify-spec](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.reify-spec.html#var-register-reify-spec) to add to spec's registry.
- [Leaky-seq detection testing framework](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.leaky-seq-detection.html) for detecting memory leaks caused by holding onto the head of sequences. An [example test suite](https://github.com/frenchy64/fully-satisfies/blob/main/src/io/github/frenchy64/fully_satisfies/leaky_seq_detection_test.clj) is provided exposing issues with clojure.core functions and verifying fixes to them.
- [Safer](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.safer.html) variants of clojure.core functions that improve thread-safety and general robustness when passed mutating collections. Includes: `butlast`, `drop-last`, `every?`, `last`, `not-every?`, `nthrest`, `partitionv-all`, `sort`, `sort-by`, `split-at`, `split-with`, `splitv-at`, `take-last`.
- [Lazier](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.lazier.html) variants of clojure.core functions that are slightly lazier when processing and/or returning lazy seqs. Includes: `bounded-count`, `cycle`, `dedupe`, `iterator-seq`, `sequence`.
- [Uniformly generalized](https://frenchy64.github.io/fully-satisfies/latest/io.github.frenchy64.fully-satisfies.uniform.html) variants of clojure.core functions such that they work consistently for all values. Includes: `partition-by`, `halt-when`.

[Latest API documentation](https://frenchy64.github.io/fully-satisfies/latest)

Expand Down
57 changes: 57 additions & 0 deletions java/io/github/frenchy64/fully_satisfies/lazier/RT.java
@@ -0,0 +1,57 @@
/**
* Copyright (c) Rich Hickey. All rights reserved.
* The use and distribution terms for this software are covered by the
* Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
* which can be found in the file epl-v10.html at the root of this distribution.
* By using this software in any fashion, you are agreeing to be bound by
* the terms of this license.
* You must not remove this notice, or any other, from this software.
**/

/* rich Mar 25, 2006 4:28:27 PM */

package io.github.frenchy64.fully_satisfies.lazier;

import clojure.lang.AFn;
import clojure.lang.ArrayChunk;
import clojure.lang.ChunkedCons;
import clojure.lang.ISeq;
import clojure.lang.LazySeq;
import java.util.Iterator;

public class RT{

////////////// Collections support /////////////////////////////////

private static final int CHUNK_SIZE = 32;
public static ISeq chunkIteratorSeq(final Iterator iter){
if(iter.hasNext()) {
return new LazySeq(new AFn() {
public Object invoke() {
Object[] arr = new Object[CHUNK_SIZE];
int n = 0;
while(n < CHUNK_SIZE && iter.hasNext()) // switch conditions to avoid realizing extra element - Ambrose
arr[n++] = iter.next();
return new ChunkedCons(new ArrayChunk(arr, 0, n),
// call chunkIteratorSequence to avoid forcing extra item - Ambrose
chunkIteratorSequence(iter));
}
});
}
return null;
}

public static ISeq chunkIteratorSequence(final Iterator iter){
// return a LazySeq instead of calling hasNext() and forcing first element - Ambrose
return new LazySeq(new AFn() {
public Object invoke() {
Object[] arr = new Object[CHUNK_SIZE];
int n = 0;
// switched condition order so hasNext() isn't called 33 times - Ambrose
while(n < CHUNK_SIZE && iter.hasNext())
arr[n++] = iter.next();
return new ChunkedCons(new ArrayChunk(arr, 0, n), chunkIteratorSequence(iter));
}
});
}
}
5 changes: 3 additions & 2 deletions project.clj
Expand Up @@ -3,7 +3,7 @@
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url "https://www.eclipse.org/legal/epl-2.0/"}
:url "https://github.com/frenchy64/fully-satisfies"
:dependencies [[org.clojure/clojure "1.11.1"]]
:dependencies [[org.clojure/clojure "1.11.2"]]
:java-source-paths ["java"]
:profiles {:bench
{:source-paths ["scripts"]
Expand All @@ -13,7 +13,8 @@
:jvm-opts ["-Djdk.attach.allowAttachSelf"]}
:gen-doc
{:jvm-opts ["--add-opens" "java.base/java.lang=ALL-UNNAMED"]}
:dev {:dependencies [[org.clojure/test.check "1.1.0"]
:dev {:jvm-opts ["-Xms64m" "-Xmx64m"] ; small heap to speed up cleaners tests
:dependencies [[org.clojure/test.check "1.1.0"]
[com.gfredericks/test.chuck "0.2.12"
:exclusions [org.clojure/clojurescript
com.andrewmcveigh/cljs-time]]
Expand Down

0 comments on commit 1b2832a

Please sign in to comment.