Gh 2553 add Iterable classes from koryphe and remove closeable Iterab…

…le, iterator and wrapper classes (#2583)

* Transferred ChainedIterable from Koryphe, added existing constructor and added a test for new constructor

* Removed the Closeable and WrappedCloseable classes and refactored the iterables/iterators to use java.io.Closeable and java.lang.Iterable instead. Also amended the tests

* Renamed the CloseableIterableDeserializer to the IterableDeserializer and removed the WrappedCloseableIterable. Tests function

* Renamed CloseableIterable to Iterable and removed the WrappedCloseableIterable and the CloseableIterable types. Tests function

* gh-2552: Removed deprecated code except getTraits and hasTrait

* gh-2552: Fix javadoc issue

* Renamed CloseableIterable to Iterable and removed the WrappedCloseableIterable and the CloseableIterable types. The GraphSerialisableTest test does not function

* Renamed CloseableIterable to Iterable and removed the WrappedCloseableIterable and the CloseableIterable types. Tests function

* Changed to try-with-resources as both io classes are auto-closeable

* The TestStore.mockStore was being set in GraphTest but reset afterwards which was causing GraphSerialisableTest to fail. Reset TestStore.mockStore for both. All Graph test now pass

* Renamed CloseableIterable to Iterable

* gh-2571: Fixed matchedVertex behaviour and added new ITs (#2572)

* gh-2571: Fixed matchedVertex behaviour and added new ITs

* gh-2571: Fixed checkstyle

* gh-2571: Updated test name

* Changed CloseableIterable to Iterable and removed the extends Wrapper, instead implemented Iterable. Fixed tests

* Converted CloseableIterable to Iterable. Fixed AbstractElementFilter bug caused by the introduction of the Koryphe version of ChainedIterable. Fixed tests

* Changed EmptyClosableIterable to EmptyIterable

* Changed the OperationDetailTest name-of-class from CloseableIterable to Iterable

* Changed the the instances and JSON values from CloseableIterable to Iterable. Test pass

* Changed the the instances of CloseableIterable to Iterable. Suppresed test class casts. Test pass

* Changed from CloseableIterable to Iterable. Deleted WrappedCloseableIterable. FederatedStoreGetTraitsTest fails

* FederatedStoreGetTraitsTest did not delete the cache service before carrying out each test causing it to fail when run as part of the maven tests. In isolation it passed. Found that the FederatedGraphStorageTraitsTest was also not clearing the cache. Added the before and after cache clearance and bumped the JUnit version of FederatedStoreGetTraitsTest from 4 to 5. Tests now pass

* Changed from CloseableIterable to Iterable and added finally to the try blocks with a close-if-auto-closeable check

* gh-2573: Replace hasTrait usages with Operation (#2574)

* gh-2573: Replace hasTrait usages with Operation

* gh-2573: Fix tests by making exception more explicit

* gh-2573: Fixed exception issues

* gh-2573: Simplified but still failing test

* gh-2573: Changed to user

* gh-2573: Changed FederatedGetTraitsHandler to return DYNAMIC_SCHEMA

* gh-2573: Use old function for FederatedOperationChainValidator

* gh-2573: Fixed FederatedStoreUtil

* gh-2573: Added Deprecation notice

* gh-2573: Cherry pick federated store tests fix from 2553

* Revert "gh-2573: Cherry pick federated store tests fix from 2553"

This reverts commit c367555.

* gh-2573: Manual fix of FederatedStoreGetTraitsTest cache

* gh-2573: Changed HasTraitHandler to not send Federated loop option

* Gh-2560 get traits operation configurable (#2579)

* Update docs links in README

* Revert "Update docs links in README"

This reverts commit 2d87bf3.

* gh-2559 added hasTrait, its handler and corresponding tests (#2561)

* gh-2559 added hasTrait, its handler and corresponding tests

* gh-2559 amended comment in HasTrait, added additional tests for the handler

* gh-2559 null check & tidy test

Co-authored-by: t92549 <80890692+t92549@users.noreply.github.com>

* Update links to Gaffer docs in all files (#2566)

Co-authored-by: t92549 <80890692+t92549@users.noreply.github.com>

* gh-2560 GetTraitsHandler is initalised with traits dependednt on store type

* gh-2560 reviewed comments

* gh-2560 unused imports

* gh-2560 review comments

* checkstyle

* gh-2560 fixed federated test

* gh-2560 fixed GraphTest

* gh-2560 fixed TestStore

* gh-2560-getTraitsOperation (#2582)

Removing redundant adding of GetTraitHandler from addAdditionalOperationHandlers() because it has its own method getGetTraitsHandler().

* gh-2560: Added fix to SingleUseMapStoreWithoutVisibilitySupport

* gh-2560: Fixed Set copy issue in GetTraitsHandler constructor

* gh-2560: Added extra protections to GetTraitsHandler

Co-authored-by: GCHQDeveloper314 <94527357+GCHQDeveloper314@users.noreply.github.com>
Co-authored-by: t92549 <80890692+t92549@users.noreply.github.com>
Co-authored-by: GCHQDev404 <45399082+GCHQDev404@users.noreply.github.com>

* gh-2552: Removed some remaining parquet properties files

* Added missed iterable try-with-resources block, reformated and tested

* Reformated code. Updated test code. All tests pass

* Reformated code. Updated test code. All tests pass

* Reformatted code. Updated test code. All tests pass

* Reformatted code. All tests pass

* Fixed NamedViewCacheTest bugs and reformatted code. All tests pass

* Reformatted code. All tests pass

* Reformatted code. All tests pass

* Reformatted code. All tests pass

* Reformatted code again. All tests pass

* Reformatted code. Changed the Operation ArrayUtils import to lang3 as accumulo did not have the other apache version. All tests pass

* Reformatted code again. All tests pass

* Reformatted code again. All tests pass

* Reformatted code. All tests pass

* Created Objects.isNull static import

* Created Objects.isNull and Objects.nonNull static imports

* Update the copyright to the correct dates

* Removed unnecessary cast

* Corrected an/a grammar error

* Removed the incorrect spacing in the imports

* Created Objects.isNull and Objects.nonNull static imports for JSONSerialiser

* Removed deleted FederatedGraphStorageTraitsTest

* Re-added incorrectly deleted empty line

* Removed the unnecessary try-finally blocks from the LimitedIterableTest

* Removed the unnecessary try-finally blocks

* Removed the unused imports

* Removed the unnecessary try-finally blocks

Co-authored-by: t92549 <80890692+t92549@users.noreply.github.com>
Co-authored-by: p3430233 <61738524+p3430233@users.noreply.github.com>
Co-authored-by: GCHQDeveloper314 <94527357+GCHQDeveloper314@users.noreply.github.com>
Co-authored-by: GCHQDev404 <45399082+GCHQDev404@users.noreply.github.com>

core/common-util/src/main/java/uk/gov/gchq/gaffer/commonutil/CloseableUtil.java

@@ -16,6 +16,8 @@
 
 package uk.gov.gchq.gaffer.commonutil;
 
+import static java.util.Objects.nonNull;
+
 /**
  * Utility class for handling {@link java.io.Closeable}s.
  */
@@ -71,7 +73,7 @@ public static void close(final AutoCloseable... closeable) {
      */
     public static void close(final AutoCloseable closeable) {
         try {
-            if (null != closeable) {
+            if (nonNull(closeable)) {
                 closeable.close();
             }
         } catch (final Exception e) {

core/common-util/src/main/java/uk/gov/gchq/gaffer/commonutil/iterable/BatchedIterable.java

@@ -18,14 +18,15 @@
 
 import uk.gov.gchq.gaffer.commonutil.CloseableUtil;
 
+import java.io.Closeable;
 import java.util.Collections;
 import java.util.Iterator;
 import java.util.NoSuchElementException;
 
 /**
- * A {@code BatchedIterable} is an iterable of batches.
- * To use, extend this class and implement the createBatch method. When you have
- * no more batches of items left, return null.
+ * A {@code BatchedIterable} is a {@link java.io.Closeable} {@link java.lang.Iterable}
+ * of batches. To use, extend this class and implement the createBatch method. When you
+ * have no more batches of items left, return null.
  * <p>
  * As a client iterators round this iterable it will first create a batch, then iterate
  * around the batch. When there are no more items in the batch createBatch will be
@@ -34,12 +35,12 @@
  *
  * @param <T> the type of items in the iterable.
  */
-public abstract class BatchedIterable<T> implements CloseableIterable<T> {
+public abstract class BatchedIterable<T> implements Closeable, Iterable<T> {
     private Iterable<T> batch;
     private Iterator<T> batchIterator;
 
     @Override
-    public CloseableIterator<T> iterator() {
+    public Iterator<T> iterator() {
         // By design, only 1 iterator can be open at a time
         closeBatch();
 
@@ -69,7 +70,7 @@ private void closeBatch() {
         }
     }
 
-    private final class BatchedIterator implements CloseableIterator<T> {
+    private final class BatchedIterator implements Closeable, Iterator<T> {
         @Override
         public boolean hasNext() {
             if (null != batchIterator && batchIterator.hasNext()) {

core/common-util/src/main/java/uk/gov/gchq/gaffer/commonutil/iterable/CachingIterable.java

@@ -18,23 +18,26 @@
 
 import uk.gov.gchq.gaffer.commonutil.CloseableUtil;
 
+import java.io.Closeable;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.Collections;
 import java.util.Iterator;
 import java.util.List;
 import java.util.NoSuchElementException;
 
+import static java.util.Objects.nonNull;
+
 /**
- * A {@link CachingIterable} is a {@link CloseableIterable} that attempts to
- * cache the iterable the first time it is read. Subsequently, when iterator is
- * called, an iterator to the cached iterable is returned. The caching is disabled
- * if the iterable size is greater than a provided max size. By default this is
- * 100,000.
+ * A {@link CachingIterable} is a {@link java.io.Closeable} {@link java.lang.Iterable}
+ * that attempts to cache the iterable the first time it is read. Subsequently, when
+ * iterator is called, an iterator to the cached iterable is returned. The caching
+ * is disabled if the iterable size is greater than a provided max size. By default
+ * this is 100,000.
  *
  * @param <T> the type of the iterable.
  */
-public class CachingIterable<T> implements CloseableIterable<T> {
+public class CachingIterable<T> implements Closeable, Iterable<T> {
     public static final int DEFAULT_MAX_SIZE = 100000;
 
     private final Iterable<T> iterable;
@@ -62,24 +65,22 @@ public CachingIterable(final Iterable<T> iterable, final int maxSize) {
     }
 
     @Override
-    public CloseableIterator<T> iterator() {
-        if (null != cachedIterable) {
-            return new WrappedCloseableIterator<>(cachedIterable.iterator());
-        }
-
-        if (tooLarge) {
-            return new WrappedCloseableIterator<>(iterable.iterator());
+    public Iterator<T> iterator() {
+        if (nonNull(cachedIterable)) {
+            return cachedIterable.iterator();
+        } else if (tooLarge) {
+            return iterable.iterator();
+        } else {
+            return new CachingIterator();
         }
-
-        return new CachingIterator();
     }
 
     @Override
     public void close() {
         CloseableUtil.close(iterable);
     }
 
-    private class CachingIterator implements CloseableIterator<T> {
+    private class CachingIterator implements Closeable, Iterator<T> {
         private final Iterator<T> iterator = iterable.iterator();
         private List<T> cache = new ArrayList<>(Math.min(DEFAULT_MAX_SIZE, maxSize));
         private boolean closed = false;

core/common-util/src/main/java/uk/gov/gchq/gaffer/commonutil/iterable/ChainedIterable.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -16,97 +16,97 @@
 
 package uk.gov.gchq.gaffer.commonutil.iterable;
 
+import org.apache.commons.lang3.ArrayUtils;
+
 import uk.gov.gchq.gaffer.commonutil.CloseableUtil;
 
+import java.io.Closeable;
+import java.util.Arrays;
+import java.util.Collections;
 import java.util.Iterator;
-import java.util.NoSuchElementException;
+
+import static java.util.Objects.isNull;
 
 /**
- * A {@code ChainedIterable} is an iterable composed of other {@link java.lang.Iterable}s.
+ * A {@code ChainedIterable} is a {@link java.io.Closeable}
+ * {@link java.lang.Iterable} composed of other {@link java.lang.Iterable}s.
  *
  * As a client iterates through this iterable, the child iterables are consumed
  * sequentially.
  *
  * @param <T> the type of items in the iterable.
  */
-public class ChainedIterable<T> implements CloseableIterable<T> {
-    private final Iterable<T>[] itrs;
-    private final int n;
+public class ChainedIterable<T> implements Closeable, Iterable<T> {
+
+    private final Iterable<? extends Iterable<? extends T>> iterables;
+
+    public ChainedIterable(final Iterable<? extends T>... iterables) {
+        this(ArrayUtils.isEmpty(iterables) ? null : Arrays.asList(iterables));
+    }
 
-    public ChainedIterable(final Iterable... itrs) {
-        if (null == itrs || 0 == itrs.length) {
-            throw new IllegalArgumentException("At least 1 iterable is required.");
+    public ChainedIterable(final Iterable<? extends Iterable<? extends T>> iterables) {
+        if (isNull(iterables)) {
+            throw new IllegalArgumentException("iterables are required");
         }
-        this.itrs = itrs;
-        n = this.itrs.length;
+        this.iterables = iterables;
     }
 
     @Override
-    public void close() {
-        for (final Iterable<T> itr : itrs) {
-            CloseableUtil.close(itr);
-        }
+    public Iterator<T> iterator() {
+        return new ChainedIterator<>(iterables.iterator());
     }
 
     @Override
-    public CloseableIterator<T> iterator() {
-        return new IteratorWrapper();
+    public void close() {
+        for (final Iterable<? extends T> iterable : iterables) {
+            CloseableUtil.close(iterable);
+        }
     }
 
-    private class IteratorWrapper implements CloseableIterator<T> {
-        private final Iterator<T>[] iterators = new Iterator[itrs.length];
-        private int index = 0;
+    private static class ChainedIterator<T> implements Closeable, Iterator<T> {
+
+        private final Iterator<? extends Iterable<? extends T>> iterablesIterator;
+
+        private Iterator<? extends T> currentIterator = Collections.emptyIterator();
+
+        ChainedIterator(final Iterator<? extends Iterable<? extends T>> iterablesIterator) {
+            this.iterablesIterator = iterablesIterator;
+        }
 
         @Override
         public boolean hasNext() {
-            return -1 != getNextIndex();
+            return getIterator().hasNext();
         }
 
         @Override
         public T next() {
-            index = getNextIndex();
-            if (-1 == index) {
-                throw new NoSuchElementException();
-            }
-
-            return getIterator(index).next();
-        }
-
-        private int getNextIndex() {
-            boolean hasNext = getIterator(index).hasNext();
-            int nextIndex = index;
-            while (!hasNext) {
-                nextIndex = nextIndex + 1;
-                if (nextIndex < n) {
-                    hasNext = getIterator(nextIndex).hasNext();
-                } else {
-                    nextIndex = -1;
-                    break;
-                }
-            }
-
-            return nextIndex;
+            return getIterator().next();
         }
 
         @Override
         public void remove() {
-            getIterator(index).remove();
+            currentIterator.remove();
         }
 
-        private Iterator<T> getIterator(final int i) {
-            if (null == iterators[i]) {
-                iterators[i] = itrs[i].iterator();
+        @Override
+        public void close() {
+            CloseableUtil.close(currentIterator);
+            while (iterablesIterator.hasNext()) {
+                CloseableUtil.close(iterablesIterator.next());
             }
-
-            return iterators[i];
         }
 
-        @Override
-        public void close() {
-            for (final Iterator<T> itr : iterators) {
-                CloseableUtil.close(itr);
+        private Iterator<? extends T> getIterator() {
+            while (!currentIterator.hasNext()) {
+                CloseableUtil.close(currentIterator);
+                if (iterablesIterator.hasNext()) {
+                    currentIterator = iterablesIterator.next().iterator();
+                } else {
+                    break;
+                }
             }
-            ChainedIterable.this.close();
+
+            return currentIterator;
         }
     }
 }

core/common-util/src/main/java/uk/gov/gchq/gaffer/commonutil/iterable/ConsumableBlockingQueue.java

@@ -26,6 +26,8 @@
 import java.util.NoSuchElementException;
 import java.util.concurrent.ArrayBlockingQueue;
 
+import static java.util.Objects.isNull;
+
 /**
  * Extension to {@link ArrayBlockingQueue} to allow consumers to iterate over
  * the queue, consuming the data, without being blocked.
@@ -71,11 +73,11 @@ public boolean equals(final Object obj) {
             return true;
         }
 
-        if (null == obj || getClass() != obj.getClass()) {
+        if (isNull(obj) || getClass() != obj.getClass()) {
             return false;
         }
 
-        final ConsumableBlockingQueue queue = (ConsumableBlockingQueue) obj;
+        final ConsumableBlockingQueue<?> queue = (ConsumableBlockingQueue<?>) obj;
 
         return new EqualsBuilder()
                 .append(toArray(), queue.toArray())