gh-2357 federatedstore federated operation merge into v2 alpha (#2769)

* gh-2357 FederatedStore FederatedOperation Operation/Graph/Merge * gh-2357 FederatedStore FederatedOperation Handlers * gh-2357 FederatedStore FederatedOperation IFederationOperation, Handlers & Tests * gh-2357 FederatedStore FederatedOperation FederatedStore * gh-2357 FederatedStore FederatedOperation removal of option KEY_OPERATION_OPTIONS_GRAPH_IDS. * gh-2357 FederatedStore FederatedOperation GetSchema & GetTraits Changes * gh-2357 FederatedStore FederatedOperation IFederatedOperation * gh-2357 FederatedStore FederatedOperation deletion of FederatedOperationChain * gh-2357 FederatedStore FederatedOperation viewValidation * gh-2357 FederatedStore FederatedOperation Merging of results * gh-2357 FederatedStore FederatedOperation Tests, CodeStyle, Refactoring & Comments * gh-2357 FederatedStore FederatedOperation.v02 Builder IO focus * gh-2357 FederatedStore FederatedOperation.v02. Removing suspected unrequired tests. * gh-2357 FederatedStore FederatedOperation.v02. Resolving TODOs * gh-2357 FederatedStore FederatedOperation.v02. Removing overly constraining MIDPUT generic * gh-2357 FederatedStore FederatedOperation.v02. Resolving TODOs * gh-2357 FederatedStore FederatedOperation.v02. checkstyle * gh-2357 FederatedStore FederatedOperation.v02. Resolving TODOs. * gh-2357 FederatedStore FederatedOperation.v02 with errors * gh-2357 FederatedStore FederatedOperation.v02 isRequestingAdmin added to IFederation * gh-2357 FederatedStore FederatedOperation.v02 IFederation builder changes. * gh-2357 FederatedStore FederatedOperation.v02 IFederation shallowClone changes. * gh-2357 FederatedStore FederatedOperation.v02 IFederation userRequestingAdminUsage changes. * gh-2357 FederatedStore FederatedOperation.v02 IFederation userRequestingAdminUsage rename. * gh-2357 FederatedStore FederatedOperation.v02 bugfix. * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. * gh-2422 FederatedStore ChangeGraphId & ChangeGraphAccess update cache + Test * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. BugHunt * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. * gh-2425 remove duplicate updateOperationInput * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. * gh-2357 FederatedStore FederatedOperation.v02 FederatedOperation is InputOutput * gh-2357 FederatedStore FederatedOperation.v02 removing TODOs. BugHunt * gh-2357 FederatedStore FederatedOperation.v02 FederatedOperation is InputOutput * fixing bug * gh-2357 FederatedStore FederatedOperation.v02 FederatedOperation is InputOutput * gh-2357 FederatedStore FederatedOperation.v02 Error Returning AccumuloAllElementsRetriever * gh-2357 FederatedStore FederatedOperation.v02 MASSIVE bug fix for IterableConcat use of ChainedIterable Constructor. * gh-2357 FederatedStore FederatedOperation.v02 IterableConcat fixes * gh-2357 FederatedStore FederatedOperation.v02 summary * gh-2357 FederatedStore FederatedOperation.v02 ChainedIterable ported over from Koryphe. BREAKING CHANGE for any ChainedIterable saved to disc/metal. * gh-2357 FederatedStore FederatedOperation.v02 checkstyle * gh-2357 FederatedStore FederatedOperation.v02 Traits * gh-2455 FederatedStore GetTraits Audit. Removing dead code and correcting tests and behaviours. * gh-2455 FederatedStore GetTraits Audit. Refactoring FederatedGetTraitsHandler to use FederatedChain. Deprecating Store.getTraits() FederatedGraphStorage.getTraits * gh-2455 FederatedStore GetTraits Audit. Refactoring FederatedGetTraitsHandler to use FederatedChain. Fixing graphSizeCount. * gh-2455 FederatedStore GetTraits Audit. Refactoring * gh-2422 FederatedStore ChangeGraphId & ChangeGraphAccess recovery info. * gh-2357 FederatedStore FederatedOperation.v02 GetSchema * gh-2357 FederatedStore FederatedOperation.v02 TODOs * gh-2357 FederatedStore FederatedOperation.v02 TODOs * gh-2357 FederatedStore FederatedOperation.v02 TODOs * gh-2357 FederatedStore FederatedOperation.v02 PR review * gh-2357 FederatedStore FederatedOperation.v02 Demo fix * gh-2357 FederatedStore FederatedOperation merge with Alpha1 fixes including a temporary class while Korphye is being released. * gh-2357 FederatedStore checkstyle * gh-2369 Remove ChainedIterable Gaffer duplicate * gh-2369 Dependency upgrade Koryphe 2.2.0 * gh-2369 Deleting Duplicate Koryphe classes * gh-2369 Deleting Duplicate Koryphe classes updating imports * Updated Koryphe version to 2.2.0 * gh-2369 corrections * gh-2369 corrections * Fix copyright dates * gh-2369 import fix * gh-2357 spotless changes * gh-2357 FederatedStore FederatedOperation merging. * gh-2357 FederatedStore FederatedOperation temporary classes in lieu of Koryphe changes. see gh-272 * gh-2357 FederatedStore FederatedOperation javadoc * gh-2357 FederatedStore FederatedOperation default merge function change. * gh-2357 FederatedStore FederatedOperation FederatedOutputIterableHandler rename and change * Updated Koryphe version to 2.3.0 * gh-2357 FederatedStore FederatedOperation default merge function change. * gh-2357 FederatedStore FederatedOperation tidy * gh-2357 FederatedStore FederatedOperation Test improvement * gh-2357 FederatedStore deprecated * gh-2357 FederatedStore checkstyle * gh-2357 FederatedStore TDD DefaultMerge of GetElements with View Aggregation * gh-2357 FederatedStore spotless * gh-2357 FederatedStore DefaultMergeFunction from util. * gh-2357 FederatedStore DefaultMergeFunction best efforts of concat. * FederatedStore Alpha4 Team Review PR * gh-2357 FederatedStore undoing changes out of scope of gh-2357 * gh-2357 FederatedStore spotless checkstyle * gh-2357 FederatedStore undoing changes out of scope of gh-2357 and spotless checkstyle * gh-2357 FederatedStore reviewing of Todos, graphIds as List not just CSV. * gh-2357 FederatedStore reviewing of Todos * PR merge hot fix * PR merge hot fix * PR hotfix formatting * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 PR changes. * gh-2357 Remove ToIterable, fix bracket typo * gh-2357 PR changes. * gh-2357 Fix regex Co-authored-by: GCHQDev404 <gchqdev404@users.noreply.github.com> Co-authored-by: t92549 <80890692+t92549@users.noreply.github.com> Co-authored-by: GCHQDeveloper314 <94527357+GCHQDeveloper314@users.noreply.github.com>
gchq · Oct 24, 2022 · c1517b1 · c1517b1
1 parent 13adef0
commit c1517b1
Show file tree

Hide file tree

Showing 118 changed files with 4,984 additions and 4,711 deletions.
diff --git a/core/data/src/main/java/uk/gov/gchq/gaffer/data/element/Edge.java b/core/data/src/main/java/uk/gov/gchq/gaffer/data/element/Edge.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -32,6 +32,9 @@
 import uk.gov.gchq.gaffer.data.element.id.EdgeId;
 
 import java.util.Comparator;
+import java.util.Map;
+
+import static java.util.Objects.isNull;
 
 /**
  * An {@code Edge} in an {@link uk.gov.gchq.gaffer.data.element.Element} containing a source, destination and a directed flag.
@@ -424,6 +427,16 @@ public Builder property(final String name, final Object value) {
             return this;
         }
 
+        public Builder properties(final Map<String, String> properties) {
+            if (isNull(properties)) {
+                properties.clear();
+            } else {
+                this.properties.putAll(properties);
+                this.properties.remove(null);
+            }
+            return this;
+        }
+
         public Edge build() {
             return new Edge(group, source, dest, directed, matchedVertex, properties);
         }

diff --git a/core/data/src/main/java/uk/gov/gchq/gaffer/data/element/Entity.java b/core/data/src/main/java/uk/gov/gchq/gaffer/data/element/Entity.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -27,6 +27,10 @@
 import uk.gov.gchq.gaffer.commonutil.ToStringBuilder;
 import uk.gov.gchq.gaffer.data.element.id.EntityId;
 
+import java.util.Map;
+
+import static java.util.Objects.isNull;
+
 /**
  * An {@code Entity} in an {@link uk.gov.gchq.gaffer.data.element.Element} containing a single vertex.
  * The vertex can be any type of {@link java.lang.Object}.
@@ -174,6 +178,16 @@ public Builder property(final String name, final Object value) {
             return this;
         }
 
+        public Builder properties(final Map<String, String> properties) {
+            if (isNull(properties)) {
+                properties.clear();
+            } else {
+                this.properties.putAll(properties);
+                this.properties.remove(null);
+            }
+            return this;
+        }
+
         public Entity build() {
             return new Entity(group, vertex, properties);
         }

diff --git a/core/graph/src/main/java/uk/gov/gchq/gaffer/graph/GraphSerialisable.java b/core/graph/src/main/java/uk/gov/gchq/gaffer/graph/GraphSerialisable.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2017-2020 Crown Copyright
+ * Copyright 2017-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -185,6 +185,11 @@ public GraphConfig getDeserialisedConfig() {
         return deserialisedConfig;
     }
 
+    @JsonIgnore
+    public String getGraphId() {
+        return getDeserialisedConfig().getGraphId();
+    }
+
     public byte[] getConfig() {
         return config;
     }

diff --git a/core/operation/src/main/java/uk/gov/gchq/gaffer/operation/Operation.java b/core/operation/src/main/java/uk/gov/gchq/gaffer/operation/Operation.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -141,11 +141,9 @@ default void addOption(final String name, final String value) {
      * @return the value of the option
      */
     default String getOption(final String name) {
-        if (isNull(getOptions())) {
-            return null;
-        }
-
-        return getOptions().get(name);
+        return isNull(getOptions())
+                ? null
+                : getOptions().get(name);
     }
 
     /**
@@ -157,13 +155,9 @@ default String getOption(final String name) {
      * @return the value of the option
      */
     default String getOption(final String name, final String defaultValue) {
-        final String rtn;
-        if (isNull(getOptions())) {
-            rtn = defaultValue;
-        } else {
-            rtn = getOptions().get(name);
-        }
-        return (isNull(rtn)) ? defaultValue : rtn;
+        return (isNull(getOptions()))
+                ? defaultValue
+                : getOptions().getOrDefault(name, defaultValue);
     }
 
     @JsonGetter("options")

diff --git a/core/store/src/main/java/uk/gov/gchq/gaffer/store/Store.java b/core/store/src/main/java/uk/gov/gchq/gaffer/store/Store.java
@@ -643,6 +643,7 @@ public String getGraphId() {
      *         {@link uk.gov.gchq.gaffer.data.element.Element}s to be stored and how to
      *         aggregate the elements.
      */
+    @Deprecated
     public Schema getSchema() {
         return schema;
     }

diff --git a/core/store/src/main/java/uk/gov/gchq/gaffer/store/operation/OperationChainValidator.java b/core/store/src/main/java/uk/gov/gchq/gaffer/store/operation/OperationChainValidator.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -17,6 +17,7 @@
 package uk.gov.gchq.gaffer.store.operation;
 
 import uk.gov.gchq.gaffer.commonutil.pair.Pair;
+import uk.gov.gchq.gaffer.data.elementdefinition.view.View;
 import uk.gov.gchq.gaffer.operation.Operation;
 import uk.gov.gchq.gaffer.operation.OperationChain;
 import uk.gov.gchq.gaffer.operation.graph.GraphFilters;
@@ -134,9 +135,9 @@ protected void validateComparables(final Operation op, final User user, final St
     }
 
     protected void validateViews(final Operation op, final User user, final Store store, final ValidationResult validationResult) {
-        if (op instanceof GraphFilters) {
+        if (shouldValidate(op)) {
             final Schema schema = getSchema(op, user, store);
-            final ValidationResult viewValidationResult = viewValidator.validate(((GraphFilters) op).getView(), schema, getStoreTraits(store));
+            final ValidationResult viewValidationResult = viewValidator.validate(getView(op), schema, getStoreTraits(store));
             if (!viewValidationResult.isValid()) {
                 validationResult.addError("View for operation "
                         + op.getClass().getName()
@@ -146,6 +147,14 @@ protected void validateViews(final Operation op, final User user, final Store st
         }
     }
 
+    protected View getView(final Operation op) {
+        return ((GraphFilters) op).getView();
+    }
+
+    protected boolean shouldValidate(final Operation op) {
+        return op instanceof GraphFilters;
+    }
+
     protected Schema getSchema(final Operation operation, final User user, final Store store) {
         return store.getSchema();
     }

diff --git a/core/store/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/GetTraitsHandler.java b/core/store/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/GetTraitsHandler.java
@@ -30,6 +30,7 @@
 import java.util.Collections;
 import java.util.Set;
 
+import static java.util.Objects.isNull;
 import static java.util.Objects.nonNull;
 import static org.apache.commons.collections4.CollectionUtils.isNotEmpty;
 
@@ -43,14 +44,16 @@ public GetTraitsHandler(final Set<StoreTrait> storeTraits) {
 
     @Override
     public Set<StoreTrait> doOperation(final GetTraits operation, final Context context, final Store store) throws OperationException {
+        Set<StoreTrait> rtn;
         if (!operation.isCurrentTraits()) {
-            return storeTraits;
-        }
-
-        if (null == currentTraits) {
-            currentTraits = Collections.unmodifiableSet(createCurrentTraits(store));
+            rtn = storeTraits;
+        } else {
+            if (isNull(currentTraits)) {
+                currentTraits = Collections.unmodifiableSet(createCurrentTraits(store));
+            }
+            rtn = currentTraits;
         }
-        return currentTraits;
+        return Sets.newHashSet(rtn);
     }
 
     private Set<StoreTrait> createCurrentTraits(final Store store) {

diff --git a/...store/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/OperationChainHandler.java b/...store/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/OperationChainHandler.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2017-2020 Crown Copyright
+ * Copyright 2017-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -13,6 +13,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
+
 package uk.gov.gchq.gaffer.store.operation.handler;
 
 import uk.gov.gchq.gaffer.operation.Operation;

diff --git a/...tore/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/OutputOperationHandler.java b/...tore/src/main/java/uk/gov/gchq/gaffer/store/operation/handler/OutputOperationHandler.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2020 Crown Copyright
+ * Copyright 2016-2022 Crown Copyright
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -24,7 +24,7 @@
 /**
  * An {@code OutputOperationHandler} defines how to handle a specific {@link Output} operations.
  */
-public interface OutputOperationHandler<OP extends Output<O>, O> extends OperationHandler<OP> {
+public interface OutputOperationHandler<OP extends Output<? extends O>, O> extends OperationHandler<OP> {
     /**
      * Execute the given {@link Output} operation.
      *

diff --git a/...GetFromEndpointOperationDeclarations.json → ...GetFromEndpointOperationDeclarations.json b/...GetFromEndpointOperationDeclarations.json → ...GetFromEndpointOperationDeclarations.json