Upsert

.gitignore

@@ -1,2 +1,2 @@
 out/
-.idea/checkstyle-idea.xml
+.idea

.gitignore.save

@@ -0,0 +1,2 @@
+out/
+.idea

README.md

@@ -14,25 +14,48 @@ Following actions are inside:
 
 ``LOOKUP BY PRIMARY KEY`` - this action will execute select query from specified table, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns only one result (a primary key is unique). 
 
+``UPSERT BY PRIMARY KEY`` - this action will execute select command from specified table, as search criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"), and execute insert command by PRIMARY KEY with specified field, if result does not found, else - action will execute update command by PRIMARY KEY with specified field. The action returns only one result row (a primary key is unique).
+
 ``DELETE BY PRIMARY KEY`` - this action will execute delete query from specified table, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns an integer value that indicates the number of rows affected, the returned value can be 0 or 1 (a primary key is unique).
 ### How works
 
 ### Requirements
 Before you can deploy any code into elastic.io **you must be a registered elastic.io platform user**. Please see our home page at [http://www.elastic.io](http://www.elastic.io) to learn how. 
 #### Environment variables
-For unit-testing
+For unit-testing is needed to specify following environment variables:
+1. Connection to MSSQL:
+ - ``CONN_USER_MSSQL`` - user login
+ - ``CONN_PASSWORD_MSSQL`` - user password
+ - ``CONN_DBNAME_MSSQL`` - DataBase name
+ - ``CONN_HOST_MSSQL`` - DataBase host
+ - ``CONN_PORT_MSSQL`` - DataBase port
+2. Connection to MySQL:
+ - ``CONN_USER_MYSQL`` - user login
+ - ``CONN_PASSWORD_MYSQL`` - user password
+ - ``CONN_DBNAME_MYSQL`` - DataBase name
+ - ``CONN_HOST_MYSQL`` - DataBase host
+ - ``CONN_PORT_MYSQL`` - DataBase port
+3. Connection to Oracle:
+ - ``CONN_USER_ORACLE`` - user login
+ - ``CONN_PASSWORD_ORACLE`` - user password
+ - ``CONN_DBNAME_ORACLE`` - DataBase name
+ - ``CONN_HOST_ORACLE`` - DataBase host
+ - ``CONN_PORT_ORACLE`` - DataBase port
+4. Connection to PostgreSQL:
+ - ``CONN_USER_POSTGRESQL`` - user login
+ - ``CONN_PASSWORD_POSTGRESQL`` - user password
+ - ``CONN_DBNAME_POSTGRESQL`` - DataBase name
+ - ``CONN_HOST_POSTGRESQL`` - DataBase host
+ - ``CONN_PORT_POSTGRESQL`` - DataBase port
 #### Others
 ## Credentials
 You may use following properties to configure a connection:
 ![image](https://user-images.githubusercontent.com/40201204/43577550-ce99efe6-9654-11e8-87ed-f3e0839d618a.png)
 You can add the authorisation methods during the integration flow design or by going to your Settings > Security credentials > REST client and adding there.
 ### DB Engine
+You are able to choose one of existing database types:
 ![image](https://user-images.githubusercontent.com/40201204/43577772-6f85bdea-9655-11e8-96e1-368493a36c9d.png)
-You are able to choose one of existing database types
-- ``MySQL`` - compatible with MySQL Server 5.5, 5.6, 5.7 and 8.0.
-- ``PostgreSQL`` - compatible with PostgreSQL 8.2 and higher
-- ``Oracle`` - compatible with Oracle Database 8.1.7 - 12.1.0.2
-- ``MSSQL`` - compatible with Microsoft SQL Server 2008 R2 and higher
+
 ### Connection URI
 In the Connection URI field please provide hostname of the server, e.g. ``acme.com``
 ### Connection port
@@ -120,12 +143,46 @@ Component supports dynamic incoming metadata - as soon as your query is in place
 
 ### LOOKUP BY PRIMARY KEY
 ![image](https://user-images.githubusercontent.com/40201204/43592505-5b6bbfe8-967e-11e8-845e-2ce8ac707357.png)
+
 The action will execute select query from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns only one result (a primary key is unique).
 Checkbox ``Don't throw Error on an Empty Result`` allows to emit an empty response, otherwise you will get an error on empty response.
 #### Input fields description
 ![image](https://user-images.githubusercontent.com/40201204/43644579-f593d1c8-9737-11e8-9b97-ee9e575a19f7.png)
 As an input metadata you will get a Primary Key field to provide the data inside as a clause value.
 
+### UPSERT BY PRIMARY KEY
+The action will execute ``SELECT`` command from a ``Tables`` dropdown field, as search criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"), and execute ``INSERT`` command by PRIMARY KEY with specified field, if result does not found, else - action will execute ``UPDATE`` command by PRIMARY KEY with specified field. The action returns only one result row (a primary key is unique).
+1. Find and select jdbc-component in the component repository
+![image](https://user-images.githubusercontent.com/16806832/44981615-c70a9d80-af7b-11e8-8055-3b553abe8212.png)
+
+2. Create new or select existing credentials
+![image](https://user-images.githubusercontent.com/16806832/44981652-e86b8980-af7b-11e8-897e-04d1fc9a93cf.png)
+
+3. Select action "Upsert Row By Primary Key" from list
+![image](https://user-images.githubusercontent.com/16806832/44981700-0d5ffc80-af7c-11e8-9ac3-aedb16e1d788.png)
+
+4. Select table from ``Table`` dropdown list
+![image](https://user-images.githubusercontent.com/16806832/44981754-38e2e700-af7c-11e8-87d3-f029a7fec8fa.png)
+
+5. Specify input data (field with red asterisk is Primary key), and click "Continue"
+![image](https://user-images.githubusercontent.com/16806832/44981854-83fcfa00-af7c-11e8-9ef2-8c06e77fed1e.png)
+
+6. Retrieving sample
+![image](https://user-images.githubusercontent.com/16806832/44983059-86f9e980-af80-11e8-8178-77e463488c7a.png)
+
+7. Retrieve sample result
+![image](https://user-images.githubusercontent.com/16806832/44982952-2ec2e780-af80-11e8-98b1-58c3adbc15b9.png)
+
+8. Click "Continue"
+![image](https://user-images.githubusercontent.com/16806832/44983101-b0b31080-af80-11e8-82d8-0e70e4b4ff97.png)
+
+9. Finish component configuration
+![image](https://user-images.githubusercontent.com/16806832/44983365-90378600-af81-11e8-9be4-4dbb39af0fdc.png)
+
+#### Input fields description
+As an input metadata you will get all fields of selected table. [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY") is required field (will mark as asterisk) and other input fields are optional.
+![image](https://user-images.githubusercontent.com/16806832/44397461-1a76f780-a549-11e8-8247-9a6f9aa3f3b4.png)
+
 ### DELETE BY PRIMARY KEY
 ![image](https://user-images.githubusercontent.com/40201204/43592505-5b6bbfe8-967e-11e8-845e-2ce8ac707357.png)
 The action will execute delete query from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns count of affected rows.
@@ -134,6 +191,17 @@ Checkbox ``Don't throw Error on an Empty Result`` allows to emit an empty respon
 ![image](https://user-images.githubusercontent.com/40201204/43644579-f593d1c8-9737-11e8-9b97-ee9e575a19f7.png)
 As an input metadata you will get a Primary Key field to provide the data inside as a clause value.
 
+## Current limitations
+1. Only tables with one [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY") is supported. You will see the message ``Table has not Primary Key. Should be one Primary Key
+``, if the selected table doesn't have a primary key. Also, you will see the message ``Composite Primary Key is not supported
+``, if the selected table has composite primary key.
+2. Only following versions of database types are supported:
+- ``MySQL`` - compatible with MySQL Server 5.5, 5.6, 5.7 and 8.0.
+- ``PostgreSQL`` - compatible with PostgreSQL 8.2 and higher
+- ``Oracle`` - compatible with Oracle Database 8.1.7 - 12.1.0.2
+- ``MSSQL`` - compatible with Microsoft SQL Server 2008 R2 and higher
+3. The current implementation of the action ``Upsert By Primary Key`` doesn't mark non-nullable fields as required fields at a dynamic metadata. In case of updating such fields with an empty value you will get SQL Exception ``Cannot insert the value NULL into...``. You should manually fill in all non-nullable fields with previous data, if you want to update part of columns in a row, even if data in that fields doesn't change. 
+
 ## Known issues
 No known issues are there yet.
 

component.json

@@ -124,6 +124,21 @@
       },
       "dynamicMetadata": "io.elastic.jdbc.PrimaryColumnNamesProvider"
     },
+    "upsertRowByPrimaryKey": {
+      "main": "io.elastic.jdbc.actions.UpsertRowByPrimaryKey",
+      "title": "Upsert Row By Primary Key",
+      "description": "Executes upsert by primary key",
+      "fields": {
+        "tableName": {
+          "viewClass": "SelectView",
+          "prompt": "Select a Table",
+          "label": "Table",
+          "required": true,
+          "model": "io.elastic.jdbc.TableNameProvider"
+        }
+      },
+      "dynamicMetadata": "io.elastic.jdbc.ColumnNamesWithPrimaryKeyProvider"
+    },
     "deleteRowByPrimaryKey": {
       "main": "io.elastic.jdbc.actions.DeleteRowByPrimaryKey",
       "title": "Delete Row By Primary Key",

src/main/java/io/elastic/jdbc/ColumnNamesProvider.java

@@ -17,7 +17,7 @@
 
 public class ColumnNamesProvider implements DynamicMetadataProvider, SelectModelProvider {
 
-  private static final Logger logger = LoggerFactory.getLogger(ColumnNamesProvider.class);
+  private static final Logger LOGGER = LoggerFactory.getLogger(ColumnNamesProvider.class);
 
   @Override
   public JsonObject getSelectModel(JsonObject configuration) {
@@ -57,8 +57,8 @@ public JsonObject getColumns(JsonObject configuration) {
     ResultSet rs = null;
     String schemaName = null;
     boolean isEmpty = true;
-    Boolean isOracle = (configuration.getString("dbEngine").equals("oracle")) ? true : false;
-    Boolean isMssql = (configuration.getString("dbEngine").equals("mssql")) ? true : false;
+    Boolean isOracle = configuration.getString("dbEngine").equals("oracle");
+    Boolean isMssql = configuration.getString("dbEngine").equals("mssql");
     try {
       connection = Utils.getConnection(configuration);
       DatabaseMetaData dbMetaData = connection.getMetaData();
@@ -71,14 +71,14 @@ public JsonObject getColumns(JsonObject configuration) {
       while (rs.next()) {
         JsonObjectBuilder field = Json.createObjectBuilder();
         String name = rs.getString("COLUMN_NAME");
-        Boolean isRequired = false;
+        Boolean isRequired;
+        Integer isNullable = (rs.getObject("NULLABLE") != null) ? rs.getInt("NULLABLE") : 1;
         if (isMssql) {
           String isAutoincrement =
               (rs.getString("IS_AUTOINCREMENT") != null) ? rs.getString("IS_AUTOINCREMENT") : "";
-          Integer isNullable = (rs.getObject("NULLABLE") != null) ? rs.getInt("NULLABLE") : 1;
           isRequired = isNullable == 0 && !isAutoincrement.equals("YES");
         } else {
-          isRequired = false;
+          isRequired = isNullable == 0;
         }
         field.add("required", isRequired)
             .add("title", name)
@@ -97,14 +97,14 @@ public JsonObject getColumns(JsonObject configuration) {
         try {
           rs.close();
         } catch (SQLException e) {
-          logger.error("Failed to close result set {}", e.toString());
+          LOGGER.error("Failed to close result set {}", e);
         }
       }
       if (connection != null) {
         try {
           connection.close();
         } catch (SQLException e) {
-          logger.error("Failed to close connection {}", e.toString());
+          LOGGER.error("Failed to close connection {}", e);
         }
       }
     }

src/main/java/io/elastic/jdbc/ColumnNamesWithPrimaryKeyProvider.java

@@ -0,0 +1,139 @@
+package io.elastic.jdbc;
+
+import io.elastic.api.DynamicMetadataProvider;
+import io.elastic.api.SelectModelProvider;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Types;
+import java.util.Map;
+import javax.json.Json;
+import javax.json.JsonObject;
+import javax.json.JsonObjectBuilder;
+import javax.json.JsonValue;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ColumnNamesWithPrimaryKeyProvider implements DynamicMetadataProvider,
+    SelectModelProvider {
+
+  private static final Logger LOGGER = LoggerFactory
+      .getLogger(ColumnNamesWithPrimaryKeyProvider.class);
+
+  @Override
+  public JsonObject getSelectModel(JsonObject configuration) {
+    JsonObjectBuilder result = Json.createObjectBuilder();
+    JsonObject properties = getColumns(configuration);
+    for (Map.Entry<String, JsonValue> entry : properties.entrySet()) {
+      result.add(entry.getKey(), entry.getKey());
+    }
+    return result.build();
+  }
+
+  /**
+   * Returns Columns list as metadata
+   */
+
+  @Override
+  public JsonObject getMetaModel(JsonObject configuration) {
+    JsonObjectBuilder result = Json.createObjectBuilder();
+    JsonObjectBuilder inMetadata = Json.createObjectBuilder();
+    JsonObjectBuilder outMetadata = Json.createObjectBuilder();
+    JsonObject properties = getColumns(configuration);
+    inMetadata.add("type", "object").add("properties", properties);
+    outMetadata.add("type", "object").add("properties", properties);
+    result.add("out", outMetadata.build()).add("in", inMetadata.build());
+    return result.build();
+  }
+
+  public JsonObject getColumns(JsonObject configuration) {
+    if (configuration.getString("tableName") == null || configuration.getString("tableName")
+        .isEmpty()) {
+      throw new RuntimeException("Table name is required");
+    }
+    String tableName = configuration.getString("tableName");
+    JsonObjectBuilder properties = Json.createObjectBuilder();
+    Connection connection = null;
+    ResultSet rs = null;
+    ResultSet rsPrimaryKeys = null;
+    String schemaName = null;
+    boolean isEmpty = true;
+    Boolean isOracle = configuration.getString("dbEngine").equals("oracle");
+    try {
+      connection = Utils.getConnection(configuration);
+      DatabaseMetaData dbMetaData = connection.getMetaData();
+      if (tableName.contains(".")) {
+        schemaName = tableName.split("\\.")[0];
+        tableName = tableName.split("\\.")[1];
+      }
+      rsPrimaryKeys = dbMetaData
+          .getPrimaryKeys(null, ((isOracle && !schemaName.isEmpty()) ? schemaName : null),
+              tableName);
+      rs = dbMetaData.getColumns(null, schemaName, tableName, "%");
+      while (rs.next()) {
+        JsonObjectBuilder field = Json.createObjectBuilder();
+        String name = rs.getString("COLUMN_NAME");
+        Boolean isRequired = false;
+        while (rsPrimaryKeys.next()) {
+          if (rsPrimaryKeys.getString("COLUMN_NAME").equals(name)) {
+            isRequired = true;
+            break;
+          }
+        }
+        field.add("required", isRequired)
+            .add("title", name)
+            .add("type", convertType(rs.getInt("DATA_TYPE")));
+        properties.add(name, field.build());
+        isEmpty = false;
+      }
+      if (isEmpty) {
+        properties.add("empty dataset", "no columns");
+      }
+
+    } catch (SQLException e) {
+      throw new RuntimeException(e);
+    } finally {
+      if (rs != null) {
+        try {
+          rs.close();
+        } catch (SQLException e) {
+          LOGGER.error("Failed to close result set {}", e);
+        }
+      }
+      if (rsPrimaryKeys != null) {
+        try {
+          rsPrimaryKeys.close();
+        } catch (SQLException e) {
+          LOGGER.error("Failed to close result set {}", e);
+        }
+      }
+      if (connection != null) {
+        try {
+          connection.close();
+        } catch (SQLException e) {
+          LOGGER.error("Failed to close connection {}", e);
+        }
+      }
+    }
+    return properties.build();
+  }
+
+  /**
+   * Converts JDBC column type name to js type according to http://db.apache.org/ojb/docu/guides/jdbc-types.html
+   *
+   * @param sqlType JDBC column type
+   * @url http://db.apache.org/ojb/docu/guides/jdbc-types.html
+   */
+  private String convertType(Integer sqlType) {
+    if (sqlType == Types.NUMERIC || sqlType == Types.DECIMAL || sqlType == Types.TINYINT
+        || sqlType == Types.SMALLINT || sqlType == Types.INTEGER || sqlType == Types.BIGINT
+        || sqlType == Types.REAL || sqlType == Types.FLOAT || sqlType == Types.DOUBLE) {
+      return "number";
+    }
+    if (sqlType == Types.BIT || sqlType == Types.BOOLEAN) {
+      return "boolean";
+    }
+    return "string";
+  }
+}

src/main/java/io/elastic/jdbc/JdbcCredentialsVerifier.java

@@ -10,27 +10,27 @@
 
 public class JdbcCredentialsVerifier implements CredentialsVerifier {
 
-  private static final Logger logger = LoggerFactory.getLogger(JdbcCredentialsVerifier.class);
+  private static final Logger LOGGER = LoggerFactory.getLogger(JdbcCredentialsVerifier.class);
 
   @Override
   public void verify(JsonObject configuration) throws InvalidCredentialsException {
 
-    logger.info("About to connect to database using given credentials");
+    LOGGER.info("About to connect to database using given credentials");
 
     Connection connection = null;
 
     try {
       connection = Utils.getConnection(configuration);
-      logger.info("Successfully connected to database. Credentials verified.");
+      LOGGER.info("Successfully connected to database. Credentials verified.");
     } catch (Exception e) {
       throw new InvalidCredentialsException("Failed to connect to database", e);
     } finally {
       if (connection != null) {
-        logger.info("Closing database connection");
+        LOGGER.info("Closing database connection");
         try {
           connection.close();
         } catch (SQLException e) {
-          logger.error("Failed to closed database connection", e);
+          LOGGER.error("Failed to closed database connection", e);
         }
       }
     }