Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres Bulk Merge Issue #73

Closed
plsft opened this issue Oct 10, 2022 · 1 comment
Closed

Postgres Bulk Merge Issue #73

plsft opened this issue Oct 10, 2022 · 1 comment

Comments

@plsft
Copy link

plsft commented Oct 10, 2022

Hello, I'm trying to use BulkMerge with a response from JSON web API call.
I created a dataTable from the result set and am trying to dynamically create the table then a PK, then bulk merge, however I keep getting same error regarding primary key.

 IList<dynamic> table = JsonConvert.DeserializeObject<List<dynamic>>(Convert.ToString(json!.response.results));
            DataTable dataTable = JsonConvert.DeserializeObject<DataTable>(Convert.ToString(json.response.results));
            var columns = dataTable.Columns.OfType<DataColumn>().Select(c => c.ColumnName).ToList();
            dataTable.PrimaryKey = new DataColumn[] { dataTable.Columns["policy_id"] }; 
            _connection = new NpgsqlConnection(_configuration.GetConnectionString("DbConnection"));
            _connection.Open();

            var sqlCreateCommand = $"CREATE TABLE IF NOT EXISTS public.crm_{module} (";
            sqlCreateCommand +=  string.Join(" ", (from dc in dataTable.Columns.Cast<DataColumn>() select $"{dc.ColumnName}" + $" {dc.DataType.ToPgSqlDataType(dc.ColumnName)},").ToArray()).RemoveTrailingComma() + ");";
            var cmd = new NpgsqlCommand(sqlCreateCommand, (NpgsqlConnection)_connection);
            cmd.ExecuteNonQuery();

            var sqlCreatePK =  $@"do $$ begin if not exists (select 1 from information_schema.table_constraints where table_name = 'crm_{module}' and constraint_type = 'PRIMARY KEY') then ALTER TABLE crm_{module} ADD PRIMARY KEY ({dataTable.PrimaryKey[0].ColumnName}); end if; end; $$";
            cmd.CommandText = sqlCreatePK;
            cmd.ExecuteNonQuery(); 

            var bulk = new BulkOperation((DbConnection)_connection)
            {
                DestinationTableName = module, 
                AllowDuplicateKeys = false, 
                AutoMapOutputIdentity = true,
                BatchSize = 1000,
                UseInternalTransaction =  true,
                AllowUpdatePrimaryKeys = false,
                AutoMapIdentityName = "policy_id",
                AutoMapKeyName = "policy_id;lead_id"
            };

            await bulk.BulkMergeAsync(dataTable, CancellationToken.None);
            
            cmd.Dispose(); 


            return bulk.RowsAffected;

The table does have a primary key and I'm setting the value in the call, but I still get this exception:


System.Exception: An error occured, no primary key could be found or resolved.
   at Z.BulkOperations.BulkOperation.()
   at Z.BulkOperations.BulkOperation.Execute()
   at System.Threading.Tasks.Task.InnerInvoke()
@plsft
Copy link
Author

plsft commented Oct 10, 2022

I was able to solve this issue. The destination table was incorrect and was missing the PK.

@plsft plsft closed this as completed Oct 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant