Handling batch request exception

[nepgya2803]

I'm in a case where I have to post all items in a JSON file to the SP list. I intend to group each 20 items into 1 batch. The thing seems to work well but sometimes I get an exception that

PnP.Core.ClientException: Batchrequest 5494125c-75cc-45c4-acaf-b031798fe88c was not found in one of the current batches
at PnP.Core.Services.BatchClient.GetBatchByBatchRequestId(Guid id)
at PnP.Core.Model.SharePoint.ListItem.<.ctor>b__2_0(FromJson input)
at PnP.Core.Services.JsonMappingHelper.FromJsonRest(TransientObject pnpObject, EntityInfo entity, ApiResponse apiResponse, Func2 fromJsonCasting) at PnP.Core.Services.JsonMappingHelper.FromJson(TransientObject pnpObject, EntityInfo entity, ApiResponse apiResponse, Func2 fromJsonCasting)
at PnP.Core.Services.JsonMappingHelper.MapJsonToModel(BatchRequest batchRequest)
at PnP.Core.Services.BatchClient.ProcessSharePointRestBatchResponse(SPORestBatch restBatch, String batchResponse, Dictionary`2 headers)
at PnP.Core.Services.BatchClient.ExecuteSharePointRestBatchAsync(Batch batch)
at PnP.Core.Services.BatchClient.ExecuteBatch(Batch batch)
at PnP.Core.Services.PnPContext.ExecuteAsync(Batch batch, Boolean throwOnError)

Also, the total number of items on the Sharepoint list exceeds the number in my JSON file (2356 vs 2000). I don't know which part I did wrong, also is there any way to handle the batch request like removing the item that caused an exception from the batch and putting it into another patch to re-process later?

 int addCount = 0;
 int updateCount = 0;
 string viewXml = CAMLQueryGenerator();
 var listExistingItem = await GetWithCalmQuery(myList, viewXml);

 var addBatch = new Batch();
 var updateBatch = new Batch();
 foreach (var user in Data)
 {
     try
     {
         // Execute the upsert procedure
         if (!listExistingItem.Any()
             || !listExistingItem.Where(p => p["EmployeeNumber"].ToString() == user.personIdExternal).Any())
         {
             if (addCount == 20)
             {
                 await this._context.ExecuteAsync(addBatch);
                 addBatch = new Batch();
                 addCount = 0;
             }

             await Add(this._context, myList, user, addBatch);
             addCount++;
         }
         else
         {
             if (updateCount == 20)
             {
                 await this._context.ExecuteAsync(updateBatch);
                 updateBatch = new Batch();
             }

             await Update(this._context, myList, viewXml, user, updateBatch);
             updateCount++;
         }
     }
     catch (Exception err)
     {
         Console.WriteLine(err.ToString());
         Console.WriteLine("Here is user ID employee: " + user.personIdExternal);
     }
 }

 await this._context.ExecuteAsync(addBatch);
 await this._context.ExecuteAsync(updateBatch);

[jansenbe]

@ngqtrieu99 : see https://pnp.github.io/pnpcore/using-the-sdk/basics-batching.html#handling-batch-failures. You can execute the batch without it throwing errors, instead you get back a collection of errors and you can decide how to handle them (e.g. re submit via a new batch).

Also you don't have to worry about the number of requests in the batch, if a batch is too big we'll automatically split it into multiple batch calls (see https://pnp.github.io/pnpcore/using-the-sdk/basics-batching.html#batch-limits)

[nepgya2803]

Thanks @jansenbe , but about the exception above, do you have any idea why it happened? I've checked the ID of addBatch but it's different from the ID in the exception when this code is executed even though I haven't renewed it
await this._context.ExecuteAsync(addBatch);

[jansenbe]

@ngqtrieu99 : Not sure, but would recommend to first remove the batch grouping to 20 from your code and see if the issue still is there

[nepgya2803]

I changed the batch = new batch() to batch = context.NewBatch() and somehow it works now, a bit weird but okay. Thank you for your help.