Composite Resources

[ghost]

@timrogers If this isn't the right place to ask, excuse me. But, to assist someone who takes the initiative to add composite resource API calls to restforce, can you explain what files would be created and where ? And what would have to be added to existing files. I think some high-level direction would help, as I poked around a bit and wasn't sure where someone would begin.

Reference, composite resource calls: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_composite_composite.htm

A couple of possibly relevant issues:

• Creating parent and children records in one query #311
• Support For SObject Tree / Nested Objects / CRUD 200 Records In One Call #258

[nicholassewitz]

I'd help on this as well

[kennyb5000]

I have added this into my own project using monkey patching (not recommended). It works to send the request but the error handling is messy because of the nature of the calls. I also implemented composite tree and it works well enough.

module Restforce::Concerns::API
  def composite_tree(*args)
    composite_tree!(*args)
  rescue *exceptions
    false
  end

  def composite_tree!(sobject, attrs)
    response = api_post("composite/tree/#{sobject}", attrs)
    response.body['results']
  end

  def composite(*args)
    composite!(*args)
  rescue *exceptions
    false
  end

  def composite!(attrs)
    response = api_post('composite/', attrs)
    errors = []
    response.body['compositeResponse'].each do |result|
      message = result['body'][0]['message']
      case result['httpStatusCode']
      when 404
        errors << "Record Not Found: #{message}"
      when 401
        errors << "Unauthorized: #{message}"
      when 413
        errors << "HTTP 413 - Request Entity Too Large: #{message}"
      when 400...600
        errors << "ClientError: #{message}"
      end
    end
    raise Faraday::Error::ClientError.new(errors.join(',')) unless errors.empty?
    response.body['compositeResponse']
  end
end

Example Usage:

client.composite!({ compositeRequest: Array.wrap(group).compact })

# PATCH is upsert, GET is query, and POST is create/update
# External IDs can be used for PATCH requests
group = [
  {
    method: 'PATCH',
    url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_EXTERNAL_ID_FIELD/SOBJECT_EXTERNAL_ID",
    referenceId: "Name to reference within response",
    body: {
        Field__c: 'data',
        Field2__c: 'data'
    }
  },
   {
       method: 'PATCH',
       url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID",
       referenceId: "Name to reference within response",
       body: {
           Field__c: 'data',
           Field2__c: 'data'
       }
   },
   {
      method: 'POST',
      url: "/services/data/v38.0/sobjects/SOBJECT_NAME",
      referenceId: "Name to reference within response",
      body: {
          Field__c: 'data',
          Field2__c: 'data'
      }
   },
   {
      method: 'POST',
      url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID",
      referenceId: "Name to reference within response",
      body: {
          Field__c: 'data',
          Field2__c: 'data'
      }
   },
  {
      method: 'Get',
      url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID",
      referenceId: "Name to reference within response"
  }
]

Example Response: 2 Success, 1 Failure

[
  {
    "body"=>nil,
    "httpHeaders"=>{},
    "httpStatusCode"=>204,
    "referenceId"=>"07faed26-3caf-49f7-9fe0-6e8091f09cfd"
  },
  {"body"=>
   [{"errorCode"=>"NOT_FOUND",
     "message"=>
      "Provided external ID field does not exist or is not accessible:"}],
      "httpHeaders"=>{},
      "httpStatusCode"=>404,
      "referenceId"=>"07faed26-3caf-49f7-9fe0-6e8091f09cfd"
  },
  {
     "body" : {       
        "id" : "001R00000033I6AIAU",
        "success" : true,
        "errors" : [ ]
     },
     "httpHeaders" : { 
        "Location" : "/services/data/v38.0/sobjects/Account/001R00000033I6AIAU"
     },
     "httpStatusCode" : 201,
     "referenceId" : "refAccount"
  }
]

The problems I am currently running into with the composite endpoint are:

1. The composite endpoint allows multiple Sobject types so the urls have to generated when the data is built not within the method like everywhere else
2. The composite endpoint only allows 25 requests per request, I am currently doing that outside of the method but it could be moved into it
3. Error Handling has been difficult because 1 out of the 25 requests can fail while 24 go through

I am planning on working through these issues getting a PR up but don't have the time right now. If someone else want to work through the issues I mentioned let me know and we can talk through it.

[timrogers]

Great work on this! It sounds like we'd need to build an abstraction around the usual way of calling the API if we wanted to support these complex composite requests. You'd ideally, I guess, want to be able to create individual composite requests (like each item in `group` in your example) as some kind of object, wrap those up into a single Composite Request object (which limits the maximum elements to 25) and then have some kind of response object which lets you easily see what failed and what worked. It probably makes sense for you to work on it (or someone else using this functionality) rather than me as I don't understand the use cases as well as you will, but I'd love to see a PR.

…

On Thursday, August 31, 2017, kennyb5000 ***@***.***> wrote: I have added this into my own project using monkey patching. It works to send the request but the error handling is messy because of the nature of the calls. I also implemented composite tree <https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_composite_sobject_tree.htm> and it works well enough. module Restforce::Concerns::API def composite_tree(*args) composite_tree!(*args) rescue *exceptions false end def composite_tree!(sobject, attrs) response = api_post("composite/tree/#{sobject}", attrs) response.body['results'] end def composite(*args) composite!(*args) rescue *exceptions false end def composite!(attrs) response = api_post('composite/', attrs) errors = [] response.body['compositeResponse'].each do |result| message = result['body'][0]['message'] case result['httpStatusCode'] when 404 errors << "Record Not Found: #{message}" when 401 errors << "Unauthorized: #{message}" when 413 errors << "HTTP 413 - Request Entity Too Large: #{message}" when 400...600 errors << "ClientError: #{message}" end end raise Faraday::Error::ClientError.new(errors.join(',')) unless errors.empty? response.body['compositeResponse'] end end Example Usage: client.composite!({ compositeRequest: Array.wrap(group).compact }) # PATCH is upsert, GET is query, and POST is create/update # External IDs can be used for PATCH requests group = [ { method: 'PATCH', url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_EXTERNAL_ID_FIELD/SOBJECT_EXTERNAL_ID", referenceId: "Name to reference within response", body: { Field__c: 'data', Field2__c: 'data' } }, { method: 'PATCH', url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID", referenceId: "Name to reference within response", body: { Field__c: 'data', Field2__c: 'data' } }, { method: 'POST', url: "/services/data/v38.0/sobjects/SOBJECT_NAME", referenceId: "Name to reference within response", body: { Field__c: 'data', Field2__c: 'data' } }, { method: 'POST', url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID", referenceId: "Name to reference within response", body: { Field__c: 'data', Field2__c: 'data' } }, { method: 'Get', url: "/services/data/v38.0/sobjects/SOBJECT_NAME/SOBJECT_ID", referenceId: "Name to reference within response" } ] Example Response: 2 Success, 1 Failure [ { "body"=>nil, "httpHeaders"=>{}, "httpStatusCode"=>204, "referenceId"=>"07faed26-3caf-49f7-9fe0-6e8091f09cfd" }, {"body"=> [{"errorCode"=>"NOT_FOUND", "message"=> "Provided external ID field does not exist or is not accessible:"}], "httpHeaders"=>{}, "httpStatusCode"=>404, "referenceId"=>"07faed26-3caf-49f7-9fe0-6e8091f09cfd" }, { "body" : { "id" : "001R00000033I6AIAU", "success" : true, "errors" : [ ] }, "httpHeaders" : { "Location" : "/services/data/v38.0/sobjects/Account/001R00000033I6AIAU" }, "httpStatusCode" : 201, "referenceId" : "refAccount" } ] The problems I am currently running into with the composite endpoint are: 1. The composite endpoint allows multiple Sobject types so the urls have to generated when the data is built not within the method like everywhere else 2. The composite endpoint only allows 25 requests per request, I am currently doing that outside of the method but it could be moved into it 3. Error Handling has been difficult because 1 out of the 25 requests can fail while 24 go through I am planning on working through these issues getting a PR up but don't have the time right now. If someone else want to work through the issues I mentioned let me know and we can talk through it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#316 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHFpnwmCxvexV1UjI_pg11myZTujqvyks5sduiMgaJpZM4Ohdol> .

-- -- *Tim Rogers* GoCardless.com <https://gocardless.com>

[kennyb5000]

@timrogers Thanks for the suggestions. I will dig further into this when I can, hopefully sometime later this month (now looking like later this year sadly). If anyone else wants to take a crack at it please do.