[PDE-4989] feat(schema): Add support for bulk writes

packages/schema/docs/build/schema.md

@@ -24,6 +24,7 @@ This is automatically generated by the `npm run docs` command in `zapier-platfor
 * [/BasicHookOperationSchema](#basichookoperationschema)
 * [/BasicOperationSchema](#basicoperationschema)
 * [/BasicPollingOperationSchema](#basicpollingoperationschema)
+* [/BulkObjectSchema](#bulkobjectschema)
 * [/BulkReadSchema](#bulkreadschema)
 * [/BulkReadsSchema](#bulkreadsschema)
 * [/CreateSchema](#createschema)
@@ -419,7 +420,7 @@ Represents the fundamental mechanics of a create.
 Key | Required | Type | Description
 --- | -------- | ---- | -----------
 `resource` | no | [/KeySchema](#keyschema) | Optionally reference and extends a resource. Allows Zapier to automatically tie together samples, lists and hooks, greatly improving the UX. EG: if you had another trigger reusing a resource but filtering the results.
-`perform` | **yes** | oneOf([/RequestSchema](#requestschema), [/FunctionSchema](#functionschema)) | How will Zapier get the data? This can be a function like `(z) => [{id: 123}]` or a request like `{url: 'http...'}`.
+`perform` | no (with exceptions, see description) | oneOf([/RequestSchema](#requestschema), [/FunctionSchema](#functionschema)) | How will Zapier get the data? This can be a function like `(z) => [{id: 123}]` or a request like `{url: 'http...'}`. Exactly one of `perform` or `performBulk` must be defined. If you choose to define `bulk` and `performBulk`, you must omit `perform`.
 `performResume` | no | [/FunctionSchema](#functionschema) | A function that parses data from a perform (which uses z.generateCallbackUrl()) and callback request to resume this action.
 `performGet` | no | oneOf([/RequestSchema](#requestschema), [/FunctionSchema](#functionschema)) | How will Zapier get a single record? If you find yourself reaching for this - consider resources and their built-in get methods.
 `inputFields` | no | [/DynamicFieldsSchema](#dynamicfieldsschema) | What should the form a user sees and configures look like?
@@ -428,6 +429,8 @@ Key | Required | Type | Description
 `lock` | no | [/LockObjectSchema](#lockobjectschema) | **INTERNAL USE ONLY**. Zapier uses this configuration for internal operation locking.
 `throttle` | no | [/ThrottleObjectSchema](#throttleobjectschema) | Zapier uses this configuration to apply throttling when the limit for the window is exceeded.
 `shouldLock` | no | `boolean` | Should this action be performed one at a time (avoid concurrency)?
+`bulk` | no (with exceptions, see description) | [/BulkObjectSchema](#bulkobjectschema) | Zapier uses this configuration for writing in bulk with `performBulk`.
+`performBulk` | no (with exceptions, see description) | [/FunctionSchema](#functionschema) | A function to write in bulk with. `bulk` and `performBulk` must either both be defined or neither. Additionally, only one of `perform` or `performBulk` can be defined. If you choose to define `perform`, you must omit `bulk` and `performBulk`.
 
 #### Examples
 
@@ -593,6 +596,34 @@ Key | Required | Type | Description
 
 -----
 
+## /BulkObjectSchema
+
+Zapier uses this configuration for writing in bulk.
+
+#### Details
+
+* **Type** - `object`
+* [**Source Code**](https://github.com/zapier/zapier-platform/blob/zapier-platform-schema@15.7.2/packages/schema/lib/schemas/BulkObjectSchema.js)
+
+#### Properties
+
+Key | Required | Type | Description
+--- | -------- | ---- | -----------
+`groupedBy` | **yes** | `array` | The list of keys of input fields to group bulk-write with. The actual user data provided for the fields will be used during execution. Note that a required input field should be referenced to get user data always.
+`limit` | **yes** | `integer` | The maximum number of items to call performBulk with. **Note** that it is capped by the platform to prevent exceeding the [AWS Lambda's request/response payload size quota of 6 MB](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#function-configuration-deployment-and-execution). Also, the execution is time-bound; we recommend reducing it upon consistent timeout.
+
+#### Examples
+
+* `{ groupedBy: [ 'workspace', 'sheet' ], limit: 100 }`
+
+#### Anti-Examples
+
+* `{ groupedBy: [], limit: 100 }` - _Empty groupedBy list provided: `[]`._
+* `{ groupedBy: [ 'workspace' ] }` - _Missing required key: `limit`._
+* `{ limit: 1 }` - _Missing required key: `groupedBy`._
+
+-----
+
 ## /BulkReadSchema
 
 How will Zapier fetch resources from your application?

packages/schema/exported-schema.json

@@ -1351,26 +1351,49 @@
       },
       "additionalProperties": false
     },
+    "BulkObjectSchema": {
+      "id": "/BulkObjectSchema",
+      "description": "Zapier uses this configuration for writing in bulk.",
+      "type": "object",
+      "required": ["groupedBy", "limit"],
+      "properties": {
+        "groupedBy": {
+          "description": "The list of keys of input fields to group bulk-write with. The actual user data provided for the fields will be used during execution. Note that a required input field should be referenced to get user data always.",
+          "type": "array",
+          "minItems": 1
+        },
+        "limit": {
+          "description": "The maximum number of items to call performBulk with. **Note** that it is capped by the platform to prevent exceeding the [AWS Lambda's request/response payload size quota of 6 MB](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#function-configuration-deployment-and-execution). Also, the execution is time-bound; we recommend reducing it upon consistent timeout.",
+          "type": "integer"
+        }
+      },
+      "additionalProperties": false
+    },
     "BasicCreateActionOperationSchema": {
       "id": "/BasicCreateActionOperationSchema",
       "description": "Represents the fundamental mechanics of a create.",
       "type": "object",
-      "required": ["perform"],
       "properties": {
         "resource": {
           "description": "Optionally reference and extends a resource. Allows Zapier to automatically tie together samples, lists and hooks, greatly improving the UX. EG: if you had another trigger reusing a resource but filtering the results.",
           "$ref": "/KeySchema"
         },
         "perform": {
-          "description": "How will Zapier get the data? This can be a function like `(z) => [{id: 123}]` or a request like `{url: 'http...'}`.",
+          "description": "How will Zapier get the data? This can be a function like `(z) => [{id: 123}]` or a request like `{url: 'http...'}`. Exactly one of `perform` or `performBulk` must be defined. If you choose to define `bulk` and `performBulk`, you must omit `perform`.",
           "oneOf": [
             {
               "$ref": "/RequestSchema"
             },
             {
               "$ref": "/FunctionSchema"
             }
-          ]
+          ],
+          "docAnnotation": {
+            "required": {
+              "type": "replace",
+              "value": "no (with exceptions, see description)"
+            }
+          }
         },
         "performResume": {
           "description": "A function that parses data from a perform (which uses z.generateCallbackUrl()) and callback request to resume this action.",
@@ -1417,6 +1440,26 @@
         "shouldLock": {
           "description": "Should this action be performed one at a time (avoid concurrency)?",
           "type": "boolean"
+        },
+        "bulk": {
+          "description": "Zapier uses this configuration for writing in bulk with `performBulk`.",
+          "$ref": "/BulkObjectSchema",
+          "docAnnotation": {
+            "required": {
+              "type": "replace",
+              "value": "no (with exceptions, see description)"
+            }
+          }
+        },
+        "performBulk": {
+          "description": "A function to write in bulk with. `bulk` and `performBulk` must either both be defined or neither. Additionally, only one of `perform` or `performBulk` can be defined. If you choose to define `perform`, you must omit `bulk` and `performBulk`.",
+          "$ref": "/FunctionSchema",
+          "docAnnotation": {
+            "required": {
+              "type": "replace",
+              "value": "no (with exceptions, see description)"
+            }
+          }
         }
       },
       "additionalProperties": false

packages/schema/lib/functional-constraints/bulkWriteConstraints.js

@@ -0,0 +1,98 @@
+'use strict';
+
+const _ = require('lodash');
+const jsonschema = require('jsonschema');
+
+const bulkWriteConstraints = (definition) => {
+  const errors = [];
+  const actionType = 'creates';
+
+  if (definition[actionType]) {
+    _.each(definition[actionType], (actionDef) => {
+      if (actionDef.operation && actionDef.operation.bulk) {
+        if (!actionDef.operation.performBulk) {
+          errors.push(
+            new jsonschema.ValidationError(
+              'must contain property "performBulk" because property "bulk" is present.',
+              actionDef.operation,
+              '/BasicCreateActionOperationSchema',
+              `instance.${actionType}.${actionDef.key}.operation`,
+              'missing',
+              'performBulk'
+            )
+          );
+        }
+
+        if (actionDef.operation.perform) {
+          errors.push(
+            new jsonschema.ValidationError(
+              'must not contain property "perform" because it is mutually exclusive with property "bulk".',
+              actionDef.operation,
+              '/BasicCreateActionOperationSchema',
+              `instance.${actionType}.${actionDef.key}.operation`,
+              'invalid',
+              'perform'
+            )
+          );
+        }
+
+        if (actionDef.operation.bulk.groupedBy) {
+          const requiredInputFields = [];
+          const inputFields = _.get(actionDef, ['operation', 'inputFields'], []);
+          inputFields.forEach((inputField) => {
+            if (inputField.required) {
+              requiredInputFields.push(inputField.key);
+            }
+          });
+
+          actionDef.operation.bulk.groupedBy.forEach((field, index) => {
+            if (!requiredInputFields.includes(field)) {
+              errors.push(
+                new jsonschema.ValidationError(
+                  `cannot use optional or non-existent inputField "${field}".`,
+                  actionDef.operation.bulk,
+                  '/BulkObjectSchema',
+                  `instance.${actionType}.${actionDef.key}.operation.bulk.groupedBy[${index}]`,
+                  'invalid',
+                  'groupedBy'
+                )
+              );
+            }
+          });
+        }
+      }
+
+      if (actionDef.operation && actionDef.operation.performBulk) {
+        if (!actionDef.operation.bulk) {
+          errors.push(
+            new jsonschema.ValidationError(
+              'must contain property "bulk" because property "performBulk" is present.',
+              actionDef.operation,
+              '/BasicCreateActionOperationSchema',
+              `instance.${actionType}.${actionDef.key}.operation`,
+              'missing',
+              'bulk'
+            )
+          );
+        }
+
+        if (actionDef.operation.perform) {
+          errors.push(
+            new jsonschema.ValidationError(
+              'must not contain property "perform" because it is mutually exclusive with property "performBulk".',
+              actionDef.operation,
+              '/BasicCreateActionOperationSchema',
+              `instance.${actionType}.${actionDef.key}.operation`,
+              'invalid',
+              'perform'
+            )
+          );
+        }
+      }
+    });
+  }
+
+  return errors;
+};
+
+module.exports = bulkWriteConstraints;

packages/schema/lib/functional-constraints/index.js

@@ -18,6 +18,8 @@ const checks = [
   require('./matchingKeys'),
   require('./labelWhenVisible'),
   require('./uniqueInputFieldKeys'),
+  require('./bulkWriteConstraints'),
+  require('./requirePerformConditionally'),
 ];
 
 const runFunctionalConstraints = (definition, mainSchema) => {

packages/schema/lib/functional-constraints/requirePerformConditionally.js

@@ -0,0 +1,30 @@
+'use strict';
+
+const _ = require('lodash');
+const jsonschema = require('jsonschema');
+
+const requirePerformConditionally = (definition) => {
+  const errors = [];
+  const actionType = 'creates';
+
+  if (definition[actionType]) {
+    _.each(definition[actionType], (actionDef) => {
+      if (actionDef.operation && !actionDef.operation.bulk && !actionDef.operation.performBulk && !actionDef.operation.perform) {
+        errors.push(
+          new jsonschema.ValidationError(
+            'requires property "perform".',
+            actionDef.operation,
+            '/BasicCreateActionOperationSchema',
+            `instance.${actionType}.${actionDef.key}.operation`,
+            'required',
+            'perform'
+          )
+        );
+      }
+    });
+  }
+
+  return errors;
+};
+
+module.exports = requirePerformConditionally;

packages/schema/lib/schemas/BasicCreateActionOperationSchema.js

@@ -3,6 +3,9 @@
 const makeSchema = require('../utils/makeSchema');
 
 const BasicActionOperationSchema = require('./BasicActionOperationSchema');
+const BulkObjectSchema = require('./BulkObjectSchema');
+const FunctionSchema = require('./FunctionSchema');
+const RequestSchema = require('./RequestSchema');
 
 // TODO: would be nice to deep merge these instead
 // or maybe use allOf which is built into json-schema
@@ -20,7 +23,45 @@ BasicCreateActionOperationSchema.properties.shouldLock = {
   type: 'boolean',
 };
 
+BasicCreateActionOperationSchema.properties.perform = {
+  description:
+    "How will Zapier get the data? This can be a function like `(z) => [{id: 123}]` or a request like `{url: 'http...'}`. Exactly one of `perform` or `performBulk` must be defined. If you choose to define `bulk` and `performBulk`, you must omit `perform`.",
+  oneOf: [{ $ref: RequestSchema.id }, { $ref: FunctionSchema.id }],
+  docAnnotation: {
+    required: {
+      type: 'replace', // replace or append
+      value: 'no (with exceptions, see description)',
+    },
+  },
+};
+
+BasicCreateActionOperationSchema.properties.bulk = {
+  description:
+    'Zapier uses this configuration for writing in bulk with `performBulk`.',
+  $ref: BulkObjectSchema.id,
+  docAnnotation: {
+    required: {
+      type: 'replace', // replace or append
+      value: 'no (with exceptions, see description)',
+    },
+  },
+};
+
+BasicCreateActionOperationSchema.properties.performBulk = {
+  description:
+    'A function to write in bulk with. `bulk` and `performBulk` must either both be defined or neither. Additionally, only one of `perform` or `performBulk` can be defined. If you choose to define `perform`, you must omit `bulk` and `performBulk`.',
+  $ref: FunctionSchema.id,
+  docAnnotation: {
+    required: {
+      type: 'replace', // replace or append
+      value: 'no (with exceptions, see description)',
+    },
+  },
+};
+
+delete BasicCreateActionOperationSchema.required;
+
 module.exports = makeSchema(
   BasicCreateActionOperationSchema,
-  BasicActionOperationSchema.dependencies
+  BasicActionOperationSchema.dependencies.concat(BulkObjectSchema)
 );

packages/schema/lib/schemas/BulkObjectSchema.js

@@ -0,0 +1,48 @@
+'use strict';
+
+const makeSchema = require('../utils/makeSchema');
+
+module.exports = makeSchema({
+  id: '/BulkObjectSchema',
+  description:
+    'Zapier uses this configuration for writing in bulk.',
+  type: 'object',
+  required: ['groupedBy', 'limit'],
+  properties: {
+    groupedBy: {
+      description:
+        'The list of keys of input fields to group bulk-write with. The actual user data provided for the fields will be used during execution. Note that a required input field should be referenced to get user data always.',
+      type: 'array',
+      minItems: 1,
+    },
+    limit: {
+      description:
+        'The maximum number of items to call performBulk with. **Note** that it is capped by the platform to prevent exceeding the [AWS Lambda\'s request/response payload size quota of 6 MB](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#function-configuration-deployment-and-execution). Also, the execution is time-bound; we recommend reducing it upon consistent timeout.',
+      type: 'integer',
+    },
+  },
+  examples: [
+    {
+      groupedBy: ['workspace', 'sheet'],
+      limit: 100,
+    },
+  ],
+  antiExamples: [
+    {
+      example: {
+        groupedBy: [],
+        limit: 100,
+      },
+      reason: 'Empty groupedBy list provided: `[]`.',
+    },
+    {
+      example: {groupedBy: ['workspace']},
+      reason: 'Missing required key: `limit`.',
+    },
+    {
+      example: {limit: 1},
+      reason: 'Missing required key: `groupedBy`.',
+    },
+  ],
+  additionalProperties: false,
+});