Improve diff outputs #870
Comments
AllanOricil
changed the title
|
Hi @AllanOricil ! Thanks for raising this issue and thanks for contributing in making this project better! I'm not familiar much with the CDK output. I don't know if its output is a standard or if it is just beautiful. I think it could be driven by a parameter, pretty much like the I also wonder if this could be the output of a |
|
In cdk, there is a special command for that output with both resource diffs and role/profile diffs:
Both commands don't need output config. That output is the "human" one. |
|
In my opinion it makes sense to create a lib that does it, based on the output of sgd. Once this lib is ready, then a pre-deploy hook and a command would need to be created to consume it. |
|
Thinking out loud. This could be a CLI plugin that looks at a package and destructive changes XML files and essentially pretty-prints them? In the past I've simply printed their contents as part of the CI job and called it good, if there are lots of lines I might have a threshold for summarization. (X CustomObject, Y PermissionSet) |
Yes. It has to show a tree like view of what has changed.
I think that showing a summary of what is and what will be can give more confidence when approving deployments, than simply displaying the contents of the manifests. What do you think? |
|
The more I think of it, the more I think it deserves its own plugin. From what I see in the "SDK" this plugin should consider the usage of the MetadataResolver from SDR. The whole work would be to create this CDK transformer IMHO (if the outputter already exists) |
|
I was not thinking about using constructs. But maybe using constructs would be the most accurate way of implementing this feature, since aws cdk works on top of this standard. However it would be necessary to create constructs for all metadata types, and then use cdk ou terraform to synthesize manifests, and perform deployments using the tolling API. As an example, Mongodb and Github, have constructs to create resources for their products. If someone does it for SF, sf cli could even be deprecated in my opinion. It would also be something good to do since SF products can now be purchased from the AWS marketplace. I do believe it would be cool if one of you that work for sf to create a demo showing how sf deploymemts could be done with IaC, representing metadata as constructs, using cdk, terraform or even polumi. These guys who signed this partnership with AWS would probably buy the idea of having a single tool. Specially because the constructs modules and tools have way more people working on it. I just realized that deployment metadata with constructs would also easily allow the creation of resources across different clouds that seamlessly work with salesforce. For example, if one needs to interact with AI models that are not part of Salesforce offerings, or even just a lambda function to overcome Salesforce limits, Salesforce constructs could be used to do the provisioning and integration between services. These constructs would result in resources that are ready to interact with salesforce services with a proprietary connector. Salesforce would profit on it. |
|
A simple demo would be creating one stack with constructs that create the following resources:
This would demonstrate how companies can leverage sf to do what they do with vercel, for example. This demo can go one step further if this stack includes resources from AWS and Heroku. For example, deploy a simple service in heroku, call a lambda function or an AI model from AWS. That would be freaking cool. And I think it could save money for companies because the same person that is expert in IaaC could now easily talk with a SF developer. Of course sf resources would still be serialized as metadata, but now teams working in different stacks would be able to talk the same language using Constructs. |
|
Anybody interested in writting a poc? I don't have time to do it alone. I would just like to find someone important which we can demo it too. I don't want to go through some asholes who still ideas and give no credit. |
IMHO the right person to contact for this kind of idea is Philippe Ozil (@pozil). |
|
This is how mongodb atlas did: They created constructs for their resources https://github.com/mongodb/awscdk-resources-mongodbatlas And enabled cloudformation to deploy them https://github.com/mongodb/mongodbatlas-cloudformation-resources I'm just not sure if they are deploying resources in aws. I think it works for them because their resources are deployed in aws infrastructure. For Salesforce it wouldn't work, I think 😕 it would be necessary to check with aws if this is possible. For me it makes sense because it could facilitate the integration between aws resources and salesforce stuff. It would have worked for that product of Salesforce lambda functions. Instead of deploying to salesforce, they could've been deployed to aws. But that product failed. |
|
After reading this https://docs.aws.amazon.com/prescriptive-guidance/latest/best-practices-cdk-typescript-iac/constructs-best-practices.html I understood we could create a "custom resource" to instruct cloudformation to deploy our custom construct to salesforce instead of aws. So it could work, I think. |
|
Chatgpt agrees it is possible haha Sure, you can create a custom resource in AWS CloudFormation that uses a Lambda function to interact with the Salesforce API to create or manage objects. Here's a detailed guide on how to achieve this: Step-by-Step Implementation
1. Define the Custom Resource in CDKFirst, we will create a custom resource construct in CDK that triggers a Lambda function. This Lambda function will handle interactions with Salesforce.
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as cr from '@aws-cdk/custom-resources';
interface SalesforceCustomResourceProps {
clientId: string;
clientSecret: string;
username: string;
password: string;
securityToken: string;
salesforceObject: {
type: string;
fields: Record<string, any>;
uniqueField: string;
uniqueFieldValue: any;
};
}
export class SalesforceCustomResource extends cdk.Construct {
constructor(scope: cdk.Construct, id: string, props: SalesforceCustomResourceProps) {
super(scope, id);
const onEvent = new lambda.Function(this, 'OnEventHandler', {
runtime: lambda.Runtime.NODEJS_14_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
environment: {
CLIENT_ID: props.clientId,
CLIENT_SECRET: props.clientSecret,
USERNAME: props.username,
PASSWORD: props.password,
SECURITY_TOKEN: props.securityToken,
SALESFORCE_OBJECT: JSON.stringify(props.salesforceObject),
},
});
const provider = new cr.Provider(this, 'SalesforceProvider', {
onEventHandler: onEvent,
});
new cdk.CustomResource(this, 'SalesforceResource', {
serviceToken: provider.serviceToken,
});
}
}2. Implement the Lambda FunctionThe Lambda function will be responsible for authenticating with Salesforce and creating or updating the specified object.
const axios = require('axios');
exports.handler = async function(event, context) {
const {
CLIENT_ID,
CLIENT_SECRET,
USERNAME,
PASSWORD,
SECURITY_TOKEN,
SALESFORCE_OBJECT
} = process.env;
const salesforceObject = JSON.parse(SALESFORCE_OBJECT);
const baseUrl = 'https://login.salesforce.com';
try {
// Authenticate with Salesforce
const authResponse = await axios.post(`${baseUrl}/services/oauth2/token`, null, {
params: {
grant_type: 'password',
client_id: CLIENT_ID,
client_secret: CLIENT_SECRET,
username: USERNAME,
password: `${PASSWORD}${SECURITY_TOKEN}`,
},
});
const { access_token, instance_url } = authResponse.data;
// Check the event type (Create/Update/Delete)
if (event.RequestType === 'Create' || event.RequestType === 'Update') {
// Fetch the current state of the object from Salesforce
const existingObjectResponse = await axios.get(`${instance_url}/services/data/v52.0/query`, {
params: {
q: `SELECT Id, ${Object.keys(salesforceObject.fields).join(', ')} FROM ${salesforceObject.type} WHERE ${salesforceObject.uniqueField} = '${salesforceObject.uniqueFieldValue}'`,
},
headers: {
Authorization: `Bearer ${access_token}`,
},
});
const existingObjects = existingObjectResponse.data.records;
if (existingObjects.length > 0) {
// Update the existing object if differences are found
const existingObject = existingObjects[0];
const differences = findDifferences(existingObject, salesforceObject.fields);
if (Object.keys(differences).length > 0) {
await axios.patch(`${instance_url}/services/data/v52.0/sobjects/${salesforceObject.type}/${existingObject.Id}`, differences, {
headers: {
Authorization: `Bearer ${access_token}`,
'Content-Type': 'application/json',
},
});
return { PhysicalResourceId: existingObject.Id };
} else {
console.log('No differences found, no update necessary.');
return { PhysicalResourceId: existingObject.Id };
}
} else {
// Create a new object if it doesn't exist
const createResponse = await axios.post(`${instance_url}/services/data/v52.0/sobjects/${salesforceObject.type}/`, salesforceObject.fields, {
headers: {
Authorization: `Bearer ${access_token}`,
'Content-Type': 'application/json',
},
});
return { PhysicalResourceId: createResponse.data.id };
}
} else if (event.RequestType === 'Delete') {
// Handle delete operation if necessary
// (Not implemented here but can be added as needed)
}
return { PhysicalResourceId: event.PhysicalResourceId };
} catch (error) {
console.error('Error:', error);
throw error;
}
};
function findDifferences(existingObject, desiredFields) {
const differences = {};
for (const [key, value] of Object.entries(desiredFields)) {
if (existingObject[key] !== value) {
differences[key] = value;
}
}
return differences;
}3. Create the CDK StackNow, let's create the CDK stack that uses our custom resource.
#!/usr/bin/env node
import * as cdk from '@aws-cdk/core';
import { SalesforceStack } from '../lib/salesforce-stack';
const app = new cdk.App();
new SalesforceStack(app, 'SalesforceStack', {
env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
});
import * as cdk from '@aws-cdk/core';
import { SalesforceCustomResource } from './salesforce-custom-resource';
export class SalesforceStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
new SalesforceCustomResource(this, 'SalesforceCustomResource', {
clientId: 'YOUR_CLIENT_ID',
clientSecret: 'YOUR_CLIENT_SECRET',
username: 'YOUR_USERNAME',
password: 'YOUR_PASSWORD',
securityToken: 'YOUR_SECURITY_TOKEN',
salesforceObject: {
type: 'Account',
fields: {
Name: 'New Account Name',
// Add other fields as needed
},
uniqueField: 'Name',
uniqueFieldValue: 'New Account Name',
},
});
}
}Synthesize and DeployRun the following commands to synthesize and deploy your stack: cdk synth
cdk deployExplanation
When you deploy the stack, the custom resource will trigger the Lambda function, which will authenticate with Salesforce and create or update the specified Salesforce object based on the defined fields. This approach leverages AWS CDK's ability to integrate with external systems using custom resources and Lambda functions. |
|
I hope chatpgt 4o isn't hallucinating Yes, you can create a custom resource with a specific namespace like Steps to Create a Custom Resource with a Custom Namespace
1. Create the Lambda Function to Handle the Custom ResourceThis Lambda function will handle the creation, update, and deletion of Salesforce metadata.
const fs = require('fs');
const path = require('path');
const jsforce = require('jsforce');
const archiver = require('archiver');
exports.handler = async function(event, context) {
const {
CLIENT_ID,
CLIENT_SECRET,
USERNAME,
PASSWORD,
SECURITY_TOKEN
} = process.env;
try {
// Authenticate with Salesforce
const conn = new jsforce.Connection({
oauth2: {
clientId: CLIENT_ID,
clientSecret: CLIENT_SECRET,
loginUrl: 'https://login.salesforce.com'
}
});
await conn.login(USERNAME, PASSWORD + SECURITY_TOKEN);
if (event.RequestType === 'Create' || event.RequestType === 'Update') {
const metadataDir = path.join(__dirname, 'metadata');
const zipFilePath = '/tmp/package.zip';
// Zip the metadata directory
await zipDirectory(metadataDir, zipFilePath);
// Read the zip file into a buffer
const zipBuffer = fs.readFileSync(zipFilePath);
// Deploy the metadata
const deployResult = await conn.metadata.deploy(zipBuffer, { singlePackage: true }).complete();
if (deployResult.success) {
console.log('Deployment succeeded.');
return { PhysicalResourceId: deployResult.id };
} else {
console.error('Deployment failed:', deployResult.details);
throw new Error('Deployment failed.');
}
} else if (event.RequestType === 'Delete') {
// Handle delete operation if necessary
// (Not implemented here but can be added as needed)
}
return { PhysicalResourceId: event.PhysicalResourceId };
} catch (error) {
console.error('Error:', error);
throw error;
}
};
async function zipDirectory(sourceDir, outPath) {
const archive = archiver('zip', { zlib: { level: 9 } });
const stream = fs.createWriteStream(outPath);
return new Promise((resolve, reject) => {
archive
.directory(sourceDir, false)
.on('error', err => reject(err))
.pipe(stream);
stream.on('close', () => resolve());
archive.finalize();
});
}2. Create a Custom Resource ProviderThe custom resource provider is defined using the
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as cr from '@aws-cdk/custom-resources';
import { SalesforceMetadataConstruct } from './salesforce-metadata-construct';
export class SalesforceStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Define Salesforce metadata
const metadataConstruct = new SalesforceMetadataConstruct(this, 'SalesforceMetadata', {
metadata: [
{
type: 'CustomObject',
members: ['MyCustomObject__c']
},
// Add more metadata types and members as needed
],
});
// Create the Lambda function
const onEvent = new lambda.Function(this, 'OnEventHandler', {
runtime: lambda.Runtime.NODEJS_14_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
environment: {
CLIENT_ID: 'YOUR_CLIENT_ID',
CLIENT_SECRET: 'YOUR_CLIENT_SECRET',
USERNAME: 'YOUR_USERNAME',
PASSWORD: 'YOUR_PASSWORD',
SECURITY_TOKEN: 'YOUR_SECURITY_TOKEN',
},
});
const provider = new cr.Provider(this, 'SalesforceProvider', {
onEventHandler: onEvent,
});
// Define the custom resource with a specific namespace
new cdk.CustomResource(this, 'SalesforceResource', {
serviceToken: provider.serviceToken,
resourceType: 'Custom::Salesforce::Metadata',
properties: {
// Add any properties required for your custom resource
},
});
}
}3. Define the Custom Resource with a Custom NamespaceIn the Full Example of CDK CodeHere's the complete example to create a custom resource with the namespace
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as cr from '@aws-cdk/custom-resources';
import { SalesforceMetadataConstruct } from './salesforce-metadata-construct';
export class SalesforceStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Define Salesforce metadata
const metadataConstruct = new SalesforceMetadataConstruct(this, 'SalesforceMetadata', {
metadata: [
{
type: 'CustomObject',
members: ['MyCustomObject__c']
},
// Add more metadata types and members as needed
],
});
// Create the Lambda function
const onEvent = new lambda.Function(this, 'OnEventHandler', {
runtime: lambda.Runtime.NODEJS_14_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda'),
environment: {
CLIENT_ID: 'YOUR_CLIENT_ID',
CLIENT_SECRET: 'YOUR_CLIENT_SECRET',
USERNAME: 'YOUR_USERNAME',
PASSWORD: 'YOUR_PASSWORD',
SECURITY_TOKEN: 'YOUR_SECURITY_TOKEN',
},
});
const provider = new cr.Provider(this, 'SalesforceProvider', {
onEventHandler: onEvent,
});
// Define the custom resource with a specific namespace
new cdk.CustomResource(this, 'SalesforceResource', {
serviceToken: provider.serviceToken,
resourceType: 'Custom::Salesforce::Metadata',
properties: {
// Add any properties required for your custom resource
},
});
}
}Synthesize and DeployRun the following commands to synthesize and deploy your stack: cdk synth
cdk deployExplanation
When you deploy this stack, the CloudFormation template will include the custom resource with the specified namespace, and the Lambda function will handle deploying the Salesforce metadata. |
|
It seems to be a lot of work... |
|
It would be a huge amount of work to make it work really well because it would be necessary to create L1 constructs for every single metadata type, including constraints. The latter is not really necessary because the metadata api would fail if wrong metadata is provided. However, implementing it would improve DX because it would allow devs to know what is wrong during Synthesis. Meaning they wouldn't wait for the api to return the error. I think that the minimum stuff can be done for a poc, just to enable a super simple deployment, like an object with a field. I will try chat gpt suggestion this week and share the repo with you, and whoever else is interested |
|
@scolladon I started it here https://github.com/AllanOricil/cdk-salesforce-iac-poc but I'm already facing some problems |
|
I close this enhancement as it is dealt somewhere else 👍 |
Is your proposal related to a problem?
CDK has a really nice diff output. It can be used to let people know exactly what is going to be deployed to an AWS account. It is very useful in pipelines that have approval processes.
Besides showing resources changes, CDK also summarizes changes to IAM Roles and User permissions in a table.
I believe that both features would ease pipeline approval processes reviews.
Describe a solution you propose
Describe alternatives you've considered
Currently people can use git diffs or just print the generated manifests. However, both are not that easy to analyze. Plus, because metadata is serialized as xml, git diff algorithms don't work well with it and it is really hard to read them.
Additional context
N/A
The text was updated successfully, but these errors were encountered: