Skip to content

Commit

Permalink
feat: add gcp support
Browse files Browse the repository at this point in the history
  • Loading branch information
almeidabbm committed Nov 28, 2023
1 parent f69262d commit 0a1e19a
Show file tree
Hide file tree
Showing 10 changed files with 165 additions and 80 deletions.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# <p align="center"><img src="./static/images/slauth-logo.png" alt="slauth.io logo"/></p>

CLI that scans repositories and generates the necessary IAM Policies for the service to run.
CLI that scans repositories and generates the necessary IAM Permissions for the service to run.

If you need any help getting started or have any questions, please join our [Slack Community](https://join.slack.com/t/slauthiocommunity/shared_invite/zt-268nxuwyd-Vav8lYJdiP44Kt8lQSSybg)

Expand All @@ -19,19 +19,19 @@ npm install -g @slauth.io/slauth

#### Scan command

The scan command will look for any `aws-sdk` calls in your git repository and generate the necessary policies for it.
The scan command will look for any calls of your Cloud Provider `sdk` in your git repository and generate the necessary permissions for it.

```bash
slauth scan -p aws ../path/to/my/repository
```

> Note: By default the `scan` command will print the generated policies to `stdout`. Use `-o,--output-file` option to specify a file to output to.
> Note: By default the `scan` command will print the result to `stdout`. Use `-o,--output-file` option to specify a file to output to.
**Result:**

The result of the scan command is an array of AWS IAM Policy Documents.
If the resource is not explicit in the code (e.g. comes from a variable), we use a placholder for it.
Before deploying the policies, you will have to **manually** change these placeholders with the correct resources the service will try to interact with.
The result of the scan command is an array of IAM Permissions.

> Note: For `aws` cloud provider, if the resource is not explicit in the code (e.g. comes from a variable), we use a placholder for it. Before deploying the policies, you will have to **manually** change these placeholders with the correct resources the service will try to interact with.
```bash
Detected Policies:
Expand Down Expand Up @@ -93,7 +93,7 @@ Detected Policies:

##### Available options

- `-p, --cloud-provider <cloudProvider>` select the cloud provider you would like to generate policies for (choices: "aws")
- `-p, --cloud-provider <cloudProvider>` select the cloud provider you would like to generate policies for (choices: "aws", "gcp")
- `-m, --openai-model <openaiModel>` select the openai model to use (choices: "gpt-3.5-turbo-16k", "gpt-4-32k")
- `-o, --output-file <outputFile>` write generated policies to a file instead of stdout

Expand All @@ -118,7 +118,7 @@ Slauth being a CLI, it can be easily integrated in your CI/CD pipelines.

#### Github Action Example

In this GitHub action workflow we install Slauth, run it and then output the resulting policies to an artifact which can then be downloaded so the policies can be used in your IaC.
In this GitHub action workflow we install Slauth, run it and then output the result to an artifact which can then be downloaded so it can be used in your IaC.

```yaml
name: scan
Expand Down
34 changes: 17 additions & 17 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"typescript": "^5.3.2"
},
"dependencies": {
"@slauth.io/langchain-wrapper": "^1.0.5",
"@slauth.io/langchain-wrapper": "^1.3.0",
"cli-spinners": "^2.9.1",
"commander": "^11.1.0",
"dotenv": "^16.3.1"
Expand Down
76 changes: 23 additions & 53 deletions src/commands/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,9 @@ import showAsyncSpinner from '../utils/show-async-spinner';
import { yellow, red, green } from '../utils/colors';
import writeToFile from '../utils/write-to-file';
import isGitRepository from '../utils/is-git-repository';
import {
getStatementsFromCode,
getPoliciesFromStatements,
CloudProviders,
OpenAIModels,
} from '@slauth.io/langchain-wrapper';
import { CloudProviders, OpenAIModels } from '@slauth.io/langchain-wrapper';
import ScannerStrategies from '../utils/scanner-strategies';
import Scanner from '../utils/scanner';

const scanCommand = new Command();
scanCommand
Expand Down Expand Up @@ -39,20 +36,20 @@ scanCommand
.action(async (pathArg, { cloudProvider, openaiModel, outputFile }) => {
try {
const fullPath = path.resolve(process.cwd(), pathArg);
const policies = await scan(fullPath, cloudProvider, openaiModel);
const result = await scan(fullPath, cloudProvider, openaiModel);

if (policies) {
const policiesJsonString = JSON.stringify(policies, null, 2);
if (result) {
const resultJSONString = JSON.stringify(result, null, 2);

if (outputFile) {
const fullOutFilePath = path.resolve(process.cwd(), outputFile);
await writeToFile(fullOutFilePath, policiesJsonString);
await writeToFile(fullOutFilePath, resultJSONString);
console.log(`${green('Wrote to file:')} ${fullOutFilePath}`);
return;
}

console.log(green('Detected Policies:\n'));
console.log(policiesJsonString);
console.log(green(`Detected ${getResultType(cloudProvider)}:\n`));
console.log(resultJSONString);
} else {
console.log(yellow('No policies have been detected'));
}
Expand All @@ -61,6 +58,17 @@ scanCommand
}
});

function getResultType(cloudProvider: keyof typeof CloudProviders) {
switch (cloudProvider) {
case 'aws':
return 'Policies';
case 'gcp':
return 'Permissions';
default:
return 'Result';
}
}

async function scan(
fullPath: string,
cloudProvider: keyof typeof CloudProviders,
Expand All @@ -80,47 +88,9 @@ async function scan(
readDirectoryPromise
);

const fileDocs = await readDirectoryPromise;

const statementsPromises = Promise.all(
fileDocs.map(async doc => {
return await getStatementsFromCode(
doc.pageContent,
cloudProvider,
modelName
);
})
);

await showAsyncSpinner(
{
spinner: spinners.dots,
text: yellow(
'Scanning for aws-sdk calls (this process might take a few minutes)'
),
},
statementsPromises
);

const statements = (await statementsPromises).flat();

const policiesPromise = getPoliciesFromStatements(
statements,
cloudProvider,
modelName
);

await showAsyncSpinner(
{
spinner: spinners.dots,
text: yellow(
'Generating policies (this process might take a few minutes)'
),
},
policiesPromise
);

return await policiesPromise;
const scanner = new Scanner(ScannerStrategies[cloudProvider]);
const codeSnippets = (await readDirectoryPromise).map(doc => doc.pageContent);
return await scanner.scan(codeSnippets, modelName);
}

export default scanCommand;
8 changes: 8 additions & 0 deletions src/types/scanner-strategy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { OpenAIModels } from '@slauth.io/langchain-wrapper';

export default interface ScannerStrategy {
scan(
codeSnippets: string[],
modelName?: keyof typeof OpenAIModels
): Promise<unknown[] | undefined>;
}
4 changes: 3 additions & 1 deletion src/utils/read-directory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ async function listFiles(directoryOrFilePath: string): Promise<string[]> {
});
}

export default async function readDirectory(dirPath: string) {
export default async function readDirectory(
dirPath: string
): Promise<{ pageContent: string }[]> {
const files = await listFiles(dirPath);
const textSplitter =
new LangChain.TextSplitters.RecursiveCharacterTextSplitter({
Expand Down
44 changes: 44 additions & 0 deletions src/utils/scanner-strategies/aws.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import { OpenAIModels, Services } from '@slauth.io/langchain-wrapper';
import ScannerStrategy from '../../types/scanner-strategy';
import showAsyncSpinner from '../show-async-spinner';
import spinners from 'cli-spinners';
import { yellow } from '../colors';

export default class AWSScanner implements ScannerStrategy {
async scan(codeSnippets: string[], modelName: keyof typeof OpenAIModels) {
const statementsPromises = Promise.all(
codeSnippets.map(async snippet => {
return await Services.aws.getStatementsFromCode(snippet, modelName);
})
);

await showAsyncSpinner(
{
spinner: spinners.dots,
text: yellow(
'Scanning for aws-sdk calls (this process might take a few minutes)'
),
},
statementsPromises
);

const statements = (await statementsPromises).flat();

const policiesPromise = Services.aws.getPoliciesFromStatements(
statements,
modelName
);

await showAsyncSpinner(
{
spinner: spinners.dots,
text: yellow(
'Generating policies (this process might take a few minutes)'
),
},
policiesPromise
);

return await policiesPromise;
}
}
29 changes: 29 additions & 0 deletions src/utils/scanner-strategies/gcp.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import { OpenAIModels, Services } from '@slauth.io/langchain-wrapper';
import ScannerStrategy from '../../types/scanner-strategy';
import showAsyncSpinner from '../show-async-spinner';
import spinners from 'cli-spinners';
import { yellow } from '../colors';

export default class GCPScanner implements ScannerStrategy {
async scan(codeSnippets: string[], modelName: keyof typeof OpenAIModels) {
const permissionsPromise = Promise.all(
codeSnippets.map(async snippet => {
return await Services.gcp.getPermissionsFromCode(snippet, modelName);
})
);

await showAsyncSpinner(
{
spinner: spinners.dots,
text: yellow(
'Scanning for google-cloud sdk calls (this process might take a few minutes)'
),
},
permissionsPromise
);

const permissions = (await permissionsPromise).flat();

return permissions.length ? permissions : undefined;
}
}
15 changes: 15 additions & 0 deletions src/utils/scanner-strategies/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import GCPScanner from './gcp';
import AWSScanner from './aws';
import { CloudProviders } from '@slauth.io/langchain-wrapper';
import ScannerStrategy from '../../types/scanner-strategy';

type ScannerStrategiesType = {
[k in CloudProviders]: ScannerStrategy;
};

const ScannerStrategies: ScannerStrategiesType = {
gcp: new GCPScanner(),
aws: new AWSScanner(),
};

export default ScannerStrategies;
17 changes: 17 additions & 0 deletions src/utils/scanner.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { OpenAIModels } from '@slauth.io/langchain-wrapper';
import ScannerStrategy from '../types/scanner-strategy';

export default class Scanner {
private strategy: ScannerStrategy;

constructor(strategy: ScannerStrategy) {
this.strategy = strategy;
}

public async scan(
codeSnippets: string[],
modelName?: keyof typeof OpenAIModels
) {
return await this.strategy.scan(codeSnippets, modelName);
}
}

0 comments on commit 0a1e19a

Please sign in to comment.