Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: max-length config #194

Merged
merged 11 commits into from
Apr 25, 2023
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,15 @@ Default: `10000` (10 seconds)
aicommits config set timeout=20000 # 20s
```

#### max-length
The maximum character length of the generated commit message.

Default: `50`

```sh
aicommits config set max-length=100
```

## How it works

This CLI tool runs `git diff` to grab all your latest code changes, sends them to OpenAI's GPT-3, then returns the AI generated commit message.
Expand Down
6 changes: 3 additions & 3 deletions src/commands/aicommits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,8 @@ export default async (
throw new KnownError('No staged changes found. Stage your changes manually, or automatically stage all changes with the `--all` flag.');
}

detectingFiles.stop(`${getDetectedMessage(staged.files)}:\n${
staged.files.map(file => ` ${file}`).join('\n')
}`);
detectingFiles.stop(`${getDetectedMessage(staged.files)}:\n${staged.files.map(file => ` ${file}`).join('\n')
}`);

const { env } = process;
const config = await getConfig({
Expand All @@ -58,6 +57,7 @@ export default async (
config.locale,
staged.diff,
config.generate,
config['max-length'],
config.timeout,
config.proxy,
);
Expand Down
1 change: 1 addition & 0 deletions src/commands/prepare-commit-msg-hook.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ export default () => (async () => {
config.locale,
staged!.diff,
config.generate,
config['max-length'],
config.timeout,
config.proxy,
);
Expand Down
13 changes: 12 additions & 1 deletion src/utils/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ const configParsers = {

parseAssert('locale', locale, 'Cannot be empty');
parseAssert('locale', /^[a-z-]+$/i.test(locale), 'Must be a valid locale (letters and dashes/underscores). You can consult the list of codes in: https://wikipedia.org/wiki/List_of_ISO_639-1_codes');

return locale;
},
generate(count?: string) {
Expand Down Expand Up @@ -78,6 +77,18 @@ const configParsers = {
const parsed = Number(timeout);
parseAssert('timeout', parsed >= 500, 'Must be greater than 500ms');

return parsed;
},
'max-length'(maxLength?: string) {
if (!maxLength) {
return 50;
}

parseAssert('max-length', /^\d+$/.test(maxLength), 'Must be an integer');

const parsed = Number(maxLength);
parseAssert('max-length', parsed >= 20, 'Must be greater than 20 characters');

return parsed;
},
} as const;
Expand Down
36 changes: 32 additions & 4 deletions src/utils/openai.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import https from 'https';
import type { ClientRequest, IncomingMessage } from 'http';
import type { CreateChatCompletionRequest, CreateChatCompletionResponse } from 'openai';
import { type TiktokenModel } from '@dqbd/tiktoken';
import {
TiktokenModel,
// eslint-disable-next-line camelcase
encoding_for_model,
} from '@dqbd/tiktoken';
import createHttpsProxyAgent from 'https-proxy-agent';
import { KnownError } from './error.js';

Expand Down Expand Up @@ -100,18 +104,42 @@ const sanitizeMessage = (message: string) => message.trim().replace(/[\n\r]/g, '

const deduplicateMessages = (array: string[]) => Array.from(new Set(array));

const getPrompt = (locale: string, diff: string) => `Write a git commit message in present tense for the following diff without prefacing it with anything. Do not be needlessly verbose and make sure the answer is concise and to the point. The response must be in the language ${locale}:\n${diff}`;
const getPrompt = (locale: string, diff: string, length: number) => `Write a git commit message in present tense for the following diff without prefacing it with anything. Do not be needlessly verbose and make sure the answer is concise and to the point. The response must be no longer than ${length} characters. The response must be in the language ${locale}:\n${diff}`;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we pad it by 5 in the prompt limit(${length}) too? RN we are only padding it in char to token conversion. But padding here might result in 50+ characters in a commit message eventually.


const generateStringFromLength = (length: number) => {
let result = '';
const highestTokenChar = 'z';
for (let i = 0; i < length; i += 1) {
result += highestTokenChar;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the highest token character? I think we can just use that instead of generating a random value every time.

return result;
};

const getTokens = (prompt: string, model: TiktokenModel) => {
const encoder = encoding_for_model(model);
const tokens = encoder.encode(prompt).length;
// Free the encoder to avoid possible memory leaks.
encoder.free();
return tokens;
};

export const generateCommitMessage = async (
apiKey: string,
model: TiktokenModel,
locale: string,
diff: string,
completions: number,
length: number,
timeout: number,
proxy?: string,
) => {
const prompt = getPrompt(locale, diff);
const prompt = getPrompt(locale, diff, length);

// Padded by 5 for more room for the completion.
const stringFromLength = generateStringFromLength(length + 5);

// The token limit is shared between the prompt and the completion.
const maxTokens = getTokens(stringFromLength + prompt, model);

try {
const completion = await createChatCompletion(
Expand All @@ -126,7 +154,7 @@ export const generateCommitMessage = async (
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
max_tokens: 200,
max_tokens: maxTokens,
stream: false,
n: completions,
},
Expand Down
34 changes: 34 additions & 0 deletions tests/specs/cli/commits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,35 @@ export default testSuite(({ describe }) => {

const { stdout: commitMessage } = await git('log', ['--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});

test('Generated commit message must be under 20 characters', async () => {
const { fixture, aicommits } = await createFixture({
...files,
'.aicommits': `${files['.aicommits']}\nmax-length=20`,
});

const git = await createGit(fixture.path);

await git('add', ['data.json']);

const committing = aicommits();
committing.stdout!.on('data', (buffer: Buffer) => {
const stdout = buffer.toString();
if (stdout.match('└')) {
committing.stdin!.write('y');
committing.stdin!.end();
}
});

await committing;

const { stdout: commitMessage } = await git('log', ['--pretty=format:%s']);
console.log('20 Committed with:', commitMessage, commitMessage.length);
expect(commitMessage.length <= 20).toBe(true);

await fixture.rm();
});
Expand Down Expand Up @@ -84,6 +113,7 @@ export default testSuite(({ describe }) => {

const { stdout: commitMessage } = await git('log', ['-n1', '--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});
Expand Down Expand Up @@ -123,6 +153,7 @@ export default testSuite(({ describe }) => {

const { stdout: commitMessage } = await git('log', ['--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});
Expand Down Expand Up @@ -157,6 +188,7 @@ export default testSuite(({ describe }) => {
const { stdout: commitMessage } = await git('log', ['--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage).toMatch(japanesePattern);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});
Expand Down Expand Up @@ -217,6 +249,7 @@ export default testSuite(({ describe }) => {

const { stdout: commitMessage } = await git('log', ['--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});
Expand Down Expand Up @@ -248,6 +281,7 @@ export default testSuite(({ describe }) => {

const { stdout: commitMessage } = await git('log', ['--oneline']);
console.log('Committed with:', commitMessage);
expect(commitMessage.length <= 50).toBe(true);

await fixture.rm();
});
Expand Down
45 changes: 44 additions & 1 deletion tests/specs/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,18 @@ export default testSuite(({ describe }) => {
expect(stderr).toMatch('Invalid config property OPENAI_KEY: Must start with "sk-"');
});

await test('set config file', async () => {
await aicommits(['config', 'set', openAiToken]);

const configFile = await fs.readFile(configPath, 'utf8');
expect(configFile).toMatch(openAiToken);
});

await test('get config file', async () => {
const { stdout } = await aicommits(['config', 'get', 'OPENAI_KEY']);
expect(stdout).toBe(openAiToken);
});

await test('reading unknown config', async () => {
await fs.appendFile(configPath, 'UNKNOWN=1');

Expand Down Expand Up @@ -57,6 +69,38 @@ export default testSuite(({ describe }) => {
});
});

await describe('max-length', ({ test }) => {
test('must be an integer', async () => {
const { stderr } = await aicommits(['config', 'set', 'max-length=abc'], {
reject: false,
});

expect(stderr).toMatch('Must be an integer');
});

test('must be at least 20 characters', async () => {
const { stderr } = await aicommits(['config', 'set', 'max-length=10'], {
reject: false,
});

expect(stderr).toMatch(/must be greater than 20 characters/i);
});

test('updates config', async () => {
const defaultConfig = await aicommits(['config', 'get', 'max-length']);
expect(defaultConfig.stdout).toBe('max-length=50');

const maxLength = 'max-length=60';
await aicommits(['config', 'set', maxLength]);

const configFile = await fs.readFile(configPath, 'utf8');
expect(configFile).toMatch(maxLength);

const get = await aicommits(['config', 'get', 'max-length']);
expect(get.stdout).toBe(maxLength);
});
});

rocktimsaikia marked this conversation as resolved.
Show resolved Hide resolved
await test('set config file', async () => {
await aicommits(['config', 'set', openAiToken]);

Expand All @@ -66,7 +110,6 @@ export default testSuite(({ describe }) => {

await test('get config file', async () => {
const { stdout } = await aicommits(['config', 'get', 'OPENAI_KEY']);

expect(stdout).toBe(openAiToken);
});

Expand Down