Skip to content

Commit

Permalink
Merge pull request #75 from tracehubpm/stabilization
Browse files Browse the repository at this point in the history
feat(#73): huge code duplication removed, objects refactored, packages
  • Loading branch information
h1alexbel committed Apr 9, 2024
2 parents 6178928 + d113943 commit 5735467
Show file tree
Hide file tree
Showing 44 changed files with 315 additions and 440 deletions.
2 changes: 1 addition & 1 deletion .pdd
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
--source=.
--verbose
--exclude src/pdd-prompt.ts
--exclude src/prompts/pdd-prompt.ts
--exclude target/**
--exclude xargs/**
--rule min-words:20
Expand Down
28 changes: 22 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,10 +134,26 @@ format using [JSON Packing Method](#json-packing-method):
#### Cap Top 3

Some analysis results contains many problems.
Consider this example:

```json
{
"size": 6,
"problems": [
"1. Lack of a clear description: The report lacks a clear and concise description of the problem. It simply states there are typos but does not specify what the typos are or how they impact the system.",
"2. Missing steps to reproduce: There are no steps provided to reproduce the issue. This makes it difficult for developers to identify if they have fixed the issue correctly.",
"3. No severity level: The severity level of the issue is not stated. This is important information for developers to prioritize how soon the issue should be resolved.",
"4. Lack of environment details: The report does not mention which environment this issue occurs in (e.g., which version of the software, which operating system).",
"5. Use of shorthand: The term 'take a look here' is used, which is not clear or professional. It is best to avoid using shorthand or colloquial language in formal documentation.",
"6. Incomplete code block: The code block is not complete (it is cut off after the relevant lines). This makes it difficult for developers to understand the context of the issue."
]
}
```

In order to make programmers not ignore the feedback reports by this action,
we **minimize** amount of problems to just 3 or less.
LLM at this stage picks the most important problems from previous analysis
and adds them into new response:
and adds them into new response:

```json
{
Expand Down Expand Up @@ -223,16 +239,16 @@ them into JSON object:
}
```

In the [UML](https://en.wikipedia.org/wiki/Unified_Modeling_Language) notation, the full process looks like this:

![method.svg](/doc/method.svg)

#### JSON Packing Method

LLMs often produce suboptimal results when directly prompted to output in JSON format.
That's why we let LLM "think" in English and ask to summarize JSON only at the final step of the operation.
At this stage we pack previous LLM response to JSON object format.

In the [UML](https://en.wikipedia.org/wiki/Unified_Modeling_Language) notation, the process internals look like this:

![method.svg](/doc/method.svg)

### Puzzle (PDD) Analysis

This action supports analysis not only for issues created manually, but also for puzzles, a.k.a `todo` in your code.
Expand All @@ -247,7 +263,7 @@ Issue is treated as puzzle if it satisfies the following regex:
The puzzle `(.+)` from #(\d+) has to be resolved:.+
```

Then we are parsing the issue to find a tree path where puzzle is hidden.
Then we parse the issue to find a tree path where puzzle is hidden.

This one
```text
Expand Down
17 changes: 9 additions & 8 deletions src/chat-gpt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,38 +32,39 @@ export class ChatGpt implements Model {
* Ctor.
* @param open Open AI
* @param model Model name
* @param system System prompt
* @param prompt User prompt
* @param temperature Temperature
* @param max Max new tokens
*/
constructor(
private readonly open: OpenAI,
private readonly model: string,
private readonly system: Scalar<string>,
private readonly prompt: Scalar<string>,
private readonly temperature: number,
private readonly max: number
) {
this.open = open;
this.model = model;
}

async analyze() {
async analyze(system: Scalar<string>, user: Scalar<string>) {
const response = await this.open.chat.completions.create({
model: this.model,
temperature: 0.5,
temperature: this.temperature,
max_tokens: this.max,
messages: [
{
role: "system",
content: this.system.value()
content: system.value()
},
{
role: "user",
content: this.prompt.value()
content: user.value()
}
]
});
return response.choices[0].message.content?.trim();
}

name(): string {
return this.model;
}
}
33 changes: 0 additions & 33 deletions src/context-expert.ts

This file was deleted.

38 changes: 0 additions & 38 deletions src/context-prompt.ts

This file was deleted.

14 changes: 7 additions & 7 deletions src/deep-infra.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,22 +31,18 @@ export class DeepInfra implements Model {
* Ctor.
* @param token Token
* @param model Model
* @param system System prompt
* @param prompt User prompt
* @param temperature Temperature
* @param max Max new tokens
*/
constructor(
private readonly token: string,
private readonly model: string,
private readonly system: Scalar<string>,
private readonly prompt: Scalar<string>,
private readonly temperature: number,
private readonly max: number
) {
}

async analyze() {
async analyze(system: Scalar<string>, prompt: Scalar<string>) {
const response = await fetch(
'https://api.deepinfra.com/v1/openai/chat/completions', {
method: 'POST',
Expand All @@ -57,11 +53,11 @@ export class DeepInfra implements Model {
messages: [
{
role: "system",
content: this.system.value()
content: system.value()
},
{
role: "user",
content: this.prompt.value()
content: prompt.value()
}
],
}),
Expand All @@ -76,4 +72,8 @@ export class DeepInfra implements Model {
);
return answer.choices[0].message.content;
}

name(): string {
return this.model;
}
}
2 changes: 1 addition & 1 deletion src/feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
import {Comment} from "./comment";
import {Comment} from "./github/comment";
import {Covered} from "./covered";
import {WithSummary} from "./with-summary";
import * as core from "@actions/core";
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
14 changes: 4 additions & 10 deletions src/example.ts → src/goal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,18 +23,12 @@
*/

/**
* Example of analysis summary.
* Goal.
*/
export class Example {
export interface Goal {

/**
* Example of analysis summary as string.
* Execute the goal.
*/
value(): string {
return `
* <...>
* <...>
* <...>
`;
}
exec(): Promise<any>;
}
58 changes: 58 additions & 0 deletions src/goals/named-goal.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/*
* The MIT License (MIT)
*
* Copyright (c) 2023-2024 Tracehub.git
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included
* in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
import {Goal} from "../goal";

/**
* Simple named goal.
*/
export class NamedGoal implements Goal {

/**
* Ctor.
* @param name Name of the goal
* @param model Model
* @param system System
* @param user User
*/
constructor(
private readonly name: string,
private readonly model: Model,
private readonly system: Scalar<string>,
private readonly user: Scalar<string>
) {
}

async exec(): Promise<any> {
console.log(
`Running ${this.name} goal`
);
const response = await this.model.analyze(this.system, this.user);
console.log(
`${this.name} completed, response from ${this.model.name()}:
${response}
`
);
return response;
}
}
Loading

1 comment on commit 5735467

@0pdd
Copy link
Collaborator

@0pdd 0pdd commented on 5735467 Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Puzzle 69-bb4151df disappeared from src/main.ts), that's why I closed #73. Please, remember that the puzzle was not necessarily removed in this particular commit. Maybe it happened earlier, but we discovered this fact only now.

Please sign in to comment.