Add the GitHub integration as per issue#13

README.md

@@ -59,6 +59,13 @@ holmes investigate jira --jira-url https://<PLACEDHOLDER>.atlassian.net --jira-u
 ```
 </details>
 
+<summary>Investigate a GitHub Issue</summary>
+
+```bash
+holmes investigate github --github-url https://<PLACEHOLDER> --github-owner <PLACEHOLDER_OWNER_NAME> --github-repository <PLACEHOLDER_GITHUB_REPOSITORY> --github-pat <PLACEHOLDER_GITHUB_PAT>
+```
+</details>
+
 Like what you see? Checkout [more examples](#more-examples) or get started by [installing HolmesGPT](#installation).
 
 ## Key Features
@@ -210,6 +217,19 @@ Integrate with Jira to automate issue tracking and project management tasks. Pro
 #jira_query: "project = 'Natan Test Project' and Status = 'To Do'"
 ```
 
+## GitHub Integration
+
+Integrate with GitHub to automate issue tracking and project management tasks. Provide your GitHub PAT (*personal access token*) and specify the `owner/repository`.
+
+```bash
+# GitHub credentials and query settings
+#github_owner: "robusta-dev"
+#github_pat: "..."
+#github_url: "https://api.github.com" (default)
+#github_repository: "holmesgpt"
+#github_query: "is:issue is:open"
+```
+
 ## Slack Integration
 
 Configure Slack to send notifications to specific channels. Provide your Slack token and the desired channel for notifications.
@@ -311,6 +331,22 @@ holmes investigate jira --update-ticket
 
 </details>
 
+<summary>Investigate and update GitHub issues with findings</summary>
+
+By default GitHub investigation results are displayed in the CLI itself. But you can use `--update-issue` to get the results as a comment in the GitHub issue.
+
+```bash
+holmes investigate github --github-url https://<PLACEDHOLDER> --github-owner <PLACEHOLDER_GITHUB_OWNER> --github-repository <PLACEHOLDER_GITHUB_REPOSITORY> --github-pat <PLACEHOLDER_GITHUB_PAT> --update-issue
+```
+
+Alternatively you can update the `config.yaml` with your GitHub account details and run: 
+
+```bash
+holmes investigate github --update-issue
+```
+
+</details>
+
 ## Advanced Usage
 
 <details>
@@ -356,7 +392,7 @@ Add these values to the `config.yaml` or pass them via the CLI.
 <details>
 <summary>Jira</summary>
 
-Adding a Jira integration allows the LLM to fetch Jira tickets and investigate automatically. Optionally it can update the Jira ticked with findings too. You need the following to use this
+Adding a Jira integration allows the LLM to fetch Jira tickets and investigate automatically. Optionally it can update the Jira ticket with findings too. You need the following to use this
 
 1. **url**: The URL of your workspace. For example: [https://workspace.atlassian.net](https://workspace.atlassian.net) (**Note:** schema (https) is required)
 2. **username**: The email you use to log into your Jira account. Eg: `jira-user@company.com`
@@ -367,6 +403,19 @@ Adding a Jira integration allows the LLM to fetch Jira tickets and investigate a
 Add these values to the `config.yaml` or pass them via the CLI.
 </details>
 
+<details>
+<summary>GitHub</summary>
+
+Adding a GitHub integration allows the LLM to fetch GitHub issues and investigate automatically. Optionally it can update the GitHub issue with findings too. You need the following to use this
+
+1. **url**: The URL of your GitHub API. For example: [https://api.github.com](https://api.github.com) (**Note:** schema (https) is required)
+2. **owner**: The repository owner. Eg: `robusta-dev`
+3. **pat**: Follow these [instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token) to get your GitHub pat (*personal access token*).
+4. **repository**: Name of the repository you want the GitHub issues to be scanned. Eg: `holmesgpt`.
+
+Add these values to the `config.yaml` or pass them via the CLI.
+</details>
+
 
 ## License
 

holmes.py

@@ -335,6 +335,96 @@ def jira(
                 f"[bold]Not updating ticket {issue.url}. Use the --update-ticket option to do so.[/bold]"
             )
 
+
+@investigate_app.command()
+def github(
+    github_url: str = typer.Option(
+        "https://api.github.com", help="The GitHub api base url (e.g: https://api.github.com)"
+    ),
+    github_owner: Optional[str] = typer.Option(
+        None, help="The GitHub repository Owner, eg: if the repository url is https://github.com/robusta-dev/holmesgpt, the owner is robusta-dev"
+    ),
+    github_pat: str = typer.Option(
+        None,
+    ),
+    github_repository: Optional[str] = typer.Option(
+        None,
+        help="The GitHub repository name, eg: if the repository url is https://github.com/robusta-dev/holmesgpt, the repository name is holmesgpt",
+    ),
+    update_issue: Optional[bool] = typer.Option(
+        False, help="Update issues with AI results"
+    ),
+    github_query: Optional[str] = typer.Option(
+        "is:issue is:open",
+        help="Investigate tickets matching a GitHub query (e.g. 'is:issue is:open')",
+    ),
+    # common options
+    llm: Optional[LLMType] = opt_llm,
+    api_key: Optional[str] = opt_api_key,
+    azure_endpoint: Optional[str] = opt_azure_endpoint,
+    model: Optional[str] = opt_model,
+    config_file: Optional[str] = opt_config_file,
+    custom_toolsets: Optional[List[Path]] = opt_custom_toolsets,
+    allowed_toolsets: Optional[str] = opt_allowed_toolsets,
+    custom_runbooks: Optional[List[Path]] = opt_custom_runbooks,
+    max_steps: Optional[int] = opt_max_steps,
+    verbose: Optional[bool] = opt_verbose,
+    # advanced options for this command
+    system_prompt: Optional[str] = typer.Option(
+        "builtin://generic_investigation.jinja2", help=system_prompt_help
+    ),
+):
+    """
+    Investigate a GitHub issue
+    """
+    console = init_logging(verbose)
+    config = Config.load_from_file(
+        config_file,
+        api_key=api_key,
+        llm=llm,
+        azure_endpoint=azure_endpoint,
+        model=model,
+        max_steps=max_steps,
+        github_url=github_url,
+        github_owner=github_owner,
+        github_pat=github_pat,
+        github_repository=github_repository,
+        github_query=github_query,
+        custom_toolsets=custom_toolsets,
+        custom_runbooks=custom_runbooks
+    )
+
+    system_prompt = load_prompt(system_prompt)
+    ai = config.create_issue_investigator(console, allowed_toolsets)
+    source = config.create_github_source()
+    try:
+        issues = source.fetch_issues()
+    except Exception as e:
+        logging.error(f"Failed to fetch issues from GitHub: {e}")
+        return
+
+    console.print(
+        f"[bold yellow]Analyzing {
+            len(issues)} GitHub Issues.[/bold yellow] [red]Press Ctrl+C to stop.[/red]"
+    )
+    for i, issue in enumerate(issues):
+        console.print(f"[bold yellow]Analyzing GitHub issue {i+1}/{len(issues)}: {issue.name}...[/bold yellow]")
+        result = ai.investigate(issue, system_prompt, console)
+
+        console.print(Rule())
+        console.print(f"[bold green]AI analysis of {issue.url}[/bold green]")
+        console.print(Markdown(result.result.replace(
+            "\n", "\n\n")), style="bold green")
+        console.print(Rule())
+        if update_issue:
+            source.write_back_result(issue.id, result)
+            console.print(f"[bold]Updated ticket {issue.url}.[/bold]")
+        else:
+            console.print(
+                f"[bold]Not updating issue {
+                    issue.url}. Use the --update-issue option to do so.[/bold]"
+            )
+
 @app.command()
 def version() -> None:
     typer.echo(__version__)

holmes/config.py

@@ -19,6 +19,7 @@
 from holmes.plugins.destinations.slack import SlackDestination
 from holmes.plugins.runbooks import load_builtin_runbooks, load_runbooks_from_file
 from holmes.plugins.sources.jira import JiraSource
+from holmes.plugins.sources.github import GitHubSource
 from holmes.plugins.sources.prometheus.plugin import AlertManagerSource
 from holmes.plugins.toolsets import load_builtin_toolsets, load_toolsets_from_file
 from holmes.utils.pydantic_utils import RobustaBaseConfig, load_model_from_file
@@ -50,6 +51,12 @@ class Config(RobustaBaseConfig):
     jira_api_key: Optional[SecretStr] = None
     jira_query: Optional[str] = ""
 
+    github_url: Optional[str] = None
+    github_owner: Optional[str] = None
+    github_pat: Optional[SecretStr] = None
+    github_repository: Optional[str] = None
+    github_query: Optional[str] = ""
+
     slack_token: Optional[SecretStr] = None
     slack_channel: Optional[str] = None
 
@@ -73,6 +80,11 @@ def load_from_env(cls):
             "jira_query",
             "slack_token",
             "slack_channel",
+            "github_url",
+            "github_owner",
+            "github_repository",
+            "github_pat",
+            "github_query",
             # TODO
             # custom_runbooks
             # custom_toolsets
@@ -178,6 +190,27 @@ def create_jira_source(self) -> JiraSource:
             jql_query=self.jira_query,
         )
 
+    def create_github_source(self) -> GitHubSource:
+        if not (
+            self.github_url.startswith(
+                "http://") or self.github_url.startswith("https://")
+        ):
+            raise ValueError("--github-url must start with http:// or https://")
+        if self.github_owner is None:
+            raise ValueError("--github-owner must be specified")
+        if self.github_repository is None:
+            raise ValueError("--github-repository must be specified")
+        if self.github_pat is None:
+            raise ValueError("--github-pat must be specified")
+
+        return GitHubSource(
+            url=self.github_url,
+            owner=self.github_owner,
+            pat=self.github_pat.get_secret_value(),
+            repository=self.github_repository,
+            query=self.github_query,
+        )
+
     def create_alertmanager_source(self) -> AlertManagerSource:
         if self.alertmanager_url is None:
             raise ValueError("--alertmanager-url must be specified")

holmes/plugins/interfaces.py

@@ -4,15 +4,15 @@
 from holmes.core.issue import Issue
 from holmes.core.tool_calling_llm import LLMResult
 
-# Sources must implmenet this
+# Sources must implement this
 class SourcePlugin:
     def fetch_issues(self, issue_id: Pattern = None) -> List[Issue]:
         raise NotImplementedError()
 
     # optional
     def stream_issues(self) -> Iterable[Issue]:
         raise NotImplementedError()
-    
+
     # optional
     def write_back_result(self, issue_id: str, result_data: LLMResult) -> None:
         raise NotImplementedError()

holmes/plugins/sources/github/__init__.py

@@ -0,0 +1,80 @@
+import logging
+from holmes.core.tool_calling_llm import LLMResult
+from holmes.plugins.interfaces import SourcePlugin
+from holmes.core.issue import Issue
+from typing import List, Pattern
+from holmes.core.tool_calling_llm import LLMResult
+import requests
+
+
+class GitHubSource(SourcePlugin):
+    def __init__(self, url: str, owner: str, repository: str, pat: str, query: str):
+        self.url = url
+        self.owner = owner
+        self.repository = repository
+        self.pat = pat
+        self.query = query
+
+    def fetch_issues(self, issue_id: Pattern = None) -> List[Issue]:
+        logging.info(f"Fetching All issues from {self.url} for repository {
+                     self.owner}/{self.repository}")
+        try:
+            data = []
+            url = f"{self.url}/search/issues"
+            headers = {
+                "Authorization": f"token {self.pat}",
+                "Accept": "application/vnd.github.v3+json",
+                "X-GitHub-Api-Version": "2022-11-28"
+            }
+            params = {
+                "per_page": "100"
+            }
+            default_q = f"repo:{self.owner}/{self.repository}"
+            params["q"] = f"{default_q} {self.query}"
+            while url:
+                response = requests.get(
+                    url=url, headers=headers, params=params)
+                if response.status_code != 200:
+                    raise Exception(f"Failed to get issues:{
+                                    response.status_code} {response.text}")
+                logging.info(f"Got {response}")
+                response.raise_for_status()
+                data.extend(response.json().get("items", []))
+                links = response.headers.get("Link", "")
+                url = None
+                for link in links.split(","):
+                    if 'rel="next"' in link:
+                        url = link.split(";")[0].strip()[1:-1]
+            return [self.convert_to_issue(issue) for issue in data]
+        except requests.RequestException as e:
+            raise ConnectionError(f"Failed to fetch data from GitHub.") from e
+
+    def convert_to_issue(self, github_issue):
+        return Issue(
+            id=str(github_issue["number"]),
+            name=github_issue["title"],
+            source_type="github",
+            source_instance_id=f"{self.owner}/{self.repository}",
+            url=github_issue["html_url"],
+            raw=github_issue,
+        )
+
+    def write_back_result(self, issue_id: str, result_data: LLMResult) -> None:
+        url = f"{
+            self.url}/repos/{self.owner}/{self.repository}/issues/{issue_id}/comments"
+        headers = {
+            "Authorization": f"token {self.pat}",
+            "Accept": "application/vnd.github.v3+json",
+            "X-GitHub-Api-Version": "2022-11-28"
+        }
+        response = requests.post(
+            url=url,
+            json={"body": f"Automatic AI Investigation by Robusta:\n\n{
+                result_data.result}\n"},
+            headers=headers
+        )
+
+        response.raise_for_status()
+        data = response.json()
+        logging.debug(f"Posted comment to issue #{
+                      issue_id} at {data['html_url']}")