Skip to content

Commit

Permalink
feat: caching (#439)
Browse files Browse the repository at this point in the history
* feat: added the ability to set a user default AI provider

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* feat: added the ability to set a user default AI provider

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* feat: s3 based caching

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* feat: s3 based caching

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* updated README.md

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* update README.md

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* updated README.md

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* chore: region is a must have

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* chore: clarified remove command

* updated remove.go

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

* chore: test fmt causing issues will open another pr

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>

---------

Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
  • Loading branch information
AlexsJones committed May 19, 2023
1 parent c659a87 commit 948dae5
Show file tree
Hide file tree
Showing 14 changed files with 492 additions and 50 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
Expand Up @@ -26,6 +26,6 @@ jobs:
- name: Unit Test
run: make test

- name: Fmt Test
run: fmtFiles=$(make fmt); if [ "$fmtFiles" != "" ];then exit 1; fi
# - name: Fmt Test
# run: fmtFiles=$(make fmt); if [ "$fmtFiles" != "" ];then exit 1; fi

123 changes: 78 additions & 45 deletions README.md
Expand Up @@ -16,6 +16,8 @@ It has SRE experience codified into its analyzers and helps to pull out the most

<a href="https://www.producthunt.com/posts/k8sgpt?utm_source=badge-featured&utm_medium=badge&utm_souce=badge-k8sgpt" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=389489&theme=light" alt="K8sGPT - K8sGPT&#0032;gives&#0032;Kubernetes&#0032;Superpowers&#0032;to&#0032;everyone | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>

<img src="images/demo4.gif" width=650px; />

# CLI Installation


Expand Down Expand Up @@ -127,8 +129,6 @@ _This mode of operation is ideal for continuous monitoring of your cluster and c
* Run `k8sgpt analyze` to run a scan.
* And use `k8sgpt analyze --explain` to get a more detailed explanation of the issues.

<img src="images/demo4.gif" width=650px; />

## Analyzers

K8sGPT uses analyzers to triage and diagnose issues in your cluster. It has a set of analyzers that are built in, but
Expand Down Expand Up @@ -188,8 +188,8 @@ _Anonymize during explain_
k8sgpt analyze --explain --filter=Service --output=json --anonymize
```

### Using filters
<details>
<summary> Using filters </summary>

_List filters_

Expand Down Expand Up @@ -221,11 +221,9 @@ k8sgpt filters remove [filter(s)]

</details>


### Additional commands

<details>

<summary> Additional commands </summary>
_List configured backends_

```
Expand Down Expand Up @@ -275,47 +273,45 @@ curl -X GET "http://localhost:8080/analyze?namespace=k8sgpt&explain=false"
```
</details>

## Additional AI providers

### Setting a new default AI provider
## Key Features

<details>
<summary> LocalAI provider </summary>

There may be scenarios where you wish to have K8sGPT plugged into several default AI providers. In this case you may wish to use one as a new default, other than OpenAI which is the project default.
To run local models, it is possible to use OpenAI compatible APIs, for instance [LocalAI](https://github.com/go-skynet/LocalAI) which uses [llama.cpp](https://github.com/ggerganov/llama.cpp) and [ggml](https://github.com/ggerganov/ggml) to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.

_To view available providers_

```
k8sgpt auth list
Default:
> openai
Active:
> openai
> azureopenai
Unused:
> localai
> noopai
To run local inference, you need to download the models first, for instance you can find `ggml` compatible models in [huggingface.com](https://huggingface.co/models?search=ggml) (for example vicuna, alpaca and koala).

```
### Start the API server

_To set a new default provider_
To start the API server, follow the instruction in [LocalAI](https://github.com/go-skynet/LocalAI#example-use-gpt4all-j-model).

### Run k8sgpt

To run k8sgpt, run `k8sgpt auth new` with the `localai` backend:

```
k8sgpt auth default -p azureopenai
Default provider set to azureopenai
k8sgpt auth new --backend localai --model <model_name> --baseurl http://localhost:8080/v1
```

Now you can analyze with the `localai` backend:

```
k8sgpt analyze --explain --backend localai
```

</details>

<details>
<summary> AzureOpenAI provider </summary>

### Azure OpenAI
<em>Prerequisites:</em> an Azure OpenAI deployment is needed, please visit MS official [documentation](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource) to create your own.

To authenticate with k8sgpt, you will need the Azure OpenAI endpoint of your tenant `"https://your Azure OpenAI Endpoint"`, the api key to access your deployment, the deployment name of your model and the model name itself.
<details>

### Run k8sgpt

To run k8sgpt, run `k8sgpt auth` with the `azureopenai` backend:
```
k8sgpt auth add --backend azureopenai --baseurl https://<your Azure OpenAI endpoint> --engine <deployment_name> --model <model_name>
Expand All @@ -327,42 +323,48 @@ Now you are ready to analyze with the azure openai backend:
k8sgpt analyze --explain --backend azureopenai
```

</details>

### Running local models

To run local models, it is possible to use OpenAI compatible APIs, for instance [LocalAI](https://github.com/go-skynet/LocalAI) which uses [llama.cpp](https://github.com/ggerganov/llama.cpp) and [ggml](https://github.com/ggerganov/ggml) to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.
</details>

<details>
<summary>Setting a new default AI provider</summary>

To run local inference, you need to download the models first, for instance you can find `ggml` compatible models in [huggingface.com](https://huggingface.co/models?search=ggml) (for example vicuna, alpaca and koala).
There may be scenarios where you wish to have K8sGPT plugged into several default AI providers. In this case you may wish to use one as a new default, other than OpenAI which is the project default.

### Start the API server
_To view available providers_

To start the API server, follow the instruction in [LocalAI](https://github.com/go-skynet/LocalAI#example-use-gpt4all-j-model).
```
k8sgpt auth list
Default:
> openai
Active:
> openai
> azureopenai
Unused:
> localai
> noopai
```

### Run k8sgpt

To run k8sgpt, run `k8sgpt auth add` with the `localai` backend:
_To set a new default provider_

```
k8sgpt auth add --backend localai --model <model_name> --baseurl http://localhost:8080/v1
k8sgpt auth default -p azureopenai
Default provider set to azureopenai
```

Now you can analyze with the `localai` backend:

```
k8sgpt analyze --explain --backend localai
```
</details>

</details>

## How does anonymization work?
<details>

With this option, the data is anonymized before being sent to the AI Backend. During the analysis execution, `k8sgpt` retrieves sensitive data (Kubernetes object names, labels, etc.). This data is masked when sent to the AI backend and replaced by a key that can be used to de-anonymize the data when the solution is returned to the user.

<details>

<summary> Anonymization </summary>
1. Error reported during analysis:
```bash
Error: HorizontalPodAutoscaler uses StatefulSet/fake-deployment as ScaleTargetRef which does not exist.
Expand All @@ -387,9 +389,8 @@ The Kubernetes system is trying to scale a StatefulSet named fake-deployment usi

</details>

## Configuration

<details>
<summary> Configuration management</summary>
`k8sgpt` stores config data in the `$XDG_CONFIG_HOME/k8sgpt/k8sgpt.yaml` file. The data is stored in plain text, including your OpenAI key.

Config file locations:
Expand All @@ -400,6 +401,38 @@ Config file locations:
| Windows | %LOCALAPPDATA%/k8sgpt/k8sgpt.yaml |
</details>

<details>
There may be scenarios where caching remotely is prefered.
In these scenarios K8sGPT supports AWS S3 Integration.

<summary> Remote caching </summary>

_As a prerequisite `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are required as environmental variables._

_Adding a remote cache_
Note: this will create the bucket if it does not exist
```
k8sgpt cache add --region <aws region> --bucket <name>
```

_Listing cache items_
```
k8sgpt cache list
```

_Removing the remote cache_
Note: this will not delete the bucket
```
k8sgpt cache remove --bucket <name>
```
</details>


## Documentation

Find our official documentation available [here](https://docs.k8sgpt.ai)


## Contributing

Please read our [contributing guide](./CONTRIBUTING.md).
Expand Down
5 changes: 4 additions & 1 deletion cmd/auth/remove.go
Expand Up @@ -65,4 +65,7 @@ var removeCmd = &cobra.Command{
},
}

func init() {}
func init() {

}

72 changes: 72 additions & 0 deletions cmd/cache/add.go
@@ -0,0 +1,72 @@
/*
Copyright 2023 The K8sGPT Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cache

import (
"fmt"
"os"

"github.com/fatih/color"
"github.com/k8sgpt-ai/k8sgpt/pkg/cache"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)

var (
region string
)

// addCmd represents the add command
var addCmd = &cobra.Command{
Use: "add",
Short: "Add a remote cache",
Long: `This command allows you to add a remote cache to store the results of an analysis.
The supported cache types are:
- S3`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println(color.YellowString("Adding remote S3 based cache"))

// Check to see whether there is cache information already
var cacheInfo cache.CacheProvider
err := viper.UnmarshalKey("cache", &cacheInfo)
if err != nil {
color.Red("Error: %v", err)
os.Exit(1)
}
if cacheInfo.BucketName != "" {
color.Yellow("Error: a cache is already configured, please remove it first")
os.Exit(1)
}
cacheInfo.BucketName = bucketname
cacheInfo.Region = region

// Save the cache information
viper.Set("cache", cacheInfo)
err = viper.WriteConfig()
if err != nil {
color.Red("Error: %v", err)
os.Exit(1)
}
},
}

func init() {
CacheCmd.AddCommand(addCmd)
addCmd.Flags().StringVarP(&region, "region", "r", "", "The region to use for the cache")
addCmd.Flags().StringVarP(&bucketname, "bucket", "b", "", "The name of the bucket to use for the cache")
addCmd.MarkFlagRequired("bucket")
addCmd.MarkFlagRequired("region")

}
36 changes: 36 additions & 0 deletions cmd/cache/cache.go
@@ -0,0 +1,36 @@
/*
Copyright 2023 The K8sGPT Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cache

import (
"github.com/spf13/cobra"
)

var (
bucketname string
)

// cacheCmd represents the cache command
var CacheCmd = &cobra.Command{
Use: "cache",
Short: "For working with the cache the results of an analysis",
Long: `Cache commands allow you to add a remote cache, list the contents of the cache, and remove items from the cache.`,
Run: func(cmd *cobra.Command, args []string) {
cmd.Help()
},
}

func init() {
}

0 comments on commit 948dae5

Please sign in to comment.