Skip to content

Commit

Permalink
config gen initial work
Browse files Browse the repository at this point in the history
Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added gen subcmd to config cmd

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

add test for aws regions option list

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

refactored based on main hadeployconfig model

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

prompt moved

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

on-prem flow completed

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

custom certs added to ha onprem

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix the config change based on main branch

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added password and paragraph support in prompt

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

hardware cal added

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

aws flow added and cert via file added

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added AWS Provision Config Gen with few tests

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

aws ha deployment working

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix custom certs

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

regex fix

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix fqdn regex

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

regex made common

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added test for aws cidr managed flow

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

changed ioutil to os in config.go cmd

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added test for AWS config gen flow

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

unit tests added

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

pg and os limit set to min 3

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added more unit test

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix complexity

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix code smells

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added tests

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

fix pipeline

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added more unit tests

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

revendor fix .bldr.toml

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

simplified code

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

removed not used code from prompt lib

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

increased unit test coverage

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

added unit tests for Init func in hadeployconfig

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

make config gen command visible

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

code review comment fix

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

PR comment change

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>

removed comment

Signed-off-by: Vivek Yadav <vivek.yadav@progress.com>
  • Loading branch information
vivek-yadav committed Jul 20, 2023
1 parent dfe7fd2 commit 3d7b667
Show file tree
Hide file tree
Showing 31 changed files with 8,175 additions and 8 deletions.
3 changes: 3 additions & 0 deletions .bldr.toml
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,7 @@ paths = [
"lib/db/*",
"lib/executil/*",
"lib/grpc/*",
"lib/hardwarecal/*",
"lib/httputils/*",
"lib/io/*",
"lib/logger/*",
Expand All @@ -455,6 +456,7 @@ paths = [
"lib/platform/config/*",
"lib/platform/pg/*",
"lib/platform/sys/*",
"lib/pmt/*",
"lib/proc/*",
"lib/product/*",
"lib/proxy/*",
Expand All @@ -464,6 +466,7 @@ paths = [
"lib/stringutils/*",
"lib/systemresource/*",
"lib/tls/*",
"lib/toml/*",
"lib/tracing/*",
"lib/user/*",
"lib/userutils/*",
Expand Down
91 changes: 86 additions & 5 deletions components/automate-cli/cmd/chef-automate/config.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package main

import (
"io/ioutil"
"reflect"
"regexp"
"strings"
Expand All @@ -19,6 +18,9 @@ import (
"github.com/chef/automate/components/automate-cli/pkg/status"
"github.com/chef/automate/components/automate-deployment/pkg/cli"
"github.com/chef/automate/components/automate-deployment/pkg/client"
"github.com/chef/automate/lib/config/genconfig"
"github.com/chef/automate/lib/io/fileutils"
"github.com/chef/automate/lib/pmt"
"github.com/chef/automate/lib/stringutils"
"github.com/chef/toml"
"github.com/imdario/mergo"
Expand Down Expand Up @@ -60,6 +62,10 @@ func init() {
configCmd.AddCommand(showConfigCmd)
configCmd.AddCommand(patchConfigCmd)
configCmd.AddCommand(setConfigCmd)
configCmd.AddCommand(genConfigCmd)

//config gen flags
genConfigCmd.Flags().BoolVarP(&configCmdFlags.overwriteFile, "overwrite", "O", false, "Overwrite existing config.toml")

//config show flags
showConfigCmd.Flags().BoolVarP(&configCmdFlags.overwriteFile, "overwrite", "O", false, "Overwrite existing config.toml [Standalone]")
Expand Down Expand Up @@ -148,6 +154,17 @@ var showConfigCmd = &cobra.Command{
},
}

var genConfigCmd = &cobra.Command{
Use: "gen [/path/to/write/config.toml]",
Short: "generate the Automate HA configuration",
Long: "Prompt based Config Generation command. It will output the config in the provided file, if file path is not provided then it will print on STDOUT.",
RunE: runGenCmd,
Args: cobra.RangeArgs(0, 2),
Annotations: map[string]string{
docs.Tag: docs.BastionHost,
},
}

var patchConfigCmd = &cobra.Command{
Use: "patch path/to/config.toml",
Short: "patch the Chef Automate configuration", Long: "Apply a partial Chef Automate configuration to the deployment. It will take the partial configuration, merge it with the existing configuration, and apply and required changes.",
Expand All @@ -169,6 +186,70 @@ var setConfigCmd = &cobra.Command{
},
}

func runGenCmd(cmd *cobra.Command, args []string) error {
fsu := fileutils.NewFileSystemUtils()
p := pmt.PromptFactory(os.Stdin, os.Stdout, fsu)
g := genconfig.GenConfigImpFactory(p)

err := g.GenConfigWithPrompts()
if err != nil {
return status.Wrap(
err,
status.FailedToGenConfig,
"Failed to generate config with given input values to the prompts.",
)
}
t, err := g.Toml()
if err != nil {
return status.Wrap(
err,
status.MarshalError,
"Marshaling configuration to TOML failed",
)
}

// Handle writing to a file if a path was given
if len(args) > 0 && args[0] != "" {
outFile, err := filepath.Abs(args[0])
if err != nil {
return status.Annotate(err, status.FileAccessError)
}

err = checkConfigGenFileExist(outFile, fsu)
if err != nil {
return err
}
err = fsu.WriteFile(outFile, t, 0644)
if err != nil {
return status.Wrap(err, status.FileAccessError, fmt.Sprint("Failed to write to file: ", outFile))
}
} else {
writer.Println("")
writer.Println(string(t))
}

status.GlobalResult = t
return nil
}

func checkConfigGenFileExist(outFile string, fsu fileutils.FileUtils) error {
if _, err := fsu.Stat(outFile); err == nil {
if configCmdFlags.overwriteFile {
return nil
}
p := pmt.PromptFactory(os.Stdin, os.Stdout, fsu)
ok, err := p.Confirm(fmt.Sprintf("%s file already exists. Do you wish to overwrite it?", outFile), "yes", "no")
if err != nil {
return status.Wrap(err, status.PromptFailed, err.Error())
}
if !ok {
err = errors.New("failed to confirm overwrite")
return status.Annotate(err, status.FileAccessError)
}
}
return nil
}

func runShowCmd(cmd *cobra.Command, args []string) error {

if isA2HARBFileExist() {
Expand Down Expand Up @@ -821,7 +902,7 @@ func getMergedOpensearchInterface(rawOutput string, pemFilePath string, remoteSe
return "", err
}

pemBytes, err := ioutil.ReadFile(pemFilePath) // nosemgrep
pemBytes, err := os.ReadFile(pemFilePath)
if err != nil {
return "", err
}
Expand All @@ -844,7 +925,7 @@ func getMergedPostgresqlInterface(rawOutput string, pemFilePath string, remoteSe
return "", err
}

pemBytes, err := ioutil.ReadFile(pemFilePath) // nosemgrep
pemBytes, err := os.ReadFile(pemFilePath)
if err != nil {
return "", err
}
Expand Down Expand Up @@ -961,7 +1042,7 @@ func getDecodedConfig(input string, remoteService string) (interface{}, error) {

// getConfigForArgsPostgresqlAndOpenSearch gets the requested config from the args provided for postgresql or opensearch
func getConfigForArgsPostgresqlOrOpenSearch(args []string, remoteService string) (interface{}, error) {
pemBytes, err := ioutil.ReadFile(args[0]) // nosemgrep
pemBytes, err := os.ReadFile(args[0])
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -1070,7 +1151,7 @@ func createTomlFileFromConfig(config interface{}, tomlFile string) (string, erro

func parseAndRemoveRestrictedKeysFromSrcFile(srcString string) (string, error) {

tomlbyt, _ := ioutil.ReadFile(srcString) // nosemgrep
tomlbyt, _ := os.ReadFile(srcString)
destString := string(tomlbyt)
var dest dc.AutomateConfig
if _, err := toml.Decode(destString, &dest); err != nil {
Expand Down
60 changes: 60 additions & 0 deletions components/automate-cli/cmd/chef-automate/hardwarecal.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package main

import (
"os"

"github.com/chef/automate/components/automate-cli/pkg/docs"
"github.com/chef/automate/components/automate-cli/pkg/status"
"github.com/chef/automate/lib/hardwarecal"
"github.com/chef/automate/lib/io/fileutils"
"github.com/chef/automate/lib/pmt"
"github.com/spf13/cobra"
)

func init() {
RootCmd.AddCommand(hardwareCalCmd())
}

func hardwareCalCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "hardware-cal COMMAND",
Short: "Estimate Hardware Requirements for given inputs, this calculator is for Chef Automate HA",
Long: "Estimate Hardware Requirements for given inputs, this calculator is for Chef Automate HA",
Annotations: map[string]string{
docs.Tag: docs.BastionHost,
},
Hidden: true,
}

run := &cobra.Command{
Use: "run",
Short: "This will trigger the hardware calculator to ask input values, based on which it will provide hardware requirements for Chef Automate HA",
Long: "This will trigger the hardware calculator to ask input values, based on which it will provide hardware requirements for Chef Automate HA",
RunE: runCal,
Annotations: map[string]string{
docs.Tag: docs.BastionHost,
},
Hidden: true,
}

cmd.AddCommand(run)
return cmd
}

func runCal(*cobra.Command, []string) error {
cw := writer
fsu := &fileutils.FileSystemUtils{}
p := pmt.PromptFactory(os.Stdin, os.Stdout, fsu)
hc := hardwarecal.NewHardwareCalPrompt(p, cw)
err := hc.Run()
if err != nil {
return status.Wrap(
err,
status.HardwareCalError,
"Failed to Estimate Hardware Requirements",
)
}

cw.Println("* These numbers are just for estimation, based on our test environment conditions. \n* Please do your performance analysis based on your environment and usages.")
return nil
}
6 changes: 6 additions & 0 deletions components/automate-cli/pkg/status/error.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ const (
const (
VerifyChecksError = 125
ConfigVerifyError = 124
PromptFailed = 126
FailedToGenConfig = 127
HardwareCalError = 128
IpAccessError = 123
InsufficientSpaceError = 122
CalDestDirSizeError = 121
Expand Down Expand Up @@ -104,6 +107,9 @@ const (
var ErrorMetadata = map[int][]string{
VerifyChecksError: {"125", "VerifyChecksError", "Verify Checks failed"},
ConfigVerifyError: {"124", "ConfigVerifyError", "Config Verification failed"},
HardwareCalError: {"128", "HardwareCalError", "Hardware Calculator failed to provide Hardware Estimates"},
PromptFailed: {"126", "PromptFailed", "Prompt failed and exited with error"},
FailedToGenConfig: {"127", "FailedToGenConfig", "Failed to Generate config with given inputs"},
InsufficientSpaceError: {"122", "InsufficientSpaceError", "Insufficient disk space"},
CalDestDirSizeError: {"121", "CalDestDirSizeError", "Error in calculating Dest directory size"},
CalESDirSizeError: {"120", "CalESDirSizeError", "Error in calculating ES directory size"},
Expand Down
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ require (
github.com/shirou/gopsutil v3.21.11+incompatible
)

require github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e // indirect

require (
cloud.google.com/go v0.65.0 // indirect
cloud.google.com/go/storage v1.10.0 // indirect
Expand Down Expand Up @@ -183,6 +185,7 @@ require (
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect
github.com/magiconair/properties v1.8.1 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/manifoldco/promptui v0.9.0
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.18 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
Expand Down
6 changes: 6 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -172,8 +172,11 @@ github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cb
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chef/toml v0.3.1-0.20200730001027-920c30b33b5d h1:fVRXxFz5+MWhty4oarqLwi6dSkPZFYYEs5OJGLzEX5A=
github.com/chef/toml v0.3.1-0.20200730001027-920c30b33b5d/go.mod h1:WzupinluQEwU43AC1/UZItixzo5R849ewfPDB2ZMDDg=
github.com/chzyer/logex v1.1.10 h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag=
github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I=
Expand Down Expand Up @@ -560,6 +563,8 @@ github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/manifoldco/promptui v0.9.0 h1:3V4HzJk1TtXW1MTZMP7mdlwbBpIinw3HztaIlYthEiA=
github.com/manifoldco/promptui v0.9.0/go.mod h1:ka04sppxSGFAtxX0qhlYQjISsg9mR4GWtQEhdbn6Pgg=
github.com/mattn/go-colorable v0.0.0-20170327083344-ded68f7a9561/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
Expand Down Expand Up @@ -1039,6 +1044,7 @@ golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190129075346-302c3dd5f1cc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
Expand Down
21 changes: 21 additions & 0 deletions lib/config/genconfig/automatehaconfig.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package genconfig

import (
"github.com/chef/automate/lib/toml"
)

type AutomateHaConfig struct {
Fqdn string `json:"fqdn,omitempty" toml:"fqdn,omitempty" mapstructure:"fqdn,omitempty"`
}

func AutomateHaConfigFactory() *AutomateHaConfig {
return &AutomateHaConfig{}
}

func (c *AutomateHaConfig) Toml() (tomlBytes []byte, err error) {
return toml.Marshal(c)
}

func (c *AutomateHaConfig) Prompts() (err error) {
return
}

0 comments on commit 3d7b667

Please sign in to comment.