Skip to content
This repository has been archived by the owner on Jun 29, 2022. It is now read-only.

(WIP) Refactoring loading of configuration, cluster operations management. #359

Closed
wants to merge 43 commits into from
Closed

(WIP) Refactoring loading of configuration, cluster operations management. #359

wants to merge 43 commits into from

Commits on Apr 27, 2020

  1. Renaming RootConfig to ClusterConfig as part of refactoring 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    415eb0a View commit details
    Browse the repository at this point in the history

  2. Rename Config to HCLConfig 

    - This adds clarity to the name of the struct as this deals with hcl
  config.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    639640d View commit details
    Browse the repository at this point in the history

  3. Refactor LoadConfig into load and parse functions 

    - This commit has been done to make the loading and parsing of the hcl
  files more testable.

  LoadConfig currently performs the task of loading the HCL files from
  the given path and parsing them to HCLConfig go struct.

  To make the code testable and the function is broken into LoadHCLFiles
  and ParseHCLFiles.

  LoadHCLfiles is an exported function for now , because it is called
  in cmd/utils.go in getLokoConfig function and I want to keep the
  commit atomic in nature. In further commits in this PR when cmd
  package is refactored, the function will be made unexported.

  ParseHCLFiles is exported becuase this function is used to parse HCL
  files and will be used in tests where a hcl config is loaded from
  string.

  loadLokocfgPaths function signature is changed to take the path of the
  hclfiles as well as extension. This change allows the user to not only
  provide a directory containing multiple '*.lokocfg' files but also
  multiple '*.vars' files.

  LoadValuesFile is refactored and renamed to LoadValues. The change is
  done so that instead of taking the path to a single vars file.
  LoadValues now accepts a list of hcl.File objects which allows the
  ability to load values from multiple vars files.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    638b3a3 View commit details
    Browse the repository at this point in the history

  4. Remove usage of anonymous struct in Packet. 

    - To remove the usage of anonymous structs, three new fields are added
  to `config` struct with a "Raw" suffix to distinguish between the
  fields that are used as in and those used to save data after
  unmarshalling.

  Doing this removes the need to create anonymous struct which in turns
  makes the go template that it renders cleaner (i.e `.Config` is
  removed )

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    c184dec View commit details
    Browse the repository at this point in the history

  5. Remove usage of anonymous struct in AWS. 

    - To remove the usage of anonymous structs, four new fields are added
  to `config` struct with a "Raw" suffix to distinguish between the
  fields that are used as in and those used to save data after
  unmarshalling.

  Doing this removes the need to create anonymous struct which in turns
  makes the go template that it renders cleaner (i.e `.Config` is
  removed )

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    c8f1603 View commit details
    Browse the repository at this point in the history

  6. Remove usage of anonymous struct in Baremetal. 

    - To remove the usage of anonymous structs, seven new fields are added
  to `config` struct with a "Raw" suffix to distinguish between the
  fields that are used as in and those used to save data after
  unmarshalling.

  Doing this removes the need to create anonymous struct which in turns
  makes the go template that it renders cleaner (i.e `.Config` is
  removed)

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    581807c View commit details
    Browse the repository at this point in the history

  7. Move verify.go to lokomotive package. 

    - install package consists of two files:
  install.go: which deals with configuring terraform.
  verify.go: which deals with verification of Lokomotive cluster.

  install package by its name and contents of the files serve no
  purpose, hence its files should be moved around to correct packages.

- This commit deals with moving the verify.go file to lokomotive
  package.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    f8d7a12 View commit details
    Browse the repository at this point in the history

  8. Move install.go to terraform package. 

    - install package now consists of one file:
  install.go: which deals with configuring terraform.

  install package by its name and contents of the files serve no
  purpose, hence its files should be moved around to correct packages.

- This commit deals with moving install.go file to terraform
  package.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    15e4814 View commit details
    Browse the repository at this point in the history

  9. Make unnecessary exported functions as unexported. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    c3cf386 View commit details
    Browse the repository at this point in the history

  10. Move template.go to util package 

    - This commit moves the template.go from pkg/components/utl to pkg/util

- Update package references for pkg/components to include new package
  location

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    82e1a55 View commit details
    Browse the repository at this point in the history

  11. Add Render and Validate to Platform interface. 

    - Add placeholder methods for Render and Validate in all the supported
  platforms.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    70c9f2f View commit details
    Browse the repository at this point in the history

  12. LokomotiveConfig struct for managing configuration. 

    - LokomotiveConfig acts as main placeholder for Lokomotive
  configuration, once the user provided configuration in `*.lokocfg`
  files is loaded and parsed.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    faaf2d8 View commit details
    Browse the repository at this point in the history

  13. Centralize loading of HCL files in one place. 

    - HCLLoader loads and parses the Lokomotive HCL configuration files into
  LokomotiveConfig struct.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    d0652dc View commit details
    Browse the repository at this point in the history

  14. Add InitializeExecutor for terraform.Executor 

    - InitializeExecutor method returns the terraform.Executor

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    1716379 View commit details
    Browse the repository at this point in the history

  15. Add Manager interface for Lokomotive operations. 

    - Manager interface manages the operations involved in the lifecycle of
  a Lokomotive cluster.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    e88b944 View commit details
    Browse the repository at this point in the history

  16. Add implmentation of Manager interface. 

    - lokomotive provides an implmentation of the Manager interface.
- currently the interface methods are implemented as placeholders.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    cf17899 View commit details
    Browse the repository at this point in the history

  17. Add method to create terraform cluster file. 

    - createTerraformClusterfile creates the `cluster.tf` file from the
  data string.

- Update Configure method to take additional parameter of
  renderedPlatform

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    6b2c9cc View commit details
    Browse the repository at this point in the history

  18. Refactor GetComponentBody 

    - This commit refactors GetComponentBody to LoadComponentsFromHCLString

- This removes the duplication of parsing the componenet configuration
  from string.

  LoadcomponentsFromString uses existing methods to parse the component
  string to Component instance.

- Updates the tests in components pkg to reflect the same

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    1c0aa96 View commit details
    Browse the repository at this point in the history

  19. Move utility functions to lokomotive package. 

    - As part of the refactor, all Lokomotive cluster related utility
  functions are ported to lokomotive package.

  Lokomotive package holds all the code related to management of
  Lokomtive cluster.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    b51b5b5 View commit details
    Browse the repository at this point in the history

  20. Update lokomotive methods. 

    - Adds implmentations to placeholder methods.

- Moves logic around cluster operations from cmd package to here.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    fe5abb9 View commit details
    Browse the repository at this point in the history

  21. Remove not needed functions from cmd/cluster.go 

    - removing the functions that are not needed anymore as their
  implmentations are ported to pkg/lokomotive package.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    2e73fea View commit details
    Browse the repository at this point in the history

  22. Update initialize method to use new methods. 

    - intitialize now makes use of the new refactor structs and methods.
  Loads the LokomotiveConfig and returns an instance of Manager
  interface for performing cluster operations.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    11049a7 View commit details
    Browse the repository at this point in the history

  23. Update cluster-apply to use new changes. 

    - cluster-apply.go now mkes use of the refactored initialize function.

- cluster related checks and operations are encapsulated in Apply method
  of lokomotive instance.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    f2b5e3c View commit details
    Browse the repository at this point in the history

  24. Update cluster-destroy to use new changes. 

    - cluster-destry.go now mkes use of the refactored initialize function.

- cluster related checks and operations are encapsulated in Destroy method
  of lokomotive instance.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    4c2fea2 View commit details
    Browse the repository at this point in the history

  25. Update component-apply to use new changes. 

    - component-apply now mkes use of the refactored initialize function.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    aa48e04 View commit details
    Browse the repository at this point in the history

  26. Update component-render-manifest to use new changes. 

    - component-render-manifest now mkes use of the refactored initialize function

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    4ead062 View commit details
    Browse the repository at this point in the history

  27. Update health to use new changes. 

    - health now mkes use of the refactored initialize function.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    4ec3146 View commit details
    Browse the repository at this point in the history

  28. Change const to reflect consistency 

    - This commit changes the const `componentName` to `name` to reflect
  consistency across const across all components

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    bbf785d View commit details
    Browse the repository at this point in the history

  29. Update component-delete to use new changes. 

    - Update Manager interface to include ComponentDelete
- Add implementation of ComponentDelete to lokotive struct
- Port component-delete functionality from cmd package to
  lokomotive package

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    064f9d0 View commit details
    Browse the repository at this point in the history

  30. Rename ContextLogger to Logger. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    5ddfb39 View commit details
    Browse the repository at this point in the history

  31. Move verifyCluster to utils.go 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    a397c04 View commit details
    Browse the repository at this point in the history

  32. Delete not needed utils.go 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    0020b44 View commit details
    Browse the repository at this point in the history

  33. Update Platform interface to remove LoadConfig 

    - Since loading and parsing of the confguration is done in
  config/hcl_loader.go. This method is redundant and hence removed.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    9e4687c View commit details
    Browse the repository at this point in the history

  34. Update Platform.Render method 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    5e38dce View commit details
    Browse the repository at this point in the history

  35. Remove Initialize method from Platform interface. 

    - The purpose of Initialize method was to create the terraform file for
  Terraform to execute. However with the refactor Render method renders
  the terraform template and passes the rendered string for further
  processing.

  The creation of the terraform file happens in the terraform package.

  Hnce removing the Initialize method from the interface.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    de469e4 View commit details
    Browse the repository at this point in the history

  36. Add validation of LokomotiveConfig 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    e19d449 View commit details
    Browse the repository at this point in the history

  37. Make methods unexported if they are not needed 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    3d14689 View commit details
    Browse the repository at this point in the history

  38. Add validation to all platforms. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    09ba186 View commit details
    Browse the repository at this point in the history

  39. Add unit tests for loading HCL configurations. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    b01bbac View commit details
    Browse the repository at this point in the history

  40. Add unit tests for HCLLoader. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    2377691 View commit details
    Browse the repository at this point in the history

  41. Add Unit tests for Platform. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    9dff2ce View commit details
    Browse the repository at this point in the history

  42. Disabling the 'gomnd' linter. 

    - There are many tests which make using of a random number, which is
  then flagged as Magic Random number by linter.

  This causes to add '//nolint:gomnd' in  a lot of places.

Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    e8a0019 View commit details
    Browse the repository at this point in the history

  43. Resolve linter errors. 

    Signed-off-by: Imran Pochi <imran@kinvolk.io>
    @ipochi
    ipochi committed Apr 27, 2020
    Configuration menu
    Copy the full SHA
    b13a203 View commit details
    Browse the repository at this point in the history