Implement readDirEntries method

[vasicvuk]

Implementing feature to read directories from provided directory path based on Idea here:
#253

Within the README.md I have also documented the readDir function which was not documented but available for usage.

[yxxhero]

@vasicvuk Please add unittest and e2etest for this.

[yxxhero]

@vasicvuk Thanks for your idea. I think readDirs is ambiguous. Do you have a higher function name

[yxxhero]

@vasicvuk e2e test in test/e2e/template/helmfile/tmpl_test.go

[vasicvuk]

Hi @yxxhero,

Maybe listDir or readDirsWithinDir?

Do you have some name you would like?

[yxxhero]

@vasicvuk listDir will better. Can exec meet your needs?

[vasicvuk]

@yxxhero As I understood exec is plarform specific and we are using helmfile on both Windows and Linux so it would be hard do handle both cases

[yxxhero]

@mumoshu WDYT?

[vasicvuk]

@yxxhero Maybe listDir can list both files and folders instead just folders, i think that would be better

[yxxhero]

Can you describe your application scenario in detail

[vasicvuk]

@yxxhero Sure,

We have multiple Cloud native apps built with 30-50 microservices, each application uses some of the common services we call platform and some services specific to their product.

All of the microservices have their own charts and since we standardize the helm chart with internal generator tool we have a file named global which values from are applied for every microservice.

Other then that we have a folder for each team (owner of multiple microservice), and also an override which we use to optimize the helmfile for certain client were we implement product to.

.
├── ...
├── base # Default configuration for the product
│   ├── global #  Global configuration
│   │   ├── values.yaml # Global values
│   │   └── 
│   ├── platform # Platform microservices
│   │   ├── service1.yaml # Platform microservice 1
│   │   ├── service2.yaml # Platform microservice 2
│   │   └── 
│   ├── app  # App microservices
│   │   ├── app1.yaml # App microservice 1
│   │   ├── app2.yaml # App microservice 2
│   │   └── 
│   └── 
├── override # Override configuration for the specific client 
│   ├── global #  Global override configuration
│   │   ├── values.yaml # Global values
│   │   └── 
│   ├── platform # Platform override microservices
│   │   ├── service1.yaml # Platform microservice 1
│   │   └── 
│   ├── app  # App override microservices
│   │   ├── app1.yaml # App microservice 1
│   │   └── 
│   └── 
└── ...

In override, the user can override only certain services, global values, or everything (it's up to the implementation team). Other than that we allow the option for an override file to begin with _ and then the microservice will not be deployed at all even if it is defined in base.

We implemented everything from the description above, but currently, we need to have a list of folders in YAML so we know what folders are there (app, platform, third-party) and we want to automate also this by having an option to list folders from code too.

[yxxhero]

@vasicvuk ok. Thanks very much. Perfect this PR according to your idea. I feel it is really useful.

[vasicvuk]

@yxxhero, Thanks, I have changed the function to listDir and added unit tests for both readDir and listDir.

Since I am not really a GO developer, I need a little more time to understand how e2e part works, I am not sure if someone can jump in and help with e2e part?

Thanks

[vasicvuk]

@yxxhero I managed to add e2e tests also :)

[yxxhero]

@mumoshu ping

[yxxhero]

helmfile/pkg/argparser/args_test.go

Line 8 in 4e9b99d

"github.com/stretchr/testify/require"

[vasicvuk]

@yxxhero Should I change whole file then, as everything is using deepEqual currently?

[yxxhero]

@vasicvuk whole file.

[vasicvuk]

@yxxhero I changed all DeepEqual calls to use require now.

[yxxhero]

@vasicvuk Please sqush your commits. Thanks very much.

[vasicvuk]

@yxxhero Done

[yxxhero]

LGTM

[yxxhero]

@vasicvuk please fix go test error

[vasicvuk]

@yxxhero Can you help with identifying what the problem is, since i didn't change snapshot_test.go and it is showing this test as failure:

snapshot_test.go:126: helm push /home/runner/work/helmfile/helmfile/test/e2e/template/helmfile/raw-0.1.0.tgz oci://localhost:5000/myrepo: Error: failed to do request: Head "http://localhost:5000/v2/myrepo/raw/blobs/sha256:82d11697f6b3c7c5bcf619fb237648b2a8daee142dd0972f7f6244e72ea921e7": EOF
    snapshot_test.go:126: Unable to run helm push /home/runner/work/helmfile/helmfile/test/e2e/template/helmfile/raw-0.1.0.tgz oci://localhost:5000/myrepo: exit status 1
--- FAIL: TestHelmfileTemplateWithBuildCommand (10.77s)
    --- PASS: TestHelmfileTemplateWithBuildCommand/chart_need (6.33s)
    --- PASS: TestHelmfileTemplateWithBuildCommand/issue_2098_release_template_needs (0.07s)
    --- FAIL: TestHelmfileTemplateWithBuildCommand/oci_need (4.37s)

Seems to me that the local registry is not running and that it was a temporary test failure? Can we maybe retry the pipeline?

[mumoshu]

Thanks for your efforts!
Probably that's a flaky test added via #239. Let me rerun it and see if the error disappears.

[vasicvuk]

@mumoshu Yes, now they are passing :)