Development sandbox make fails

[MCP-LTS]

Hello!

I created a virtualbox to test dev environment and even with docker tools even with nix-shell tried many times in clean install at step "make"

i get the following error:

1st time running make

INFO[0051] Cluster 'homelab-dev' created successfully!  
INFO[0051] You can now use it like this:                
kubectl cluster-info
k3d kubeconfig get homelab-dev > kubeconfig.yaml
make[1]: Leaving directory '/home/vip/homelab/metal'
make -C bootstrap
make[1]: Entering directory '/home/vip/homelab/bootstrap'
kubectl create namespace argocd --dry-run=client --output=yaml \
        | kubectl apply -f -
namespace/argocd created
cd argocd && ./apply.sh
error: error validating "STDIN": error validating data: invalid object to validate; if you choose to ignore these errors, turn validation off with --validate=false
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applications.argoproj.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applicationsets.argoproj.io" not found
make[1]: *** [Makefile:11: argocd] Error 1
make[1]: Leaving directory '/home/vip/homelab/bootstrap'
make: *** [Makefile:18: bootstrap] Error 2

2nd time trying running make

statefulset.apps/argocd-application-controller created
ingress.networking.k8s.io/argocd-server created
resource mapping not found for name: "argocd-application-controller" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-repo-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-redis" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io condition met
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io condition met
cd root && ./apply.sh
Error from server (NotFound): namespaces "gitea" not found
applicationset.argoproj.io/bootstrap created
applicationset.argoproj.io/system created
applicationset.argoproj.io/platform created
applicationset.argoproj.io/apps created
make[1]: Leaving directory '/home/vip/homelab/bootstrap'
make -C test filter=Smoke
make[1]: Entering directory '/home/vip/homelab/test'
gotestsum --format testname -- -timeout 30m -run "Smoke"
go: downloading github.com/khuedoan/terratest v0.0.0-20220921151310-016fe8e1536b
...
...
...

continuing after few minutes make fails.

If you need any other details ill be happy to send you and help fix it.

[khuedoan]

The NotFound warning is safe to ignore, but do you have more logs on the second run? I'm guessing it's something like:

=== FAIL: . TestSmoke/vault (330.04s)
    retry.go:59: 'Wait for ingress vault to be provisioned.' unsuccessful after 10 retries
    --- FAIL: TestSmoke/vault (330.04s)

=== FAIL: . TestSmoke/argocd-server (330.05s)
    retry.go:59: 'Wait for ingress argocd-server to be provisioned.' unsuccessful after 10 retries
    --- FAIL: TestSmoke/argocd-server (330.05s)

=== FAIL: . TestSmoke (0.00s)

Could you also please attach the full log?

[khuedoan]

If your issue is the same as the above, I've increased the timeout to avoid that c4c2f8e.
You can rebase the dev branch to latest and try again.

[khuedoan]

Related note: I attempted to speed up the installation by using SkipDryRunOnMissingResource ¹, but it doesn't work because that option is resource specific ², so I had to revert it.

Footnotes

1. https://github.com/khuedoan/homelab/commit/377b02a6eab8322a1ac2b5b1d4166639193165ab ↩

2. https://github.com/argoproj/gitops-engine/blob/e284fd71cb96fc2767d3ae79b26e0533ad8edc4a/pkg/sync/sync_context.go#L734 ↩

[MCP-LTS]

i rebased the dev branch to latest as you suggested and after trying again, i attach the log of the 2 "make" tries
make-error.log

note this: "resource mapping not found for name: "argocd-application-controller" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1""

The VM FYI has 12GB RAM and 10CPUs although ram is not problem disk input output and cpu at some points for long time goes really high.

If you think still is a matter of timeout i can manually change the value to a really big number that you suggest and check again.

Hope helps. You can ask me to try anything else you think might help.

[locmai]

The ServiceMonitor CRD is installed at the Prometheus/monitoring installation step. ArgoCD could skip it with the SkipDryRunOnMissingResource but not with Helm. The fix could either be:

• Install the Prometheus CRD separately before the ArgoCD
• Disable the ArgoCD ServiceMonitor and apply it later after the Prometheus chart installed

[khuedoan]

The resource mapping not found for name:... no matches for kind... in version... lines, as I mentioned earlier, are just warnings; we don't need them on the first install and are safe to ignore.

You can check application status during the installation process by running:

./scripts/argocd-admin-password
kubectl port-forward -n argocd service/argocd-server 8000:80

Then go to http://localhost:8000 and login with the admin account (you can still check it even if the test has timed out).

[MCP-LTS]

Hello happy new year !

i finally managed with 2 runs again but worked...

maybe locmai's solution is more clear but since i chose to play with that project's concept especially since i try it, i didn't want to make tweaks that break the logic of the project.

after the second run with the following file, after reading and trying various solutions the typical old solution "use 2 ArgoCD apps one for CRDs" etc ..., finally reading an article made me take the right path. i don't want to publish - advertise where i got the idea , in pm ok i can give you the link where i got the idea....

what worked was

the file "homelab/system/monitoring-system/Chart.yaml" since argo 2.5 and later ... works the ServerSideApply=true
no "repository" no "version" like helm dependencies typical thing ok ? its a tweak ok? not a permanent solution. It takes a long time the first run without a reason i guess or we should use a "kubectl wait" .... but finally at the second run worked. maybe the first run no need to wait so much ...

apiVersion: v2
name: kube-prometheus-stack
version: 0.0.0
dependencies:
  - name: kube-prometheus-stack
    helm:
#just in case ...
      skipCrds: false
      values: |-
        prometheusOperator:
           admissionWebhooks:
             failurePolicy: Ignore

    repoURL: https://prometheus-community.github.io/helm-charts
    targetRevision: 43.1.4
    syncPolicy:
      syncOptions:
        - ServerSideApply=true
        - CreateNamespace=true

make2.log
make1.log

attaching the 2 runs logs

enjoy!

[khuedoan]

I disabled ServiceMonitor on first apply to avoid the confusion caused by the warning cd39632 (as it occurred again in another issue #102)

[khuedoan]

Closing due to inactivity, please re-open if you still have the issue.