Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Multiple sources for applications #10432

Merged
merged 36 commits into from
Dec 16, 2022
Merged

feat: Multiple sources for applications #10432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
1ca3e74
feat: support multiple sources for application
ishitasequeira May 17, 2022
7d35c89
fix bug introduced after rebase
ishitasequeira Sep 15, 2022
dd24796
rebase with master
ishitasequeira Sep 16, 2022
41fe9b3
fix index out of bounds error
ishitasequeira Sep 26, 2022
b37f979
ui fixes
ishitasequeira Sep 26, 2022
91bde73
add revisions to SyncOperation for rollback
ishitasequeira Sep 27, 2022
756040e
change Source to *ApplicationSource in ApplicationSpec
ishitasequeira Sep 30, 2022
a735c61
fix env variable read logic for ValueFiles
ishitasequeira Oct 16, 2022
1c3a961
update multiple sources doc
ishitasequeira Oct 24, 2022
92c21b5
add repository lock and checkout target revision
ishitasequeira Nov 7, 2022
a5803d2
checkout all sources before generating manifest
ishitasequeira Nov 9, 2022
85a2d64
update logic for returning ManifestResponse to avoid nil pointer issues
ishitasequeira Nov 29, 2022
ae9a737
fix nil reference and key mismatch bugs; add more logs (#6)
crenshaw-dev Dec 2, 2022
08d0070
update grpc field numbers
ishitasequeira Dec 9, 2022
09128c6
path resolution tests (#12)
crenshaw-dev Dec 9, 2022
186242c
add regex check for value of source.ref
ishitasequeira Dec 9, 2022
35488cf
add webhook tests
crenshaw-dev Dec 13, 2022
36d97d3
fix env variable read logic for ValueFiles
ishitasequeira Oct 16, 2022
10267e2
add repository lock and checkout target revision
ishitasequeira Nov 7, 2022
803da2f
checkout all sources before generating manifest
ishitasequeira Nov 9, 2022
50ebea5
generate mock reposerverclient
ishitasequeira Nov 10, 2022
2eaeb2e
address comments
ishitasequeira Nov 27, 2022
e2e7b9a
update logic for returning ManifestResponse to avoid nil pointer issues
ishitasequeira Nov 29, 2022
2789d4a
fix nil reference and key mismatch bugs; add more logs (#6)
crenshaw-dev Dec 2, 2022
cf74005
update grpc field numbers
ishitasequeira Dec 9, 2022
bc9e4ca
add regex check for value of source.ref
ishitasequeira Dec 9, 2022
8c3e437
Rebase with master
ishitasequeira Dec 13, 2022
b0142d7
Added unit tests (#15)
ishitasequeira Dec 13, 2022
e2d2267
fix application parameters tab, rebased UI changes, tests
ishitasequeira Dec 14, 2022
316d985
More tests (#16)
crenshaw-dev Dec 14, 2022
4c9ac12
unlock the values file repo before doing a symlink check (#22)
crenshaw-dev Dec 15, 2022
b51136d
multi source docs (#21)
crenshaw-dev Dec 16, 2022
a746d36
add e2e tests for multiple sources and fix UI lint (#23)
ishitasequeira Dec 16, 2022
7d0ecb9
rebase with master
ishitasequeira Dec 16, 2022
a47c6da
rebase with master
ishitasequeira Dec 16, 2022
bbb9240
fix lint
ishitasequeira Dec 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 14 additions & 14 deletions applicationset/controllers/applicationset_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -684,7 +684,7 @@ func TestCreateOrUpdateInCluster(t *testing.T) {
Template: argov1alpha1.ApplicationSetTemplate{
Spec: argov1alpha1.ApplicationSpec{
Project: "project",
Source: argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Source: &argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Destination: argov1alpha1.ApplicationDestination{Server: "server", Namespace: "namespace"},
},
},
Expand Down Expand Up @@ -721,7 +721,7 @@ func TestCreateOrUpdateInCluster(t *testing.T) {
},
Spec: argov1alpha1.ApplicationSpec{
Project: "project",
Source: argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Source: &argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Destination: argov1alpha1.ApplicationDestination{Server: "server", Namespace: "namespace"},
},
},
Expand All @@ -741,7 +741,7 @@ func TestCreateOrUpdateInCluster(t *testing.T) {
},
Spec: argov1alpha1.ApplicationSpec{
Project: "project",
Source: argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Source: &argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Destination: argov1alpha1.ApplicationDestination{Server: "server", Namespace: "namespace"},
},
Status: argov1alpha1.ApplicationStatus{
Expand Down Expand Up @@ -918,7 +918,7 @@ func TestRemoveFinalizerOnInvalidDestination_FinalizerTypes(t *testing.T) {
},
Spec: argov1alpha1.ApplicationSpec{
Project: "project",
Source: argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Source: &argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
// Destination is always invalid, for this test:
Destination: argov1alpha1.ApplicationDestination{Name: "my-cluster", Namespace: "namespace"},
},
Expand Down Expand Up @@ -1080,7 +1080,7 @@ func TestRemoveFinalizerOnInvalidDestination_DestinationTypes(t *testing.T) {
},
Spec: argov1alpha1.ApplicationSpec{
Project: "project",
Source: argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Source: &argov1alpha1.ApplicationSource{Path: "path", TargetRevision: "revision", RepoURL: "repoURL"},
Destination: c.destinationField,
},
}
Expand Down Expand Up @@ -1589,7 +1589,7 @@ func TestValidateGeneratedApplications(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{},
Spec: argov1alpha1.ApplicationSpec{
Project: "default",
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://url",
Path: "/",
TargetRevision: "HEAD",
Expand All @@ -1612,7 +1612,7 @@ func TestValidateGeneratedApplications(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{},
Spec: argov1alpha1.ApplicationSpec{
Project: "default",
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://url",
Path: "/",
TargetRevision: "HEAD",
Expand All @@ -1636,7 +1636,7 @@ func TestValidateGeneratedApplications(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{},
Spec: argov1alpha1.ApplicationSpec{
Project: "DOES-NOT-EXIST",
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://url",
Path: "/",
TargetRevision: "HEAD",
Expand All @@ -1659,7 +1659,7 @@ func TestValidateGeneratedApplications(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{},
Spec: argov1alpha1.ApplicationSpec{
Project: "default",
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://url",
Path: "/",
TargetRevision: "HEAD",
Expand All @@ -1682,7 +1682,7 @@ func TestValidateGeneratedApplications(t *testing.T) {
ObjectMeta: metav1.ObjectMeta{},
Spec: argov1alpha1.ApplicationSpec{
Project: "default",
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://url",
Path: "/",
TargetRevision: "HEAD",
Expand Down Expand Up @@ -1811,7 +1811,7 @@ func TestReconcilerValidationErrorBehaviour(t *testing.T) {
Namespace: "argocd",
},
Spec: argov1alpha1.ApplicationSpec{
Source: argov1alpha1.ApplicationSource{RepoURL: "https://github.com/argoproj/argocd-example-apps", Path: "guestbook"},
Source: &argov1alpha1.ApplicationSource{RepoURL: "https://github.com/argoproj/argocd-example-apps", Path: "guestbook"},
Project: "default",
Destination: argov1alpha1.ApplicationDestination{Server: "{{.url}}"},
},
Expand Down Expand Up @@ -1954,7 +1954,7 @@ func TestGenerateAppsUsingPullRequestGenerator(t *testing.T) {
},
},
Spec: argov1alpha1.ApplicationSpec{
Source: argov1alpha1.ApplicationSource{
Source: &argov1alpha1.ApplicationSource{
RepoURL: "https://testurl/testRepo",
TargetRevision: "{{.head_short_sha}}",
},
Expand All @@ -1973,7 +1973,7 @@ func TestGenerateAppsUsingPullRequestGenerator(t *testing.T) {
},
},
Spec: v1alpha1.ApplicationSpec{
Source: v1alpha1.ApplicationSource{
Source: &v1alpha1.ApplicationSource{
RepoURL: "https://testurl/testRepo",
TargetRevision: "089d92cb",
},
Expand Down Expand Up @@ -2113,7 +2113,7 @@ func TestPolicies(t *testing.T) {
},
},
Spec: argov1alpha1.ApplicationSpec{
Source: argov1alpha1.ApplicationSource{RepoURL: "https://github.com/argoproj/argocd-example-apps", Path: "guestbook"},
Source: &argov1alpha1.ApplicationSource{RepoURL: "https://github.com/argoproj/argocd-example-apps", Path: "guestbook"},
Project: "default",
Destination: argov1alpha1.ApplicationDestination{Server: "https://kubernetes.default.svc"},
},
Expand Down
6 changes: 3 additions & 3 deletions applicationset/utils/utils_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ func TestRenderTemplateParams(t *testing.T) {
Namespace: "default",
},
Spec: argoappsv1.ApplicationSpec{
Source: argoappsv1.ApplicationSource{
Source: &argoappsv1.ApplicationSource{
Path: "",
RepoURL: "",
TargetRevision: "",
Expand Down Expand Up @@ -220,7 +220,7 @@ func TestRenderTemplateParamsGoTemplate(t *testing.T) {
Namespace: "default",
},
Spec: argoappsv1.ApplicationSpec{
Source: argoappsv1.ApplicationSource{
Source: &argoappsv1.ApplicationSource{
Path: "",
RepoURL: "",
TargetRevision: "",
Expand Down Expand Up @@ -511,7 +511,7 @@ func TestRenderTemplateParamsFinalizers(t *testing.T) {

emptyApplication := &argoappsv1.Application{
Spec: argoappsv1.ApplicationSpec{
Source: argoappsv1.ApplicationSource{
Source: &argoappsv1.ApplicationSource{
Path: "",
RepoURL: "",
TargetRevision: "",
Expand Down
74 changes: 74 additions & 0 deletions assets/swagger.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5831,6 +5831,10 @@
"plugin": {
"$ref": "#/definitions/v1alpha1ApplicationSourcePlugin"
},
"ref": {
"description": "Ref is reference to another source within sources field. This field will not be used if used with a `source` tag.",
"type": "string"
},
"repoURL": {
"type": "string",
"title": "RepoURL is the URL to the repository (Git or Helm) that contains the application manifests"
Expand Down Expand Up @@ -6068,6 +6072,13 @@
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sources": {
"type": "array",
"title": "Sources is a reference to the location of the application's manifests or chart",
"items": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
}
},
"syncPolicy": {
"$ref": "#/definitions/v1alpha1SyncPolicy"
}
Expand Down Expand Up @@ -6118,6 +6129,13 @@
"type": "string",
"title": "SourceType specifies the type of this application"
},
"sourceTypes": {
"type": "array",
"title": "SourceTypes specifies the type of the sources included in the application",
"items": {
"type": "string"
}
},
"summary": {
"$ref": "#/definitions/v1alpha1ApplicationSummary"
},
Expand Down Expand Up @@ -6415,6 +6433,13 @@
},
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sources": {
"type": "array",
"title": "Sources is a reference to the application's multiple sources used for comparison",
"items": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
}
}
}
},
Expand Down Expand Up @@ -7757,8 +7782,22 @@
"type": "string",
"title": "Revision holds the revision the sync was performed against"
},
"revisions": {
"type": "array",
"title": "Revisions holds the revision of each source in sources field the sync was performed against",
"items": {
"type": "string"
}
},
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sources": {
"type": "array",
"title": "Sources is a reference to the application sources used for the sync operation",
"items": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
}
}
}
},
Expand Down Expand Up @@ -8059,9 +8098,23 @@
"description": "Revision is the revision (Git) or chart version (Helm) which to sync the application to\nIf omitted, will use the revision specified in app spec.",
"type": "string"
},
"revisions": {
"description": "Revisions is the list of revision (Git) or chart version (Helm) which to sync each source in sources field for the application to\nIf omitted, will use the revision specified in app spec.",
"type": "array",
"items": {
"type": "string"
}
},
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sources": {
"type": "array",
"title": "Sources overrides the source definition set in the application.\nThis is typically set in a Rollback operation and is nil during a Sync operation",
"items": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
}
},
"syncOptions": {
"type": "array",
"title": "SyncOptions provide per-sync sync-options, e.g. Validate=false",
Expand Down Expand Up @@ -8107,8 +8160,22 @@
"type": "string",
"title": "Revision holds the revision this sync operation was performed to"
},
"revisions": {
"type": "array",
"title": "Revisions holds the revision this sync operation was performed for respective indexed source in sources field",
"items": {
"type": "string"
}
},
"source": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
},
"sources": {
"type": "array",
"title": "Source records the application source information of the sync, used for comparing auto-sync",
"items": {
"$ref": "#/definitions/v1alpha1ApplicationSource"
}
}
}
},
Expand Down Expand Up @@ -8163,6 +8230,13 @@
"type": "string",
"title": "Revision contains information about the revision the comparison has been performed to"
},
"revisions": {
"type": "array",
"title": "Revisions contains information about the revisions of multiple sources the comparison has been performed to",
"items": {
"type": "string"
}
},
"status": {
"type": "string",
"title": "Status is the sync state of the comparison"
Expand Down
7 changes: 6 additions & 1 deletion cmd/argocd/commands/admin/app.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -401,7 +401,12 @@ func reconcileApplications(
return nil, err
}

res := appStateManager.CompareAppState(&app, proj, app.Spec.Source.TargetRevision, app.Spec.Source, false, false, nil)
sources := make([]v1alpha1.ApplicationSource, 0)
revisions := make([]string, 0)
sources = append(sources, app.Spec.GetSource())
revisions = append(revisions, app.Spec.GetSource().TargetRevision)

res := appStateManager.CompareAppState(&app, proj, revisions, sources, false, false, nil, false)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though CompareAppState target revisions and sources as slices, looks like we're only passing in the first target revision and source. Is that intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we are sending only the first source to CompareAppState from the cmd as we are not supporting CLI in this PR for multi-source apps.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly worried about this. If someone is using this command in a pipeline, this change could be silently-breaking for them. Can we just throw an error if the command encounters a multi-source app?

items = append(items, appReconcileResult{
Name: app.Name,
Conditions: app.Status.Conditions,
Expand Down
1 change: 1 addition & 0 deletions cmd/argocd/commands/admin/app_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ func TestGetReconcileResults_Refresh(t *testing.T) {
Namespace: "default",
},
Spec: v1alpha1.ApplicationSpec{
Source: &v1alpha1.ApplicationSource{},
Project: "default",
Destination: v1alpha1.ApplicationDestination{
Server: v1alpha1.KubernetesInternalAPIServerAddr,
Expand Down