Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for builder-jammy-full, and made it default #2659

Merged
merged 4 commits into from
Oct 20, 2023

Conversation

andreas-kupries
Copy link
Contributor

@andreas-kupries andreas-kupries commented Oct 16, 2023

@andreas-kupries
Copy link
Contributor Author

Note: rails and wordpress require updates to work ...

rails simply goes 404.

wordpress in apps, cli-apps does not stage:

buildpack       BP_PHP_VERSION -> "8.0.x" 
  🕞  [stage-namespace-793386-f2b1fc7eca946581b893e034f6f927936b26dfbl] buildpack       <unknown>      -> "" 
  🕞  [stage-namespace-793386-f2b1fc7eca946581b893e034f6f927936b26dfbl] buildpack  
  🕞  [stage-namespace-793386-f2b1fc7eca946581b893e034f6f927936b26dfbl] buildpack failed to satisfy "php" dependency version constraint "8.0.x": no compatible versions on "io.buildpacks.stacks.jammy" stack. Supported versions are: [8.1.23, 8.1.24, 8.2.10, 8.2.11] 

@codecov
Copy link

codecov bot commented Oct 18, 2023

Codecov Report

Attention: 36 lines in your changes are missing coverage. Please review.

Files Coverage Δ
internal/api/v1/info.go 76.00% <100.00%> (+7.03%) ⬆️
internal/cli/server.go 84.37% <100.00%> (+0.50%) ⬆️
internal/helmchart/constants.go 100.00% <ø> (ø)
internal/api/v1/application/stage.go 75.09% <75.83%> (-1.30%) ⬇️

... and 18 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

note: dropped hard-coded user/group information.
note: staging scripts to use based on chosen builder image (glob matching, details in reference docs)
chore: switched to builder-jammy-full as default builder.
note: The old bionic based builder is still supported, via a custom set of staging scripts.
chore: updated testsuite (wordpress, rails require bionic builder for now)
chore: extended action tracing in server and scripts
chore: factored assembly of stage env into separate function to avoid `funlen` warning on `newJobRun`.
Copy link
Member

@enrichman enrichman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me! Pretty nice. I would just make a couple of small refactoring:

since the staging script ConfigMap contains a bit of information it will be probably easier to handle it as a small struct. It a bit error prone to use the Data["xxx"] because you could get some errors with just a typo, and also you have to know/remember the type that you are getting to cast it properly (such the uid/guid).

So I would create something like this with a constructor:

type StagingScriptConfig struct {
	UserID        int
	GroupID       int
	Base          string
	DownloadImage string
	UnpackImage   string
	Builder       string
	BuilderImage  string
}

func NewStagingScriptConfig(config *v1.ConfigMap) (*StagingScriptConfig, error) {
	stagingScript := &StagingScriptConfig{
		Base: config.Data["base"],
		// ...
	}

	groupID, err := strconv.ParseInt(config.Data["groupID"], 10, 64)
	if err != nil {
		return nil, err
	}
	stagingScript.GroupID = int(groupID)

	// ...

	return stagingScript, nil
}

and as a personal preference I would also move out the fallback check.
It will do one more loop in case of a missing match, but it's much more readable (and it should be just a rare case). Also performance-wise I don't think it makes any difference.

	stagingScripts := []*StagingScriptConfig{}
	for _, configmap := range configmapList.Items {
		stagingScript, err := NewStagingScriptConfig(configMap)
		if err != nil {
			return nil, err
		}
		stagingScripts = append(stagingScripts, stagingScript)

		// check for matching staging script
		matched, err := filepath.Match(pattern, stagingScript.Builder)
		if err != nil {
			return nil, err
		}
		if matched {
			logger.Info("locate staging scripts - match", "name", configmap.Name, "for", pattern, "builder", builder)
			return stagingScript, nil
		}
	}

	// no matches, find fallback
	for _, stagingScript := range stagingScripts {
		if stagingScript.Builder == "*" || stagingScript.Builder == "" {
			logger.Info("locate staging scripts - using fallback", "name", fallback.Name)
			return stagingScript, nil
		}
	}

	return nil, fmt.Errorf("no matches, no fallback")

@andreas-kupries
Copy link
Contributor Author

That small comment needs quite a bit of code work ...

Still, most of it is ok and will work on that.

The search logic of yours however is semantically different.
If the fallback is marked with * it will never run the second loop because it will match in the first.
Thing is, I am unsure about the order I get the configmaps in, and I really want the fallback to be effectively at the end of the list.
Thus why I simply stash it when found in my loop, and used only when I leave my loop with nothing matched.
In your code I fear that the * fallback can match before and thus preempt a more specific pattern.
Which is not desirable.

…lly.

this simplified the resolution of `base` references too.
search kept as-is
@andreas-kupries
Copy link
Contributor Author

Comment addressed with commit fb2c9b0.
An analogous commit in docs documents the deterministic search order.

Copy link
Member

@enrichman enrichman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small comment.

internal/api/v1/application/stage.go Show resolved Hide resolved
more tracing (full set of specs, explicit note about base/inheritance)
@andreas-kupries andreas-kupries merged commit c1b8d50 into main Oct 20, 2023
17 checks passed
@andreas-kupries andreas-kupries deleted the 2153-jammy branch October 20, 2023 11:59
@mmartin24
Copy link
Contributor

mmartin24 commented Nov 21, 2023

Checked in:

Platform: k3s
Kubernetes Version: v1.26.10+k3s2
Epinio Server Version: v1.11.0-rc1
Epinio Client Version: v1.11.0-rc1
OIDC enabled: true

Check 1 --builder-image left to blank points to paketobuildpacks/builder-jammy-full:0.3.290

epinio push -n sample1 -p /home/mmartin/suse/epinio-repos/epinio/assets/sample-app      

App show output:

✔️  Details:
|         KEY          |                                       VALUE                                        |
|----------------------|------------------------------------------------------------------------------------|
| Origin               | /home/mmartin/suse/epinio-repos/epinio/assets/sample-app                           |
| Created              | 2023-11-21 13:58:27 +0100 CET                                                      |
| Status               | 1/1                                                                                |
| Username             | admin                                                                              |
| Running StageId      | 79f8736037adc003                                                                   |
| Last StageId         | 79f8736037adc003                                                                   |
| Age                  | 31s                                                                                |
| Internal Route       | rsample1-c37bca4afb8ff7f52f450b04c1973f37dfde48db.workspace.svc.cluster.local:8080 |
| Active Routes        |                                                                                    |
|                      | sample1.192.168.16.3.omg.howdoi.website                                            |
| App Chart            | standard                                                                           |
| Builder Image        | paketobuildpacks/builder-jammy-full:0.3.290                                        |

Check 2 builder image paketobuildpacks/builder-jammy-full:latest works with correct environment var

Fails:

epinio push --builder-image 'paketobuildpacks/builder-jammy-full:latest' -n sample2 -p /home/mmartin/suse/epinio-repos/epinio/assets/sample-app    

Works:

epinio push -n cat -p /home/mmartin/suse/epinio-repos/epinio/assets/sample-app --builder-image paketobuildpacks/builder-jammy-full:latest --env CNB_PLATFORM_API=0.7

Note: opened separate issue here to address this: #2715

Check 3 -Bionic builder full image works

epinio push --builder-image 'paketobuildpacks/builder:full' -n sampleapp-bionic-full -p /home/mmartin/suse/epinio-repos/epinio/assets/sample-app

App show output:

  Details:
|         KEY          |                                              VALUE                                               |
|----------------------|--------------------------------------------------------------------------------------------------|
| Origin               | /home/mmartin/suse/epinio-repos/epinio/assets/sample-app                                         |
| Created              | 2023-11-21 15:18:27 +0100 CET                                                                    |
| Status               | 1/1                                                                                              |
| Username             | admin                                                                                            |
| Running StageId      | 8b0788238d9cc81b                                                                                 |
| Last StageId         | 8b0788238d9cc81b                                                                                 |
| Age                  | 3m10s                                                                                            |
| Internal Route       | rsampleapp-bionic-full-c0e16a3f585894b2a01bb8a4005e53c7b7ff2f45.workspace.svc.cluster.local:8080 |
| Active Routes        |                                                                                                  |
|                      | sampleapp-bionic-full.192.168.16.3.omg.howdoi.website                                            |
| App Chart            | standard                                                                                         |
| Builder Image        | paketobuildpacks/builder:full                                                                    |
| Desired Instances    |                                                                                                1 |
| Bound Configurations |                                                                                                  |
| Environment          |                                                                                                  |
| Chart Values         |                                                                                                  |

Check 4 - Wordpress app works

# Create mysql-svc
epinio service create mysql-dev mycustom-service


# Push wordpress
epinio push --name testapp-wordpress-git --git https://github.com/epinio/example-wordpress  -e BP_PHP_VERSION=8.1.x -e BP_PHP_SERVER=nginx -e BP_PHP_WEB_DIR=wordpress -e DB_HOST=x8e5ee833a0f2faebaf5c4171baca-mysql  -e SERVICE_NAME=mycustom-service

# Bind it
epinio service bind mycustom-service testapp-wordpress

Output curl checking html:

curl -kL "https://testapp-wordpress-git.192.168.16.3.omg.howdoi.website" | head -10

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
<!DOCTYPE html>
1<html lang="en-US" xml:lang="en-US">
0<head>
0	<meta name="viewport" content="width=device-width" />
 	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 	<meta name="robots" content="noindex,nofollow" />
9	<title>WordPress &rsaquo; Installation</title>
84	<link rel='stylesheet' id='dashicons-css'  href='https://testapp-wordpress-git.192.168.16.3.omg.howdoi.website/wp-includes/css/dashicons.min.css?ver=5.7.2' type='text/css' media='all' />
7<link rel='stylesheet' id='buttons-css'  href='https://testapp-wordpress-git.192.168.16.3.omg.howdoi.website/wp-includes/css/buttons.min.css?ver=5.7.2' type='text/css' media='all' />
 <link rel='stylesheet' id='forms-css'  href='https://testapp-wordpress-git.192.168.16.3.omg.howdoi.website/wp-admin/css/forms.min.css?ver=5.7.2' type='text/css' media='all' />
100  9847    0  9847    0     0  59336      0 --:--:-- --:--:-- --:--:--     0

App show output:

✔️  Details:
|         KEY          |                                              VALUE                                               |
|----------------------|--------------------------------------------------------------------------------------------------|
| Origin               | https://github.com/epinio/example-wordpress                                                      |
|                      | @ fe70da3e5b884115261cb8672342370802636023                                                       |
|                      | (on main)                                                                                        |
| Created              | 2023-11-21 16:13:44 +0100 CET                                                                    |
| Status               | 1/1                                                                                              |
| Username             | admin                                                                                            |
| Running StageId      | b7d38e75c69fcfe0                                                                                 |
| Last StageId         | b7d38e75c69fcfe0                                                                                 |
| Age                  | 5m44s                                                                                            |
| Internal Route       | rtestapp-wordpress-git-16d535b42e4795917616be64d11554f4950e0a2b.workspace.svc.cluster.local:8080 |
| Active Routes        |                                                                                                  |
|                      | testapp-wordpress-git.192.168.16.3.omg.howdoi.website                                            |
| App Chart            | standard                                                                                         |
| Builder Image        | paketobuildpacks/builder-jammy-full:0.3.290                                                      |
| Desired Instances    |                                                                                                1 |
| Bound Configurations | x8e5ee833a0f2faebaf5c4171baca-mysql                                                              |
| Environment          |                                                                                                  |
|   - BP_PHP_SERVER    | nginx                                                                                            |
|   - BP_PHP_VERSION   | 8.1.x                                                                                            |
|   - BP_PHP_WEB_DIR   | wordpress                                                                                        |
|   - DB_HOST          | x8e5ee833a0f2faebaf5c4171baca-mysql                                                              |
|   - SERVICE_NAME     | mycustom-service                                                                                 |
| Chart Values         |                                                                                                  |

@andreas-kupries andreas-kupries changed the title Support builder-jammy-full Added support for builder-jammy-full, and made it default Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants