Fix whitespace handling with gcr service accounts in mind. #221

jacobstr · 2020-05-29T22:37:16Z

At some point in time I followed the instructions here to generate a static config.json using a service account authenticated against GCR for use in our CI flows.

The encoded password is actually json and the embedded ":" causes it to be split across multiple array elements. That's undesirable in this case and ends up getting manifest as:

FATA[0004] Bad response status: 401 Unauthorized >>

Because the config loader ends up skipping the element in https://github.com/ivanilves/lstags/blob/master/docker/config/config.go#L108

jacobstr · 2020-05-29T22:50:50Z

Note: #211 suggests using this exact strategy. Folks attempting that will surely run into this caveat.

When encoding json service accounts, the base64 encoded json may contain newlines. These resulted in > 2 elements in the strings.Split. This modification makes the function resilient to differences whitespace within the embedded service account.json. --- drive-by fix to error string generation We're using a formatting string, but not the `f` variant of `Sprint`. --- address linter complaint -> use fmt.Errorf

jacobstr · 2020-05-29T23:03:25Z

Looks like there's a bit more to this still. Going to close an re-open.

The approach of base64 encoding the json-like structure appears to work fine. The shape of the moving parts (username, password) was largely reverse engineered by: 1. Generating a service account with gcr access and saving to `key.json`. 2. `cat key.json | docker login -u _json_key --password-stdin https://gcr.io` 3. `cat `~/.docker/config.json` should now contain somethin like:``` "gcr.io" : { "auth" : "..." } The password should typically itself be a json-encoded service account i.e. the contents of `key.json` above. We've replaced the parent commits sprintf json-encoding by using the upstream types. Note: we're panicking on the error here vs. propagating it. This is partially laziness, the perceived unlikihood, and finally the desire to keep this change set as straightforward as possible. ```

jacobstr · 2020-06-04T20:28:02Z

docker/config/config_test.go

 	"testing"
 )

 var configFile = "../../fixtures/docker/config.json"

 func TestGetRegistryAuth(t *testing.T) {
 	examples := map[string]string{
-		"registry.company.io":     "eyAidXNlcm5hbWUiOiAidXNlcjEiLCAicGFzc3dvcmQiOiAicGFzczEiIH0=",
-		"registry.hub.docker.com": "eyAidXNlcm5hbWUiOiAidXNlcjIiLCAicGFzc3dvcmQiOiAicGFzczIiIH0=",
+		"registry.company.io":     "eyJ1c2VybmFtZSI6Il9qc29uX2tleSIsInBhc3N3b3JkIjoicGFzczEifQ==",


The json encoding via json.Marshal strips some of the whitespace, resulting in this base64 difference.

jacobstr · 2020-06-04T20:30:05Z

docker/config/config.go

@@ -55,15 +55,19 @@ func (c *Config) GetCredentials(registry string) (string, string, bool) {
 }

 func getAuthJSONString(username, password string) string {
-	if username == "_json_key" {
-		return fmt.Sprintf("%s:%s", username, password)
+	b, err := json.Marshal(types.AuthConfig{


Considering using an anonymous struct here as well to reduce the dependencies, but I wanted to denote where the fields come from - it may assist future contributors in discovering what authn knobs are available.

jacobstr · 2020-06-04T20:32:29Z

docker/config/config.go

@@ -55,15 +55,19 @@ func (c *Config) GetCredentials(registry string) (string, string, bool) {
 }

 func getAuthJSONString(username, password string) string {
-	if username == "_json_key" {


I tried to grok this a bit - 005c3ac mentions that it was done to fix gcr authentication, but I could not get GCR working end to end until I made these changes.

It appeared to work initially - list operations against GCR succeeded. I saw subsequent push failures however which led to my original retraction of the MR.

jacobstr · 2020-06-04T20:33:44Z

docker/config/config.go

+	// happen. We preserve the non error-propagating API for callers, but want
+	// some visibility into this that's better than simply swallowing the error.
+	if err != nil {
+		panic(err)


Potentially a bit lazy here, but I didn't want to mess with your API's to much by changing the signature here.

Cool, if I see this panic popping out anywhere, I would change, but anyway, this should never happen. 🙂

jacobstr · 2020-06-04T20:35:43Z

docker/config/config.go

@@ -103,7 +107,7 @@ func Load(fileName string) (*Config, error) {
 		}

 		authenticationToken := string(b)
-		usernameAndPassword := strings.Split(authenticationToken, ":")
+		usernameAndPassword := strings.SplitN(authenticationToken, ":", 2)


This was the initial change I made in order to fix the decoding of the base64 data I gleaned from my ~/.docker/config.json. The commit message in e33c4e2 discusses how I replicated and tested this:

The shape of the moving parts (username, password) was largely reverse engineered by: 1. Generating a service account with gcr access and saving to `key.json`. 2. `cat key.json | docker login -u _json_key --password-stdin https://gcr.io` 3. `cat `~/.docker/config.json` should now contain somethin like:``` "gcr.io" : { "auth" : "..." }

ivanilves · 2020-06-05T10:01:35Z

Thank you for this contribution! I'll review it ASAP.

ivanilves · 2020-06-07T08:44:27Z

docker/config/config_test.go

 	examples := map[string]string{
 		"registry.company.io":     "user1:pass1",
 		"registry.hub.docker.com": "user2:pass2",
+		"us.gcr.io":               fmt.Sprintf("%s:%s", "_json_key", string(gcrJSONKey)),


jacobstr force-pushed the koobz/fix-gcr-sa branch from c4b119c to b447b8f Compare May 29, 2020 22:50

jacobstr force-pushed the koobz/fix-gcr-sa branch from b447b8f to a10732b Compare May 29, 2020 22:58

jacobstr closed this May 29, 2020

jacobstr reopened this Jun 3, 2020

jacobstr force-pushed the koobz/fix-gcr-sa branch 3 times, most recently from e349ad2 to 7a69f1f Compare June 4, 2020 18:00

jacobstr force-pushed the koobz/fix-gcr-sa branch from 7a69f1f to e33c4e2 Compare June 4, 2020 19:59

jacobstr commented Jun 4, 2020

View reviewed changes

ivanilves self-assigned this Jun 5, 2020

ivanilves added the enhancement label Jun 5, 2020

ivanilves reviewed Jun 7, 2020

View reviewed changes

ivanilves approved these changes Jun 7, 2020

View reviewed changes

ivanilves merged commit 8706a12 into ivanilves:master Jun 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix whitespace handling with gcr service accounts in mind. #221

Fix whitespace handling with gcr service accounts in mind. #221

jacobstr commented May 29, 2020 •

edited

Loading

jacobstr commented May 29, 2020

jacobstr commented May 29, 2020

jacobstr Jun 4, 2020

jacobstr Jun 4, 2020

jacobstr Jun 4, 2020 •

edited

Loading

jacobstr Jun 4, 2020

ivanilves Jun 7, 2020

jacobstr Jun 4, 2020

ivanilves commented Jun 5, 2020

ivanilves Jun 7, 2020

Fix whitespace handling with gcr service accounts in mind. #221

Fix whitespace handling with gcr service accounts in mind. #221

Conversation

jacobstr commented May 29, 2020 • edited Loading

jacobstr commented May 29, 2020

jacobstr commented May 29, 2020

jacobstr Jun 4, 2020

Choose a reason for hiding this comment

jacobstr Jun 4, 2020

Choose a reason for hiding this comment

jacobstr Jun 4, 2020 • edited Loading

Choose a reason for hiding this comment

jacobstr Jun 4, 2020

Choose a reason for hiding this comment

ivanilves Jun 7, 2020

Choose a reason for hiding this comment

jacobstr Jun 4, 2020

Choose a reason for hiding this comment

ivanilves commented Jun 5, 2020

ivanilves Jun 7, 2020

Choose a reason for hiding this comment

jacobstr commented May 29, 2020 •

edited

Loading

jacobstr Jun 4, 2020 •

edited

Loading