GetSchemaVersionRequest in AWSSchemaRegistryClient does not specify a registry name

[dexter-mh-lee]

Hi.

I've been trying to figure out the minimum permissions required to set up Glue schema registry based serialization/deserialization (avro) as we consume/produce events to kafka.

Realized through line by line debugging that the getSchemaVersionRequest does not specify a registry name (https://github.com/awslabs/aws-glue-schema-registry/blob/master/common/src/main/java/com/amazonaws/services/schemaregistry/common/AWSSchemaRegistryClient.java#L222). This requires me to add the following wide permissions for my application to work.

        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "glue:GetSchemaVersion"
            ],
            "Resource": [
                "*"
            ]
        }

I would like to instead just give such permission to the specific registry I am using right now.

Is it possible to specify a registry name in this request?

[mohitpali]

Edited - Reviewing the document now, it says that "You can limit the registries that can read by using the Resource clause. But if access to all registries is required then it can be achieved by specifying "*" for the appropriate portions of the ARN."

https://docs.aws.amazon.com/glue/latest/dg/schema-registry-gs.html#schema-registry-gs1b

        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "glue:GetSchemaVersion"
            ],
            "Resource": [
               "arn:aws:glue:aws-region:123456789012:registry/registryname-1"
            ]
        }

Is specifying registry arn not working ?

[dexter-mh-lee]

Is this something that could be fixed? Why does getSchemaVersion require access to everything?

[mohitpali]

Oops, updated my comment after reviewing the document closely.

[dexter-mh-lee]

No. So every other permissions, I can set to only point to the specific registry, but for getSchemaVersion, it throws an authentication error if I set the registry name.

[mohitpali]

Can you please paste the error here ? I will check my resources and will get back on this.

[dexter-mh-lee]

So initially used the following policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "glue:GetRegistry",
                "glue:CreateSchema",
                "glue:UpdateSchema",
                "glue:GetSchema",
                "glue:ListSchemas",
                "glue:RegisterSchemaVersion",
                "glue:GetSchemaByDefinition",
                "glue:GetSchemaVersion",
                "glue:GetSchemaVersionsDiff",
                "glue:ListSchemaVersions",
                "glue:CheckSchemaVersionValidity",
                "glue:PutSchemaVersionMetadata"
            ],
            "Resource": [
                "arn:aws:glue:*:---------:schema/*",
                "arn:aws:glue:us-west-2:---------:registry/demo-shared"
            ]
        }
    ]
}

I get the following error
Caused by: software.amazon.awssdk.services.glue.model.AccessDeniedException: User: arn:aws:sts::---------:assumed-role/EKSWorkerNodeIAM-demo/i-0672c1632cc37421e is not authorized to perform: glue:GetSchemaVersion (Service: Glue, Status Code: 400, Request ID: fb1de2eb-6b86-4f37-b13c-1b700cc2443f, Extended Request ID: null)

[dexter-mh-lee]

Once I change it to

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "glue:GetRegistry",
                "glue:CreateSchema",
                "glue:UpdateSchema",
                "glue:GetSchema",
                "glue:ListSchemas",
                "glue:RegisterSchemaVersion",
                "glue:GetSchemaByDefinition",
                "glue:GetSchemaVersion",
                "glue:GetSchemaVersionsDiff",
                "glue:ListSchemaVersions",
                "glue:CheckSchemaVersionValidity",
                "glue:PutSchemaVersionMetadata"
            ],
            "Resource": [
                "arn:aws:glue:*:----------:schema/*",
                "arn:aws:glue:us-west-2:----------:registry/demo-shared"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "glue:GetSchemaVersion"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

it works

[mohitpali]

I confirmed within the team that this is a known issue. We will look into this and will require some work.

[dexter-mh-lee]

Thanks a lot!

[dexter-mh-lee]

@mohitpali Any updates on this one?

[mohitpali]

We are looking into this and unfortunately this would require some work. We will work on prioritizing this but we don't have a timeline as of now. As a workaround, you could continue using * permissions.

[fr-sano]

@mohitpali
Actually, I faced the same problem.
My company doesn't allow me to use * permissions.

Can you take a look at this patch?
The basic idea and the actual modifications are so simple.
In this patch, when we specify the registry name and the schema name for the consumer side as we do for the producer side, instead of specifying the schema version id for GetSchemaVersion, it specifies the registry name, the schema name and the version number for it, after detecting the version number using ListSchemaVersions.
We can avoid using the full access privilege, by doing like this.
I hope you can consider to apply this once.

0001-Don-t-use-GetSchemaVersion-with-the-schema-version-i.patch.gz

[fr-sano]

Actually I came up with another idea to include the version number in the Kafka messages.
If it's possible it might be better from the performance perspective.
But we cannot do it because GetSchemaByDefinition doesn't return the version number, though I don't know why.

[blacktooth]

Sorry we couldn't get to this, we will prioritize this against our existing backlog items.

[jonsinfinity1]

I'm experiencing the same behavior. Is this issue still not resolved?

[rupeshmore85]

We are experiencing same issue! Unfortunately we cannot specify * resource

[AmsterdamFilho]

+1

[srikanthvpai]

+1

[ciandt-andregc]

+1

[amcquistan]

Seriously this is needed. I thought AWS was a security first organization? Simply telling customers to use * for reading any and all schemas / data contracts isn't an acceptable solution

[chriline]

+1

[kothapet]

Wow still not fixed after 3+ years and its security issue.. Anyways issues here are very helpful when other folks run into the same issue.
thanks @dexter-mh-lee.

[Tr1monster]

+1 this should be fixed

[polster]

Hey guys - same problem encountered during integration of the Glue Schema Registry, where we wanted to apply least privilege for the job principle...

Exception:

Caused by: software.amazon.awssdk.services.glue.model.AccessDeniedException: User: arn:aws:iam::XXXXXXXX:user/YYYYYYY is not authorized to perform: glue:GetSchemaVersion because no identity-based policy allows the glue:GetSchemaVersion action

Only workaround so far (terraform in our case):
statement { actions = [ "glue:GetSchemaVersion" ] resources = [ "*" ] }

As mentioned above already, we would be more than happy if this can be fixed may be in 2024 😄