Error("missing field `identity_id`", ...) when invoking with temporary credentials

[m-mueller678]

I ran into an issue trying to invoke a lambda function from the aws CLI using temporary credentials. The function apperas to fail in deserialization before running my handler.

What I did

This is the CLI call. The credentials were obtained from GetCredentialsForIdentity via the rust sdk. The only login passed was "cognito-idp.eu-west-1.amazonaws.com/eu-west-1_pjUAfaskc" along with the id_token. The credentials have since expired.

AWS_ACCESS_KEY_ID=ASIATCLJQXJNHNIM7LHF AWS_SECRET_ACCESS_KEY=P5XBfU0cJrKQKWvv7bkZbgWKnC+ssAhaBLb64oj9  AWS_SESSION_TOKEN=IQoJb3JpZ2luX2VjEGMaCWV1LXdlc3QtMSJIMEYCIQDglPA69qBrUH/GFoXcWGXtFlc5eVO8tV30jdmrH2Ju/QIhAOG+9wH9Rfr+xpxiPe4OIFmiXTd9ebAXYgZ+hM4wNXHFKsQECEsQAhoMMjExMjExNTY1NjU4Igy+ddKh41qJRLiuvRwqoQSlC8DtZtamNQDY4SfFS2vyMCIcTw9/Bn/holfgxD4OxjLAjJA6MsC8P8b97yX3j5QMogPh5yHrszEVRaSkrELIoHnuGRUH1gGC7XtC2ssCW8nIWNBvpnU2W1L7lA2wzrTs+qD7TaR4Iy3pOub5AAOZZZLVN0Ux2pcKYP1ESopfikOyAvctUeXoh29rTOilQ4YebIJg1KOJHB2pr6Uf6ZzKMtFiSM2HbgObeceee77NNXSdXq4BWga0YmsMiFg9E1dPPxWYL0IHotv4jXTnk3uG1621jiqJwy8K6IuoyzzBoYD5df4cxkd79BC4GdBE699vE01D4aBBseekje7SOOAj1NvM6IiukkO0V7p22ouiia2I3Pa+03krJIMouSEOQ3rQX+vzEej9bDl4KCigIbxXVOanRR39bZkd8z4K2iP6ICsagmiU/FXXXDdK3K7jbB3DIBZdGoxqoA+f8Kuj5oheOeIjJU7r8v9m35LY49An6sX2tmHojlpWZby4eHjkgi2fpLCEtvW/juDvAlRSsEEOu9U3mjnUh9XO1cATB0Y3zrMC7c/4hYylc8+NdlynWB+syaZXfn6fgpsl+q8x6HvnYqzj+ZqE6jmBi9WNMg30I4uziZILMnwS9pvFA9UumZtcmZMcR4oWX5uQQLHYdLfT8/rpSZYZr0W9uUweM+/mDnK9/7h/fSsrbiBOxoYlHrjie4Y2hr8XM7jvAgWo0okTYjC93vWaBjqEAvhwcikOoQDoU4x+FQEQxgcIimHzh6KDWBmrlUFqq3d7Uf62D5gCYx2Lw/nGp2NqEAoUIDQlTBETm9GhigPPqsHlC0eY/0X/vJAxGsILYC2npOZF2nUMM7FLeLfidn5OG88JYIhghs9DUBdWSV6FCu5jiJGM78+uWRKDYmgcv3D7n2+sE1PIWH2jg9G0uBl1QztdD2KhWh8LFcMhv1LQasXBGRkOsex8zIrCs12L7hWywfvHjB7UZMunRK5RtM3UKFvs1FGGt2PhuCp71H5kqFSWokwdZyOE63wa++pcmmnDmKa0aBVLsFpisccP6A5F99Km0Ccll9opIManYM/AD5xNYODf aws lambda invoke --function-name archiver-server out-file --region eu-west-1 --log-type Tail

this is what I got back:

{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "LogResult": "U1RBUlQgUmVxdWVzdElkOiBhYmFkYWExYy00YTZmLTQ3MGYtYTE5Ny1iOWJhZWM0NjJlOGIgVmVyc2lvbjogJExBVEVTVApFcnJvcjogRXJyb3IoIm1pc3NpbmcgZmllbGQgYGlkZW50aXR5X2lkYCIsIGxpbmU6IDEsIGNvbHVtbjogMTQzKQpSZXF1ZXN0SWQ6IGFiYWRhYTFjLTRhNmYtNDcwZi1hMTk3LWI5YmFlYzQ2MmU4YiBFcnJvcjogUnVudGltZSBleGl0ZWQgd2l0aCBlcnJvcjogZXhpdCBzdGF0dXMgMQpSdW50aW1lLkV4aXRFcnJvcgpFTkQgUmVxdWVzdElkOiBhYmFkYWExYy00YTZmLTQ3MGYtYTE5Ny1iOWJhZWM0NjJlOGIKUkVQT1JUIFJlcXVlc3RJZDogYWJhZGFhMWMtNGE2Zi00NzBmLWExOTctYjliYWVjNDYyZThiCUR1cmF0aW9uOiA5LjA4IG1zCUJpbGxlZCBEdXJhdGlvbjogMTAgbXMJTWVtb3J5IFNpemU6IDEyOCBNQglNYXggTWVtb3J5IFVzZWQ6IDUgTUIJCg==",
    "ExecutedVersion": "$LATEST"
}

base64 decoding the log shows there's some deserialization issue. Notably the handler function never gets to run.

START RequestId: abadaa1c-4a6f-470f-a197-b9baec462e8b Version: $LATEST
Error: Error("missing field `identity_id`", line: 1, column: 143)
RequestId: abadaa1c-4a6f-470f-a197-b9baec462e8b Error: Runtime exited with error: exit status 1
Runtime.ExitError
END RequestId: abadaa1c-4a6f-470f-a197-b9baec462e8b
REPORT RequestId: abadaa1c-4a6f-470f-a197-b9baec462e8b	Duration: 9.08 ms	Billed Duration: 10 ms	Memory Size: 128 MB	Max Memory Used: 5 MB	

The code

This is the lambda code:
main.rs

use lambda_runtime::{service_fn, LambdaEvent, Error};
use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt().with_max_level(tracing::Level::TRACE).init();
    let func = service_fn(func);
    lambda_runtime::run(func).await?;
    Ok(())
}

async fn func(event: LambdaEvent<serde_json::Value>) -> Result<Value, Error> {
    println!("invoked stdout!");
    eprintln!("invoked stderr!");
    Ok(json!({ "value": event.payload ,"identity": event.context.identity }))
}

Cargo.toml

[package]
name = "archiver-server"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
lambda_http = "0.7"
lambda_runtime = "0.7"
tokio = { version = "1", features = ["macros"] }
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3.16", default-features = false, features = ["fmt"] }
serde_json = "1.0.87"

relevant section from Cargo.lock

[[package]]
name = "lambda_runtime"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b64c1a62e7f43f7c3aed77806c182a338acbed3d95995380d6a9c1eb8650767"

[calavera]

According to the documentation, the identity_id is required in every request, but it looks like the SDK doesn't enforce the requirement and it's allowing you to send the request without it:

[IdentityId](https://docs.aws.amazon.com/cognitoidentity/latest/APIReference/API_GetCredentialsForIdentity.html#API_GetCredentialsForIdentity_RequestSyntax)

    A unique identifier in the format REGION:GUID.

    Type: String

    Length Constraints: Minimum length of 1. Maximum length of 55.

    Pattern: [\w-]+:[0-9a-f-]+

    Required: Yes

Unless I'm mistaken, you should probably open an issue in the SDK repo to let the maintainers know, and set that identity_id field in the SDK call.

[m-mueller678]

I do specify the identity id to the GetCredentialsForIdentity call. And I do obtain valid credentials. I just tried using the same AWS CLI command (with the same credentials) to call a Node.js hello world lambda, and that does work.

For completeness, this is how I get the credentials:

#[derive(Debug)]
struct CognitoCredentialsProvider;

impl ProvideCredentials for CognitoCredentialsProvider {
    fn provide_credentials<'a>(&'a self) -> aws_types::credentials::future::ProvideCredentials<'a> where Self: 'a {
        aws_types::credentials::future::ProvideCredentials::new(async {
            let auth:OauthTokenResponse = auth().await.map_err(CredentialsError::unhandled)?;
            let config = config(None).await;
            let client = aws_sdk_cognitoidentity::Client::new(&config);
            let get_id = client.get_id()
                .identity_pool_id(IDENTITY_POOL_ID)
                .logins(COGNITO_LOGINS_KEY, auth.extra_fields().id_token.secret())
                .send().await.map_err(CredentialsError::unhandled)?;
            let identity_id = get_id.identity_id().ok_or_else(|| CredentialsError::unhandled("aws returned no identity"))?;
            let credentials = client.get_credentials_for_identity()
                .logins(COGNITO_LOGINS_KEY, auth.extra_fields().id_token.secret())
                .identity_id(dbg!(identity_id))
                .send().await.map_err(CredentialsError::unhandled)?;
            let credentials = credentials.credentials().ok_or_else(|| CredentialsError::unhandled("aws returned no credentials for identity"))?.clone();
            let expiration = credentials.expiration().ok_or_else(|| CredentialsError::unhandled("missing expiration"))?;
            let expiration = SystemTime::try_from(*expiration).map_err(|e| CredentialsError::unhandled(e))?;
            Ok(
                dbg!(Credentials::new(
                    credentials.access_key_id().ok_or_else(|| CredentialsError::unhandled("missing access key"))?,
                    dbg!(credentials.secret_key().ok_or_else(|| CredentialsError::unhandled("missing secret key"))?),
                    dbg!(Some(credentials.session_token().ok_or_else(|| CredentialsError::unhandled("missing session token"))?.into())),
                    Some(expiration),
                    "CognitoOauth",
                ))
            )
        })
    }
}

where auth is obtained from https://archiver.auth.eu-west-1.amazoncognito.com/oauth2/token

[calavera]

I just tried using the same AWS CLI command (with the same credentials) to call a Node.js hello world lambda, and that does work.

Can you print the context.identity inside this function call to see if the identity_id field includes any information? from the error in the description, the payload doesn't have that information. My guess is that the Node runtime just ignores the lack of data and sets that field to null or undefined.

The question is, is that field optional, and therefore we should set it as so or set a default, or there is something else going on?

[m-mueller678]

Not quite sure what function call you're referring to, I'll assume the node.js lambda.
Here It says identity is for mobile apps, which confuses me a bit.
Anyway, context.identity is

{
  "cognitoIdentityId":"eu-west-1:ed162e0f-2fd7-4ef7-9e41-aab400f34d88",
  "cognitoIdentityPoolId":"eu-west-1:eb55aa58-021a-4d13-b084-72a7eff35f48"
}

[calavera]

Thanks for the example @m-mueller678 , that's exactly what we needed. It looks like we're not parsing the identity correctly because we expect identityId, not cognitoIdentityId. That should be an easy fix.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for the maintainers of this repository to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.