Error listing tasks, InvalidParameterException: ownerId longer than 36

[bryceray1121]

ecs-cli compose -p "production_prod-ecscpu1_alerts" -f "/data/application/workspaces/production_prod-ecscpu1_alerts/artisan-run-alerts.yml"

RuntimeError: b'time="2016-04-06T20:53:58Z" level=error msg="Error listing tasks" error="InvalidParameterException: ownerId longer than 36.\n\tstatus code: 400, request id: af2c9d53-fc39-11e5-8d9c-871efba49583" request={\n Cluster: "prod-ecscpu1",\n DesiredStatus: "RUNNING",\n StartedBy: "ecscompose-production_prod-ecscpu1_alerts"\n} \ntime="2016-04-06T20:53:58Z" level=fatal msg="InvalidParameterException: ownerId longer than 36.\n\tstatus code: 400, request id: af2c9d53-fc39-11e5-8d9c-871efba49583" \n',

Can't seem to get around this error and appears to be no documentation on what ownerId might be.

[uttarasridhar]

ecs-cli uses the "started-by" field to determine (and list) the tasks started by this compose project.
By default, it generates the string for started-by as "ecscompose-projectname", in this case that would be "ecscompose-production_prod-ecscpu1_alerts", which has 41 characters > 36 words.

With the v0.3.0 release, we have made the prefix for "started-by" configurable by supplying --compose-project-name-prefix to configure command.

Example:
You can specify an empty string (--compose-project-name-prefix "") with this option to omit the default prefix ("ecscompose-").

`ecs-cli configure --region us-west-2 --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --cluster ecs-cli --compose-project-name-prefix ""`

By setting it to empty, all the successive tasks started by ecs-cli compose would have started-by field as "project-name", which is "production_prod-ecscpu1_alerts"

You can find more information in the docs.

Hope that helps.

[alextes]

This limit just turned into a pretty nasty problem for us @uttarasridhar. These names have been in use for months before we needed the ecs-cli so changing them is painful (not that that's your fault!).

I see AWS ECS allows for 255 characters when setting a Task Definition Name (family). Is the 36 char limit then set by the ecs-cli? And if so why? I think there are very valid use-cases for longer names.

In case the thinking behind the limit is indeed that no task needs a longer name and you want to save bytes or something, let me shed some light on how our names are over 36 chars:

The core of the name is pretty much supplies-messaging-v0. It's name_of_app-name_of_sub_app-version. Since our app works using two protocols (HTTP, AMQP) it is a single codebase with two executables where about 70% of the code is shared. Hence the app, sub_app distinction. Add to that the cluster distinction (staging, production), and whether we're running the main task or its pre/post scripts (main, pre, post) and you end up with main-production-supplies-messaging-v0. 37 characters.

[yinshiua]

@alextes The reason why you are running into this issue is because the cli calls StartTask with the StartedBy field (StartedBy has a limit of 36 characters, see doc.

We need to set this StartedBy field because ecs-cli compose ps uses this tag to filter out tasks that are unrelated to the current project. To solve this problem, we will need to avoid setting the StartedBy field, but also find away to continue to support ecs-cli compose ps.

[baldwinsung]

@uttarasridhar thank you. used --compose-project-name-prefix "" . we are no longer having the StartedBy: errors.

[alextes]

@yinshiua thank you for linking the doc!

If a task is started by an Amazon ECS service, then the startedBy parameter contains the deployment ID of the service that starts it.

It sounds like I should have a chat with our friendly DevOps people to see if we can do some mapping of project-prefix + project-name to some ID we can pass to your CLI for use as the startedBy property. Is it correct to say that the way to deal with this limit is to still use project-prefix + project-name but drop semantic correctness and actually pass some identifier instead?

[jdno]

I'm the friendly DevOps person @alextes mentioned. ;)

The problem for us is that compose-project-name-prefix is used both for startedBy and the task definition's name. As @alextes described, we prefix our services and task definitions by their stage (staging, production) and the type of command they run (main, pre, post). By doing this, we get individual task definitions for the different environments and can better track changes to our services and their configuration. Task definition names can be as long as 255 characters, so for that it has worked really well. We are running into issues with a few of our services that have longer names, where this naming scheme conflicts with the 36 character limit for startedBy.

Looking at the documentation for ecs-cli configure, --compose-project-name-prefix only mentions the task definition:

Specifies the prefix to add to an ECS task definition that is registered from a compose file.

Intuitively, I would've expected a different parameter to set startedBy. With the different limits for task definitions and started-by, this makes the most sense to me.

The way we are using the ecs-cli is pretty hardcore and most likely far outside the indented scope of the project (it grew organically 😈 ). So far, we were successful in hacking around the limitations of the cli, but I expect us to hit a dead end sooner or later. I just need to understand if this is it...

Would it be possible to add a different parameter to ecs-cli configure, or are we the only ones affected by this issue?

[yinshiua]

@jdno Thanks for the detail explanation. I think this is a valid concern. We should not be limiting the length of Task Definition name because of limits in StartedBy.

We are actually thinking of deprecating the prefixes since most customers just set it to empty and other various reasons, so I would be opposed to add another parameter in the ecs-cli configure.

We can get around with this by either using the hashed Task Definition name in StartedBy or increase the limit in StartedBy. I will keep you posted on which direction we want to go here shortly.

[jdno]

Thanks a lot. 👍 Let me know if i can help.

[yinshiua]

@jdno We don't have plans to increase the limit for StartedBy, we can use the TaskDefinitionName hash for the StartedBy field. Feel free to take a crack at this. Thanks!