Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TRANSFORM_INVALID_GROUP_CHARS doesn't document valid group patterns #56930

Open
xarses opened this issue May 24, 2019 · 71 comments · May be fixed by #55474

Comments

@xarses
Copy link

commented May 24, 2019

SUMMARY

With the addition of the TRANSFORM_INVALID_GROUP_CHARS. Other than reading the source, it was not clear which characters must be avoided going forward, only that the warning (with -vvvv) points out which characters you currently use that are invalid.

Please clarify that you are pushing the names to be valid python vars. This is missing from the documentation for the cfg option, warning and online documentation

(d241794#diff-b77962b6b54a830ec373de0602918318R122)

There appears to be no mention of this on https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.8.html either.

ISSUE TYPE
  • Documentation Report
COMPONENT NAME

group

ANSIBLE VERSION
ansible 2.8.0
  config file = /home/awoodward/ansible-skynet/ansible.cfg
  configured module search path = [u'/home/awoodward/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.5 (default, Apr  9 2019, 14:30:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
CONFIGURATION

n/a

OS / ENVIRONMENT

n/a

ADDITIONAL INFORMATION

n/a

@ansibot

This comment has been minimized.

Copy link
Contributor

commented May 24, 2019

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

commented May 24, 2019

@danielmotaleite

This comment has been minimized.

Copy link

commented May 28, 2019

i started to get this warning but found no references in the porting guide and no references how or what to fix.

most of my warnings are from the ec2.py, where the instance_id used the - (eg: i-033f62b586143dff7) and the regions (eg: eu-central-1c) , so we have no real fix for this ones

Finally, this broke some of my playbooks, where i used when: ansible_hostname in groups['varnish'] and the ansible_hostname is varnish-eu-central-1c-001 .
In the past this worked fine, now i need to use inventory_hostname to get varnish_eu_central_1c_001 and get a match vs the groups['varnish']

So this needs at least and urgently a warning in the porting guide that inventory_hostname and groups[] may be returning different data

@bcoca bcoca self-assigned this May 28, 2019

@bcoca bcoca referenced a pull request that will close this issue May 28, 2019
@ssbarnea

This comment has been minimized.

Copy link
Contributor

commented May 31, 2019

What was the reasoning behind dropping the dash from group names? I really struggle to find good reason for that, especially as this will require lots of code refactoring.

@sivel

This comment has been minimized.

Copy link
Member

commented May 31, 2019

@ssbarnea For one thing, we are making a push to only allow variable names, and other similar keys, that are valid python identifiers. To explain a little further about group names, it causes a problem for users trying to use "dot syntax" such as groups.foo-group, which doesn't do what the user expects. The number of issues and support requests caused by small problems like these has caused us to go down a path to safe guard names to ensure that problems like this do not occur.

For those wanting to keep what we consider invalid characters, can opt out of this functionality.

@caleb-s-cullen

This comment has been minimized.

Copy link

commented May 31, 2019

What must we do to opt out of this functionality? Our local Ansible deployment scripts are littered with hyphen-containing group names. We don't use them with dot notation, of course. But changing all of them would be a truly monumental task. I would prefer to opt out and at the same time encourage my team to avoid using hyphens in the future and where possible to convert hyphens to underscores, tho' that last part is not always as straightforward as it might seem.

So, does one simply set force_valid_group_names = false in ansible.cfg ? That seems right based on d241794#diff-fd24ad93fbc32f454761746c1ac908f2

@jamescassell

This comment has been minimized.

Copy link
Contributor

commented Jun 3, 2019

What must we do to opt out of this functionality? Our local Ansible deployment scripts are littered with hyphen-containing group names. We don't use them with dot notation, of course. But changing all of them would be a truly monumental task. I would prefer to opt out and at the same time encourage my team to avoid using hyphens in the future and where possible to convert hyphens to underscores, tho' that last part is not always as straightforward as it might seem.

So, does one simply set force_valid_group_names = false in ansible.cfg ? That seems right based on d241794#diff-fd24ad93fbc32f454761746c1ac908f2

export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=never or export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore -- the latter is not in the docs yet: #57318

@al-x

This comment has been minimized.

Copy link
Contributor

commented Jun 4, 2019

Thanks, James. Since people are coming to this issue to follow-up on the warning message, I'm including information I think may be useful:

To disable the ≥2.10 group name auto-transformation more portably/permanently until such time as you may be ready to clear out invalid groups from your inventory, add force_valid_group_names = never to the [defaults] INI-section of ansible.cfg.

To see all groups and invalid characters which trigger the warning (perhaps so you can target them for phase-out), you can do something like this ansible CLI no-op:

ansible-inventory -vvvv --host=localhost 2>&1 | grep replacing

These invalid characters are (as of 2019-06-04) defined as a constant, INVALID_VARIABLE_NAMES, in:
https://github.com/ansible/ansible/blob/devel/lib/ansible/constants.py#L119
as '^[\d\W]|[^\w]',
that is: any leading non-alpha character OR any character other than alpha-numeric and underscore.
(I hope I got that right)

If you find deprecation warnings annoying, you may also disable permanently them for any ansible- command or the ansible ad-hoc command by adding deprecation_warnings = False to the same [defaults] section of ansible.cfg, but I would recommend against that (since you may miss important news), and instead use inline shell environment variables like this:
ANSIBLE_DEPRECATION_WARNINGS=False ansible-inventory --host=localhost

The inventory parsing [WARNING]s won't go away, however. There's no specific config or env var to turn off all warnings (yet?), but if it really bugs you, you can just send all stderr to /dev/null (insert "best practice" caveats here):

2>/dev/null ansible-inventory --host=localhost

Hope this helps someone, somewhere.

@ssbarnea

This comment has been minimized.

Copy link
Contributor

commented Jun 4, 2019

I only find deprecation warning messages annoying when they do not provide a migration path. Considering that the space is limited and that the remediation is likely to need update I would find very useful to provide links to tickets which can document solutions, workarounds,...

An approach like this could save extra work needed for improving incomplete warnings messages as we would not have to update the message, backport it to few versions back.

PS. Disabling deprecations warnings is something I would not recommend to anyone, maybe only if a project is already facing its ultimate fate ;)

@bpleines

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2019

i started to get this warning but found no references in the porting guide and no references how or what to fix.

most of my warnings are from the ec2.py, where the instance_id used the - (eg: i-033f62b586143dff7) and the regions (eg: eu-central-1c) , so we have no real fix for this ones

Finally, this broke some of my playbooks, where i used when: ansible_hostname in groups['varnish'] and the ansible_hostname is varnish-eu-central-1c-001 .
In the past this worked fine, now i need to use inventory_hostname to get varnish_eu_central_1c_001 and get a match vs the groups['varnish']

So this needs at least and urgently a warning in the porting guide that inventory_hostname and groups[] may be returning different data

I'd like to echo statement about the warning being generated by the EC2 dynamic inventory script. I noticed that there is an ec2.ini configuration setting to disable grouping hosts by instance_id (group_by_instance_id = False), but setting that didn't resolve the warning for me like I anticipated it would - I made sure I cleared the local inventory cache.

Any workarounds for EC2 dynamic inventory specifically?

@jamescassell

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2019

These invalid characters are (as of 2019-06-04) defined as a constant, INVALID_VARIABLE_NAMES, in:
https://github.com/ansible/ansible/blob/devel/lib/ansible/constants.py#L119
as '^[\d\W]|[^\w]',
that is: any leading non-alpha character OR any character other than alpha-numeric and underscore.
(I hope I got that right)

Sounds accurate to me. You should submit a docs PR with that info.

If you find deprecation warnings annoying, you may also disable permanently them for any ansible- command or the ansible ad-hoc command by adding deprecation_warnings = False to the same [defaults] section of ansible.cfg, but I would recommend against that (since you may miss important news), and instead use inline shell environment variables like this:
ANSIBLE_DEPRECATION_WARNINGS=False ansible-inventory --host=localhost

The inventory parsing [WARNING]s won't go away, however. There's no specific config or env var to turn off all warnings (yet?), but if it really bugs you, you can just send all stderr to /dev/null (insert "best practice" caveats here):

The undocumented ignore option provides this functionality. Docs PR here: #57318

Starting in 2.8.2, this deprecation warning will be squashed if you explicitly set any of choices.

@maglub

This comment has been minimized.

Copy link

commented Jun 5, 2019

Where does the ansible dev team discuss this type of decisions? It is very hard for us users to understand the reasoning to this. If it is pure "python style" reasoning, rather than a practical reasoning, it might be worth reconsidering? If a dash in group names break things in future releases of ansible, it might rather be a problem with the implementation, more than the naming of a group?

To me, this sounds more like a cosmetic change, than something that has been thought through properly.

A group name is not a variable name, it is the content of such. A hyphen/dash is just a character, that also happens to be an extremely popular way of grouping information in a naming convention. Compared to the exclamation point or the star, it does not have a special meaning in a limit clause.

The cost to mitigate this issue is enorm, given that thousands of sites has to not only change the group names in the inventories, but also go through all their playbooks and home grown roles, and test them all again.

If there is any way at all for "peasants" to make their voice heard, I would love to chip in my opinion and try and understand how this idea came to be.

@jamescassell

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2019

I've come to understand that the change was made to ansible because users made mistakes such as trying to use groups.group-name rather than groups['group-name']. AIUI, it is a change purely for the purpose of reducing support issues. (I'm personally opposed to the change.)

The old behavior will not go away; it will just become unavailable without explicitly choosing the old behavior.

@maglub

This comment has been minimized.

Copy link

commented Jun 5, 2019

Sad to hear.

My use case, is that I am embedding the command "ansible-inventory" in a Vagrant file, in a way where it is impolite to put things in ansible.cfg, and that it would help to be able to be able to override the behavior as a command line option (not environment variable).

Usually changes like this are due to good intentions, but might not always lead to the outcome one has in mind.

vitabaks added a commit to vitabaks/postgresql_cluster that referenced this issue Jun 5, 2019
fixed: "Not replacing invalid character(s) "set([u'-'])" in group nam…
…e (postgres-cluster)"

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature
will be removed in version 2.10.

ansible/ansible#56930
@MarkusTeufelberger

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2019

My issue with this change is that group names now become somewhat "special" - dashes are allowed in host names, but not in group names which makes it a bit weird considering at the start of a playbook in the hosts: section I could write host and/or group names.

@jamescassell

This comment has been minimized.

Copy link
Contributor

commented Jul 30, 2019

I did some digging.

this functionality was originally added in PR #52748 allegedly to support the feature request #40581

one description of the goal: #52748 (comment)

first version of THIS symptom (though different cause): #51844

@maglub

This comment has been minimized.

Copy link

commented Jul 31, 2019

Man, I have been reading #52748 so many times now.

As I understand it, group names were previously sanitised in the plugins and the core, and someone (for whatever reason, as it is still not at all clear to me why) decided that group names should follow python variable naming conventions.

So #52748 pushed the sanitation into the inventory instead, which breaks things for so many people. Especially people who use clever naming conventions, for example in AWS, Azure, etc, to map hosts to groups.

If we were to use the same standard/naming convention for host names, we would for sure lose momentum and lose users.

A group name is a name, not a variable. Using dashes in group names (just as in host names) makes sense. The translation (sanitation) should not need to be done on an inventory level (by us, the users), and in the best of all worlds actually never.

I really don't see the benefit of enforcing this. The discussion seems to cover also "." and ":", which some people like using in group names. Personally I don't use them, but I also don't see the harm in doing so.

As long as cloud providers re using dashes in their meta information, we should be able to use them for grouping. Actually, that should not even be the driver. If I feel like naming a group a-b-c-d-e, it should not really be an issue. It is a very useful delimiter.

This thread does not seem to attract any attention from devs or maintainers, though. I think we are speaking to deaf ears.

Devs/Maintainers: Please, pretty please allow dashes in group names!

@bcoca

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

To clarify some misconceptions, part of them were to due to my mistakes and making initial messages unclear, the latest versions have fixes to some of the issues people keep bringing up here, other fixes are still incomming:

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default. This 'default' is what is being deprecated, the default will be 'safe' in 2.11, but you will always have an option to 'opt in' to the old behaviour.

And to explain how and why we got here:

First, group names were ALWAYS sanitized, they just had different inconsistent rules, depending on the inventory type you were using, scripts were all over the place, YAML and INI formats did each their own thing. The major change was 'centralizing and normalizing sanitation', this was decided back in 2.4 but was not totally implemented until 2.8. The intent was to provide a norm or baseline all could use safely in Ansilbe, that said we recognize there are many people using 'unsafe' or 'invalid' characters for variables so we made this configurable, not only globally but also by some of the inventory plugins.

The initial implementation had some issues and a lot of discussion (no we don't decide this in hiding, we have public meetings in irc to which you are all welcome, https://github.com/ansible/community/blob/master/meetings/README.md) and a lot of the feedback was incorporated (These are also logged so you can go back to see the discussions and reasoning, but to avoid 'log dumpster diving', I'll explain most of issues below). After going out in 2.8 we got another round of feedback and we have been fixing the some bugs, like always getting a deprecation, not just when using the default and specially with the wording of the documentation and warnings.

  • 'Why Python names?'
    Mostly because Ansible uses Python and JInja (which also uses Python) and some uses of groups (mostly in our early examples, but also plenty of 3rd party ones) can create errors in playbooks, i.e stuff: '{{ groups.gropup-name-with-dash ... }}' or worse a group.name.with.dots. This creates confusion for many users that want to use the Jinja feature of 'dot notation for variable access' and this is why the default should be safe for all users. Most people on this post might disagree with this, but this is a real issue for a lot of people and should not be a 'trap' waiting for new or old Ansible users. Those that 'opt out' are then responsible for avoiding the breaking usage in other parts of Ansible.

  • 'What if I liked that each inventory had different sanitation?'
    Well, you can still turn off the 'central' sanitation and enable the one for your specific inventory source, most popular new inventory plugins that substitute the old scripts have had options added to emulate the script behaviour, worst case scenario, you can still use the inventory scripts .

  • 'Why not hostnames/are hostnames next?'
    Like for groups, sanatization of hostnames has always existed, but has not changed, hostnames have a different requirements, like being DNS resolvable
    for network connections or a valid path for chroot connections. Also, fortunately, there are little to no examples of using hostnames in dot notation, this has not been a common practice and would become a problem if people suddenly started using them, but unlike group names this is something we have avoided until now. If it becomes a problem going forward ... I don't see a good solution either.

Note that this specific ticket (not good enough description/information) is something I'm addressing already, hope to have fix out soon. As for the rest of the discussion, the devs don't use Github as a forum, some tickets devolve into that, the previous ticket that is closed and also has a long thread was ignored until recently, mostly because devs filter out closed issues and expect discussions in IRC the mailing list or new issues.

I hope this addresses all the major issues, as always we are open for discussion, feel free to drop by ML or IRC, we just avoid using github since it is not a good place for such things.

@haeringer

This comment has been minimized.

Copy link

commented Jul 31, 2019

Thanks a lot for the clarification.

@skyscooby

This comment has been minimized.

Copy link
Contributor

commented Aug 1, 2019

Appreciate you taking the time to explain even though it would have been far simpler to stop supporting dot notation and deprecate that support over a few releases. Fewer people use that vs the amount of people who have invalid chars in their group names. Se la vie

@bcoca

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

@skyscooby the problem is that it is not Ansible doing that, it is Jinja.

@loop-evgeny

This comment has been minimized.

Copy link

commented Aug 2, 2019

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default.

OK, this is good to know, thanks for the clarification. However, the user experience really needs to be improved. You have the "curse of knowledge". Just try to imagine yourselves in the shoes of a user who sees this:

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be
user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

That is a long, long way from

[DEPRECATION WARNING] Group name 'my-servers' contains '-', which will become invalid by default from Ansible 2.11. Set force_valid_group_names in ansible.cfg or the ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS environment variable to "ignore" to suppress this. See https://docs.ansible.com/something for more information.

... which is what I, as a user, would have really wanted to see. It would have saved me over an hour. Now multiply by that by the number of people who have or will run into this issue.

@ramilehti

This comment has been minimized.

Copy link

commented Aug 2, 2019

Having dashes and dots be invalid in group-names is not a sensible default. People will always have them in their group names. To require them to set yet another variable in a config file to enable sensible behaviour is IMHO indefensible.

@Tronde

This comment has been minimized.

Copy link
Contributor

commented Aug 2, 2019

Thanks @bcoca for your comment above. It's much appreciated.

Although I'm not happy with the decision I understand that a discussion took place and a decision was made. If the decision is still open for debate it should be continued at the mailling list or on irc but may not capture this issue.

For the topic of this issue I would like to find the following information in the official documentation and the porting guides, to be aware of this change.

  • What are group names that are valid by default?
  • What to do to keep using group names including dashes, hyphens, dots and colons?

Because we use dashes in all our group and host names and we won't change that. So I have to opt-in and change my ansible.cfg every time I setup a new installation/enviroment. That's unfortunate to me but I have to deal with it somehow. The least I would expect is that this is documented in accordingly.

To continue the discussion whether this change is wise or not, I opened a thread in the Ansible Development group.

Best regards,
Tronde

@ssbarnea

This comment has been minimized.

Copy link
Contributor

commented Aug 2, 2019

I want to thanks to all contributors on this issue. Based on what I read here I decided to write a blog post https://docs.sbarnea.com/ansible/naming-hosts-and-groups -- Hopefully it would summarize what users need to do.

@jctanner

This comment has been minimized.

Copy link
Member

commented Aug 4, 2019

@loop-evgeny I agree that we as the core team do have the "curse of knowledge" and it inhibits us from creating docs and errors that are useful to everyone. We also completely rely on the community to help us shape ansible and keep it simple for as many users as possible, so when folks have recommendations on improving our docs and our error/warning messages, I always encourage them to help us out by sending a pullrequest. The message you point out is held in the following file and we would greatly appreciate it if you could send us a PR with the suggested change ...

display.deprecated('The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default,'
' this will change, but still be user configurable on deprecation', version='2.10')

@loop-evgeny

This comment has been minimized.

Copy link

commented Aug 5, 2019

@jctanner Ordinarily I'd be happy to submit a PR to improve a free and useful program that I use. However, the general attitude of the Ansible developers towards usability, their eagerness to close as "working as intended" issues that I consider to be self-evident bugs (even if design bugs) and the fact that Ansible currently has 2025 (that's two thousand!) open PRs gives me very little confidence that my work will not go to waste. If you really want to "rely on the community", as you say, then a substantial culture change is needed IMHO.

@mowgli

This comment has been minimized.

Copy link
Contributor

commented Aug 6, 2019

Hmm.. That chance hit me too.

Unfortunately we use network names as group names and that is not easily changable. I would love to opt out for the dot syntax sugar for group names as that is nothing I ever used (Although I use it with other variables).

It would be preferable to use ansible-playbook whatever.yaml -l some.network.to.use in future. Using any other than the network as group name would reduce the usecase massively.

pneerincx added a commit to pneerincx/league-of-robots that referenced this issue Aug 8, 2019
@Tronde

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

Hi,
I'm somewhat confused at the moment. Could somebody tell me what I have to set in ansible.cfg to allow invalid chars in group names in the future, please?

@equinox0815

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

force_valid_group_names = ignore

@sunshine69

This comment has been minimized.

Copy link

commented Aug 14, 2019

What is the future version of ansible regrading to this issues? Will at some time ansible will reject all dashes in group name without a work around using force_valid_group_names ? (without hearing feedback from users who has suffered by the change, and who has never suffered the issues by using hyphen in group names)

Sorry Just read the comments from @bcoca and happy to see that I would be able to use hyphen if I want to in the future - that is enough for me.

@cesarjorgemartinez

This comment has been minimized.

Copy link

commented Aug 14, 2019

Hi,
I see the same warning, but, I don't understand what I should change, and if we should change it.
Is it something related to python?
Howto resolve?

If I ignore with force_valid_group_names = ignore, it would be with that necessary, and when I upgrade to Ansible >= 2.10?

Regards,
Cesar Jorge

@equinox0815

This comment has been minimized.

Copy link
Contributor

commented Aug 14, 2019

If i understand this correctly. The only thing that is deprecated is the automatic transformation of group names. This means it should be totally fine to set force_valid_group_names = ignore after 2.10 and beyond.

It should also be totally fine to continue to use dashes and whatever you want in group names. What Ansible won't do in the future is to sanatize them so you can use the dotted notation even for "invalid" group names. For example:

Your inventory contains a group named foo-bar.xyz. Now you want to write a template that creates a list of hosts that belong to that group:

{% for host in groups['foo-bar.xyz'] %}
{{ host }}
{% endfor %}

Mind that the following version of the template would not work:

{% for host in groups.foo-bar.xyz %}
{{ host }}
{% endfor %}

This is because the - and the . have special meaning in this case. The dotted notation however would be totally fine if your group would have the name foo_bar_xyz because the template then becomes:

{% for host in groups.foo_bar_xyz %}
{{ host }}
{% endfor %}

Which is of course totally fine.

In an attempt to make things easier for users, Ansible apparently always did some sanitation for group names. This means it was (and until 2.10 still is) possible to use foo_bar_xyz in the above examples even though the group actually is called foo-bar.xyz. Personally i don't think this makes things easier at all and it seems that the core team now also agrees with that.
So their next attempt to tackle the issue is to make it impossible to have "invalid" group names in the first place. However as far as i understand it will always be possible to opt-out of this limitation by setting force_valid_group_names = ignore.

Long story short it is actually two different changes that have been intertwined[1] with each other. Whence the confusing name and wording of the warning.

Again this is only how i understand the issue. Please correct me if i'm wrong!

[1] for further details see RFC1925 Paragraph 2, Point (5)

@geerlingguy

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2019

I just wanted to leave my 2¢ since I'm firmly on the side of dashes > underscores. Just doing a quick Google search on the matter, almost universally underscores are labeled as inferior to dashes for any kind of pathing and labeling purposes. They are also more difficult to work with in many text editors and filesystems.

Even if that were not the case, the fact that I see dashes used for things like server labels and groups in the wild a lot more often than underscores means this will be yet another thing I'll have to make sure to add to all my and my clients' ansible.cfg files (I tend to have one per playbook).

I have no problem with Ansible trying to force a stricter default where it improves the experience, but first you came for the dashes in my role names (and sometimes allowed singular exceptions for older roles that were grandfathered in), then you came for dashes in my collections (they are not allowed in any way, shape or form), and now you've come for dashes in my inventory!

It's a war against dashes out there... and I want to draw a line somewhere—in this case, it's the one place where it's actually impossible for me to prevent people from using dashes, as many of the dynamic inventory providers create groups based on server names and labels, and many (if not most) organizations seem to label things using dashes (e.g. us-east-1a, not us_east_1a).

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed. It's not super pure and holistic, but we're not all Rust developers here...

@markgoddard

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2019

Very well put, Jeff. I strongly agree with you here - this change will be so disruptive, and rather than just requiring a one-off migration, will change the workflow of a huge number of users. Ansible will no longer work out of the box.

Hostnames cannot include an underscore, so in a sane world, inventory_hostname would not be forced to. This means that our inventories will now look quite inconsistent, with hostnames that cannot contain underscores, and groups that cannot contain hyphens.

Please, just don't switch the default.

https://en.m.wikipedia.org/wiki/Hostname

@Tronde

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2019

Hi,
I totally agree with Jeff here.

But as @bcoca stated above most of the developers don't look at these discussions here on a regular basis and this issue may not be the right place to discuss the change because it is about the correct documentation.

For discussion please join the thread Is changing the default setting of TRANSFORM_INVALID_GROUP_CHARS a good idea? in Google Groups.

@apple4ever

This comment has been minimized.

Copy link
Contributor

commented Aug 26, 2019

Great points Jeff.

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

This is the big takeaway for me from all these discussions. I understand the problem that needs to be solved, but the solution seems to be the opposite of what is needed. It makes things easier for support but hard for the users - that is a backwards solution.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed.

This here is the best solution, not breaking things that have been used for years.

hmadison pushed a commit to hmadison/lobsters-ansible that referenced this issue Aug 28, 2019
Hunter Madison
[Ansible 2 Deprecation] Don't use dashes in inventory.
ansible/ansible#56930 Appears to be the
best documentation around the issue.
hmadison pushed a commit to hmadison/lobsters-ansible that referenced this issue Aug 29, 2019
Hunter Madison
[Ansible 2 Deprecation] Don't use dashes in inventory.
ansible/ansible#56930 Appears to be the
best documentation around the issue.
@nwinkler

This comment has been minimized.

Copy link

commented Sep 9, 2019

Great comment from @geerlingguy - spot on!

I'd like to add that as a user of Ansible, why should I need to know what valid Python syntax is? Having used Ansible for a long time now, I understand that Ansible (and its modules) is written in Python, but why should I need to care about that? Exposing that fact to the end user is just bad design.

Similar to only allowing valid JavaScript/Ruby/.NET/whatever notation for things like usernames in a web application. Why would the end user care which language the application is written in?

In addition to that, introducing breaking changes is a difficult topic, I try to avoid that if possible. If I have to make a change, I typically leave in the old, existing behavior as the default, and let people opt-in for the new behavior. Why wasn't this done here? Why do I have to either change my configuration, or worse my whole inventory? Why not the other way around?

hmadison added a commit to hmadison/lobsters-ansible that referenced this issue Sep 15, 2019
[Ansible 2 Deprecation] Don't use dashes in inventory.
ansible/ansible#56930 Appears to be the
best documentation around the issue.
hmadison added a commit to hmadison/lobsters-ansible that referenced this issue Sep 15, 2019
[Ansible 2 Deprecation] Don't use dashes in inventory.
ansible/ansible#56930 Appears to be the
best documentation around the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.