Skip to content

add toml module#56615

Merged
dwoz merged 5 commits into
saltstack:masterfrom
mchugh19:toml
Apr 26, 2020
Merged

add toml module#56615
dwoz merged 5 commits into
saltstack:masterfrom
mchugh19:toml

Conversation

@mchugh19

@mchugh19 mchugh19 commented Apr 13, 2020

Copy link
Copy Markdown
Contributor

What does this PR do?

Leverages the existing toml serializer to add a toml renderer. Also updates the serializer from pytoml to the supported toml library.

What issues does this PR fix or reference?

Updated library fixes: #56323
Fixes: #14802

Previous Behavior

Serializer used old pytoml library and no toml renderer.

New Behavior

Now you can write salt states in toml:

#!jinja|toml

{% set myvar = "sometext" %}

[["some id"."test.nop"]]
name = "{{ myvar }}"
[["some id"."test.nop"]]
txt = "hello"
[["some id"."test.nop"]]
wow = "thing"

Docs & changelog written?

Yes & No

  • Docs
  • Changelog

Tests written?

Yes

Commits signed with GPG?

No

@mchugh19 mchugh19 requested a review from a team as a code owner April 13, 2020 09:08
@ghost ghost requested a review from DmitryKuzmenko April 13, 2020 09:08
@mchugh19

Copy link
Copy Markdown
Contributor Author

Looks like this may require adding the toml dependency. What's the appropriate way of marking that in the test suite?

@Ch3LL Ch3LL removed the request for review from a team April 15, 2020 14:11
@mchugh19

Copy link
Copy Markdown
Contributor Author

re-run pr-ubuntu1804-py3

1 similar comment
@mchugh19

Copy link
Copy Markdown
Contributor Author

re-run pr-ubuntu1804-py3

@mchugh19

Copy link
Copy Markdown
Contributor Author

Almost all working. Ubuntu seems pretty stuck with

An un-handled exception was caught by salt's testing global exception handler:
07:11:39         RuntimeError: Failed to load filename map: while constructing a mapping
07:11:39           in "/tmp/kitchen/testing/tests/filename_map.yml", line 1, column 1
07:11:39         found conflicting ID 'salt/utils/vt.py'

@dwoz dwoz merged commit 6e0efd0 into saltstack:master Apr 26, 2020
myii added a commit to myii/ssf-formula that referenced this pull request May 2, 2020
* https://tree.taiga.io/project/myii-saltstack-formulas/task/187
* https://travis-ci.org/github/myii/telegraf-formula/builds/681360474 (before)
* https://travis-ci.org/github/myii/telegraf-formula/builds/682200993 (after)

Quoting from Slack #testing channel:

> Final issue with this test run: the `telegraf-formula` is failing
> completely, due to `toml` issues:
> https://travis-ci.org/github/myii/telegraf-formula/builds/681360474.
> I'm vaguely aware that something has changed in this regard for Sodium,
> so probably the formula needs to change?
>
> ```
> ID: telegraf/config/install
> Function: file.managed
>     Name: /etc/telegraf/telegraf.conf
>   Result: False
>  Comment: Unable to manage file: Jinja error: module 'toml' has no attribute 'dumps'
>    Traceback (most recent call last):
>      File "/usr/local/lib/python3.7/dist-packages/salt/serializers/toml.py", line 63, in serialize
>        return toml.dumps(obj, **options)
>    AttributeError: module 'toml' has no attribute 'dumps'
>
>    During handling of the above exception, another exception occurred:
>
>    Traceback (most recent call last):
>      File "/usr/local/lib/python3.7/dist-packages/salt/utils/templates.py", line 400, in render_jinja_tmpl
>        output = template.render(**decoded_context)
>      File "/usr/local/lib/python3.7/dist-packages/jinja2/environment.py", line 1090, in render
>        self.environment.handle_exception()
>      File "/usr/local/lib/python3.7/dist-packages/jinja2/environment.py", line 832, in handle_exception
>        reraise(*rewrite_traceback_stack(source=source))
>      File "/usr/local/lib/python3.7/dist-packages/jinja2/_compat.py", line 28, in reraise
>        raise value.with_traceback(tb)
>      File "<template>", line 6, in top-level template code
>      File "/usr/local/lib/python3.7/dist-packages/salt/modules/slsutil.py", line 224, in serialize
>        return _get_serialize_fn(serializer, "serialize")(obj, **kwargs)
>      File "/usr/local/lib/python3.7/dist-packages/salt/serializers/toml.py", line 65, in serialize
>        raise SerializationError(error)
>    salt.serializers.SerializationError: module 'toml' has no attribute 'dumps'
> ```
>
> ---
>
> toml update was to migrate from pytoml to toml library:
> saltstack/salt#56615
>
> Should have been a painless change, but I guess not. There weren't any
> tests for the serializer, so I made one. If I get some spare time this
> week, I'll see if I can take a look. There's not much to the serializer
@sagetherage sagetherage added the ZRelease-Sodium retired label label May 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ZRelease-Sodium retired label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Write States in TOML

6 participants