3000 release annouced the deprecation of json_decode_dict is not equalivant as tojson

[sjorge]

Description of Issue

With the 3000rc dropping the deprecation of json_decode_dict in the next release was announced. The new way to do it is to use tojson.

In prepare my states and pillars before upgrading from 2018.2.3 to 3000rc2 I noticed issues when replacing json_decode_dict with tojson.

json_decode_dict would output an in the order it was received, tojson will output the keys sorted alphabetically!

Setup

{% load_yaml as x %}
z: some
y: text
x: here
{% endload %}
/tmp/tojson.out:
  file.managed:
    - contents: |
        {{ x|tojson }}

/tmp/json_decode_dict.out:
  file.managed:
    - contents: |
        {{ x|json_decode_dict }}

Steps to Reproduce Issue

I provided a basic test state to see it in action, notice the different order between tosjon and json_decode_dict

Versions Report

Salt Version:
           Salt: 2019.2.3

Dependency Versions:
           cffi: 1.11.5
       cherrypy: 17.3.0
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: 1.6.1
       M2Crypto: 0.30.1
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.6.6
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.15 (default, Oct 23 2018, 17:11:49)
   python-gnupg: 0.4.3
         PyYAML: 3.13
          PyZMQ: 17.1.2
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 5.1.1
            ZMQ: 4.2.5

System Versions:
           dist:
         locale: UTF-8
        machine: i86pc
        release: 5.11
         system: SunOS
        version: Not Installed

[xeacott]

I was looking at the documentation and it doesn't appear to have anything indicating that the output between those 2 flags are different, just that tojson is suppose to replace it. Is having them sorted unwanted behavior?

Here's the docs page for reference too... https://jinja.palletsprojects.com/en/2.10.x/templates/#tojson

[sjorge]

Depending on what output we're trying to cast, order might be imporant. For me a lot of things I had in a specific order got sorted alphabetically. e.g. some share templates showed up after the share that used them, resulting in errors in the smb.conf

[xeacott]

Understood, so it seems quite undesirable especially when it isn't clear to the user. Thanks for the report!

[waynew]

@sjorge json_decode_dict is actually an incorrectly named function, which is why it was deprecated in 2018.3.0.

To retain the same functionality, use json_encode_dict:

{% load_yaml as x %}
z: some
y: text
x: here
{% endload %}
/tmp/tojson.out:
  file.managed:
    - contents: |
        {{ x|json_encode_dict }}     # Note `en`code

/tmp/json_decode_dict.out:
  file.managed:
    - contents: |
        {{ x|json_decode_dict }}

The tojson filter provides a different functionality, and actually comes from the Jinja library.

[sjorge]

@waynew should the release ntoes be updated then? It says the json_decode_dict is deprecated and to use tojson instead... ?

Edit: yep using json_encode_dict gets the correct behavior again.

[waynew]

If one wanted the correct behavior of a function called json_decode_dict, tojson is the right filter. What json_decode_dict did originally is not json_decode_dict, it was doing json_encode_dict.

Basically you could think of it as being called "undo" when it should have been "do".

And for most cases, people were looking for "do", and tojson is what they're wanting.

I'm going to go ahead and close this issue, but reproducing it has exposed a potential issue when migrating from 2 to 3.