[BUG] pillar.stack inconsistent pillar.items/pillar.get and saltenv/pillarenv

[morgana2313]

Description

When using opts.pillarenv and opts.saltenv in pillar.stack, pillar.items and pillar.get give inconsistent answers.

I suspect that's because they're called with inconsistent saltenv and pillarenv values in __opts__:

# salt-call pillar.items
local:
    ----------
    foo:
        ----------
        opts.pillarenv:
            master
        opts.saltenv:
            None

# salt-call pillar.get foo
local:
    ----------
    opts.pillarenv:
        None
    opts.saltenv:
        master

Setup

master.conf:

ext_pillar:
  - stack:
      opts:saltenv:
        master: /tmp/foo.cfg
      opts:pillarenv:
        master: /tmp/foo.cfg

/tmp/foo.cfg:

foo.yml

/tmp/foo.yml:

foo:
  saltenv: {{ saltenv }}
  pillarenv: {{ pillarenv }}
  opts.pillarenv: {{ __opts__['pillarenv'] }}
  opts.saltenv: {{ __opts__['saltenv'] }}

Steps to Reproduce the behavior

An extra debug line (diff below) in stack.py sheds some light:

salt-call pillar.get foo calls ext_pillar.stack with __opts__['pillarenv']=None.

[INFO    ] ext_pillar.stack: __opts__['saltenv']=master __opts__['pillarenv']=None pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'None', 'opts.saltenv': 'master'}}

salt-call pillar.items does the same, but the result is discarded. Then ext_pillar.stack is called a second time with __opts__['pillarenv']=master
The results from the second call are used as the result:

[INFO    ] ext_pillar.stack: __opts__['saltenv']=master __opts__['pillarenv']=None pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'None', 'opts.saltenv': 'master'}}
[INFO    ] ext_pillar.stack: __opts__['saltenv']=None __opts__['pillarenv']=master pillar:{'foo': {'saltenv': None, 'pillarenv': None, 'opts.pillarenv': 'master', 'opts.saltenv': 'None'}}

diff --git a/salt/pillar/stack.py b/salt/pillar/stack.py
index 57935021e6..f69d762b2a 100644
--- a/salt/pillar/stack.py
+++ b/salt/pillar/stack.py
@@ -413,6 +413,7 @@ def ext_pillar(minion_id, pillar, *args, **kwargs):
             log.info('Ignoring pillar stack cfg "%s": file does not exist', cfg)
             continue
         stack = _process_stack_cfg(cfg, stack, minion_id, pillar)
+    log.info(f"ext_pillar.stack: __opts__['saltenv']={__opts__['saltenv']} __opts__['pillarenv']={__opts__['pillarenv']} pillar:{stack}")
     return stack

Expected behavior
I would expect pillar.items and pillar.get to give consistent results. And I do not understand why pillar.items makes two calls.

The supplied test.conf is a workaround for this bug; when removing either the opts.pillarenv or opts.saltenv sections (and restarting the salt-master) pillar.items or pillar.get gives an empty result.

Versions Report

salt --versions-report

Salt Version:
          Salt: 3005+197.g1a546537b1

Dependency Versions:
          cffi: 1.15.1
      cherrypy: Not Installed
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.4
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.15.0
        pygit2: Not Installed
        Python: 3.10.6 (main, Aug  2 2022, 15:11:03) [GCC 7.5.0]
  python-gnupg: 0.4.1
        PyYAML: 3.12
         PyZMQ: 20.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.3

System Versions:
          dist: ubuntu 18.04 Bionic Beaver
        locale: utf-8
       machine: x86_64
       release: 4.15.0-191-generic
        system: Linux
       version: Ubuntu 18.04 Bionic Beaver```
</details>