Fix Set Modules Issue

[lapp0]

Fixes #54

• fix test case so it correctly fails if set module is used
• minor improvements to code and tests
• dependency injection inget_option_tree
• import set module by changing lib.nix

Currently the evalModuleStub changes result in no position being returned.

Error for set modules before this change:

E               error: while evaluating 'get_modules_defined_attrs' at /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:48:31, called from (string):1:1:
E               while evaluating 'collectDeclarationPositions' at /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:12:42, called from /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:55:5:
E               while evaluating 'evalModuleStub' at /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:21:20, called from /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:53:36:
E               attempt to call something which is not a function but a set, at /build/k204x5pv8a4s0ach7h0g08wibc09gdgd-source/nixui/nix/lib.nix:21:33

Output before this change:

[{'loc': ['environment', 'etc'], 'position': {'column': 3, 'file': '/build/ffjx8jqw8fn6ijcqdd0nrqrcb5x73gja-source/nixui/tests/sample/configuration.nix', 'line': 104}}, {'loc': ['environment', 'systemPackages'], 'position': {'column': 3, 'file': '/build/ffjx8jqw8fn6ijcqdd0nrqrcb5x73gja-source/nixui/tests/sample/configuration.nix', 'line': 167}}...

Output after this change:

[{'loc': ['appstream', 'enable'], 'position': None}, {'loc': ['assertions'], 'position': None}

Need to look more into how https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/eval-config.nix and https://github.com/NixOS/nixpkgs/blob/a3df3d05e5db17b26ff886f53a58761a547561cc/lib/modules.nix works to fix this issue and merge.

[lapp0]

Note:

Fails when running nix run nix-gui -- -c "/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix

nix_eval.get_modules_defined_attrs(module_path) includes

system.stateVersion {'position': {'column': 5, 'file': '/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix', 'line': 6}}

However tests pass for set_configuration.nix and nix_eval.get_modules_defined_attrs(module_path) includes

system.stateVersion {'position': {'column': 5, 'file': '/build/2vpbxrgrxln02faiv476hcwgpn8j9js3-source/nixui/tests/sample/set_configuration.nix', 'line': 5}}

For some reason unsafeGetAttrPos returns a different line in tests vs actually running the program

[lapp0]

Normal shell:

nix-instantiate --eval -E 'builtins.unsafeGetAttrPos "stateVersion" (import /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix).system'       
{ column = 5; file = "/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix"; line = 6; }

nix develop shell

 nix-instantiate --eval -E 'builtins.unsafeGetAttrPos "stateVersion" (import /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix).system'
warning: unknown setting 'experimental-features'
{ column = 5; file = "/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix"; line = 5; }

[lapp0]

Normal shell:

nix-shell -p nix-info --run "nix-info -m" | grep version                                                                                            
 - version: `nix-env (Nix) 2.4pre20210922_bcd73eb`

nix develop shell:

nix-shell -p nix-info --run "nix-info -m" | grep version
version: `nix-env (Nix) 2.3.12`

Edit: this discrepancy in version has been resolved, but the issue persists per the next comment

[lapp0]

After changing checkInputs for nix-gui to use pkgs.nixUnstable, nix develop and the tests all have the output

nix-instantiate --eval -E 'builtins.unsafeGetAttrPos "stateVersion" (import /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix).system
system.stateVersion {'position': {'column': 5, 'file': '/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix', 'line': 6}}

However, inside a debug shell within nix-gui, we get the output

subprocess.check_output(['nix-instantiate', '--eval', '-E', """builtins.unsafeGetAttrPos "stateVersion" (import /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix).system"""])
b'{ column = 5; file = "/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix"; line = 5; }\n'

Comparing Environments

def main():
    import subprocess
    print(subprocess.check_output([
        'nix-info', '-m'
    ]).decode())

as well as running nix-shell -p nix-info "nix-info -m" in a normal shell result in

 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.67, NixOS, 21.05.git.70904d4a992 (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20211006_53e4794`
 - channels(andrew): `"home-manager-21.05, nixos-20.09.4404.a13824fe5e3"`
 - nixpkgs: `/etc/nixos/nixpkgs`

Comparing nix-instantiate versions

subprocess.check_output(['nix-instantiate', '--version'])
b'nix-instantiate (Nix) 2.4pre20211006_53e4794\n'

normal shell:

nix-instantiate --version
nix-instantiate (Nix) 2.4pre20211006_53e4794

Comparing file hash

subprocess.check_output(['sha1sum', '/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix'])
b'fd73eff1df5a417f45f8fd81353039360f46d0a7  /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix\n

sha1sum /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix
fd73eff1df5a417f45f8fd81353039360f46d0a7  /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix

Summary

The environment described by nix-info -m is consistent between my shell and nix-gui as described by subprocess..

nix-gui tests, nix develop, and my shell all show line 6, however nix run shows line 5.

Not sure what else could be different resulting in different output...

[lapp0]

I've made the environments consistent, however now there is a discrepancy between different ways of running unsafeGetAttrPos for set_configuration.nix

Test case nix_eval.get_modules_defined_attrs (line=5)

The following test passes:

@pytest.mark.datafiles(SAMPLES_PATH)
def test_get_modules_defined_attrs_set_configuration():
    module_path = os.path.abspath(os.path.join(SAMPLES_PATH, 'set_configuration.nix'))
    result = nix_eval.get_modules_defined_attrs(module_path)
    assert result[attribute.Attribute.from_string('system.stateVersion')]['position']['column'] == 5
    assert result[attribute.Attribute.from_string('system.stateVersion')]['position']['line'] == 5

nix-gui run environment via pdb - running nix_instantiate_eval, builtins.unsafeGetAttrPos directly (line=5)

nix_eval.nix_instantiate_eval('builtins.unsafeGetAttrPos "stateVersion" (import /home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix).system', strict=True)
{'column': 5, 'file': '/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix', 'line': 5}

nix-gui run environment via pdb - catching exception in parser.get_key_value_nodes (line=6)

        (...)
        try:
            key_node, value_node = [e for e in definition_node.elems if isinstance(e, syntax_tree.Node)]
        except:
            import pdb;pdb.set_trace()
         (...)

(Pdb) attribute
Attribute('system.stateVersion')
(Pdb) attr_data
{'position': {'column': 5, 'file': '/home/andrew/p/nix-gui/nixui/tests/sample/set_configuration.nix', 'line': 6}}

sample configuration.nix (not set_configuration.nix)

No problem here, program and tests run fine, indicating consistent output.

[lapp0]

Next step, go over every single environment variable in both test environment and run environment.

[lapp0]

This was actually a diskcache issue. nix_eval.get_modules_defined_attrs was cached and returning the wrong result.