Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: add nix-ast-dump #16

Merged
merged 1 commit into from
May 27, 2023
Merged

tools: add nix-ast-dump #16

merged 1 commit into from
May 27, 2023

Conversation

inclyc
Copy link
Member

@inclyc inclyc commented May 16, 2023

also add an ast dump utility, produces indented output:

[user@system]$ build/nix-ast-dump ~flakes/shell.nix
 ExprLambda: ({ pkgs ? (let lock = (((builtins).fromJSON ((builtins).readFile ~flakes/flake.lock))).nodes.nixpkgs.locked; nixpkgs = (fetchTarball { sha256 = (lock).narHash; url = ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz"); }); in (import nixpkgs { overlays = [ ]; })), ... }: ((pkgs).mkShell { NIX_CONFIG = "extra-experimental-features = nix-command flakes repl-flake"; nativeBuildInputs = (with pkgs; [ (nix) (home-manager) (git) (sops) (gnupg) (age) ]); }))
  ExprLet: (let lock = (((builtins).fromJSON ((builtins).readFile ~flakes/flake.lock))).nodes.nixpkgs.locked; nixpkgs = (fetchTarball { sha256 = (lock).narHash; url = ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz"); }); in (import nixpkgs { overlays = [ ]; }))
   ExprAttrs: { lock = (((builtins).fromJSON ((builtins).readFile ~flakes/flake.lock))).nodes.nixpkgs.locked; nixpkgs = (fetchTarball { sha256 = (lock).narHash; url = ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz"); }); }
    ExprSelect: (((builtins).fromJSON ((builtins).readFile ~flakes/flake.lock))).nodes.nixpkgs.locked
     ExprCall: ((builtins).fromJSON ((builtins).readFile ~flakes/flake.lock))
      ExprCall: ((builtins).readFile ~flakes/flake.lock)
       ExprPath: ~flakes/flake.lock
       ExprSelect: (builtins).readFile
        ExprVar: builtins
      ExprSelect: (builtins).fromJSON
       ExprVar: builtins
    ExprCall: (fetchTarball { sha256 = (lock).narHash; url = ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz"); })
     ExprAttrs: { sha256 = (lock).narHash; url = ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz"); }
      ExprConcatStrings: ("https://github.com/nixos/nixpkgs/archive/" + (lock).rev + ".tar.gz")
       ExprString: "https://github.com/nixos/nixpkgs/archive/"
       ExprSelect: (lock).rev
        ExprVar: lock
       ExprString: ".tar.gz"
      ExprSelect: (lock).narHash
       ExprVar: lock
     ExprVar: fetchTarball
   ExprCall: (import nixpkgs { overlays = [ ]; })
    ExprVar: nixpkgs
    ExprAttrs: { overlays = [ ]; }
     ExprList: [ ]
    ExprVar: import
  ExprCall: ((pkgs).mkShell { NIX_CONFIG = "extra-experimental-features = nix-command flakes repl-flake"; nativeBuildInputs = (with pkgs; [ (nix) (home-manager) (git) (sops) (gnupg) (age) ]); })
   ExprAttrs: { NIX_CONFIG = "extra-experimental-features = nix-command flakes repl-flake"; nativeBuildInputs = (with pkgs; [ (nix) (home-manager) (git) (sops) (gnupg) (age) ]); }
    ExprString: "extra-experimental-features = nix-command flakes repl-flake"
    ExprWith: (with pkgs; [ (nix) (home-manager) (git) (sops) (gnupg) (age) ])
     ExprVar: pkgs
     ExprList: [ (nix) (home-manager) (git) (sops) (gnupg) (age) ]
      ExprVar: nix
      ExprVar: home-manager
      ExprVar: git
      ExprVar: sops
      ExprVar: gnupg
      ExprVar: age
   ExprSelect: (pkgs).mkShell
    ExprVar: pkgs
# Shell for bootstrapping flake-enabled nix and other tooling
{ pkgs ? # If pkgs is not defined, instanciate nixpkgs from locked commit
  let
    lock = (builtins.fromJSON (builtins.readFile ./flake.lock)).nodes.nixpkgs.locked;
    nixpkgs = fetchTarball {
      url = "https://github.com/nixos/nixpkgs/archive/${lock.rev}.tar.gz";
      sha256 = lock.narHash;
    };
  in
  import nixpkgs { overlays = [ ]; }
, ...
}: pkgs.mkShell {
  NIX_CONFIG = "extra-experimental-features = nix-command flakes repl-flake";
  nativeBuildInputs = with pkgs; [
    nix
    home-manager
    git

    sops
    gnupg
    age
  ];
}

@inclyc inclyc marked this pull request as draft May 16, 2023 06:10
@inclyc
Copy link
Member Author

inclyc commented May 16, 2023

TODO: add regression tests.

Base automatically changed from nixd/ast-library to main May 16, 2023 12:37
@inclyc inclyc force-pushed the nixd/ast-dump-tool branch 2 times, most recently from 81864c6 to 2c46b03 Compare May 17, 2023 10:55
@inclyc inclyc marked this pull request as ready for review May 27, 2023 12:33
@inclyc inclyc merged commit 2cff60d into main May 27, 2023
@inclyc inclyc deleted the nixd/ast-dump-tool branch May 27, 2023 12:39
inclyc pushed a commit that referenced this pull request Apr 21, 2024
While bumping my system flake I ran into a test failing due to an
out-of-bounds access. It got caught by `_GLIBCXX_ASSERTIONS`. I guess
the not-so-recent
[bump](https://www.github.com/NixOS/nixpkgs/pull/298047) of meson in
nixpkgs brought in this
[commit](https://www.github.com/mesonbuild/meson/pull/12683) from meson
1.4.0.

Here's the failing test logs and a backtrace:

```bash
nixd-nightly> >>> MALLOC_PERTURB_=222 LD_LIBRARY_PATH=/build/3gwj8jwd4ypvcahxqv3cn2fq617fm7v2-source/build/libnixbc:/build/3gwj8jwd4ypvcahxqv3cn2fq617fm7v2-source/build/libnixf:/build/3gwj8jwd4ypvcahxqv3cn2fq617fm7v2-source/build/libbc ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 /build/3gwj8jwd4ypvcahxqv3cn2fq617fm7v2-source/build/libnixf/unit-libnixf-parse
nixd-nightly>  ✀
nixd-nightly> stdout:
nixd-nightly> Running main() from /build/source/googletest/src/gtest_main.cc
nixd-nightly> [==========] Running 90 tests from 2 test suites.
nixd-nightly> [----------] Global test environment set-up.
nixd-nightly> [----------] 15 tests from LexerTest
nixd-nightly> [ RUN      ] LexerTest.Integer
nixd-nightly> stderr:
nixd-nightly> /nix/store/NN152MGPDFCW92B3B03G9KMNLPD9JRIS-gcc-13.2.0/include/c++/13.2.0/string_view:258: constexpr const std::basic_string_view<_CharT, _Traits>::value_type& std::basic_string_view<_CharT, _Traits>::operator[](size_type) const [with _CharT = char; _Traits = std::char_traits<char>; const_reference = const char&; size_type = long unsigned int]: Assertion '__pos < this->_M_len' failed.
```

lldb backtrace:
```c++
(lldb) process launch --environment LD_LIBRARY_PATH=/home/xokdvium/Work/Programming/nixd/build/libnixf:/home/xokdvium/Work/Programming/nixd/build/libbc:/home/xokdvium/Work/Programming/nixd/build/libnixbc:/nix/store/fivm0vvpkfm933fd5gqj3dzni0pp50cg-pipewire-1.0.4-jack/lib ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 MALLOC_PERTURB_=239 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1
Process 433026 launched: '/home/xokdvium/Work/Programming/nixd/build/libnixf/unit-libnixf-parse' (x86_64)
Running main() from /build/source/googletest/src/gtest_main.cc
[==========] Running 90 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 15 tests from LexerTest
[ RUN      ] LexerTest.Integer
/nix/store/NN152MGPDFCW92B3B03G9KMNLPD9JRIS-gcc-13.2.0/include/c++/13.2.0/string_view:258: constexpr const std::basic_string_view<_CharT, _Traits>::value_type& std::basic_string_view<_CharT, _Traits>::operator[](size_type) const [with _CharT = char; _Traits = std::char_traits<char>; const_reference = const char&; size_type = long unsigned int]: Assertion '__pos < this->_M_len' failed.
Process 433026 stopped
* thread #1, name = 'unit-libnixf-pa', stop reason = signal SIGABRT
    frame #0: 0x00007ffff7abfefc libc.so.6`__pthread_kill_implementation + 268
libc.so.6`__pthread_kill_implementation:
->  0x7ffff7abfefc <+268>: movl   %eax, %ebx
    0x7ffff7abfefe <+270>: negl   %ebx
    0x7ffff7abff00 <+272>: cmpl   $0xfffff000, %eax         ; imm = 0xFFFFF000
    0x7ffff7abff05 <+277>: movl   $0x0, %eax
(lldb) bt
* thread #1, name = 'unit-libnixf-pa', stop reason = signal SIGABRT
  * frame #0: 0x00007ffff7abfefc libc.so.6`__pthread_kill_implementation + 268
    frame #1: 0x00007ffff7a6fe86 libc.so.6`raise + 22
    frame #2: 0x00007ffff7a58935 libc.so.6`abort + 215
    frame #3: 0x00007ffff7d289fe libstdc++.so.6`std::__glibcxx_assert_fail(char const*, int, char const*, char const*) + 94
    frame #4: 0x00007ffff7f79f60 libnixf.so`std::basic_string_view<char, std::char_traits<char>>::operator[](this=<unavailable>, __pos=<unavailable>) const at string_view:258:2
    frame #5: 0x00007ffff7f782d6 libnixf.so`nixf::Lexer::consumeManyOf(std::basic_string_view<char, std::char_traits<char>>) [inlined] nixf::Lexer::peekUnwrap(this=0x00007fffffff73b0) const at Lexer.h:113:64
    frame #6: 0x00007ffff7f782ca libnixf.so`nixf::Lexer::consumeManyOf(this=0x00007fffffff73b0, Chars=<unavailable>) at Lexer.cpp:57:33
    frame #7: 0x00007ffff7f79258 libnixf.so`nixf::Lexer::lexNumbers() [inlined] nixf::Lexer::consumeManyDigits(this=0x00007fffffff73b0) at Lexer.h:81:25
    frame #8: 0x00007ffff7f7923f libnixf.so`nixf::Lexer::lexNumbers(this=0x00007fffffff73b0) at Lexer.cpp:195:30
    frame #9: 0x00007ffff7f79587 libnixf.so`nixf::Lexer::lex(this=0x00007fffffff73b0) at Lexer.cpp:471:15
    frame #10: 0x0000000000408a77 unit-libnixf-parse`(anonymous namespace)::LexerTest_Integer_Test::TestBody(this=0x00000000004844b0) const at Lexer.cpp:32:21
    frame #11: 0x00007ffff7f0625d libgtest.so.1.14.0`void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 77
    frame #12: 0x00007ffff7eed06e libgtest.so.1.14.0`testing::Test::Run() + 206
    frame #13: 0x00007ffff7eed225 libgtest.so.1.14.0`testing::TestInfo::Run() + 405
    frame #14: 0x00007ffff7eed457 libgtest.so.1.14.0`testing::TestSuite::Run() + 519
    frame #15: 0x00007ffff7efc9e7 libgtest.so.1.14.0`testing::internal::UnitTestImpl::RunAllTests() + 1143
    frame #16: 0x00007ffff7eed65b libgtest.so.1.14.0`testing::UnitTest::Run() + 123
    frame #17: 0x00007ffff7f230c0 libgtest_main.so.1.14.0`main + 64
    frame #18: 0x00007ffff7a5a10e libc.so.6`__libc_start_call_main + 126
    frame #19: 0x00007ffff7a5a1c9 libc.so.6`__libc_start_main@@GLIBC_2.34 + 137
    frame #20: 0x00000000004073a5 unit-libnixf-parse`_start + 37
```

Maybe it's a good idea to bump nixpkgs for the new default meson
behavior
or enable stdlib assertions with bounds-checking explicitly?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant