Full syntax_tree binding for node-syntax-tree interface

[aaronlippold]

We are looking to update our OCLIF cli https://saf-cli.mitre.org/ inspec-delta utiliy and ts-inspec-objects library https://github.com/mitre/ts-inspec-objects in the MITRE SAF CLI to help ease upkeep of our ruby DSL inspec profiles which are generated from XCCDF security benchmarks.

These tools used to be in ruby but we moved to typescript to ease end-user use and adoption.

We are looking to use the Web Assembler Ruby work to use ASTs to parse, update and lint our profiles rather then recreate that body of work in TS/JS and throw away years of work and more importantly keep running into parse errors, corner cases when there are 'tools that do that already' but we also face the challenge of mixing ruby and TS/JS.

Is there any reason only part of the syntax_tree was not fully bound here? We can update it but wanted to ask if there was a reason not all parts were added and it would not work well before we looked into it and perhaps submitted a PR.

[kddnewton]

Hi @aaronlippold -

The ruby virtual machine is accessible through the main export, so you should be able to rubyVM.eval whatever it is that you need. If you're looking for additional functionality I'm happy to add it in or accept PRs. What APIs are you looking to have added?

[aaronlippold]

Hi, we are working out the pattern workflow we are interested in. I will post it here in the morning to give you and idea and that should outline the additional functionality needed as well as advice for some of the other tools and libraries we would need to use to fill out the process.

If you are interested in learning more about what the MITRE Security Automation Framework does and our efforts in the community I would be happy talk with you about it and of course you can hit https://saf.mitre.org to take a peek.

[aaronlippold]

The first iteration of the delta project was with Cerner and MITRE - https://github.com/cerner/inspec_delta - which was ruby based but used regex to do a lot of the matching. Given the end goals - perhaps one approach could be integrate AST into the original ruby work, use the ruby traversal and code generation libraries, package it and then wrap it in the RubyVM so we can 'include' it into the OCLIF SAF CLI.

What are your thoughts given we don't want to reinvent the wheel and or continuously chase bugs in a hand rolled ruby parser written in JS.

[aaronlippold]

The last part of our goal is to also produce cops so that we can lint our final ruby inspec profiles to a style and structure that we establish for the community and likely integrate into the chefstyle gem.

Given that is the case - we would likely be doing work with the rubocop-ast gem as well which I see you already developed a transform for.

[em-c-rod]

Thank you for your response, @kddnewton . Here is the diagram for what we are trying to accomplish.

[camdenmoors]

Hi, thanks for being willing to add functionality! We may have found a bug, not completely sure yet. Perhaps you could tell us if there's something we're doing wrong. The library seems to be throwing an error when processing when processing multiple levels of recursion, for example:

Using AST on this Ruby code does not throw an error:

control 'SV-230238' do
  title "RHEL 8 must prevent system daemons from using Kerberos for
  authentication."
  desc  "Unapproved mechanisms that are used for authentication to the
cryptographic module are not verified and therefore cannot be relied upon to
provide confidentiality or integrity, and DoD data may be compromised.
    RHEL 8 systems utilizing encryption are required to use FIPS-compliant
mechanisms for authenticating to cryptographic modules.
    The key derivation function (KDF) in Kerberos is not FIPS compatible.
Ensuring the system does not have any keytab files present prevents system
daemons from using Kerberos for authentication.  A keytab is a file containing
pairs of Kerberos principals and encrypted keys.
    FIPS 140-2 is the current standard for validating that mechanisms used to
access cryptographic modules utilize authentication that meets DoD
requirements. This allows for Security Levels 1, 2, 3, or 4 for use on a
general-purpose computing system.
  "
  desc  'rationale', ''
  desc  'check', "
    Verify that RHEL 8 prevents system daemons from using Kerberos for
authentication.
    If the system is a server utilizing krb5-server-1.17-18.el8.x86_64 or
newer, this requirement is not applicable.
    If the system is a workstation utilizing
krb5-workstation-1.17-18.el8.x86_64 or newer, this requirement is not
applicable.
    Check if there are available keytabs with the following command:
    $ sudo ls -al /etc/*.keytab
    If this command produces any file(s), this is a finding.
  "
  desc 'fix', "
    Configure RHEL 8 to prevent system daemons from using Kerberos for
authentication.
    Remove any files with the .keytab extension from the operating system.
  "
  impact 0.5
  tag severity: 'medium'
  tag gtitle: 'SRG-OS-000120-GPOS-00061'
  tag gid: 'V-230238'
  tag rid: 'SV-230238r646862_rule'
  tag stig_id: 'RHEL-08-010161'
  tag fix_id: 'F-32882r567461_fix'
  tag cci: ['CCI-000803']
  tag nist: ['IA-7']

  krb5_server = package('krb5-server')
  krb5_workstation = package('krb5-workstation')
  if 1 
    if 1 

    end
  end
end

However, this code

control 'SV-230238' do
  title "RHEL 8 must prevent system daemons from using Kerberos for
  authentication."
  desc  "Unapproved mechanisms that are used for authentication to the
cryptographic module are not verified and therefore cannot be relied upon to
provide confidentiality or integrity, and DoD data may be compromised.
    RHEL 8 systems utilizing encryption are required to use FIPS-compliant
mechanisms for authenticating to cryptographic modules.
    The key derivation function (KDF) in Kerberos is not FIPS compatible.
Ensuring the system does not have any keytab files present prevents system
daemons from using Kerberos for authentication.  A keytab is a file containing
pairs of Kerberos principals and encrypted keys.
    FIPS 140-2 is the current standard for validating that mechanisms used to
access cryptographic modules utilize authentication that meets DoD
requirements. This allows for Security Levels 1, 2, 3, or 4 for use on a
general-purpose computing system.
  "
  desc  'rationale', ''
  desc  'check', "
    Verify that RHEL 8 prevents system daemons from using Kerberos for
authentication.
    If the system is a server utilizing krb5-server-1.17-18.el8.x86_64 or
newer, this requirement is not applicable.
    If the system is a workstation utilizing
krb5-workstation-1.17-18.el8.x86_64 or newer, this requirement is not
applicable.
    Check if there are available keytabs with the following command:
    $ sudo ls -al /etc/*.keytab
    If this command produces any file(s), this is a finding.
  "
  desc 'fix', "
    Configure RHEL 8 to prevent system daemons from using Kerberos for
authentication.
    Remove any files with the .keytab extension from the operating system.
  "
  impact 0.5
  tag severity: 'medium'
  tag gtitle: 'SRG-OS-000120-GPOS-00061'
  tag gid: 'V-230238'
  tag rid: 'SV-230238r646862_rule'
  tag stig_id: 'RHEL-08-010161'
  tag fix_id: 'F-32882r567461_fix'
  tag cci: ['CCI-000803']
  tag nist: ['IA-7']

  krb5_server = package('krb5-server')
  krb5_workstation = package('krb5-workstation')
  if 1 
    if 1 
        if 1 
            if 1 
                if 1 
                    if 1 
                        if 1 
                            if 1 
                                if 1 

                                end
                            end
                        end
                    end
                end
            end
        end
    end
  end
end

Throws this error:

wasm://wasm/08bd1846:1


RuntimeError: unreachable
    at wasm://wasm/08bd1846:wasm-function[116]:0x8391
    at wasm://wasm/08bd1846:wasm-function[137]:0x91a1
    at RbAbiGuest.rbEvalStringProtect (/Users/sfdcop/Documents/node-syntax-tree/test/node_modules/node-syntax-tree/dist/index.cjs:8321:56)
    at evalRbCode (/Users/sfdcop/Documents/node-syntax-tree/test/node_modules/node-syntax-tree/dist/index.cjs:8928:36)
    at RubyVM.eval (/Users/sfdcop/Documents/node-syntax-tree/test/node_modules/node-syntax-tree/dist/index.cjs:8802:12)
    at /Users/sfdcop/Documents/node-syntax-tree/test/index.js:8:35

Initial thoughts was lack of memory, however, there is no indication of OOM errors or similar. We're using this code to read the file:

const { default: createSyntaxTree } = require("node-syntax-tree");
const fs = require('fs')

createSyntaxTree().then((syntaxTree) => {
    const source = JSON.stringify(JSON.stringify(fs.readFileSync('/Users/sfdcop/Documents/syntax_tree/control.rb', 'utf-8')))

    const res = syntaxTree.rubyVM.eval(
        `source = JSON.parse(${source})
        SyntaxTree::Visitor::JSONVisitor.new.visit(SyntaxTree::HANDLERS['.rb'].parse(source)).to_json`
    ).toString()

    console.log(res)
});

[aaronlippold]

@kddnewton any thoughts on the above issue or how we can discover the root cause?

[kateinoigakukun]

We recently improved error diagnostics for the RuntimeError: unreachable in 0.5.0. I guess it may be helpful for further investigation.

[kddnewton]

@aaronlippold sorry it took me so long to respond. The very first thing I would do is make sure that the file can parse as expected in syntax tree. If that works just fine, make sure it can serialize. If that still works, then I would attempt to upgrade the deps on this package and try again.

[kddnewton]

We're deprecating this in favor of using the WASM bindings to https://github.com/ruby/prism.