Runs shell unit test frameworks (bats, shunit2, etc.)

[andychu]

https://github.com/bats-core/bats-core/tree/master/libexec/bats-core

It appears to be pretty commonly used:

https://github.com/bats-core/bats-core/wiki/Projects-Using-Bats

Someone told me a basic example worked, but I also noticed that the code uses printf -v, so it must not work entirely.

Note that bats changed maintainers sometime in 2016:

sstephenson/bats#150 (comment)

[andychu]

This would be a great issue for someone who uses bats! Tell me what goes wrong :)

The underlying motivation is that I want OSH to have some kind of testing story, and it's good to be familiar with what people already use. I haven't used bats myself though!

[andychu]

more feedback about bats here:

https://lobste.rs/s/jrzobc/better_bashing_through_technology#c_rtyj0d

Idea: we could parse bats literally with

shopt --set oil:basic
shopt --unset parse_at {
   @test "foo" {
     echo hi
   }
}

shopt --unset parse_at {
  source mytest.sh
}

Or just have people change it to

%test "foo" { 
  echo hi
}

could be .test, +test, ^test, etc. I guess the thing to do would be to patch bats to allow %test in addition to @test ... hm.

Yeah it would be a one character change here: https://github.com/bats-core/bats-core/blob/master/libexec/bats-core/bats#L74

used here:

https://github.com/bats-core/bats-core/blob/master/libexec/bats-core/bats-preprocess

[andychu]

Many more test frameworks listed here: https://news.ycombinator.com/item?id=26314160

• shunit2, etc.
• shellspec, which we already run

[andychu]

This is important because running unit tests could be the first thing that users do to port!

[andychu]

Many examples of bats here: https://github.com/dolthub/dolt/tree/master/bats

However they don't appear to have a continuous build to verify it? Or at least no badge

[andychu]

Nice article that compares ShellSpec, BATS, and shUnit2

https://lobste.rs/s/2soumt/writing_unit_tests_mocks_for_unix_shells

I think the summary is that

• ShellSpec has a DSL they didn't like
• BATS not compatible with busybox ash?
• shUnit2 is good

[andychu]

Note that bats uses export -f, so we're not likely to run it soon

https://lobste.rs/s/2soumt/writing_unit_tests_mocks_for_unix_shells

[bar-g]

Um, since existing testing frameworks don't seem to be readily supported, how would you imagine/picture/sketch a good testing practice when using the oil language?

It could be quite useful to devise a plan quite early on, to support pioneer users in staying on top of changes, while the oil language is still undergoing considerable stabilizing and debugging phase.

One pattern that stuck out to me as mixing and matching quite well from some quick tests (within a single source file) up to a large testing suite's directory tree (with execution concurrency and process isolation) was the way Nim does it: https://nim-lang.github.io/Nim/unittest.html

The test file example there even seemed to remember me of hay in some way... but of course, nim is still missing the direct "command language" of a shell, even though nim is itself already written in nim, and compiles to C, C++, and more. It even supports being compiled into a nim-script run-time interpreter.

[andychu]

Sorry for the delay ... I'm sort of torn between writing a new test framework and running old ones.

A barrier to this is that I don't use any of these test frameworks :-P

Oil itself has a lot of shell tests, and we can definitely use a better shell to write those !

---

I think this work will have to come after argparse

• argparse builtin: Replacement / enhancement for getopts #469

That is a similar problem in that Oil language features are necessary to support a rich interface

i.e. we can either write arg parsers and test frameworks in Oil, or in the meta-language Python

So that is probably the first thing to work out

[bar-g]

(Thanks, as you know I try to check in here from time to time, but can't on on all news and discussions.)

torn between writing a new test framework and running old ones.

Ok, but with hay being a thing now (is it better to say hay-markup?, hay-config/stanzas? or hay-??? to steer clear from referring to a pile of straws?) , now i don't see any old framework could usefully support hay, not even just for sample data. And hay seems so useful in writing tests even beyond just "fixtures": https://guides.rubyonrails.org/testing.html#the-low-down-on-fixtures

So I'd guess it may inevitably end up needing some own oil testing conventions and hooks similar to https://nim-lang.github.io/Nim/unittest.html