Hi @ianfixes ,
the project is super nice! About the test folder, can you expand on that? Do you expect files under test to appear in the menu (as they would if the extension was .ino)?
Or do you simply would like to introduce the test folder as a standard container for tests?

I don't require any UI interaction. I think I'm almost better off if the IDE completely ignores the contents of test/ because I don't think the capability to test an Arduino library should be associated with my system in particular. I just feel that there should be a standardized place to keep unit tests (nearer the code than a subfolder of extras/) and the user is free to choose from any number of Arduino unit test paradigms, to find one that works for them.

That said, if you do want to consider using arduino_ci as a standard method for unit test / CI support, I would recommend bundling g++ with the Arduino IDE install (not avr-gcc, but something that can actually build on the host architecture -- where the tests run).

But the answer to your question is (TL;DR) just introduce test/ as a standard container.

I think what is being requested is for some text to be added to the library specification:
https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#layout-of-folders-and-files
Something like:

Tests

A tests folder can be used by the developer to store test files to be bundled with the library.

The content of the tests folder is totally ignored by the IDE.

Then ensure that the Arduino IDE complies with the specification by ignoring that folder.

As far as I know, it will almost do so already. The only exception is that any sketches under the tests folder will be listed under the File > Examples > {library name} > tests. Even without specifically incorporating the tests folder into the specification, that behavior is not what I'd expect from reading the specification, which seems to suggest that only sketches under the examples folder should appear in the File > Examples menu:
https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#library-examples

Library examples must be placed in the examples folder.

Sketches contained inside the examples folder will be shown in the Examples menu of the IDE.

In fact that behavior already violates the specification when sketches are placed under the extras folder (arduino/Arduino#5688) so that needs to be changed anyway.

Yes, this suggested text looks perfect!

I'm not sure if this would cause a related issue where the IDE would try to pull in test/*.cpp for compilation if someone included a library as a dependency.

As the arduino_ci GitHub Action takes shape, it becomes more difficult for me to change things later.

If there is any great objection to calling the directory test/ (note singular spelling, not plural), I'd value that feedback as early as you can provide it.

Opened #1125