-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider CI/testing #95
Comments
For personal projects I use https://bors.tech which lets me use Travis in a controlled way: I write |
Yes. it took me at least 6 months to develop a knack for knowing which intermittents might be my fault and which probably (hopefully) aren't. |
That sounds good. I'll check it out. |
@brendanzab just showed me his configuration for github actions. This looks really neat and probably what we want for now. I'm still intrigued by bors, but it's probably more than we need for now. https://help.github.com/en/actions/automating-your-workflow-with-github-actions |
The advantage of bors is that you don't have to sit around waiting for the green CI thing to show - you just say |
Regarding bors, I've read up on it and I'm all-on-board. But CI is a prerequisite so we'll do that first. For CI there's things we can do now, and some things we can do later. Now
LaterSome of these can be lifted out of "make test" so that we speed-up "make test" but still catch problems in CI.
Much LaterPlasma has not been tested on these combinations at all, so CI doesn't yet make sense. Or cloud hosting isn't available.
Test suite |
I'm not an expert on CI/CD. I'm fairly sure CD is not what we want. And
I'm not sure of the precise meaning of CI. So bear with me if I get
something wrong.
Compiler
Plasma has a compiler written in Mercury that has no system specific parts
beyond what Mercury has in its own runtime. it has a tiny bit of foreign
code in C but it's pretty simple.
We should test with:
Note that development versions of Mercury are currently broken on 32 bit
systems.
Runtime
The rumtime is currently fairly system agnostic, at least until C++ was
added. It uses a mix of C and C++, and should be tested with at least two
compilers. There is also a build option to enable extra asserts and other
development options.
We should test with:
If possible:
to work on something else then I'd add this to "should").
I'm also in the process of adding a "zealous" mode to the GC, which performs
a lot more checking but runs slower. And this is why I'm interested in CI.
I don't think we want this to run when people type "make test", not by
default. But it does make sense to run it for each commit / pull request.
So there's a few things that would be useful to begin testing now. Or at
the very least to think about testing. So here's where I throw it over to
you (the reader) since you probably know more about this than me.
New GC mode
So the first question, is for the new GC mode should this be enabled as a
compiler option or an environment variable? I'm imagining compiling it when
PZ_DEV is defined and enable it when an environment variable is set. I
imagine that typical CI stuff can handle this but please tell me if it
isn't.
Hosted CI?
When I've worked on other projects, things like "Travis" would run when I
opened a pull request. Has anyone worked with or set this up before? Can
it support what we need?
Alternatively I've setup Jenkins before, although I never finished the full
configuration, for a non-hosted thing I think Jenkins seems good. Does
anyone have any suggestions between hosted/non-hosted and which service to
use?
TAP
It was suggested (Bug #28, Bug #6) that Test Anything Protocol (TAP) can be used to gather results from a test suite and integrate with a CI tool. Should we use this? is it the best option? What other work needs to be done to the test suite / Makefile to make them work with a CI tool?
Thanks.
Also here on the mailing list: http://www.plasmalang.org/list-archives/dev/2018-December/000027.html
The text was updated successfully, but these errors were encountered: