I'm not an expert on CI/CD. I'm fairly sure CD is not what we want. And
I'm not sure of the precise meaning of CI. So bear with me if I get
something wrong.
Compiler
Plasma has a compiler written in Mercury that has no system specific parts
beyond what Mercury has in its own runtime. it has a tiny bit of foreign
code in C but it's pretty simple.
We should test with:
- 32 and 64 bit systems,
- stable and development versions of Mercury.
Note that development versions of Mercury are currently broken on 32 bit
systems.
Runtime
The rumtime is currently fairly system agnostic, at least until C++ was
added. It uses a mix of C and C++, and should be tested with at least two
compilers. There is also a build option to enable extra asserts and other
development options.
We should test with:
- 32 and 64 bit systems
- -DPZ_DEV
If possible:
- le and be if possible
- More than one OS (Linux and OS X have been tried so far, if it was known
to work on something else then I'd add this to "should").
- Systems that assert for unaligned accesses if possible.
- valgrind / memcheck / ubcheck / etc.
I'm also in the process of adding a "zealous" mode to the GC, which performs
a lot more checking but runs slower. And this is why I'm interested in CI.
I don't think we want this to run when people type "make test", not by
default. But it does make sense to run it for each commit / pull request.
So there's a few things that would be useful to begin testing now. Or at
the very least to think about testing. So here's where I throw it over to
you (the reader) since you probably know more about this than me.
New GC mode
So the first question, is for the new GC mode should this be enabled as a
compiler option or an environment variable? I'm imagining compiling it when
PZ_DEV is defined and enable it when an environment variable is set. I
imagine that typical CI stuff can handle this but please tell me if it
isn't.
Hosted CI?
When I've worked on other projects, things like "Travis" would run when I
opened a pull request. Has anyone worked with or set this up before? Can
it support what we need?
Alternatively I've setup Jenkins before, although I never finished the full
configuration, for a non-hosted thing I think Jenkins seems good. Does
anyone have any suggestions between hosted/non-hosted and which service to
use?
TAP
It was suggested (Bug #28, Bug #6) that Test Anything Protocol (TAP) can be used to gather results from a test suite and integrate with a CI tool. Should we use this? is it the best option? What other work needs to be done to the test suite / Makefile to make them work with a CI tool?
Thanks.
Also here on the mailing list: http://www.plasmalang.org/list-archives/dev/2018-December/000027.html
I'm not an expert on CI/CD. I'm fairly sure CD is not what we want. And
I'm not sure of the precise meaning of CI. So bear with me if I get
something wrong.
Compiler
Plasma has a compiler written in Mercury that has no system specific parts
beyond what Mercury has in its own runtime. it has a tiny bit of foreign
code in C but it's pretty simple.
We should test with:
Note that development versions of Mercury are currently broken on 32 bit
systems.
Runtime
The rumtime is currently fairly system agnostic, at least until C++ was
added. It uses a mix of C and C++, and should be tested with at least two
compilers. There is also a build option to enable extra asserts and other
development options.
We should test with:
If possible:
to work on something else then I'd add this to "should").
I'm also in the process of adding a "zealous" mode to the GC, which performs
a lot more checking but runs slower. And this is why I'm interested in CI.
I don't think we want this to run when people type "make test", not by
default. But it does make sense to run it for each commit / pull request.
So there's a few things that would be useful to begin testing now. Or at
the very least to think about testing. So here's where I throw it over to
you (the reader) since you probably know more about this than me.
New GC mode
So the first question, is for the new GC mode should this be enabled as a
compiler option or an environment variable? I'm imagining compiling it when
PZ_DEV is defined and enable it when an environment variable is set. I
imagine that typical CI stuff can handle this but please tell me if it
isn't.
Hosted CI?
When I've worked on other projects, things like "Travis" would run when I
opened a pull request. Has anyone worked with or set this up before? Can
it support what we need?
Alternatively I've setup Jenkins before, although I never finished the full
configuration, for a non-hosted thing I think Jenkins seems good. Does
anyone have any suggestions between hosted/non-hosted and which service to
use?
TAP
It was suggested (Bug #28, Bug #6) that Test Anything Protocol (TAP) can be used to gather results from a test suite and integrate with a CI tool. Should we use this? is it the best option? What other work needs to be done to the test suite / Makefile to make them work with a CI tool?
Thanks.
Also here on the mailing list: http://www.plasmalang.org/list-archives/dev/2018-December/000027.html