Testing and validation needs better automation

[linjer]

After following issue #29 , it looks like we need people to perform certain manual tests before we are confident enough to merge PRs.

This may be easier to address with Travis + testing. What is the general status of this project in that regard?

[tatokis]

CI is on the todo list, after we switch to cmake as the primary build system, and cut a first release.

[linjer]

Ah thank you @tatokis . If this is a duplicate of another issue, feel free to close.

[auibrian]

To be fair, this worked flawlessly for me on 10.12.2. I had to upgrade to 10.12.3 to reproduce it. So whatever CI is implemented is going to need to support multiple versions of macOS.

[tatokis]

CI in the current state would be very difficult. Best we could do for now is just put up auto generated debug/testing builds.

[linjer]

@auibrian My understanding is that Travis CI does support multiple versions of OSX and XCode, based on this doc: https://docs.travis-ci.com/user/osx-ci-environment/

@tatokis What are the current challenges? Is there a tag with all the issues which need to be resolved for us to get to 1.0? I'd love to contribute if you can point me in the right direction, I would greatly appreciate it.

[ghost]

@linjer if you mean some kind of release, then it's a floating question, can be speculated on and decided either way by everyone's acceptance or someone's strong-willed decision. If you really mean 1.0, the project might not reach it in the nearest future, according to an old roadmap. If you are looking for challenges, the problems tagged as bugs are the most severe ones.

[tatokis]

ckb-next/ckb-next#12