You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This seems to be more tricky than initially thought. For DuckDB and SQLite, running SQLancer might cause the JVM to crash when a memory error is triggered (see #1). Also for the other DBMS, it might be that bugs are still triggered on their latest version, either since a new bug was introduced or because an old one has not yet been fixed. An option would be to disable checking whether SQLancer finds a bug, but then we would also not detect any regression bugs in SQLancer.
I think the optimal setup would be if the DBMS under test would, for each commit to it, pull the latest (release) version of SQLancer and run it for a while (e.g., 10 minutes), to prevent that any new bugs are introduced to the DBMS. SQLancer could, in turn, test the latest (release) version of the DBMS to check that SQLancer is not obviously broken.
It would be desirable to add smoke testing to the CI, to verify that the testing implementation for one of the DBMS is not obviously broken.
The text was updated successfully, but these errors were encountered: