Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider integration with luatest #304

Closed
Totktonada opened this issue May 18, 2021 · 2 comments · Fixed by #310
Closed

Consider integration with luatest #304

Totktonada opened this issue May 18, 2021 · 2 comments · Fixed by #310
Assignees
Labels
feature A new functionality
Milestone

Comments

@Totktonada
Copy link
Member

luatest is the testing framework for tarantool written in Lua. The nice property of luatest is how tests are organized: a test case, setup code, teardown code are all in its own functions. Aside of this, there are a bunch of asserts and helpers, including helpers for spawning new tarantool instances.

The idea is to provide ability to write tests for tarantool using luatest API and run those tests together with existing ones. So we'll able to experiment around different ways to write a test and, if we'll dedice so, perform migration to luatest piece-by-piece.

The rough idea is to teach test-run to work as luatest harness: collect *_test.lua files and run them though luatest. test-run should report results for luatest tests in the same way as for all other ones.

There are other points, where we can integrate luatest and test-run. At least, we can reimplement server.lua to track started instances though test-run (so test-run will kill them at termination -- I don't know whether there is a problem with this in luatest, and will collect RSS usage -- after #277). It is not mandatory for an initial implementation.

@Totktonada Totktonada added the feature A new functionality label May 18, 2021
@kyukhin kyukhin added the teamQ label May 20, 2021
@kyukhin kyukhin added this to the Q2-21 milestone May 20, 2021
@kyukhin
Copy link
Contributor

kyukhin commented May 20, 2021

Let's refactor a couple of more or less complicated tests from the test suite. I'd consider something in replication/ and vinyl/.

@VitaliyaIoffe VitaliyaIoffe self-assigned this May 20, 2021
VitaliyaIoffe added a commit to VitaliyaIoffe/luatest that referenced this issue Jun 9, 2021
    Use luatest assertions for checking tarantool msgpack decode/encode
    functions.

    Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to VitaliyaIoffe/luatest that referenced this issue Jun 9, 2021
Use luatest assertions for checking tarantool msgpack decode/encode
functions.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to VitaliyaIoffe/luatest that referenced this issue Jun 9, 2021
Use luatest assertions for checking tarantool msgpack decode/encode
functions.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to VitaliyaIoffe/luatest that referenced this issue Jun 9, 2021
Use luatest assertions for checking tarantool msgpack decode/encode
functions.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Was added LUATEST_DIR and updated LUA_PATH for finding luatest binary
for execution *_test.lua inside the test-run.

Part of: #304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Provide luatest server, which can find tests by luatest pattern *_test.lua
and execute them 'luatest -v <name>_test.lua -o tap --shuffle none'.

Was changed 'get_filename_by_test' for right working not only with '*.test.*',
but also with '*_test.*'.

Part of: #304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 15, 2021
Use luatest for checking correctness behavioral for encode/decode
functions.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Provide luatest server, which can find tests by luatest pattern *_test.lua
and execute them 'luatest -v <name>_test.lua -o tap --shuffle none'.

Was changed 'get_filename_by_test' for right working not only with '*.test.*',
but also with '*_test.*'.

Part of: #304
@VitaliyaIoffe VitaliyaIoffe linked a pull request Jun 15, 2021 that will close this issue
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Was added LUATEST_DIR and updated LUA_PATH for finding luatest binary
for execution *_test.lua inside the test-run.

Part of: #304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Provide luatest server, which can find tests by luatest pattern *_test.lua
and execute them 'luatest -v <name>_test.lua -o tap --shuffle none'.

Was changed 'get_filename_by_test' for right working not only with '*.test.*',
but also with '*_test.*'.

Part of: #304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Was added LUATEST_DIR and updated LUA_PATH for finding luatest binary
for execution *_test.lua inside the test-run.

Part of: #304
VitaliyaIoffe added a commit that referenced this issue Jun 15, 2021
Provide luatest server, which can find tests by luatest pattern *_test.lua
and execute them 'luatest -v <name>_test.lua -o tap --shuffle none'.

Was changed 'get_filename_by_test' for right working not only with '*.test.*',
but also with '*_test.*'.

Part of: #304
VitaliyaIoffe added a commit that referenced this issue Jun 23, 2021
Provide luatest server, which can find tests by luatest pattern *_test.lua
and execute them 'luatest -v <name>_test.lua -o tap --shuffle none'.

Was changed 'get_filename_by_test' for right working not only with '*.test.*',
but also with '*_test.*'.

Part of: #304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
Use luatest for checking correctness behavioral for encode/decode
functions.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
Use luatest for checking quorum replication.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
Use luatest for checking quorum replication.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
Use luatest for checking quorum replication.

Part of: tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 28, 2021
Add update_optimize_test.lua based on luatest framework.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 28, 2021
Add update_optimize_test.lua based on luatest framework.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 29, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 29, 2021
Add quorum_test.lua based on luatest framework.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Jun 29, 2021
Add update_optimize_test.lua based on luatest framework.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
There are different warnings after adding checks and luatest submodules.

Follows-up tarantool/test-run#304
Totktonada pushed a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
There are different warnings after adding checks and luatest submodules.

Follows-up tarantool/test-run#304
Totktonada pushed a commit to tarantool/tarantool that referenced this issue Oct 22, 2021
There are different warnings after adding checks and luatest submodules.

Follows-up tarantool/test-run#304

(cherry picked from commit 2847977)
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 25, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 25, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 25, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Delete msgpack.test.lua.

Part of tarantool/test-run#304
VitaliyaIoffe added a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Delete msgpack.test.lua.

Part of tarantool/test-run#304
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304
yanshtunder pushed a commit to tarantool/tarantool that referenced this issue Oct 26, 2021
There are different warnings after adding checks and luatest submodules.

Follows-up tarantool/test-run#304
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304

(cherry picked from commit 61bf298)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Delete msgpack.test.lua.

Part of tarantool/test-run#304

(cherry picked from commit 6e163c1)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304

(cherry picked from commit c3e231d)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304

(cherry picked from commit 61bf298)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
Use luatest for checking correctness behavioral for encode/decode
functions

Delete msgpack.test.lua.

Part of tarantool/test-run#304

(cherry picked from commit 6e163c1)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Oct 27, 2021
Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304

(cherry picked from commit c3e231d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants