Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: small test suite is not run for out-of-source build #4485

Closed
Totktonada opened this issue Sep 10, 2019 · 1 comment
Closed

test: small test suite is not run for out-of-source build #4485

Totktonada opened this issue Sep 10, 2019 · 1 comment
Assignees
@ylobankov
Labels
qa Issues related to tests or testing subsystem

Comments

@Totktonada
Copy link
Member

Tarantool version: 2.3.0-93-g664788a31.
OS version: Linux.

cd tarantool
mkdir build
cmake .. -DCMAKE_BUILD_TYPE=Debug -DENABLE_BACKTRACE=ON -DENABLE_DIST=ON && make -j
TEST_RUN_TESTS="small" make test

It was observed during moving of luajit-tap test suite to luajit repository, see #4478. The issue affects all external test suites: now it is small; but later it will be luajit-tap too.

I briefly look into this problem and it seems that test/CMakeLists.txt creates symlinks in a build directory (see 23032bb), while test-run searches for tests only in a source directory. It seems that test-run was changed at some point and we missed this change. I suspect that it was non-intentionally changed in tarantool/test-run@b42093a .

We also need to ensure that small tests are built during make: esp. that it is before make test, make test-force, make all.
@Totktonada Totktonada added the qa Issues related to tests or testing subsystem label Sep 10, 2019
@kyukhin kyukhin added this to the 2.4.1 milestone Sep 26, 2019
@kyukhin kyukhin modified the milestones: 2.4.1, 2.6.1 Apr 10, 2020
@avtikhon avtikhon self-assigned this Apr 29, 2020
@avtikhon avtikhon added this to DOING in Quality Assurance Apr 29, 2020
@avtikhon avtikhon removed this from DOING in Quality Assurance May 15, 2020
@kyukhin kyukhin modified the milestones: 2.6.1, wishlist Oct 23, 2020
@Totktonada Totktonada added teamQ and removed teamL labels Dec 12, 2021
ylobankov added a commit to ylobankov/tarantool that referenced this issue Jun 27, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
1a1f10a 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
ylobankov added a commit to ylobankov/tarantool that referenced this issue Jun 27, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
8ce5558 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
ylobankov added a commit to ylobankov/tarantool that referenced this issue Jun 27, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
de41c65 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
ylobankov added a commit to ylobankov/tarantool that referenced this issue Jun 28, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
90328e0 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
@ylobankov ylobankov self-assigned this Jun 28, 2022
@Totktonada
Copy link
Member Author

The luajit-tap test suite was converted to a separate make target in 07c83aa and those tests are not run under test-run anymore. It seems, the problem now affects only small tests.

ylobankov added a commit to ylobankov/tarantool that referenced this issue Jul 1, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
4946af5 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
ylobankov added a commit to ylobankov/tarantool that referenced this issue Jul 1, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
93e79cb 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
ylobankov added a commit that referenced this issue Jul 2, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
f8a332f 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes #4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 28426f6)
ylobankov added a commit that referenced this issue Jul 22, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
4e59013 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes #4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 28426f6)
ylobankov added a commit that referenced this issue Jul 27, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
2fe2dec 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes #4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 28426f6)
ylobankov added a commit that referenced this issue Jul 27, 2022
@ylobankov
test: fix running 'small' lib tests for OOS build
2615310 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes #4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 28426f6)
@kyukhin kyukhin removed this from the wishlist milestone Aug 25, 2022
mkokryashkin pushed a commit to mkokryashkin/tarantool that referenced this issue Sep 9, 2022
@ylobankov @mkokryashkin
test: fix running 'small' lib tests for OOS build
297cdf5 
The 'small' lib test suite was not run for out-of-source builds since
the wrong symlink was created for test binaries and test-run couldn't
find them. Now it is fixed.

When test-run loads tests, first, it searches the suite.ini file and if
it exists test-run consider the dir as a test suite. So there was sense
to create a permanent link for 'small' lib tests.

Closes tarantool#4485

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ylobankov ylobankov

Labels
qa Issues related to tests or testing subsystem
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Totktonada @ylobankov @kyukhin @avtikhon