Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid /tmp when running some tests #2129

Merged
merged 1 commit into from Nov 23, 2023
Merged

Avoid /tmp when running some tests #2129

merged 1 commit into from Nov 23, 2023

Conversation

ellert
Copy link
Contributor

@ellert ellert commented Nov 22, 2023

Many features in xrootd require file system support for extended file attributes. Tests that run tests on these features fail if the file system does not support them. The /tmp directory in many Linux installations is using a tmpfs partition. The tmpfs file system does not support extended file attributes, so some tests that use /tmp to store files fail.

This commit changes some affected tests so that they create the temporary directory containing the test files in the current working directory instead of /tmp.

Example of failure:

17/34 Test #20: XrdEc::AlignedWriteTest ...................................................***Failed    0.07 sec
You have selected:

Selected tests/
  Selected tests/MicroTest::AlignedWriteTest

Running:

.F

MicroTest.cc:506:Assertion
Test name: MicroTest::AlignedWriteTest
assertion failed
- Expression: _st.IsOK()
- [*status]: [ERROR] Internal error: std::bad_alloc

Failures !!!
Run: 1   Failure total: 1   Failures: 1   Errors: 0

The following tests FAILED:
	 20 - XrdEc::AlignedWriteTest (Failed)
	 21 - XrdEc::SmallWriteTest (Failed)
	 22 - XrdEc::BigWriteTest (Failed)
	 23 - XrdEc::VectorReadTest (Failed)
	 24 - XrdEc::IllegalVectorReadTest (Failed)
	 25 - XrdEc::AlignedWrite1MissingTest (Failed)
	 26 - XrdEc::AlignedWrite2MissingTest (Failed)
	 27 - XrdEc::AlignedWriteTestIsalCrcNoMt (Failed)
	 28 - XrdEc::SmallWriteTestIsalCrcNoMt (Failed)
	 29 - XrdEc::BigWriteTestIsalCrcNoMt (Failed)
	 30 - XrdEc::AlignedWrite1MissingTestIsalCrcNoMt (Failed)
	 31 - XrdEc::AlignedWrite2MissingTestIsalCrcNoMt (Failed)

In addition to addressing the above failures, the commit also addresses the following warnings during the XRootD::smoke-test test:

34: setfattr: /tmp/xrdfs-test-ChGSEb/01.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks
34: setfattr: /tmp/xrdfs-test-ChGSEb/02.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks
34: setfattr: /tmp/xrdfs-test-ChGSEb/03.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks

Many features in xrootd require file system support for extended file
attributes. Tests that run tests on these features fail if the file
system does not support them. The /tmp directory in many Linux
installations is using a tmpfs partition. The tmpfs file system does
not support extended file attributes, so some tests that use /tmp to
store files fail.

This commit changes some affected tests so that they create the
temporary directory containing the test files in the current working
directory instead of /tmp.

Example of failure:

17/34 Test xrootd#20: XrdEc::AlignedWriteTest ...................................................***Failed    0.07 sec
You have selected:

Selected tests/
  Selected tests/MicroTest::AlignedWriteTest

Running:

.F

MicroTest.cc:506:Assertion
Test name: MicroTest::AlignedWriteTest
assertion failed
- Expression: _st.IsOK()
- [*status]: [ERROR] Internal error: std::bad_alloc

Failures !!!
Run: 1   Failure total: 1   Failures: 1   Errors: 0

The following tests FAILED:
	 20 - XrdEc::AlignedWriteTest (Failed)
	 21 - XrdEc::SmallWriteTest (Failed)
	 22 - XrdEc::BigWriteTest (Failed)
	 23 - XrdEc::VectorReadTest (Failed)
	 24 - XrdEc::IllegalVectorReadTest (Failed)
	 25 - XrdEc::AlignedWrite1MissingTest (Failed)
	 26 - XrdEc::AlignedWrite2MissingTest (Failed)
	 27 - XrdEc::AlignedWriteTestIsalCrcNoMt (Failed)
	 28 - XrdEc::SmallWriteTestIsalCrcNoMt (Failed)
	 29 - XrdEc::BigWriteTestIsalCrcNoMt (Failed)
	 30 - XrdEc::AlignedWrite1MissingTestIsalCrcNoMt (Failed)
	 31 - XrdEc::AlignedWrite2MissingTestIsalCrcNoMt (Failed)

In addition to addressing the above failures, the commit also
addresses the following warnings during the XRootD::smoke-test test:

34: setfattr: /tmp/xrdfs-test-ChGSEb/01.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks
34: setfattr: /tmp/xrdfs-test-ChGSEb/02.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks
34: setfattr: /tmp/xrdfs-test-ChGSEb/03.ref: Operation not supported
34: Extended attributes not supported, using downloaded checksums for server checks
@ellert
Copy link
Contributor Author

ellert commented Nov 22, 2023

Additional observation:

The reason the failing XrdEc tests end up in an uncaught std::bad_alloc exception are the following lines:

int size = xattr->Get( name.c_str(), 0, 0, 0, fd );
buffer.reset( new char[size] );
int ret = xattr->Get( name.c_str(), buffer.get(), size, 0, fd );

If extended attributes are not supported xattr->Get() returns -ENOTSUP. When this small negative value is cast to a size_t in the call to new on the following line it becomes a very large integer and the request to allocate this enormous memory block fails with a std::bad_alloc exception.

Should there be some protection against here?

@amadio amadio self-assigned this Nov 22, 2023
@amadio
Copy link
Member

amadio commented Nov 22, 2023

@ellert Thanks for this, I'm aware of the problems of using /tmp, so I started fixing stuff in 8eff8a4 by skipping some checks on filesystems that don't support attributes, but indeed, my plan before the next release was to put everything in the build directory as you do here. There are still some things like admin paths to set in the configs of the servers used for testing as well to be in the build directory.

I also agree that a check should be added in the place you indicated to fail gracefully rather than trying to create a huge array and end up with std::bad_alloc.

@amadio
Copy link
Member

amadio commented Nov 22, 2023

@ellert There are some typos in the commit message (e.g. attribites), please fix and then I'll merge. Thanks!

@ellert
Copy link
Contributor Author

ellert commented Nov 23, 2023

@ellert There are some typos in the commit message (e.g. attribites), please fix and then I'll merge. Thanks!

I have corrected the typos I saw. Let me know if I missed something.

@amadio
Copy link
Member

amadio commented Nov 23, 2023

No, sorry, that was it.

@amadio amadio merged commit a19c94d into xrootd:devel Nov 23, 2023
13 checks passed
@ellert ellert deleted the avoid-tmpfs branch November 23, 2023 13:52
@amadio amadio added this to the 5.6.4 milestone Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants