HADOOP-18781. ABFS backReference passed down to streams to avoid GC closing the FS.(#5780)

[mehakmeet]

Description of PR

To avoid the ABFS instance getting closed due to GC while the streams are working, attach the ABFS instance to a backReference opaque object and passing down to the streams so that we have a hard reference while the streams are working.

How was this patch tested?

mvn -Dparallel-tests=abfs -DtestsThreadCount=8 -Dscale clean verify

[INFO] Results:
[INFO] 
[WARNING] Tests run: 141, Failures: 0, Errors: 0, Skipped: 5

[INFO] Results:
[INFO] 
[WARNING] Tests run: 586, Failures: 0, Errors: 0, Skipped: 76

[INFO] Results:
[INFO] 
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+0 🆗	mvndep	16m 29s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 36s		branch-3.3 passed
+1 💚	compile	23m 14s		branch-3.3 passed
+1 💚	checkstyle	3m 0s		branch-3.3 passed
+1 💚	mvnsite	2m 34s		branch-3.3 passed
+1 💚	javadoc	1m 29s		branch-3.3 passed
+1 💚	spotbugs	3m 52s		branch-3.3 passed
+1 💚	shadedclient	41m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 40s		the patch passed
+1 💚	compile	19m 9s		the patch passed
+1 💚	javac	19m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 59s		the patch passed
+1 💚	mvnsite	2m 32s		the patch passed
+1 💚	javadoc	1m 24s		the patch passed
+1 💚	spotbugs	4m 8s		the patch passed
+1 💚	shadedclient	41m 27s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 10s		hadoop-common in the patch passed.
+1 💚	unit	2m 29s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 58s		The patch does not generate ASF License warnings.
		247m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5830/1/artifact/out/Dockerfile
GITHUB PR	#5830
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 9b77f63afa4d 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / c9cf96a
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5830/1/testReport/
Max. process+thread count	3144 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5830/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[steveloughran]

thanks. nice bit of work. and we've learned something new which will forever taint our brains.