-
Notifications
You must be signed in to change notification settings - Fork 0
/
testing-netcat.html
7 lines (7 loc) · 7.89 KB
/
testing-netcat.html
1
2
3
4
5
6
7
<html><body>
<head>
<title>Testing netcat | Oracle Czech techie's adventures Blog</title>
<meta name="publish_date" content="2008-11-20 01:22:00">
</head>
<p>After multiple rounds of code review the netcat (or <tt>nc</tt>) test suite is now finally in the <tt>onnv-stc2</tt>gate.<br/>The test suite has <a href="http://opensolaris.org/os/community/networking/tests/nc/">its home</a> in the OpenSolaris <br/><a href="http://opensolaris.org/os/community/networking/">Networking community</a> (see<br/><a href="http://opensolaris.org/os/community/networking/tests/">the networking tests page</a> for the list of networking<br/>test suites).<br/><br/><br/>The source code is present in the<br/><a href="http://src.opensolaris.org/source/xref/test/ontest-stc2/src/suites/net/nc/"><tt>src/suites/net/nc/</tt></a><br/>directory and <tt>SUNWstc-netcat</tt> packages can be downloaded from <br/><a href="http://dlc.sun.com/osol/test/downloads/current/">OpenSolaris Download center</a>.</p><p>Before I go further, this is how it looks like when the test suite is run (the output is trimmed a bit):</p><pre>vk:honeymooners:/opt/SUNWstc-nc$ run_test nc<br/>Validating Arguments...<br/>New TET_ROOT for this run : /var/tmp/honeymooners_27828<br/>The results will be available in /var/tmp/results.27828<br/>tcc: journal file is /var/tmp/results.27828/testlog<br/>12:45:57 Execute /tests/dflag/tc_dflag<br/>12:46:04 Execute /tests/hflag/tc_hflag<br/>12:46:05 Execute /tests/kflag/tc_kflag<br/>12:46:11 Execute /tests/nflag/tc_nflag<br/>12:46:15 Execute /tests/portranges/tc_portranges<br/>12:46:23 Execute /tests/pflag/tc_pflag<br/>12:46:26 Execute /tests/sflag/tc_sflag<br/>12:46:35 Execute /tests/Uflag/tc_Uflag<br/>12:46:36 Execute /tests/vflag/tc_vflag<br/>12:46:43 Execute /tests/zflag/tc_zflag<br/>12:46:46 Execute /tests/iflag/tc_iflag<br/>12:46:59 Execute /tests/lflag/tc_lflag<br/>12:47:29 Execute /tests/rflag/tc_rflag<br/>12:48:16 Execute /tests/Tflag/tc_Tflag<br/>12:48:33 Execute /tests/uflag/tc_uflag<br/>12:48:50 Execute /tests/wflag/tc_wflag<br/>##################################################<br/>TC /tests/dflag/tc_dflag<br/>TP 1 tc_dflag PASS<br/>##################################################<br/>TC /tests/hflag/tc_hflag<br/>TP 1 tc_hflag PASS<br/>...<br/>##################################################<br/> SUMMARY <br/> ======= <br/>Number of Tests : 50<br/>PASS : 50<br/>FAIL : 0<br/>UNRESOLVED : 0<br/>UNINITIATED : 0<br/>OTHER : 0<br/>##################################################<br/>Test Logs are at /var/tmp/results.27828, Journal File = /var/tmp/results.27828/testlog <br/>vk:honeymooners:/opt/SUNWstc-nc$</pre><p>It's been almost a year since I started developing the test suite last Christmas (see <br/><a href="http://blogs.sun.com/vlad/entry/test_suite_for_netcat">the initial blog entry about nc-tet</a>).<br/>Since then, I have lost part of the source code in <br/><a href="http://blogs.sun.com/vlad/entry/zfs_is_going_to_save">hard drive crash</a>, had to redo the<br/>source tree structure, fix <tt>ksh</tt> style, fix numerous bugs in test suite code and make the test suite more robust.<br/>One might ask whether having test suite for such a simple program like nc(1) was worth the hassle.<br/>I have only one answer to that: absolutely. First, it gives a confidence of not breaking (most of; see below)<br/>existing things when changing/adding functionality and second it helped me (and I hope the others participating/observing<br/>the code review on <tt>testing-discuss</tt> too) to explore what it takes to write a test suite from scratch <br/>(I will not go here into details whether I prefer <br/><a href="http://opensolaris.org/os/community/testing/testsuites/ctifortet/">CTI-TET</a> over<br/><a href="http://opensolaris.org/os/community/testing/testsuites/stf/">STF</a> and vice versa).</p><p>The <a href="http://www.amazon.com/Beautiful-Code-Leading-Programmers-Practice/dp/0596510047/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1213904030&sr=8-1">Beautiful code</a> book (which I really recommend for anyone tinkering with any source code) contains a chapter called <i>Beautiful tests</i><br/>by Alberto Savoia. I hope that at least some of the test purposes in nc test suite have some degree of beautifulness of at least one <br/>of the ways highlighted by Alberto (1. simplicity/efficiency, 2. help making the software being tested better in terms of<br/>quality and testability, 3. breadth/thoroughness).</p><p>One of the important questions for a test suite is <a href="http://en.wikipedia.org/wiki/Code_coverage">code coverage</a> level.<br/>Obviously, for software adhering to the OpenSolaris <br/><a href="http://www.opensolaris.org/os/community/arc/policies/interface-taxonomy/">interface taxonomy model</a> it is important<br/>that the test suite exercises all of the <i>Committed</i> interfaces and execution paths around those interfaces.<br/>For nc(1) this means a subset of the command line options and their arguments (see <a href="http://www.opensolaris.org/os/community/arc/caselog/2007/389/">PSARC 2007/389</a> for the actual list).<br/>The key is certainly to test the features which are likely to break with an intrusive code change.</p><p>Very crude view of test coverage for nc(1) test suite (counting test purposes gives only very remote idea about <br/>real coverage but at least provides visual image) looks like this:</p><pre> rflag: +<br/> Tflag: +++++---<br/> pflag: +<br/> iflag: +-<br/> vflag: ++<br/> kflag: +<br/> Uflag: +-<br/> dflag: +<br/> uflag: ++-<br/> sflag: +-<br/> hflag: +<br/> nflag: +-<br/> wflag: +<br/> portranges: +---<br/> lflag: ++++++++----------</pre><p>One plus character stands for one positive test purpose, minus is negative test purpose.</p><p>Side note: the above ASCII graph was produced using <a href="http://blogs.sun.com/vlad/resource/test-coverage-graph.sh"><br/><tt>test-coverage-graph.sh</tt></a> script (which presumes certain naming scheme for test purpose files).<br/>Just pipe a file listing into the script with test purpose filenames compliant to the scheme used in <br/><a href="http://src.opensolaris.org/source/xref/test/ontest-stc2/">ontest-stc2</a> gate and it will spew out graph<br/>similar to the above.</p><p>In the above musing about code coverage I left out an important piece - why some of the features are not tested.<br/>For nc(1) the yet untested part is the SOCKS protocol support. Basically, this is because test suite environment<br/>does not contain SOCKS server to test against. There might not be many people using the <tt>-x</tt>/<tt>-X</tt><br/>options but from my own experience nothing is more frustrating than discovering some old dusty corner which had <br/>to be fixed long time ago or removed completely. So for now, on my workstation which sits behind SOCKS proxy<br/>I have the following in <tt>~/.ssh/config</tt> for a server outside corporate network which hosts my personal<br/>mailbox so it is accessed every day:</p><pre>Host bar<br/> User foo<br/> Hostname outside.kewl.org<br/> # NOTE: for nc(1) testing<br/> ProxyCommand /usr/bin/nc -x socks-proxy.foothere.bar outside.kewl.org %p<br/> ForwardAgent no<br/> ForwardX11 no</pre><p>This ensures (along with upgrades of the workstation to recent Nevada builds periodically) that SOCKS<br/>support gets tested as well. And yes, ssh-socks5-proxy-connect(1) and ssh-http-proxy-connect(1) are not<br/>really needed.</p><p>Now with the test suite in place, anybody modifying nc(1) (there are some RFEs for nc in the <br/><a href="http://opensolaris.org/os/bug_reports/oss_bite_size/">oss-bit-size</a> list and other bugfixes<br/>or features are also welcome) can have pretty high confidence that his change will not break things.<br/>Yes, this means that more nc(1) features are coming.</p>
</body></html>