Validation of Tax-Calculator Logic
Tax-Calculator computes USA federal income and payroll taxes for a sample of tax filing units in years beginning with 2013. The Python code that performs the tax calculations has been validated in several ways. During the course of development Tax-Calculator results for a number of filing units have been compared to hand calculations performed using IRS tax forms and instructions. In addition, a more systematic program of cross-model validation is part of the ongoing development effort.
The premise behind cross-model validation work is that independently developed tax-simulation models or tax-preparation software are unlikely to contain the same bug, which means looking for differences between the output from two models using the same input is an effective way to locate bugs in tax-calculation logic.
The tools included in this directory support the following validation work flow:
- Generate a random sample of tax filing units (INPUT).
- Generate OUTPUT from INPUT using Tax-Calculator.
- Obtain OUTPUT from INPUT generated by another tax program.
- Generate tax differences by comparing the two OUTPUT files.
The working assumption in our cross-model validation work is that tax differences are more likely than not to be caused by bugs in Tax-Calculator. If exploration of specific differences do confirm a bug, it is corrected and the four-step validation process is repeated again until there are no meaningful differences in the two OUTPUT files.
This four-step validation process can be repeated for different sized INPUT files that vary in the number of input variables used to specify each filing unit's attributes and in the number of filing units included in the INPUT file. A more extensive list of input variables and a larger number of filing units increase the likelihood of finding cross-model differences. In our work, each INPUT file is generated randomly to insure a wide range of filing unit attributes.
Other Tax Programs Used for Cross-Model Validation
Our goal is to repeat the four-step cross-model validation process described above using more than one other tax program with which to compare Tax-Calculator results. The details and results of the four-step process are provided in a different sub-directory for each other model. Here are links to the cross-model validation results that are currently available:
Details on Using the Validation Tools
The current version of the validation tools in this directory tree
should work on Linux or Mac OS X without any changes and without
adding any extra software. Those who want to use these validation
tools on Windows will have to do three things: (a) install an AWK
interpreter, (b) install a Tcl interpreter, and (c) translate each
test.sh bash script into a Windows batch file. The
Free Software Foundation provides a free AWK interpreter for Windows
gawk.exe). ActiveState provides a free Tcl interpreter for Windows