Version 7.6.3

Updates for 7.6.3

Some issues were found in the recent 7.6.2 release that needed to be fixed, so we are releasing a bug-fix release, 7.6.3. The changes are as follows.

1. The Class.getInstance() method in the reflection API was deprecated; a workaround was implemented for this. Needed for Causal Command.
2. Fixed an interface bug affecting 2-factor random MIM graph generation.
3. Removed extraneous algorithms, tests, and scores from the interface.
4. Adding missing manual docs for remaining algorithms, tests, and scores, so that all algorithms, tests, and scores in the Search box now have descriptions.

Version 7.6.2

Updates for 7.6.2

Note that starting with this version, all Tetrad jars will be compiled using Corretto JDK 17. Also, the language level of the project will be increased to 17.

These are the new items that impact the Tetrad interface:

1. There is a new tool for checking whether background knowledge is violated for a search, which is useful for checking ground truth knowledge for a large graph.
2. The Chi-Square and G-Square discrete tests have been improved. There is now a parameter for them, minimum expected count per cell.
3. The method for randomizing Bayes IMs has been improved to make these models more Powerful.
4. The Discrete BIC score has been corrected in places.
5. A pseudoinverse option has been added for all relevant tests and scores to avoid singularity exceptions.
6. The Markov Checker now accepts knowledge as an input. For independence facts Ind(X, Y | Z), X and Y are now constrained to come from the last tier of the knowledge, whereas Z can come from any variable in any tier.
7. In the Graph box, a menu item has been added to highlight all triangles in a graph.
8. All independence test results are now cached, including for m/d-separation, but excluding ProbabilisticTest when threshold = true (which has an element of randomness). Also, the algorithm for m/d-separation has been sped up considerably. As a result, one should be able to use PC or FCI to calculate correct CPDAGs or PAGs, respectively, for even large graphs, for instance, and Markov Checker should run faster.
9. BIC scores are available now in the interface for both continuous and discrete DAG and CPDAG models.
10. The Processing... dialog should now follow the Tetrad interface window if you move the window.
11. If you type command-Q you now get a confirm dialog. This is to address the old problem of Tetrad quitting unexpectedly while saving out a session, resulting in the session being corrupted.
12. Added a seed parameter to GRaSP, BOSS, GRaSP-FCI, BFCI, and a few other algorithms that use BOSS or GRaSP. Setting the seed to something other than -1 allows for reproducibility for those algorithms.
13. Added a feature to copy the selected dataset from a data box with multiple data models into a separate box.

Details.

1. Sped up M/D-Separation.
2. Added a tool to let users check violations of knowledge for a graph.
3. Made G-Square an option in ChiSquareTest and eliminated the previous GSquare class.
4. Added a parameter for the minimum expected count per cell, for the Chi-Square and G-Square tests, defaulting to 1.
5. Added the isValid parameter to independence results if an algorithm wants to use this information. A result is invalid if the test cannot determine whether the result is 'dependent' or 'independent.'
6. Moved docs to lib/resources.
7. Added Anderson-Darling test of U(0, 1) to the Markov checker.
8. Added IMaGES-BOSS algorithm.
9. Changed the causal order layout to judge both --> and o-> as putting nodes in a subsequent tier.
10. Adjusted the randomization of Bayes IMs to reflect, e.g., the ALARM network, where, to the extent possible, a different category in each CPT row has a high probability, and other categories have a low probability. This has much higher power.
11. Fixed the sign of the penalty term for DiscreteBIC.
12. Added the F-beta statistic.
13. Changing the default structure prior to the BDeu score to 0.
14. Re-introduced the 'pcHeuristic' parameter, which allows one to select one of the PC heuristics given in Spirtes et al., Causation, Prediction, and Search, for algorithms using the Fast Adjacency Search.
15. Made some corrections to the Discrete BIC score for the case where CPTs had rows summing to zero.
16. Added an option in the SEM BIC score to use pseudoinverses rather than inverses for calculating residual variances.
17. Added the pseudoinverse option to the Fisher Z test.
18. Extended the pseudoinverse option to all other feasible tests and scores that follow the same pattern.
19. In the Simulation box, changed the menu item for "Bayes net" to "Bayes Net (Multinomial)" and "SEM BIC" to "Linear SEM BIC" for clarity and slightly reordered the options. The underlying code for these did not change.
20. Added "local Markov check" options to the Markov checker--using a knowledge object to specify for Ind(X, Y | Z) which set X and Y come from (the last tier) and which set the Z variables come from (previous tiers).
21. Added a menu item for the graph box to highlight all triangles in a graph.
22. Added caching of independence results for all independence tests (including M/D-Sep, and except for ProbabilisticTest when threshold = true, which has an element of randomness).
23. Reverted the GrowShrink class to the published algorithm.
24. Changed the language level of the Tetrad codebase to 17 and specified that all jars starting with this version will be compiled under Corretto's JDK 17.
25. Fixed BIC score stamping for all relevant algorithms, both continuous and discrete.
26. Added a note that BIC is calculated as 2L - k ln N (or 2L - c k ln N) everywhere the term appears in the documentation.
27. Caught the event of a desktop being quit in the interface (as with command-Q on a Mac, which could not be handled in earlier versions), now displaying a confirm dialog.
28. Updated the saving/loading code in the interface to be a bit smoother.
29. Changed the wording of the "Processing..." button to "Processing (click to stop)...".
30. Added code to center the Processing... dialog above the Tetrad interface app frame even if the app is moved to a different monitor.
31. Fixed a bug in the Markov checker where not all dependencies were being processed.
32. Added a seed parameter to GRaSP, BOSS, GRaSP-FCI, BFCI, and a few other algorithms that use BOSS or GRaSP. Setting the seed to something other than -1 allows for reproducibility for those algorithms.
33. Added a feature to copy the selected dataset from a data box with multiple data models into a separate box.

Version 7.6.1

Updates for 7.6.1 (hotfix):

1. Fixed a bug introduced in 7.6.0 where, for tiered knowledge, too many edges were forbidden per tier.
2. Fixed an issue in a snapshot build (not 7.6.0) where you couldn't make a Tiers and Edges editor if you put a knowledge box on the workbench and double-clicked it.
3. Adding an instruction in the Markov Checker editor that the contents of the table can be copied and pasted into Excel.
4. Fixed a backward compatibility issue with 7.5.0 for knowledge.
5. Stamping CPDAG algorithms with both the score used to generate them (where applicable) and standard BIC.
6. Changed the name of PC-LiNGAM to BOSS-LiNGAM.
7. Fixed problem with BOSS, GRaSP, etc. code hanging with constant columns.
8. [Code] Added code to py-tetrad to run the Markov Checker, which required moving the enum ConditioningSetType to a higher level.

Version 7.6.0

Updates for 7.6.0:

1. Added parallelization code to BOSS.
2. Fixed some knowledge issues in BOSS and GRaSP.
3. Fixed sluggishness of graph display.
4. Fixed the issue in some graph layouts where cyclic graphs would throw exceptions.
5. Fixed the problem where sometimes, in IntelliJ, parameter defaults would not be available if you launch the Tetrad app.
6. Fixed a problem in which, for time lag knowledge, the knowledge was not getting into some of the FCI-style algorithms.
7. Fixed variable ordering problem for knowledge editor (1, 2, 3; not 1, 10, 2. 3).
8. Adjusted code for all independence tests so exceptions are thrown when encountering singularities.
9. Fixed SEM IM so parameter settings are used in the SEM IM editor in data simulation.
10. Added a menu item to the PlotMatrix interface to remove zeros per plot.
11. Changed the default layout to circle for # notes <= 20 and square otherwise.
12. Revised PC-LiNGAM code for undirected edges to do BOSS + non-gaussian orientation (for scalability).
13. Added PC-LiNGAM to the interface.
14. Added DirectLiNGAM to the interface.
15. Added DAGMA to the interface.
16. Added CFCI to the interface.
17. Fixed a bug in Knowledge where deleting required or forbidden edges in the Knowledge editor didn't guarantee that they were deleted from Knowledge.
18. Fixed a Knowledge bug preventing Knowledge Groups from behaving correctly.
19. Fixed a bug in the forbidden edges iterator in Knowledge that allowed edges to be oriented against knowledge.
20. Added some checks so that edges forbidden in Knowledge cannot be required and vice-versa.
21. Fixed a problem with testwise deletion for the Fisher Z test.

Code-only updates:

1. (Code) Collapsed duplicate methods for detecting semidirected paths.
2. (Code) Moved all data transform and graph transform methods into DataTransform and GraphTransform, respectively (a change made for Tetrad-FX). This corrects the continuous, discrete, and mixed data loading method in SimpleDataLoader.
3. (Code) Fixed some Javadoc errors preventing successful build.
4. (Code) Copied some layout methods from Tetrad-FX and some modified methods for loading data.
5. (Code) Factored out the method in Markov Check that generates the lists of all m-separation and m-connecting independence checks from the all sets option in the Markov Checker for use in rpy-tetrad.

Updates for py-tetrad and rpy-tetrad:

1. Added PC-LiNGAM methods to TetradSearch (available in Python and R).
2. Added DirectLiNGAM methods to TetradSearch (available in Python and R).
3. Added DAGMA methods to TetradSearch (available in Python and R).
4. Added a method to TetradSearch to check whether a graph is a legal PAG and a method to determine why not.
5. Added a method to TetradSearch to list all independence facts checked (or dependence facts checked) by the Markov Checker in the all-sets option.
6. Updated the Tetrad jar for py-tetrad.

v7.5.0

Updates for 7.5.0

General comment:

• For version 7.5.0, we will do separate releases for JDK 1.8 and JDK > 1.8 (compiled under JDK 17). It has come to our attention that many users cannot update to a recent version of Java because their machines are not self-administered, and we wish to support these users. The Tetrad codebase uses language level 8, so this is feasible.

Algorithm highlights:

• Added an implementation of DirectLiNGAM.
• Added an implementation of DAGMA.
• Added an additive nonlinear simulation method.
• Added Integration of Overlapping Datasets (IOD) as a pooled independence test that can be used in, say, FCI.
• Improved interfaces and code for PC, CPC, FAS, and other related classes.
• Generalized the Markov blanket checker to be compatible across graph types.
• Added the Restricted BOSS algorithm, which does an anteriority variable selection and then runs BOSS over that set of variables.
• Updated FGES-MB with a more up-to-date algorithm and new options.
• Did several updates/revisions of CStaR and added BOSS and Restricted BOSS as options for the CPDAG search.

Interface highlights:

• Added a menu item to the interface to show people how to make suggestions.
• Added a new Markov checker accessible from both Python and the Java interface.
• Added a new dialog showing descriptive statistics for entire datasets.
• The descriptive statistic dialog also identifies constant columns in the data and gives an example of a 2-3 variable singularity if one exists.
• Added an interactive plot matrix tool.
• Added a new popup, compatible cross-platform, to show when processes are being run in the interface.
• Added a graph manipulation option to strip null edges from bootstrapping graphs.
• Added informative exceptions when nonsingularies are encountered in test and scoring operations.
• Moved methods to compare two graphs to the algcomparison package.
• Added a new parameter to allow saving of bootstrapped graphs to be turned on or off.
• Added tools to copy coefficient matrix copy error covariance matrix' to SEM IM editor and SEM Estimator Editor.
• Wrote a paper for the CAWS conference introducing our Python and R interfaces.

Debugging highlights:

• Turned unit test back on to test serialization so that saving and loading will be more stable.
• Fixed a node equality bug causing discrepancies between Causal Command and the Tetrad GUI.
• Fixed bugs preventing simple time series from being simulated.
• Fixed and updated knowledge layouts.
• Fixed issue with large bootstrapping operations where session sizes would balloon to large sizes because bootstrapped graphs were saved.

====DETAILS====

1. Decreasing the minimum version needed for Tetrad to 1.8, so long as it is compiled under Corretto 1.8.
2. Added a parameter to BOSS to permit optional inclusion of the BES step.
3. Fixed some unit tests for the data loader that were failing on Windows; now, all tests pass on all platforms.
4. Added BES option to BFCI.
5. Added a method in Paths to calculate the maximal cliques in a graph using the Brom-Kerbosch algorithm.
6. Cleaned up PC, CPC, FAS, PcCommon, and other related classes in the search and algcomparison packages. Their parameters are slightly different now.
7. Removed PcMax and added a parameter in PC to use the Max-P heuristic for orient colliders.
8. Adjusted hyperparameters for PC, CPC, FAS, and their defaults. Please see the code and interfaces.
9. Changed 'aggressively prevent cycles' to 'meekPreventCycles' throughout the code.
10. Coded up a new Integration of Overlapping Datasets (IOD) pooled independence test and an FCI-IOD wrapper for it that takes a list of datasets with overlapping variables and generates an FCI graph for them that pools independence tests for X || Y | Z for all datasets containing the variables X, Y and all of Z, using the Fisher p-value pooling method.
11. Moved the IOD independence test from the work_in_progress to the search package.
12. Linked to the search Javadocs from the Java interface manual.
13. Coded up a new method for running processes with a popup "Stop" button that doesn't call the deprecated Thread.stop(). The "Stop" button was broken on Windows but now works on all platforms.
14. Added a graph tool to strip null edges from graphs included in graphs by the bootstrapping API. This is to allow bootstrapping graphs to be estimated.
15. Added 'copy coefficient matrix' and 'copy error covariance matrix' menu items to SEM IM editor and SEM Estimator Editor.
16. Changed the sets of "should be independent" (Markov check) and "should be dependent" (Faithfulness check) facts in the Markov Checker. For the Markov check X || Y | parents(X) is listed for dsep(X, Y | parents(X)), and the Faithfulness check, X ~|| Y | parents(X) is listed for dconn(X, Y | parents(X).
17. Relabeled tabs in Markov check to "Check Local Markov" and "Check Local Faithfulness."
18. Extracted a Python-accessible model for the Markov check in the search package and refactored the interface to use it.
19. Added a Help button to the Markov check that gives instructions for how to use the tool.
20. Added a Kolmogorov-Smirnov check of uniformity for p-values in the Markov checker. For the "Check Local Markov" tab, if the test yields p-values, this should register as Uniform (p > alpha) for good results.
21. Added the Markov Adequacy Score to the Markov Check and to the Markov Check Editor, which is zero if non-Markov and the faction of dependent tests under the m-connected condition otherwise (to give a sense of how faithful the graph is to the distribution).
22. Changed "d-separation" and "d-connection" to "m-separation" and "m-connection" throughout the codebase to emphasize the fact that d-separation is m-separation restricted to the case of DAGs.
23. Fixed problem preventing GRaSP-FCI from using m-separation to analyze an input graph.
24. Put the legal PAG check in the interface inside of a watch block so that it can be canceled for large graphs.
25. Changed name of IndTestMsep to MsepTest.
26. Added parameters to GRaSP and BOSS to disallow randomness inside the algorithm. Allowing randomness helps these algorithms avoid local optima, but the user may wish to prioritize consistent answers for the algorithms.
27. Replaced Markov blanket calculations in GraphUtils with a calculation that works for DAGs, CPDAGs, MAGs, and PAGs.
28. Added an option in the Markov Checker to display independence results for Ind(X, Y | MB(X)).
29. Generalized the markovBlanketDag method in GraphUtils to markovBlanketGraph, so that it gives the subgraph over {X} U MB(X) for any DAG, CPDAG, MAG, or PAG.
30. Fixed the displayed error for the case where you erroneously try to do bootstrapping on a covariance matrix.
31. Cleaned up the graph comparison API. (There were two such methods; collapsed these into one.)
32. Fixed a null pointer bug in the Edgewise Comparison for the case where you compared a PAG to the true DAG with latents.
33. Fixed bugs preventing simple time series from being simulated.
34. Added Bollen and Ting reference for Wishart and Delta tetrad tests to the manual (for BPC, FOFC, and FTFC). I also added the original Delta test reference to the manual (Bollen 1990).
35. Added a knowledge layout by knowledge indices in the Layout menus. Tiers for variables need to be indicated in the variable names as "X," e.g., for the last tier, and, e.g., "X:1" for the next to last tier, etc.
36. Fixed some issues with knowledge for BFCI, GRaSP-FCI, and BFCI.
37. Turned stable FAS back on.
38. Allowed control-click to substitute for right-click for graph workbench to accommodate Mac Magic Mouse.
39. Fixed the code so that if a search uses knowledge with more than one tier, a knowledge layout is used based on indices of variables ("X", "X:1", "X:2", etc.
40. Made knowledge layout work (not from knowledge indices) in the Search box.
41. Refactored GFCI, BFCI, and GRaSP-FCI to make the code more modular.
42. Fixed node hashcode and equality methods to look at the names of nodes rather than node objects.
43. Revised the hashCode and equals methods for the node classes to forgo testing object identity and instead always test for equality of names.
44. Fixed bug in Knowledge editor where if you loaded knowledge with more than 3 tiers, only 3 would be displayed.
45. Added a hashcode method for DiscreteVariable to return the hashcode of the variable's name (as with the other variable types).
46. Cleaned up equals method for Edge to use name equality only and node hashcodes.
47. Removed the NodeEquality class and all references to it in the project.
48. Added a menu to the Java Tetrad interface to allow users to submit suggestions. It contains a clickable link that takes the user to the Tetrad Issues List.
49. Moved the dangling IndependenceTest interface into the search package.
50. Removed the structure prior parameter from Degenerate Gaussian as it was not used.
51. Turned the test back on to deserialize saved serialization archives to address saving and loading issues with Tetrad sessions.
52. Added new serialization archives for tetrad-lib and tetrad-gui for 2023-06-27.
53. Placed Save... and Save As... actions for sessions in the Tetrad app inside watch blocks to reduce the possibility of session corruption due to prematurely typing control-Q (which Java can't catch).
54. Added informative exceptions in places where singularity exceptions are thrown, indicating which variables are involved in the singularities.
55. Added CompareTwoGraphs in algcomparison to compare two graphs on a list of statistics.
56. Added a printout in the Descriptive Statistics popup to pre-screen data for constant columns and singularities for 2 or 3 variables taken at a time.
57. Added a new algorithm, DirectLingam. This algorithm, like IcaLingam and the pairwise orientation rules, addresses the linear, non-Gaussian case. Reference: Shimizu, S., Inazumi, T., Sogawa, Y., Hyvarinen, A...

v7.4.0

Fixes 7.4.0

1. Refactored the Degenerate Gaussian BIC score so that it wraps SEM BIC. (Faster.)
2. Added new implementations of ICA-LiNGAM and ICA-LiNG-D.
3. Reorganized and fixed Javadoc documentation for the search package in Tetrad. This involved general code cleanup, removing unused classes, fleshing out documentation, and repackaging. All scores are now in the edu.cmu.tetrad.search.score package and all tests are now in the edu.cmu.tetrad.search.test package. All utility classes for search are now in the edu.cmu.tetrad.search.utils package. Classes not ready for publication are in the "work_in_progress" package. All documentation for public-facing classes has been updated and is currently at this location: https://www.phil.cmu.edu/tetrad-javadocs/7.4.0/edu/cmu/tetrad/search/package-summary.html.
4. The Python and R wrappers (py-tetrad, rpy-tetrad) have been updated with the above revisions. Comments on the documentation are welcome.
5. Updated and greatly optimized GRaSP, BOSS, and SP versions are now in the development branch. Knowledge of forbidden and required edges is implemented for all of these. For BOSS and SP, processing by knowledge tier is available to increase the number of variables then can be processed for, say, time series data.
6. Fixed saving and loading for graphs using the JSON format.

v7.3.4

Fixes for 7.3.4

This version will almost certainly break Tetrad sessions saved under previous versions from the interface; you should keep the datasets from those sessions if needed before using the new version and re-load them in the new session. This change was required as part of an effort to make the codebase more professional.

1. Enforced class naming convention (Capital letter followed by camel case) throughout the code.
2. Fixed null pointer problem for PcMb.
3. Fixed knowledge issue for PcMax.
4. Fixed null pointer problem for FaskVote.

v7.1.3

Fixes for 7.3.3

1. Fixed knowledge issue for PC.
2. Fixed knowledge issue for IMaGES.

v7.3.2

Fixes 7.3.2

1. Fixed a problem with DiscreteBIC (setting structure-prior to 0 didn't work).
2. Re-set up BOSS and SP for bootstrapping.
3. Added citation block to the README.
4. Added a method to save graphs in lavaan (https://lavaan.ugent.be/tutorial/) format in R.
5. Capitalized all algcomparison algorithm wrapper names.
6. Deleted unused algcomparison algorithm wrappers.
7. In py-tetrad, insisting all of the algorithms and score wrappers deal in Java objects and that datasets and graphs be translated to Python objects outside the methods for now.
8. In py-tetrad, renaming some methods for clarity for data translation.
9. Fixed a knowledge bug for BOSS, where some acyclic graphs were triggering a cyclic exception.
10. Re-added the CCD algorithm to the interface and py-tetrad per user request.
11. Added a method to run Tetrad algorithms through py-tetrad in R. Here are the instructions.
12. Fixed issues with PC, CPC, and PC-Max with knowledge not passed in through their algcomparison wrappers. This affected the 7.3.0 interface.
13. Added a unit test to ensure knowledge is passed into algcomparison wrappers for interface and algcomparison stability.
14. Fixed a couple of issues with the graph to DOT method so that Graphviz will correctly display current bootstrap graphs.
15. Fixed a problem with GRaSP-FCI and some other searches where the wrong knowledge was passed into the bootstrapping API, so bootstrap ensemble graphs wouldn't follow knowledge.

Version 7.3.0

Updates for 7.3.0.

1. Added a new Python project, py-tetrad, which shows how to integrate arbitrary Tetrad code into a Python workflow using JPype.
2. Updated the existing examples that show how to incorporate causal-cmd into Python.
3. Updated the command-line causal-compare tool to the latest Tetrad version.
4. The string "new thread" was confusingly being printed sometimes--removed this.
5. Changed the name of BicScore to DiscreteBicScore for clarity.
6. Consolidated versions of PC, CPC, and PCMAX in the code to use the more up-to-date ones.
7. Revised the constructors for CG, took structure prior out, and made it separately settable.
8. Fixed some bugs in the mixed variable tests and scores so that they can successfully be run from Python.
9. Renamed DegenerateGaussianScore--the old one as DegenerateGaussianScoreOld (deprecated), the new one as DegenerateGaussianScore.
10. Moved a new implementation of Boss into the build from another branch.
11. Rewrote the Tetrad README for clarity.
12. Fixed issue with "high" bootstrap graph not being rendered correctly.
13. Fixed calculation of true negatives for ArrowConfusion and BidirectedConfusion. (Thanks huiouyang16!)
14. Fixed an issue with BOSS where the variables were returned in an order different from the original order.
15. Switched FGES &c from storing node and model scores as "BIC" (using a BIC score) to "Score" (using the score from the constructor).
16. Fixed a problem with the Bayes estimator where correct p-values, BIC scores, etc., were not displayed.
17. Simplified the IndependenceTest interface, making default implementations for several methods in preparation for use in py-tetrad.
18. Simplified the Score interface, making default implementations for several methods in preparation for use in py-tetrad.
19. Added new knowledge functionality for the permutation algorithms BOSS, SP. Now searches are done within each tier separately, allowing for more extensive searches if tiered background knowledge is available, and, e.g., forbid within tier works as expected.