create and populate optimization_diagnostics sheet in output workbook

[kdahlquist]

"optimization_diagnostics" this would be a new worksheet that has stuff like the LSE value in it and possibly other stuff we want to also output that doesn't fit into any of the other sheets. We will create a new issue for this task.

[kdahlquist]

What should be put out in diagnostic worksheet #106 is

• LSE
• penalty
• minimum (ideal) LSE
• number of iterations (counter)
• sum of squares of error for the individual genes

[bengfitzpatrick]

@kdahlquist @trixr4kdz I am unable to push a new beta package, so I've uploaded a zip file with updated source code to lionshare. You should be able to access these by signing into lionshare and navigating to my folder. The zip file is GRNmap-beta-bgf.zip. The code includes the following updates:

(1) optimization_diagnostics sheet with the requested output data.
(1a) added code functionality to compute min LSE and SSEs of individual genes.
(2) reordering of the strain sigma sheets according to Issue #107
(3) renamed graph files to correspond to gene names.
(4) saved the diagnostic graph to a jpg.
(5) fixed a bug in the penalty computation of the production rates.

Once the code has been published to the beta branch and the tests have been run through it, we can close #106 and #107.

[tessaam]

I tried running 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph three times unsuccessfully. MATLAB never produces an output sheet. The counter got up to over 5,000,000 with no output sheet produced. For this input sheet, the counter previously only went up to 8700 before producing an output sheet.

http://openwetware.org/wiki/GRNmap_Testing_Report_GRNmap-beta-bgf_2015-06-08_TM

[bengfitzpatrick]

Thanks, @tessaam . I noticed in my testing that the number of iterations has increased on the data @GraceJohnson provided. I will look into this. I suspect our rescaling of the LSE has changed the convergence properties for the optimization routine somewhat.

In the meantime, try to increase the TolX and TolFun values. 1e-10 may be too small given the rescaling we've done on the LSE. Increase both to 1e-8 and again to 1e-6. Experiments #3 and #4 should be to decrease the MaxIter and MaxFunEval to 1e6 and again to 1e5. At 1e8 the counter could theoretically go to 100,000,000 before termination. 1e5 is 100,000. 1e5 for the max's and 1e-6 for the tol's should improve the stopping, but we need to see if the results come out right.

[tessaam]

@bengfitzpatrick Increasing the TolX and TolFun values to 1e-6 worked, but increasing them to 1e-8 did not. Decreasing MaxIter and MaxFunEval to 1e5 worked, but decreasing them to 1e6 did not.
The optimization_diagnostics sheet had fairly different values for increasing the TolX and TolFun values to 1e-6 and decreasing MaxIter and MaxFunEval to 1e5 (Results are on http://openwetware.org/wiki/GRNmap_Testing_Report_GRNmap-beta-bgf_2015-06-08_TM).

For both of the experiments that worked (2 and 4):
-Order of sheets was correct (wt_log2_optimized_expression_dcin5_log2_optimized_expression, wt_sigmas, dcin5_sigmas, optimized_production_rates, optimized_threshold_b, network_optimized_weights, optimization_diagnostics)
-Graph files were saved correspond to gene names
-Automatically saved the diagnostic graph to a .jpg (named OptimizationDiagnostic.jpg)

[bengfitzpatrick]

@tessaam can you please repeat these four tests setting fix_b to 1 (keeping fix_p at 0)? Thanks!

[tessaam]

I repeated the test fixing b-1 and keeping P-0. Increasing the TolX and TolFun values to 1e-6 and 1e-8 and decreasing MaxIter and MaxFunEval to 1e6 and 1e5 worked. The output sheets all had very similar values, except that the value of the optimization_diagnostic parameters varied between increasing the TolX and TolFun and decreasing MaxIter and MaxFunEval. The openwetware page http://openwetware.org/wiki/GRNmap_Testing_Report_GRNmap-beta-bgf_2015-06-08_TM has the updated results.

[bengfitzpatrick]

thanks, @tessaam that's great.

[kdahlquist]

optimization_diagnostics should appear like that, not camelCase

[kdahlquist]

@bengfitzpatrick is going to change the sum of squares of error for the individual genes to mean squared error so that the values for the different strains can be compared.

[kdahlquist]

still open.

[kdahlquist]

@bengfitzpatrick is working on this, this week.

[bengfitzpatrick]

fixed this: SSE is now mean square error. ready for testing once the code is uploaded. L-curve branch transmitted to @kdahlquist

[kdahlquist]

There is a change in nomenclature that is required for the optimization_diagnostics sheet. Instead of using "SSE", we need to use "MSE", which stands for Mean Squared Error. I'm attaching a screenshot for how this should look.

[trixr4kdz]

The SSE has now been changed into MSE.

[kdahlquist]

Verified that this has changed on beta branch by running one input workbook. We'll run the test in the meeting tomorrow.

[kdahlquist]

Test suite was run in the meeting, so we are closing this now.