Regarding the log prior (phi_0 in the paper)

[Hugo101]

1. about the function performInference in boosting.py during the inference:
   logPrior = log(0.5/float(1-0.5)) = 0

   However, in the java version, this is -1.8.

2. about the functions: setPos and setNeg in util.py during learning:

   • self.pos[example] = 0.5 # Set initial gradient to 0.5 for positives.
   • self.neg[example] = -0.5 # Set initial gradient to -0.5 for negatives.

   However, in the java version:

   • For positives, the prob is 1-p=1 - sigmoid(-1.8)
   • For negatives, the prob is 0-p = 0 - sigmoid(-1.8)

[hayesall]

Good catch, some additional background (some lines are removed for easier reading, line numbers from BoostSRL's current master branch are marked)

edu.wisc.cs.will.Boosting.RDN.ConditionalModelPerPredicate.java

public class ConditionalModelPerPredicate implements Serializable {
    /**
    *  Prior log probability i.e. \psi_0
    */
    private double log_prior = -1.8; // Line 49

    public void setLog_prior(double logPrior) { // Line 589
        log_prior = logPrior;
    }

Setting the log prior to 0 in this repository may be left over from when regression was implemented. The BoostSRL codebase sets the log prior to 0 when a regression model is learned.

edu.wisc.cs.will.Boosting.RDN.LearnBoostedRDN

if (cmdArgs.isLearnRegression()) {
    rdn.setLog_prior(0); // 171
}

[hayesall]

None of the unit tests in this repository check __logPrior__, so for the classification case we can simply switch the lines out.

https://github.com/starling-lab/rfgb.py/blob/288fa25613716531c0bf280f2e7fb05a6106d3e6/rfgb/boosting.py#L35

For regression we may want to do something similar to the Java codebase where the prior can be adjusted, maybe as a function in boosting or as a parameter in boosting.updateGradients.

- __logPrior__ = log(0.5/float(1-0.5))
+ __logPrior__ = -1.8
+
+ def setLogPrior(prior):
+     __logPrior__ = prior

I looked back through the original RDN-Boost paper but didn't find this specific value referenced there. @boost-starai or @gkunapuli may know where this originated from.

[boost-starai]

-1.8 is the log prior for 0.5,0.5.