MAP's fmin is unconstrained and breaks nodes by walking outside the support.

[twiecki]

Hi,

I found it very useful to just use the MAP for complex models during model exploration. However, I'm having the problem that fmin sometimes explores parameter spaces outside the support of my distributions (Uniform). In this case, rather than return -Inf, they raise an exception. I'm not sure what the best way would be to fix it, but in this case I think this behavior makes no sense, so either MAP should ignore out-side-of-support-errors or the nodes should have a flag not to raise that error.

Thanks,
Thomas

[fonnesbeck]

Hi Thomas,

Can you post a simple example with which we can debug the problem?

Thanks,
cf

[twiecki]

Example code :
https://gist.github.com/1045292

Output:
python example_breaks_map.py
t: value set to 0.603171404722
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.603171404722
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.633329974958
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.573012834486
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.542854264249
t: logp accessed.
t: Returning log-probability 0.0
t: logp accessed.
t: Returning log-probability 0.0
Current log-probability : -16.280555
t: value set to 0.482537123777
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.422219983305
t: logp accessed.
t: Returning log-probability 0.0
t: logp accessed.
t: Returning log-probability 0.0
Current log-probability : -15.619173
t: value set to 0.301585702361
t: logp accessed.
t: Returning log-probability 0.0
t: value set to 0.180951421416
t: logp accessed.
t: Returning log-probability 0.0
t: logp accessed.
t: Returning log-probability 0.0
Current log-probability : -14.340067
t: value set to -0.0603171404722
t: logp accessed.
t: Returning log-probability -1.79769313486e+308
t: value set to 0.301585702361
t: logp accessed.
t: Returning log-probability 0.0
t: logp accessed.
t: Returning log-probability 0.0
Current log-probability : -14.972344
t: value set to 0.0603171404722
t: logp accessed.
t: Returning log-probability 0.0
t: value set to -0.0603171404722
t: logp accessed.
t: Returning log-probability -1.79769313486e+308
t: logp accessed.
t: Returning log-probability -1.79769313486e+308
Traceback (most recent call last):
File "example_breaks_map.py", line 7, in
pm.MAP([t,y]).fit(verbose=1)
File "pymc/NormalApproximation.py", line 273, in fit
disp=verbose)
File "scipy/optimize/optimize.py", line 259, in fmin
callback(sim[0])
File "pymc/NormalApproximation.py", line 250, in callback
print 'Current log-probability : %f' %self.logp
File "pymc/Node.py", line 340, in _get_logp
return logp_of_set(self.stochastics | self.potentials | self.observed_stochastics)
File "pymc/Node.py", line 19, in logp_of_set
logp += obj.logp
File "pymc/PyMCObjects.py", line 832, in get_logp
raise ZeroProbability, self.errmsg + "\nValue: %s\nParents' values:%s" % (self._value, self._parents.value)
pymc.Node.ZeroProbability: Stochastic t's value is outside its support,
or it forbids its parents' current values.
Value: -0.0603171404722
Parents' values:{'upper': 1, 'lower': 0}

[twiecki]

Also, as you can see, Uniform returns a very big negative number outside the support instead of -infinity.

[twiecki]

Just noticed that the problem only exists when setting verbose=1 in MAP.fit(). If this it not done, the exception is not raised (or ignored?).

However, when running with verbose=0, it will happily find the parameter value t=-5 (i.e. outside the support). So it seems that MAP then ignores any distribution parameters of non-observed variables. Is this intended behavior?

[fonnesbeck]

That big negative number comes from FORTRAN, which is where all of our log-probabilities are calculated. It serves the same purpose as -inf.

I think the problem here is simply that logp is calculated simply for reporting, and because the value of t is outside the support, an exception is raised, which is the appropriate behavior for MCMC, but not here. I will fix this.

[fonnesbeck]

Fixed in 8423d05