Vectors of multivariate variables

[fonnesbeck]

It would be useful if we could model multiple independent multivariate variables in the same statement. For example, if I wanted four multivariate normal vectors with the same prior, I should be able to specify:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=(4,3))

but it currently returns a ValueError complaining of non-aligned matrices.

[jsalvatier]

will it be obvious what dimension is the multivariate dimension?

On Fri, May 2, 2014 at 10:16 AM, Chris Fonnesbeck
notifications@github.comwrote:

It would be useful if we could model multiple independent multivariate
variables in the same statement. For example, if I wanted four multivariate
normal vectors with the same prior, I should be able to specify:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=(4,3))

but it currently returns a ValueError complaining of non-aligned matrices.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/535
.

[fonnesbeck]

It should be intuitive, if not obvious. I think most people would expect a vector of variables, which implies that the first dimension is the number of variable elements and the remaining dimension(s) the size of each variable.

[fonnesbeck]

We at least need to be able to do the analog of this:

m = [pm.MvNormal('m_{}'.format(i), mu, Tau, value=[0]*3) for i in range(len(unique_studies))]

This has been a show-stopper for me trying to use PyMC 3 for new work, so I'm going to try to set aside some time to work on this.

Thinking about it some more, however, I think that shape is not the appropriate way to specify the dimension of a multivariate variable -- that should be reserved for the size of the vector of variables. Might be best to have:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), dim=3)

for a single variable and:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=4, dim=3)

for a vector containing 4 MvNormals of dimension 3. Better yet, we ought to be able to infer the dimension of the MvNormal from its arguments.

[jsalvatier]

That makes some sense. The words shape and dim seem very close, so it seems
confusing to have both

On Thu, May 29, 2014 at 1:30 PM, Chris Fonnesbeck
notifications@github.comwrote:

We at least need to be able to do the analog of this:

m = [pm.MvNormal('m_{}'.format(i), mu, Tau, value=[0]*3) for i in range(len(unique_studies))]

This has been a show-stopper for me trying to use PyMC 3 for new work, so
I'm going to try to set aside some time to work on this.

Thinking about it some more, however, I think that shape is not the
appropriate way to specify the dimension of a multivariate variable -- that
should be reserved for the size of the vector of variables. Might be best
to have:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), dim=3)

for a single variable and:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=4, dim=3)

for a vector containing 4 MvNormals of dimension 3. Better yet, we ought
to be able to infer the dimension of the MvNormal from its arguments.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/535#issuecomment-44581060
.

[fonnesbeck]

Perhaps using plates here would be clearer, since this is common terminology in graphical models. The we could generalize the business of generating vectors of variables.

[fonnesbeck]

Just bumping this one. I come up against it frequently in epidemiological analyses.

[aflaxman]

I like the originally proposed notation, shape=(4,3), since that will be the shape of f.value. Am I stuck in a PyMC2 way of thinking?

[fonnesbeck]

I'd be happy with that. We would just have to adopt the convention that the last dimension is always the size of the individual multivariate node, and not the size of the array containing the nodes. The tricky part comes when you have, say, a vector of Wisharts that is itself multidimensional, so the total shape could be (4,4,3,3) for a 4x4 array of 3x3 variables.

[twiecki]

I would imagine it's a rare case but can't hurt to consider it and come up with a sane way to handle. In the end, complex things will be complex in code but defaulting to the last dimensions is an easy rule to keep in mind.

[aflaxman]

+1 for shape=(4,4,3,3) to get a 4x4 array of 3x3 wisharts. So

C = pm.WishartCov('C', C=np.eye(3), n=5)
C.value.shape == (3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,3,3))
C.value.shape == (4,3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,4,3,3))
C.value.shape == (4,4,3,3)

[fonnesbeck]

Multivariate classes could have the appropriate dimension specified in the class to know how to deal with the shape argument. Wisharts will always be 2-dimensional, for example, so any remaining dimensions will always be how many wisharts are in the set. Multinomials will always be a 1-d vector, etc.

[jsalvatier]

Okay, are we agreed that when we do this the multivariate dimensions start at the back?

We could start them at the front, but the way numpy.dot works suggests at the back.

[twiecki]

Agree on defaulting to last dimension.

I wonder, is the shape argument not redundant? Seems like we can always infer it from the inputs. shape could then only add the dimensions. Personally I would find this less confusing:

C = pm.WishartCov('C', C=np.eye(3), n=5)
C.value.shape == (3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=4)
C.value.shape == (4,3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,4)))
C.value.shape == (4,4,3,3)

The 3,3 is already encoded in np.eye(3), no?

[jsalvatier]

Shape is not redundant when you want to have the same prior arguments for a
bunch of variables.

On Mon, Jul 27, 2015 at 2:14 PM Thomas Wiecki notifications@github.com
wrote:

Agree on defaulting to last dimension.

I wonder, is the shape argument not redundant? Seems like we can always
infer it from the inputs. shape could then only add the dimensions.
Personally I would find this less confusing:

C = pm.WishartCov('C', C=np.eye(3), n=5)
C.value.shape == (3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=4)
C.value.shape == (4,3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,4)))
C.value.shape == (4,4,3,3)

The 3,3 is already encoded in np.eye(3), no?

—
Reply to this email directly or view it on GitHub
#535 (comment).

[twiecki]

right, I'm only talking about the case where the input to the RV (e.g. C above) is multi-dimensional already. Then you can use shape to repeat that input arbitrarily.

pm.Normal('x', mu=[1, 2, 3], shape=2) would give a 2x3 in my proposal.

[jsalvatier]

Yeah, we could do that. I'm slightly worried that its going to make
implementation more complex. And perhaps be confusing to users. But maybe
either way is going to be confusing.

On Mon, Jul 27, 2015 at 2:23 PM Thomas Wiecki notifications@github.com
wrote:

right, I'm only talking about the case where the input to the RV (e.g. C
above) is multi-dimensional already. Then you can use shape to repeat
that input arbitrarily.

pm.Normal('x', mu=[1, 2, 3], shape=2) would give a 2x3 in my proposal.

—
Reply to this email directly or view it on GitHub
#535 (comment).

[twiecki]

I recently ran into the confusion where I wanted 2 Dirichlets of len 3, should I do:
pm.Dirichlet(np.ones((2, 3)), or should I do pm.Dirichlet(np.ones((2, 3)), shape=(2, 3)) or maybe pm.Dirichlet(np.ones((2, 3)), shape=2) or pm.Dirichlet(np.ones(3), shape=2)? Are some equivalent? I actually still don't know.

So with my proposal there's a clear rule and I don't have to remember which dimensions of the shape kwarg match to which dimensions of my input.

Why do you think it would be harder to implement?

[jsalvatier]

That does seem attractive from an API point of view.

Let me check how that plays with broadcasting rules.

[jsalvatier]

That does seem to play nicely with things.

I see two issues. Maybe we can resolve them.

First, this change will break previously working models.

Second, shape is a common argument for all distributions and this means the shape argument won't match the actual shape of the variable. I think that might not actually break anything right now, but seems like a bug waiting to happen.

Perhaps we should have a different argument, not shape for multivariate distributions, but count or dimensions or something else that is used to compute the shape. That would make it more obvious that the behavior is different.

[twiecki]

Or maybe repeat? pm.Dirichlet(np.ones(3), repeat=2) would give a 2x3.

What I also like about this is that it makes the translation from pymc2 style [pm.Dirichlet(np.ones(3)) for i in range(2)] more direct.

So if we were to change this, do we still need the shape kwarg? Do we deprecate it?

[twiecki]

And maybe we could even use theano.tensor.extra_ops.repeat(x, repeats, axis=None) for this.

[twiecki]

Theoretically we could even teach users to use repeat directly and not be concerned with all this in the API. E.g.:

from pymc3 import repeat # alias to theano.tensor.extra_ops.repeat
pm.Dirichlet(repeat(my_prior, 2)) # gives 2x3 if my_prior is shape 3

[fonnesbeck]

I don't think we should worry about breaking changes too much in a beta for such an important design decision.

I like the idea of a dim (dimension) argument that represents the shape of the variable, rather than how many of them there are:

Sigma = Wishart('Sigma', 4, np.eye(3), dim=3)
mu = pm.Normal('mu', 0, 0.001, shape=3)
x = pm.MvNormal('x', mu, Sigma, dim=3, shape=5)

which results in an x that consists of 5 multivariate normals, each of dimension 3.

[jsalvatier]

Shape currently means the actual shape of the resulting variable, and I kind of want to keep that unless there's a good reason.

[twiecki]

@PietJones You shouldn't include observed variables to be sampled.

[nouiz]

Can PyMC3 give a better user error for that case?

On Thu, May 5, 2016 at 10:21 AM, Thomas Wiecki notifications@github.com
wrote:

@PietJones https://github.com/PietJones You shouldn't include observed
variables to be sampled.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

Cool, I took that from:

http://austinrochford.com/posts/2016-02-25-density-estimation-dpm.html

After changing, now I get the following error:

Exception: ('Compilation failed (return status=1): /Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpYXDK_O/mod.cpp:27543:32: fatal error: bracket nesting level exceeded maximum of 256. 

Is there some size limit that I am not aware of? The data frame is not that large: (450, 1051)

[nouiz]

You can use this PR for a work around:

Theano/Theano#4289

On Thu, May 5, 2016 at 10:30 AM, PietJones notifications@github.com wrote:

Cool, I took that from:

http://austinrochford.com/posts/2016-02-25-density-estimation-dpm.html
http://url

After changing, now I get the following error:

Exception: ('Compilation failed (return status=1): /Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpYXDK_O/mod.cpp:27543:32: fatal error: bracket nesting level exceeded maximum of 256.

Is there some size limit that I am not aware of? The data frame is not
that large: (450, 1051)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

@nouiz Thnx for the advice, again not sure if this was what you meant that I should do, but I tried the following, and I still get the same error:

cd ~/git
git clone https://github.com/Theano/Theano
git fetch origin pull/4289/head:pr-4289
git checkout pr-4289
python setup.py develop

I then restarted my ipython/jupyter kernel and reran my code.

[nouiz]

Delete your Theano cache. If that don't fix it, you probably using the old
Theano. Uninstall Theano many times to be sure it is not installed and
reinstall as you just did.

On Thu, May 5, 2016 at 11:05 AM, PietJones notifications@github.com wrote:

@nouiz https://github.com/nouiz Thnx for the advice, again not sure if
this was what you meant that I should do, but I tried the following, and I
still get the same error:

cd ~/git
git clone https://github.com/Theano/Theano
git fetch origin pull/4289/head:pr-4289
git checkout pr-4289
python setup.py develop

I then restarted my ipython/jupyter kernel and reran my code.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.__version__"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1): /Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32: fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv, using jupyter (did restart the kernel), (don't have cuda). Sorry for the trouble.

[nouiz]

I taught that you where on windows with a GPU.

Then you have a new case.

Can you use this Theano flag: nocleanup=True then after the error send me
the file that failed compilation.

On Thu, May 5, 2016 at 12:44 PM, PietJones notifications@github.com wrote:

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.version"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1): /Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32: fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv, using
jupyter (did restart the kernel), (don't have cuda). Sorry for the trouble.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

Hi,

Find attached the mod.cpp file which failed to compile.

https://gist.github.com/PietJones/8e53946b2738008095ced8fb9ab4db44

I dont think the entire file uploaded, just in case here is google drive link:

https://drive.google.com/file/d/0B2e7WGnBljbJZnJ1T1NDU1FjS1k/view?usp=sharing

On Thu, May 5, 2016 at 1:00 PM, Frédéric Bastien notifications@github.com
wrote:

I taught that you where on windows with a GPU.

Then you have a new case.

Can you use this Theano flag: nocleanup=True then after the error send me
the file that failed compilation.

On Thu, May 5, 2016 at 12:44 PM, PietJones notifications@github.com
wrote:

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.version"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1):
/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32:
fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv, using
jupyter (did restart the kernel), (don't have cuda). Sorry for the
trouble.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[nouiz]

Update Theano to 0.8.2. I have the impression that you use an older version.

Fred

On Thu, May 5, 2016 at 1:25 PM, PietJones notifications@github.com wrote:

Hi,

Find attached the mod.cpp file which failed to compile.

On Thu, May 5, 2016 at 1:00 PM, Frédéric Bastien <notifications@github.com

wrote:

I taught that you where on windows with a GPU.

Then you have a new case.

Can you use this Theano flag: nocleanup=True then after the error send me
the file that failed compilation.

On Thu, May 5, 2016 at 12:44 PM, PietJones notifications@github.com
wrote:

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.version"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1):

/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32:
fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv,
using
jupyter (did restart the kernel), (don't have cuda). Sorry for the
trouble.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

I originally had that version of Theano, which gave the same error. The
older version I posted about above was using a specific Pull Request to see
if that would help.

P

On Fri, May 6, 2016 at 9:03 AM, Frédéric Bastien notifications@github.com
wrote:

Update Theano to 0.8.2. I have the impression that you use an older
version.

Fred

On Thu, May 5, 2016 at 1:25 PM, PietJones notifications@github.com
wrote:

Hi,

Find attached the mod.cpp file which failed to compile.

On Thu, May 5, 2016 at 1:00 PM, Frédéric Bastien <
notifications@github.com

wrote:

I taught that you where on windows with a GPU.

Then you have a new case.

Can you use this Theano flag: nocleanup=True then after the error send
me
the file that failed compilation.

On Thu, May 5, 2016 at 12:44 PM, PietJones notifications@github.com
wrote:

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.version"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1):

/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32:

fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv,
using
jupyter (did restart the kernel), (don't have cuda). Sorry for the
trouble.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
<
https://github.com/pymc-devs/pymc3/issues/535#issuecomment-217206605>

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[nouiz]

Can you confirm it was the pull request about the GpuJoin proble on windows
(
Theano/Theano#4289)?
Theano/Theano#4289

I check that code and it would have tested what I wanted to test.

Can you manually apply this diff and test again?

"""
diff --git a/theano/tensor/opt.py b/theano/tensor/opt.py
index cd74c1e..e9b44b5 100644
--- a/theano/tensor/opt.py
+++ b/theano/tensor/opt.py
@@ -6761,7 +6761,7 @@ def elemwise_max_input_fct(node):
# inputs.
if not theano.config.cxx:
return 31

• return 1024
• return 512

local_elemwise_fusion = local_elemwise_fusion_op(T.Elemwise,
"""

If it still fail, instead of a max of 512, try 256, 128, ...

Tell me the biggest number that work.

On Fri, May 6, 2016 at 9:47 AM, PietJones notifications@github.com wrote:

I originally had that version of Theano, which gave the same error. The
older version I posted about above was using a specific Pull Request to see
if that would help.

P

On Fri, May 6, 2016 at 9:03 AM, Frédéric Bastien <notifications@github.com

wrote:

Update Theano to 0.8.2. I have the impression that you use an older
version.

Fred

On Thu, May 5, 2016 at 1:25 PM, PietJones notifications@github.com
wrote:

Hi,

Find attached the mod.cpp file which failed to compile.

On Thu, May 5, 2016 at 1:00 PM, Frédéric Bastien <
notifications@github.com

wrote:

I taught that you where on windows with a GPU.

Then you have a new case.

Can you use this Theano flag: nocleanup=True then after the error
send
me
the file that failed compilation.

On Thu, May 5, 2016 at 12:44 PM, PietJones <notifications@github.com

wrote:

I tried the following.

rm -r ~/.theano*
pip uninstall theano #did this several times until there was error
cd ~/git/theano #then fetched the PR, did git checkout etc
python setup.py develop #also tried python setup.py install

python -c "import theano; print theano.version"
0.8.0.dev-410eacd379ac0101d95968d69c9ccb20ceaa88ca

Still the same problem.

Exception: ('Compilation failed (return status=1):

/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpJ01xYP/mod.cpp:27543:32:

fatal error: bracket nesting level exceeded maximum of 256.

If it helps, I am running this on a MacOSX, in a conda virtualenv,
using
jupyter (did restart the kernel), (don't have cuda). Sorry for the
trouble.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
<
https://github.com/pymc-devs/pymc3/issues/535#issuecomment-217206605>

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
<
https://github.com/pymc-devs/pymc3/issues/535#issuecomment-217210834>

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[PietJones]

Thnx for the advice, I tried all of the above, editing the file manually, removing the .theano directory, then restarting the jupyter kernel and running the code again, still get the same error. This is also further down before the actual traceback:

/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpEMrDR6  /mod.cpp:35943:32: fatal error: bracket nesting level exceeded maximum of 256
    if (!PyErr_Occurred()) {
                           ^
/Users/jq2/.theano/compiledir_Darwin-14.5.0-x86_64-i386-64bit-i386-2.7.11-64/tmpEMrDR6/mod.cpp:35943:32: note: use -fbracket-depth=N to increase maximum nesting level
1 error generated.

[nouiz]

Which new value did you try? Only 512? Can you try something like 31? If it still fait with 31, then try this diff:

diff --git a/theano/tensor/opt.py b/theano/tensor/opt.py
index a08e900..ef0821d 100644
--- a/theano/tensor/opt.py
+++ b/theano/tensor/opt.py
@@ -6724,6 +6724,8 @@ def local_add_mul_fusion(node):
                 isinstance(inp.owner.op.scalar_op, s_op)):
             l = list(node.inputs)
             l.remove(inp)
+            if len(l) + len(inp.owner.inputs) > 31:
+                return
             output_node = node.op(*(l + inp.owner.inputs))

             copy_stack_trace(node.outputs[0], output_node)

This opt could also cause this extra big Elemwise.

[PietJones]

I have tried 1024, 512, 256 and 31, they all result in the same problem.
Not sure what correction you want me to implement, as the formatting of
what you sent has been corrupted. But the changes that I tried was :

5549 def local_add_mul_fusion(node):
5550 """Fuse consecutive add or mul in one such node with more inputs.
5551
5552 It is better to fuse add/mul that way then in a Composite node as
5553 this make the inner graph of the Compiste smaller. This allow to
5554 put more computation in a Composite before hitting the max
5555 recusion limit when pickling Composite.
5556
5557 """
5558 if (not isinstance(node.op, Elemwise) or
5559 not isinstance(node.op.scalar_op, (scalar.Add, scalar.Mul))):
5560 return False
5561
5562 s_op = node.op.scalar_op.class
5563 for inp in node.inputs:
5564 if (inp.owner and
5565 isinstance(inp.owner.op, Elemwise) and
5566 isinstance(inp.owner.op.scalar_op, s_op)):
5567 l = list(node.inputs)
5568 l.remove(inp)
5569 if len(l) + len(inp.owner.inputs) > 31:
5570 return
5571 #return [node.op((l + inp.owner.inputs))]
5572 output_node = node.op((l + inp.owner.inputs))
5573 copy_stack_trace(node.ouput[0],output_node)
5574

which still gave an error:
https://gist.github.com/PietJones/26339593d2e7862ef60881ea09a817cb

On Tue, May 10, 2016 at 10:16 AM, Frédéric Bastien <notifications@github.com

wrote:

Which new value did you try? Only 512? Can you try something like 31? If
it still fait with 31, then try this diff:

diff --git a/theano/tensor/opt.py b/theano/tensor/opt.py
index a08e900..ef0821d 100644
--- a/theano/tensor/opt.py
+++ b/theano/tensor/opt.py
@@ -6724,6 +6724,8 @@ def local_add_mul_fusion(node):
isinstance(inp.owner.op.scalar_op, s_op)):
l = list(node.inputs)
l.remove(inp)

• if len(l) + len(inp.owner.inputs) > 31:

        return
  output_node = node.op(*(l + inp.owner.inputs))
  
  copy_stack_trace(node.outputs[0], output_node)
  

This opt could also cause this extra big Elemwise.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#535 (comment)

[twiecki]

@fonnesbeck I think this works for Multivariate now, right?

[fonnesbeck]

A list comprehension seems to work now, yes. Ultimately I'd like to be able to specify a vector of multivariates using the shape argument, as in the original issue, but that will be for post-3.0.

[twiecki]

I think that should also work, no? At least for 3D multivariates.

[fonnesbeck]

It runs, but does not "work":

from pymc3 import *

with Model():

    p = Dirichlet('p', np.ones(3), shape=(4,3))
    x = Multinomial('x', np.array([20, 16, 10, 5]), p, shape=(4,3))
    print('p initial:', p.tag.test_value)
    print('x initial:', x.tag.test_value)
    tr = sample(50)
    print('x final:', x.tag.test_value)

yields:

p initial: [[ 0.33333333  0.33333333  0.33333333]
 [ 0.33333333  0.33333333  0.33333333]
 [ 0.33333333  0.33333333  0.33333333]
 [ 0.33333333  0.33333333  0.33333333]]
x initial: [[7 7 7]
 [5 5 5]
 [3 3 3]
 [2 2 2]]
Assigned NUTS to p_stickbreaking_
Assigned Metropolis to x
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 287.60it/s]
x final: [[7 7 7]
 [5 5 5]
 [3 3 3]
 [2 2 2]]

So, the x's don't sum to n, yet it does not fail!

[fonnesbeck]

This is tied up in the shape refactoring. Closing.