Memory error of GrangerAnalyzer #123

dongqunxi · 2014-04-10T16:00:46Z

Dear all,
When I run a script like that:

>>>sampling_rate=1000
>>>freq_idx_G
Out[7]: array([40, 41])
>>>G.frequencies.shape[0]
Out[8]: 513
>>>g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-6-b3dd332ebe13> in <module>()
----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc in __get__(self, obj, type)
    138         # Errors in the following line are errors in setting a
    139         # OneTimeProperty
--> 140         val = self.getter(obj)
    141 
    142         setattr(obj, self.name, val)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc in causality_xy(self)
    202     @desc.setattr_on_read
    203     def causality_xy(self):
--> 204         return self._dict2arr('gc_xy')
    205 
    206     @desc.setattr_on_read

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc in _dict2arr(self, key)
    191         arr = np.empty((self._n_process,
    192                         self._n_process,
--> 193                         self.frequencies.shape[0]))
    194 
    195         arr.fill(np.nan)

MemoryError:

Can anyone give me some tips? Thanks!

arokem · 2014-04-10T23:10:39Z

Hmm. How many channels do you have in the data? How many time-points?

On Thu, Apr 10, 2014 at 9:00 AM, dongqunxi notifications@github.com wrote:

Dear all,
When I run a script like that:

sampling_rate=1000>>>freq_idx_GOut[7]: array([40, 41])>>>G.frequencies.shape[0]Out[8]: 513>>>g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------MemoryError Traceback (most recent call last) in ()----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)
/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc in get(self, obj, type)
138 # Errors in the following line are errors in setting a
139 # OneTimeProperty--> 140 val = self.getter(obj)
141
142 setattr(obj, self.name, val)
/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc in causality_xy(self)
202 @desc.setattr_on_read
203 def causality_xy(self):--> 204 return self._dict2arr('gc_xy')
205
206 @desc.setattr_on_read
/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc in _dict2arr(self, key)
191 arr = np.empty((self._n_process,
192 self._n_process,--> 193 self.frequencies.shape[0]))
194
195 arr.fill(np.nan)
MemoryError:

Can anyone give me some tips? Thanks!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/123
.

dongqunxi · 2014-04-11T08:06:10Z

three channels and 6000 time points each channel.

Best wishes,
Qunxi Dong

2014-04-11 1:10 GMT+02:00 Ariel Rokem notifications@github.com:

Hmm. How many channels do you have in the data? How many time-points?

On Thu, Apr 10, 2014 at 9:00 AM, dongqunxi notifications@github.com
wrote:

Dear all,
When I run a script like that:

sampling_rate=1000>>>freq_idx_GOut[7]: array([40,
41])>>>G.frequencies.shape[0]Out[8]: 513>>>g1 = np.mean(G.causality_xy[:,
:, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------MemoryError
Traceback (most recent call last) in
()----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc
in get(self, obj, type)
138 # Errors in the following line are errors in setting a
139 # OneTimeProperty--> 140 val = self.getter(obj)
141
142 setattr(obj, self.name, val)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc
in causality_xy(self)
202 @desc.setattr_on_read
203 def causality_xy(self):--> 204 return self._dict2arr('gc_xy')
205
206 @desc.setattr_on_read

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc
in _dict2arr(self, key)
191 arr = np.empty((self._n_process,
192 self._n_process,--> 193 self.frequencies.shape[0]))
194
195 arr.fill(np.nan)
MemoryError:

Can anyone give me some tips? Thanks!

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/123#issuecomment-40153686
.

arokem · 2014-04-11T16:14:11Z

I can't replicate this error on my laptop. It might be that your machine
just doesn't have enough memory. How much RAM are you working with here?
Would you mind sharing those time-series with me? I can try it with your
actual data. I had to generate some random data to try out the data-size.

On Fri, Apr 11, 2014 at 1:06 AM, dongqunxi notifications@github.com wrote:

three channels and 6000 time points each channel.

Best wishes,
Qunxi Dong

2014-04-11 1:10 GMT+02:00 Ariel Rokem notifications@github.com:

Hmm. How many channels do you have in the data? How many time-points?

On Thu, Apr 10, 2014 at 9:00 AM, dongqunxi notifications@github.com
wrote:

Dear all,
When I run a script like that:

sampling_rate=1000>>>freq_idx_GOut[7]: array([40,
41])>>>G.frequencies.shape[0]Out[8]: 513>>>g1 =
np.mean(G.causality_xy[:,
:, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------MemoryError

Traceback (most recent call last) in
()----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc

in get(self, obj, type)

138 # Errors in the following line are errors in setting a
139 # OneTimeProperty--> 140 val = self.getter(obj)
141
142 setattr(obj, self.name, val)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in causality_xy(self)

202 @desc.setattr_on_read
203 def causality_xy(self):--> 204 return self._dict2arr('gc_xy')
205
206 @desc.setattr_on_read

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in _dict2arr(self, key)

191 arr = np.empty((self._n_process,
192 self._n_process,--> 193 self.frequencies.shape[0]))
194
195 arr.fill(np.nan)
MemoryError:

Can anyone give me some tips? Thanks!

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40153686>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/123#issuecomment-40179835
.

dongqunxi · 2014-04-13T15:18:24Z

https://dl.dropboxusercontent.com/u/94877880/causality%20challenge%20biomag%202014/BioMag2014-Causality-Challenge.htm
My Laptop has 8G memory. I think it is enough. About the time series I got
from the above link. Maybe you can try it. Thanks

Best wishes,
Qunxi Dong

2014-04-11 18:14 GMT+02:00 Ariel Rokem notifications@github.com:

I can't replicate this error on my laptop. It might be that your machine
just doesn't have enough memory. How much RAM are you working with here?
Would you mind sharing those time-series with me? I can try it with your
actual data. I had to generate some random data to try out the data-size.

On Fri, Apr 11, 2014 at 1:06 AM, dongqunxi notifications@github.com
wrote:

three channels and 6000 time points each channel.

Best wishes,
Qunxi Dong

2014-04-11 1:10 GMT+02:00 Ariel Rokem notifications@github.com:

Hmm. How many channels do you have in the data? How many time-points?

On Thu, Apr 10, 2014 at 9:00 AM, dongqunxi notifications@github.com
wrote:

Dear all,
When I run a script like that:

sampling_rate=1000>>>freq_idx_GOut[7]: array([40,
41])>>>G.frequencies.shape[0]Out[8]: 513>>>g1 =
np.mean(G.causality_xy[:,
:, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------MemoryError

Traceback (most recent call last) in
()----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc

in get(self, obj, type)

138 # Errors in the following line are errors in setting a
139 # OneTimeProperty--> 140 val = self.getter(obj)
141
142 setattr(obj, self.name, val)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in causality_xy(self)

202 @desc.setattr_on_read
203 def causality_xy(self):--> 204 return self._dict2arr('gc_xy')
205
206 @desc.setattr_on_read

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in _dict2arr(self, key)

191 arr = np.empty((self._n_process,
192 self._n_process,--> 193 self.frequencies.shape[0]))
194
195 arr.fill(np.nan)
MemoryError:

Can anyone give me some tips? Thanks!

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40153686>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40179835>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/123#issuecomment-40221384
.

arokem · 2014-04-14T16:24:08Z

I don't know how to read that data. Could you please save the time-series
in question as an array in an .npy file and share that?

On Sun, Apr 13, 2014 at 8:18 AM, dongqunxi notifications@github.com wrote:

https://dl.dropboxusercontent.com/u/94877880/causality%20challenge%20biomag%202014/BioMag2014-Causality-Challenge.htm
My Laptop has 8G memory. I think it is enough. About the time series I got
from the above link. Maybe you can try it. Thanks

Best wishes,
Qunxi Dong

2014-04-11 18:14 GMT+02:00 Ariel Rokem notifications@github.com:

I can't replicate this error on my laptop. It might be that your machine
just doesn't have enough memory. How much RAM are you working with here?
Would you mind sharing those time-series with me? I can try it with your
actual data. I had to generate some random data to try out the
data-size.

On Fri, Apr 11, 2014 at 1:06 AM, dongqunxi notifications@github.com
wrote:

three channels and 6000 time points each channel.

Best wishes,
Qunxi Dong

2014-04-11 1:10 GMT+02:00 Ariel Rokem notifications@github.com:

Hmm. How many channels do you have in the data? How many
time-points?

On Thu, Apr 10, 2014 at 9:00 AM, dongqunxi notifications@github.com

wrote:

Dear all,
When I run a script like that:

sampling_rate=1000>>>freq_idx_GOut[7]: array([40,
41])>>>G.frequencies.shape[0]Out[8]: 513>>>g1 =
np.mean(G.causality_xy[:,
:, freq_idx_G], -1)

it met the following memory error (freq_idx_G=):

---------------------------------------------------------------------------MemoryError

Traceback (most recent call last) in
()----> 1 g1 = np.mean(G.causality_xy[:, :, freq_idx_G], -1)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/descriptors.pyc

in get(self, obj, type)

138 # Errors in the following line are errors in setting a
139 # OneTimeProperty--> 140 val = self.getter(obj)
141
142 setattr(obj, self.name, val)

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in causality_xy(self)

202 @desc.setattr_on_read
203 def causality_xy(self):--> 204 return self._dict2arr('gc_xy')
205
206 @desc.setattr_on_read

/home/qdong/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/nitime/analysis/granger.pyc

in _dict2arr(self, key)

191 arr = np.empty((self._n_process,
192 self._n_process,--> 193 self.frequencies.shape[0]))
194
195 arr.fill(np.nan)
MemoryError:

Can anyone give me some tips? Thanks!

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40153686>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40179835>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/nipy/nitime/issues/123#issuecomment-40221384>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/123#issuecomment-40310024
.

dongqunxi · 2014-04-15T07:45:50Z

@arokem, I have found the reason of this error.
If the array is a shape like (6000, 2), it meets the memory error.
When I transpose it as (2, 6000), it works well. Thanks for your attention and patience.

arokem · 2014-04-15T14:19:51Z

Yes - that makes sense. Time should always be on the last dimension. Happy to hear that it's now working for you

arokem closed this as completed Apr 15, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory error of GrangerAnalyzer #123

Memory error of GrangerAnalyzer #123

dongqunxi commented Apr 10, 2014

arokem commented Apr 10, 2014

dongqunxi commented Apr 11, 2014

arokem commented Apr 11, 2014

dongqunxi commented Apr 13, 2014

arokem commented Apr 14, 2014

dongqunxi commented Apr 15, 2014

arokem commented Apr 15, 2014

Memory error of GrangerAnalyzer #123

Memory error of GrangerAnalyzer #123

Comments

dongqunxi commented Apr 10, 2014

arokem commented Apr 10, 2014

dongqunxi commented Apr 11, 2014

arokem commented Apr 11, 2014

dongqunxi commented Apr 13, 2014

arokem commented Apr 14, 2014

dongqunxi commented Apr 15, 2014

arokem commented Apr 15, 2014