Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bayesian RSA to the package (reprsimil) #98

Merged
merged 63 commits into from Sep 22, 2016
Merged
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
e4ab872
a
Sep 2, 2016
06c869a
deleted: brsa/test.txt
Sep 2, 2016
45f5627
return to origin
Sep 2, 2016
39441a9
improving codes to past test
Sep 5, 2016
762735b
more changes in brsa.py to fit testing requirements
Sep 6, 2016
00ed548
keep removing trailing whitespace and reducing complexity :)
Sep 6, 2016
1465422
passed brainiak test, ready for pull request. There is no example yet.
Sep 7, 2016
f77e4a2
URL for preprint added
Sep 7, 2016
e4ff88f
Added note to warn about the assumption of spatial independence of noise
Sep 7, 2016
b0fc653
removed trailing whitespace
Sep 7, 2016
8b7232a
some updates to docstring. But it still seems to have problem
Sep 8, 2016
e84cd76
Added a test for read_design in utils.py. Corrected the docstring err…
Sep 8, 2016
79da035
fixed a mistake of the standard deviation of Gaussian Process
Sep 8, 2016
1f5775a
reversed some unintended change to utils.py
Sep 8, 2016
72993e0
modified: brainiak/reprsimil/brsa.py
Sep 11, 2016
df1212f
modified: brainiak/reprsimil/brsa.py
Sep 12, 2016
feffa12
intermittant push. A few suggestions by Mike are fixed: distance for …
Sep 13, 2016
33d1230
fixing formatting errors
Sep 13, 2016
2e62141
fixed some errors of the previous commit
Sep 13, 2016
cd92ca8
The 3 steps of fitting are now separated into 3 functions. Additional…
Sep 15, 2016
4a17697
minor change to test code
Sep 16, 2016
e18827b
minor change to test code
Sep 16, 2016
54b23cb
minor changes to some ocmments
Sep 16, 2016
890dc15
Merge branch 'master' into add_brsa
lcnature Sep 16, 2016
54e8035
Added more documentation in utils.read_design and reprsimil.brsa. Red…
Sep 19, 2016
048b6c8
minor update to comments in brsa.py
Sep 19, 2016
08eb888
Merge branch 'add_brsa' of https://github.com/lcnature/brainiak into …
Sep 19, 2016
f189941
Merge branch 'master' into add_brsa
lcnature Sep 19, 2016
787020d
improving codes to past test
Sep 5, 2016
99aa7c2
more changes in brsa.py to fit testing requirements
Sep 6, 2016
4b58738
keep removing trailing whitespace and reducing complexity :)
Sep 6, 2016
d67008b
passed brainiak test, ready for pull request. There is no example yet.
Sep 7, 2016
5350ddc
URL for preprint added
Sep 7, 2016
063a0c1
Added note to warn about the assumption of spatial independence of noise
Sep 7, 2016
f77e228
removed trailing whitespace
Sep 7, 2016
99dcd7c
some updates to docstring. But it still seems to have problem
Sep 8, 2016
dc57582
Added a test for read_design in utils.py. Corrected the docstring err…
Sep 8, 2016
b56a476
fixed a mistake of the standard deviation of Gaussian Process
Sep 8, 2016
4dd6091
reversed some unintended change to utils.py
Sep 8, 2016
374cf64
modified: brainiak/reprsimil/brsa.py
Sep 11, 2016
19f76ff
modified: brainiak/reprsimil/brsa.py
Sep 12, 2016
c61db99
intermittant push. A few suggestions by Mike are fixed: distance for …
Sep 13, 2016
7d7254f
fixing formatting errors
Sep 13, 2016
dc441a3
fixed some errors of the previous commit
Sep 13, 2016
c524178
Use logging in SRM (#100)
TuKo Sep 14, 2016
8d5128b
Call `git clean` for docs before building
mihaic Sep 14, 2016
92de13a
The 3 steps of fitting are now separated into 3 functions. Additional…
Sep 15, 2016
607f371
minor change to test code
Sep 16, 2016
d8d3a9f
minor change to test code
Sep 16, 2016
2ff640e
minor changes to some ocmments
Sep 16, 2016
19e5ce9
Added more documentation in utils.read_design and reprsimil.brsa. Red…
Sep 19, 2016
9c8b0da
minor update to comments in brsa.py
Sep 19, 2016
cbdfd76
reducing perturbation for gradient test
Sep 19, 2016
b174e7d
trying to solve divergence
Sep 19, 2016
d927f19
updating test code
Sep 19, 2016
f5cf7cf
delete temporary files
Sep 19, 2016
59ada9c
use numdifftools for gradient testing in brsa
Sep 19, 2016
9b1e30a
use numdifftools for gradient testing in brsa
Sep 19, 2016
68a4600
use numdifftools for gradient testing in brsa
Sep 19, 2016
b8fa72a
correcting some comments and deleting some old codes
Sep 20, 2016
00403b8
changing some parameters in test script to see why travis-CI fails
Sep 20, 2016
3f5933f
naming change to utils.ReadDesign and a few functions in BRSA
Sep 21, 2016
6884290
Merge branch 'master' into add_brsa
lcnature Sep 22, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 15 additions & 0 deletions brainiak/reprsimil/__init__.py
@@ -0,0 +1,15 @@
# Copyright 2016 Mingbo Cai, Princeton Neuroscience Instititute,
# Princeton University
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""A Bayesian method to perform Representational Similarity Analysis"""
1,340 changes: 1,340 additions & 0 deletions brainiak/reprsimil/brsa.py

Large diffs are not rendered by default.

168 changes: 166 additions & 2 deletions brainiak/utils/utils.py
@@ -1,4 +1,4 @@
# Copyright 2016 Intel Corporation
# Copyright 2016 Intel Corporation, Princeton University
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -13,6 +13,9 @@
# limitations under the License.
import numpy as np
import logging
import re
import warnings
import os.path

"""
Some utility functions that can be used by different algorithms
Expand Down Expand Up @@ -49,7 +52,6 @@ def from_sym_2_tri(symm):
"""convert a 2D symmetric matrix to an upper
triangular matrix in 1D format


Parameters
----------

Expand All @@ -71,6 +73,7 @@ def from_sym_2_tri(symm):

def fast_inv(a):
"""to invert a 2D matrix

Parameters
----------

Expand Down Expand Up @@ -100,3 +103,164 @@ def fast_inv(a):
except np.linalg.linalg.LinAlgError:
logging.exception('Error from np.linalg.solve')
raise


def cov2corr(cov):
"""Calculate the correlation matrix based on a
covariance matrix

Parameters
----------

cov: 2D array

Returns
-------

corr: 2D array
correlation converted from the covarince matrix


"""
assert cov.ndim == 2, 'covariance matrix should be 2D array'
inv_sd = 1 / np.sqrt(np.diag(cov))
corr = cov * inv_sd[None, :] * inv_sd[:, None]
return corr


class read_design:
Copy link
Member

@mihaic mihaic Sep 20, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class names should be use the CapWords capitalization convention:
https://www.python.org/dev/peps/pep-0008/#class-names
While for the code that closely follows the formulas in the paper we can disregard this rule, please follow it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

""" A class which has the ability of reading in design matrix in .1D file,
generated by AFNI's 3dDeconvolve.

Parameters
----------

fname: string, the address of the file to read.

include_orth: Boollean, whether to include "orthogonal" regressors in
the design matrix which are usually head motion parameters. All
the columns of design matrix are still going to be read in, but
the attribute cols_used will reflect whether these orthogonal
regressors are to be included for furhter analysis.

include_pols: Boollean, whether to include polynomial regressors in
the design matrix which are used to capture slow drift of signals.
This will be reflected in the indices in the attribute cols_used.

Attributes
----------

design: 2d array. The design matrix read in from the csv file.

n_col: number of total columns in the design matrix.

column_types: 1d array. the types of each column in the design matrix.
0 for orthogonal regressors (usually head motion parameters),
-1 for polynomial basis (capturing slow drift of signals),
values > 0 for stimulus conditions

n_basis: scalar. The number of polynomial bases in the designn matrix.

n_stim: scalar. The number of stimulus conditions.

n_orth: scalar. The number of orthogoanal regressors (usually head
motions)

StimLabels: list. The names of each column in the design matrix.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguments not documented here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: worth checking if this and readAFNI below are duplicating nipype functionality.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think AFNI still does not provide easy python tool for reading relevant information from 1D files. Basically the support for python is very limited: https://afni.nimh.nih.gov/afni/community/board/read.php?1,82973,82973#msg-82973

def __init__(self, fname=None, include_orth=False, include_pols=False):
if fname is None:
# fname is the name of the file to read in the design matrix
self.design = np.zeros([0, 0])
self.n_col = 0
# number of columns (conditions) in the design matrix
self.column_types = np.ones(0)
self.n_basis = 0
self.n_stim = 0
self.n_orth = 0
self.StimLabels = []
else:
# isAFNI = re.match(r'.+[.](1D|1d|txt)$', fname)
filename, ext = os.path.splitext(fname)
# We assume all AFNI 1D files have extension of 1D or 1d or txt
if ext in ['.1D', '.1d', '.txt']:
self.readAFNI(fname=fname)

self.include_orth = include_orth
self.include_pols = include_pols
self.cols_used = np.where(self.column_types == 1)[0]
if self.include_orth:
self.cols_used = np.sort(
np.append(self.cols_used, np.where(self.column_types == 0)[0]))
if self.include_pols:
self.cols_used = np.sort(np.append(
self.cols_used, np.where(self.column_types == -1)[0]))
self.design_used = self.design[:, self.cols_used]
if not self.include_pols:
# baseline is not included, then we add a column of all 1's
self.design_used = np.insert(self.design_used, 0, 1, axis=1)
self.n_TR = np.size(self.design_used, axis=0)

def readAFNI(self, fname):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method names should be lowercase, with optional underscores for readability:
https://www.python.org/dev/peps/pep-0008/#method-names-and-instance-variables

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Done

# Read design file written by AFNI
self.n_basis = 0
self.n_stim = 0
self.n_orth = 0
self.StimLabels = []
self.design = np.loadtxt(fname, ndmin=2)
with open(fname) as f:
all_text = f.read()

find_n_column = re.compile(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really basic error checking would be nice (just so that it doesn't fail silently if it doesn't find what it needs).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I realized that there is a checking a few lines below, for whether the number of conditions read from the header matches the number of columns.

r'^#[ ]+ni_type[ ]+=[ ]+"(?P<n_col>\d+)[*]', re.MULTILINE)
n_col_found = find_n_column.search(all_text)
if n_col_found:
self.n_col = int(n_col_found.group('n_col'))
if self.n_col != np.size(self.design, axis=1):
warnings.warn(
'The number of columns in the design matrix'
+ 'does not match the header information')
self.n_col = np.size(self.design, axis=1)
else:
self.n_col = np.size(self.design, axis=1)

self.column_types = np.ones(self.n_col)
# default that all columns are conditions of interest

find_ColumnGroups = re.compile(
r'^#[ ]+ColumnGroups[ ]+=[ ]+"(?P<CGtext>.+)"', re.MULTILINE)
CG_found = find_ColumnGroups.search(all_text)
if CG_found:
CG_text = re.split(',', CG_found.group('CGtext'))
curr_idx = 0
for CG in CG_text:
split_by_at = re.split('@', CG)
if len(split_by_at) == 2:
# the first tells the number of columns in this condition
# the second tells the condition type
n_this_cond = int(split_by_at[0])
self.column_types[curr_idx:curr_idx + n_this_cond] = \
int(split_by_at[1])
curr_idx += n_this_cond
elif len(split_by_at) == 1 and \
not re.search('..', split_by_at[0]):
# Just a number, and not the type like '1..4'
self.column_types[curr_idx] = int(split_by_at[0])
curr_idx += 1
else: # must be a single stimulus condition
split_by_dots = re.split('\..', CG)
n_this_cond = int(split_by_dots[1])
self.column_types[curr_idx:curr_idx + n_this_cond] = 1
curr_idx += n_this_cond
self.n_basis = np.sum(self.column_types == -1)
self.n_stim = np.sum(self.column_types > 0)
self.n_orth = np.sum(self.column_types == 0)

find_StimLabels = re.compile(
r'^#[ ]+StimLabels[ ]+=[ ]+"(?P<SLtext>.+)"', re.MULTILINE)
StimLabels_found = find_StimLabels.search(all_text)
if StimLabels_found:
self.StimLabels = \
re.split(r'[ ;]+', StimLabels_found.group('SLtext'))
else:
self.StimLabels = []
1 change: 1 addition & 0 deletions requirements-dev.txt
@@ -1,6 +1,7 @@
coverage
flake8
flake8-print
numdifftools
pytest
pytest-cython
restructuredtext-lint
Expand Down