Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Connection Generator Interface #483

Merged
merged 72 commits into from
Oct 16, 2017
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
c3ebdc1
- Checking for nosetest xunit support now uses python-free directory …
heplesser Apr 26, 2016
81a7b80
Parse compiler output for number of warnings and errors.
tammoippen Feb 16, 2016
dfa85ea
Add `VERBOSE=1` to make and also add VERBOSE=1 to parsing process
tammoippen Apr 13, 2016
5a09f25
Rename `file` variable to file_name
tammoippen Apr 25, 2016
d6f5f83
make standard bash empty string check
tammoippen Apr 25, 2016
559f19e
remove file comments
tammoippen Apr 25, 2016
21673cb
PEP8ify warnings part of the code.
tammoippen Apr 25, 2016
7cb0a6c
Update parse_travis_log.py
tammoippen Apr 26, 2016
aa89e2f
Create separate sli_readline target
tammoippen Apr 26, 2016
ba492b9
Comment sli_readline
tammoippen Apr 26, 2016
ec4228b
Remove extra file cg_connect.{h,cpp} and add functionality to conngen…
jougs Apr 28, 2016
0ce01ad
Refactor cmake includes into functions
tammoippen May 18, 2016
15a6ae7
Pre-set variables for cross-compiling
tammoippen May 23, 2016
d97f9d2
fix typo in warn message
tammoippen Jun 10, 2016
4a495c2
Merge branch 'master' into HEAD
jougs Jun 23, 2016
be137b7
Merge remote-tracking branch 'tammoippen/refactor_cmake' into cgconne…
jougs Jun 23, 2016
2ad63c6
Fix wording
jougs Jun 23, 2016
4cdf1fa
Refactor CGConnect to take gidcollections
jougs Jun 27, 2016
38e956b
Add CGConnect as additional SeeAlso
jougs Jun 27, 2016
14bcb73
Add SLI test for CGConnect
jougs Jun 27, 2016
7cce626
First draft of the Connection Generator documentation
jougs Jun 27, 2016
f1769dd
Merge branch 'master' into XXX_cgconnect_gidcollection
jougs Aug 10, 2016
3fb44f2
Install csa with libneurosim support for Travis CI
jougs Aug 10, 2016
b0002e4
Fix PYTHONPATH
jougs Aug 10, 2016
37238a1
Clean up .travis.yml and move libraries out of the NEST clone
jougs Aug 11, 2016
8e44cd1
Updated ConnGen test
jougs Aug 12, 2016
04cd821
Extended the documentation of the Connection Generator Interface
jougs Aug 12, 2016
8c9cbc0
Add availability to documentation
jougs Aug 17, 2016
e61e3d5
Merge branch 'master' of git://github.com/nest/nest-simulator into re…
jougs Sep 14, 2016
666d13c
Formatting
jougs Sep 14, 2016
19b7523
Remove obsoletion warning for SLIHOME environment variable
jougs Sep 14, 2016
6b34935
Add handling of environment variable SLI_PATH
jougs Sep 14, 2016
64dc2df
Add handling of environment variable NEST_MODULES
jougs Sep 14, 2016
a747327
Adding documentation on environment variables read by NEST
jougs Sep 14, 2016
a43e421
Fix English
jougs Sep 14, 2016
0136fd7
Made manualtest automatic
jougs Sep 14, 2016
0adc52b
Formatting
jougs Sep 15, 2016
9342411
Remove redundant check for locality
jougs Sep 28, 2016
3344b9b
Merge branch 'master' of git://github.com/nest/nest-simulator into re…
jougs Sep 28, 2016
41442b8
Fix Travis YAML file
jougs Sep 28, 2016
c785823
Fixed formatting
jougs Sep 29, 2016
2a66348
Merge branch 'master' of git://github.com/nest/nest-simulator into re…
jougs Oct 26, 2016
a1bd9fe
Fix ordering of list
jougs Oct 28, 2016
9770a71
Merge branch 'master' into refactor_conngen_interface
jougs Feb 24, 2017
44e8881
Updated and extended documentation for CGConnect
jougs Feb 24, 2017
5874b78
Use the Git repositories of csa and libneurosim
jougs Feb 24, 2017
2a774a2
Added const classifiers where possible
jougs Feb 24, 2017
292f690
Added +1 to the definition of num_elements and corrected places of usage
jougs Feb 24, 2017
4ba0af1
Marked SLI functions of low-level interface as internal
jougs Feb 24, 2017
77915a1
Fixed wrong type in documentation
jougs Feb 24, 2017
29d9d3a
Replace map by list comprehension and make print Py3 compatible
jougs Feb 24, 2017
b3e2b89
Actually clone the csa and libneurosim repositories
jougs Feb 25, 2017
49d1503
Merge branch 'master' of git://github.com/nest/nest-simulator into re…
jougs Sep 4, 2017
fbd8434
Fixed typo
jougs Sep 4, 2017
bd2fdfe
Removed stray word
jougs Sep 4, 2017
84d73c1
Add comments explaining the test procedure and expected outcome
jougs Sep 4, 2017
1541185
Split error checking unit test into two distinct functions
jougs Sep 4, 2017
fb3e1af
Formatting
jougs Sep 4, 2017
fe44089
Create RangeSets centrally in cg_set_mask
jougs Sep 5, 2017
0066013
Fix documentation
jougs Sep 5, 2017
43e8d6a
Minor cleanup and additions of const where possible
jougs Sep 5, 2017
12a3e9a
Merge branch 'master' into refactor_conngen_interface
jougs Sep 6, 2017
a411748
Merge branch 'master' into refactor_conngen_interface
jougs Sep 18, 2017
a835bd4
Converted conngen_interface SLI test file to an example
jougs Sep 18, 2017
80c9d62
Merge branch 'master' of git://github.com/nest/nest-simulator into re…
jougs Oct 13, 2017
9601223
Add Doxygen comment for cg_connect()
jougs Oct 13, 2017
70d77c4
Remove copying of values
jougs Oct 13, 2017
156b88b
Add check that w_idx and d_idx are either 0 or 1 and differ
jougs Oct 13, 2017
892c7fe
Added const and fixed formatting
jougs Oct 16, 2017
1cf4325
Set LD_LIBRARY_PATH so csa library is found
jougs Oct 16, 2017
5898e37
Updated comments in csa test and remove obsolete libcsa test
jougs Oct 16, 2017
1e423f5
Fixed TravisCI config so that CSA ist actually built and installed
jougs Oct 16, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 30 additions & 13 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
language: generic


#For Ubuntu Trusty (14.04 LTS)
sudo: required
dist: trusty

env:
matrix:
# We don't have to run a full matrix here, because most of the options are
# independent. This means that we can test them together in the full build.
# independent. This means we only need a minimal and maximal configuration
# and some additional jobs for interdependent options.
# See also https://docs.travis-ci.com/user/caching/#Caches-and-build-matrices
- xTHREADING=0 xMPI=0 xGSL=0 xLTDL=0 xREADLINE=0 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # minimal
- xTHREADING=0 xMPI=1 xGSL=0 xLTDL=1 xREADLINE=1 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only MPI
- xTHREADING=1 xMPI=0 xGSL=0 xLTDL=1 xREADLINE=1 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only threading
- xTHREADING=1 xMPI=0 xGSL=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # Python & Threading
- xTHREADING=0 xMPI=1 xGSL=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # Python & MPI
- xTHREADING=0 xMPI=0 xGSL=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only Python
- xTHREADING=1 xMPI=1 xGSL=1 xLTDL=1 xREADLINE=1 xPYTHON=1 xMUSIC=1 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # full
- xTHREADING=0 xMPI=0 xGSL=0 xLIBNEUROSIM=0 xLTDL=0 xREADLINE=0 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # minimal
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not related to this PR, but it looks like you do want a matrix for Python, mpi and threading. In the current setup mpi, threading and python are not tested in isolation.

You would then only need a single manual job

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of what you suggest was done in #409 and #640. More is to come in #789.

- xTHREADING=0 xMPI=1 xGSL=0 xLIBNEUROSIM=0 xLTDL=1 xREADLINE=1 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only MPI
- xTHREADING=1 xMPI=0 xGSL=0 xLIBNEUROSIM=0 xLTDL=1 xREADLINE=1 xPYTHON=0 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only threading
- xTHREADING=1 xMPI=0 xGSL=0 xLIBNEUROSIM=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # Python & Threading
- xTHREADING=0 xMPI=1 xGSL=0 xLIBNEUROSIM=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # Python & MPI
- xTHREADING=0 xMPI=0 xGSL=0 xLIBNEUROSIM=0 xLTDL=0 xREADLINE=0 xPYTHON=1 xMUSIC=0 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # only Python
- xTHREADING=1 xMPI=1 xGSL=1 xLIBNEUROSIM=1 xLTDL=1 xREADLINE=1 xPYTHON=1 xMUSIC=1 xSTATIC_ANALYSIS=1 CACHE_NAME=JOB # full
matrix:
# do notify immediately about it when a job of a build fails.
fast_finish: true
Expand All @@ -29,16 +29,17 @@ before_install:
- cd $HOME/build
- echo $PATH

# update package repository status (-qq is more quiet)
# Update package repository status
- sudo rm -rf /var/lib/apt/lists/*
- ls /etc/apt/sources.list.d/
- sudo apt-get update -qq

# remove any obsolete libraries
# Remove obsolete packages
- sudo apt-get autoremove

# from installation webpage http://www.nest-simulator.org/installation-2/#Standard_configuration
- sudo apt-get install -y build-essential cmake libltdl7-dev libreadline6-dev libncurses5-dev libgsl0-dev python-all-dev python-numpy python-scipy python-matplotlib ipython
# Install dependencies (see http://www.nest-simulator.org/installation)
- sudo apt-get install -y build-essential cmake libltdl7-dev libreadline6-dev libncurses5-dev
libgsl0-dev python-all-dev python-numpy python-scipy python-matplotlib ipython

# Install pip and cython
- wget https://bootstrap.pypa.io/get-pip.py
Expand All @@ -61,6 +62,22 @@ before_install:
# process json output from github api
- sudo apt-get install -y jq

# Install libneurosim
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that libneurosim is always installed? I don't see a conditional on xLIBNEUROSIM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is in accordance with the handling of all other dependencies, I would not want to change this here and rather do this in a refactoring of the whole dependency handling. A good opportunity is probably when we move to a Docker-based setup.

- git clone https://github.com/INCF/libneurosim.git
- cd libneurosim
- ./autogen.sh
- ./configure --prefix=$HOME/.cache/libneurosim.install --with-mpi
- make
- make install
- cd $HOME/build

# Install csa
- git clone https://github.com/INCF/csa.git
- cd csa
- ./autogen.sh
- ./configure --with-libneurosim=$HOME/.cache/libneurosim.install --prefix=$HOME/.cache/csa.install
- cd $HOME/build

# Install MUSIC
- sudo apt-get install -y python-mpi4py
- git clone https://github.com/INCF/MUSIC.git music
Expand Down
8 changes: 8 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,12 @@ else
CONFIGURE_READLINE="-Dwith-readline=OFF"
fi

if [ "$xLIBNEUROSIM" = "1" ] ; then
CONFIGURE_LIBNEUROSIM="-Dwith-libneurosim=$HOME/.cache/libneurosim.install"
else
CONFIGURE_LIBNEUROSIM="-Dwith-libneurosim=OFF"
fi



NEST_VPATH=build
Expand Down Expand Up @@ -228,6 +234,7 @@ cmake \
$CONFIGURE_GSL \
$CONFIGURE_LTDL \
$CONFIGURE_READLINE \
$CONFIGURE_LIBNEUROSIM \
..
echo "======= Configure NEST end ======="

Expand All @@ -240,6 +247,7 @@ make install
echo "======= Install NEST end ======="

echo "======= Test NEST start ======="
export PYTHONPATH=$HOME/.cache/csa.install/lib/python2.7/site-packages:$PYTHONPATH
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard coded python version

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w-klijn, indeed, but this has already been reworked and improved in the latest version.

make installcheck
echo "======= Test NEST end ======="

Expand Down
5 changes: 1 addition & 4 deletions conngen/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,7 @@
# You should have received a copy of the GNU General Public License
# along with NEST. If not, see <http://www.gnu.org/licenses/>.
set( conngen_sources
conngenmodule.h
conngenmodule.cpp
cg_connect.h
cg_connect.cpp
conngenmodule.h conngenmodule.cpp
conngendatum.h conngendatum.cpp
conngen.h conngen.cpp
)
Expand Down
Loading