Merge master into v1.5 #362

Accidentally added when merging changes from master branch.

The removal of the bindings lets us clean out the Mixins so we can rewrite the class in a much cleaner fashion. The File.open() function now simply calls the File() constructor.

ProxyLists are going to be removed but for now, I need to duplicate this definition to avoid an import cycle.

Was only supported in HDF5 backend.

- Dimension - EntityWithSources - Property - Section

HDF5 object wrappers (h5group and h5dataset) submodule is now called HDF5.

Top level files should retain original dates: 2015 for group.py and 2014 for the rest.

Travis issue causes packages to be built from source, which times out.

HDF5 object wrappers (h5group and h5dataset) submodule is now called HDF5.

More efficient versions of ProxyList and RefProxyList

Before it was assumed that if the group property is None then the underlying HDF5 group does not exist, since instantiating a group sets the property in such cases. With the new Container class, a long living H5Group object may be created that does not have a group prop set initially but ends up being created in another place (presumably, from a _create_obj() method).

LinkContainer requires reference to parent Block containers. Container and LinkContainer __init__ docstrings added. LinkContainer reimplements certain functions due to difference in indexing links in the backend.

With parent backend (H5Group) reference, it can't instantiate new NIX objects properly when loading from the backend. LinkContainer bugfixes - append after all checks pass - return super getitem call

Objects retrieved from LinkContainers would have a '_parent' reference to the linking parent (e.g., Group) instead of the location where they were created (e.g., Block). This is fixed now. The objects are created with a reference to the 'itemstore' parent.

Currently fails because LinkContainer doesn't handle Sources properly.

Subclasses LinkContainer

Creating test files in temporary directories avoids polluting the user's project directories with leftover nix files. More importantly, each setUp() function creates its own temporary directory, which allows tests to be run concurrently (use pytest -n auto).

Entity as common base class. No more EntityWithMetadata and EntityWithSources

The create_metadata_prop() function returns a property for setting, getting, and deleting metadata properties. We set it in the constructor of each object that can point to metadata (Block, Group, DataArray, Tag, MultiTag, Source)

The LinkContainer for Sources requires a slightly different append() function, so we override the LinkContainer base class.

Check and test removed.

Instead of converting back and forth between slices and (count, offset) pairs, we now work directly with slice objects or indices as they are provided.

The fix got lost in the merging and flattening of the classes. Bringing it back.

H5Py throws ValueError for out-of-bounds indexing. We catch it and throw IndexError instead, reusing the error message from H5Py. This also makes indexing more flexible. Slices now work the same as in numpy, so no slice is ever out-of-bounds. The corresponding test has been updated accordingly. This is an updated version of PR G-Node#303, which was a fix for G-Node#299.

Returns (id, item) tuples

The create_metadata_prop() function had some limitations which could cause issues. Every object now implements a metadata property getter, setter, and deleter explicitly. No fancy central md property, no baseclass for EntityWithMetadata, just plain properties on each class that supports it.

Getters, appenders, deleters, contains checkers.

StopIteration should not be raised manually by __iter__. See PEP 479 (Python 3.7).

- C++ files use nix::Variant instead of old nix::Value - Python functions write values directly and don't compare against nix.Value collections. - Compile script uses nixio as library name.

Files can now be opened using the 'with' statement for context management.

When creating a property using create_property(), strings are identified as Sequence and not converted to lists.

Numpy arrays are not Sequence types, but they are Iterable.

Setting an existing property to a single string (no list) breaks when np.shape(str) returns () and tries to resize the existing dataset. Including test for the issue.

Same name for both H5Group and H5DataSet

This can almost certainly be optimised, but it's more correct than the previous implementation, which risked infinite recursion.

Both of these require a different delete method that first collects their descendants and then searches through the entire tree to find references in order to delete them.

DimensionContainer failed to set the dimension properly when initialising a Dimension that was being loaded from the file. It was looking for an "index" attribute and was returning None. Should be using the name of the HDF5 Group object. Added tests to catch regressions. Wrapping index in int() when returning and when setting the object attribute to catch unwanted behaviour.

Let it be freeform. Could do X.Y.Z.beta, or X.Y.Z.dev.

+ simplifications and fixes

Updated NIX_LIBDIR

For uniformity with odML

Addressing issue G-Node#330 (partially) Use write_data() when creating labels instead of create_dataset(). The former creates the dataset if necessary and resizes is if it already exists.

Closes G-Node#330 Use write_data() when creating ticks instead of create_dataset(). The former creates the dataset if necessary and resizes is if it already exists.

Support optionally setting dimension attributes when creating (append_<>_dimension) a new dimension: - SetDimension: labels - RangeDimension: label, unit - SampledDimension: label, unit, offset Closes G-Node#331

help help

Squashed commits: * initial change in no-binding branch * start writing valid_da * valid da with units and poly * def val-functions and val dimension corectly * add check_file and check_blk function * del "is not None"/ Add poly & ex_origin * add check_for_basics function to perform common id, name type checks * adding loops * setting up validate file structure * finish blk and grp check and dict_struct * validate da * finish check_da function * finish check dim rdim setdim samdim exp sorted ticks * 1st draft * extent and pos * start testing files * writing test * not append, but eq to list/ finish test_blk * finish test_check_grp * finish test_data_array * testing tags * add get ref units function as seperate func * finish test tag * finish test_tag * finish test_mt * add test_prop * some small fixes * change the struct of props/dims/feas in dict * write the dim_test * all test done * changed param and add obj_ref * PEP8 fixes * add error_count for file.validate() * PEP8 fixes * cleanup

1D position arrays are handled as a special case. This used to work before because the DataArray would allow indexing with more indexes than the length of the array itself. The new DataArray indexing function doesn't allow this and tries to follow numpy array indexing rules. Test added to catch regression.

Always convert uncertainty to float for compatibility with odML

Use homebrew addon instead of manually invoking. See https://docs.travis-ci.com/user/installing-dependencies/#installing-packages-on-os-x

Importing abstract base classes directly from collections will cause an error in Python 3.8. Currently, in 3.7, it raises a warning. In 3.8, they will have to be imported from collections.abc. Wrapping in exception catcher for compatibility with older versions.

Squashed commits: * initial change in no-binding branch * start writing valid_da * valid da with units and poly * def val-functions and val dimension corectly * add check_file and check_blk function * del "is not None"/ Add poly & ex_origin * add check_for_basics function to perform common id, name type checks * adding loops * setting up validate file structure * finish blk and grp check and dict_struct * validate da * finish check_da function * finish check dim rdim setdim samdim exp sorted ticks * 1st draft * extent and pos * start testing files * writing test * not append, but eq to list/ finish test_blk * finish test_check_grp * finish test_data_array * testing tags * add get ref units function as seperate func * finish test tag * finish test_tag * finish test_mt * add test_prop * some small fixes * change the struct of props/dims/feas in dict * write the dim_test * all test done * changed param and add obj_ref * PEP8 fixes * add error_count for file.validate() * PEP8 fixes * cleanup

1D position arrays are handled as a special case. This used to work before because the DataArray would allow indexing with more indexes than the length of the array itself. The new DataArray indexing function doesn't allow this and tries to follow numpy array indexing rules. Test added to catch regression.

Importing abstract base classes directly from collections will cause an error in Python 3.8. Currently, in 3.7, it raises a warning. In 3.8, they will have to be imported from collections.abc. Wrapping in exception catcher for compatibility with older versions.

* finish write_to_pandas df * write_to_csv working! but very verbose * write_to_csv improvement/ cleanup variable * PEP8 fixes * some attr adpations after rebase * remove pandas

… err for len(col) not correct/ del col_idx for unit.setter

* fixed broken links eddited install instruction to install nixio no bindings * deleted all NIX c++ dependencies and information, added new "advance install" * added mising link to NIXPy * added beta version install instructions, fixed stable version install instruction

* small fix to append_column/ short col too long * write_df_compat sole test * fix bug for np.string not converted to vlen_str * Revert "write_df_compat sole test" in this branch This reverts commit a1c0ab1.

When looking for NIX to compile cross-compatibility tests, search both in /usr and /usr/local directories, as well as any directories supplied by the user through the env vars (NIX_LIBDIR, NIX_INCDIR, BOOST_LIBDIR, BOOST_INCDIR)

Old way is deprecated and produces warnings. Now building in conftest and sending bindir to tests via fixture. Tests are disabled by being marked with 'skip' if the build fails and force-compat is not specified.

The compatibility tests are no longer run by default when NIX is detected. They can be run by specifying --nix-compat. If specified, the tests are compiled and any errors will cause failure of the compatibility tests. If the option is not specified, the tests are skipped, even if NIX is available. One reason for this change is that the compatibility tests were always compiled when NIX was available, even when they are unselected, and compilation slows down the tests. With this change, they are only compiled and run when explicitly enabled.

Rebasing of no-bindings-dev branch off master had leftover files that were modified in master after the merge-base. Removing them here.

Counterpart to 779ee10 from before 1.5 dev rebase.

Counterpart to cca977c from before 1.5 dev rebase.

Bindings removal and internals rewrite: Merge to master

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge master into v1.5 #362

Merge master into v1.5 #362

Commits on Jan 17, 2019

Commits on Jan 18, 2019

Commits on Jan 21, 2019