-
Notifications
You must be signed in to change notification settings - Fork 35
/
changelog.txt
1066 lines (765 loc) · 45.2 KB
/
changelog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
=========
Changelog
=========
The **signac** package follows `semantic versioning <https://semver.org/>`_.
Version 2
=========
[2.2.0] -- 2024-02-13
---------------------
Added
+++++
- Official support for Python 3.12 (#957).
- ``Job.cached_statepoint`` - cached and read only access to job state points. Faster than ``Job.statepoint`` (#975).
Changed
+++++++
- Restrict allowable tar file features in Python 3.12 (#957).
- Linked views now can contain spaces and other characters except directory separators (#926).
- Linked views now can be created on Windows, if 'Developer mode' is enabled (#430, #958).
- Increase performance for many usage patterns (#975).
Fixed
+++++
- Fixed parsing of ``$not`` query expressions on command line (#970).
Removed
+++++++
- Unused ``ruamel.yaml`` dependency (#981).
[2.1.0] -- 2023-07-12
---------------------
Changed
+++++++
- signac's "synced collections" backend used for reading/writing JSON files has been moved to a separate package called ``synced_collections``. See https://github.com/glotzerlab/synced_collections (#924).
[2.0.0] -- 2023-03-30
---------------------
Added
+++++
- Official support for Python 3.11 (#875).
- ``H5Store`` related errors are now included in the public API (#775).
- Users can now access the project to which a job belongs with the ``Job.project`` attribute (#808).
Changed
+++++++
- Functions raising ``AssertionError`` now raise ``RuntimeError`` (#612).
- State points and documents require keys to be of type ``str`` (#719).
- The keyword ``_id`` of the Job constructor has been renamed to ``id_`` (#681).
- The signac schema to use version 2. This update involves a number of changes:
- The project configuration is now stored at .signac/config instead of signac.rc or .signacrc (#678).
- Schema versions are now identified by simple integers rather than semantic versioning style versions (#688).
- Projects no longer have names and are identified solely by their root directories. This change also affects a number of public APIs where names are including, most prominently ``signac.init_project`` and ``Project.init_project``. Projects can now be constructed with just a root directory rather than a preloaded config (#677, #684, #706).
- Project workspaces are no longer configurable, but are instead always defined as a subdirectory of the Project's root directory called ``workspace`` (#714).
- Rather than searching upwards until the root, ``load_config`` will only load configuration files in the specified directory, which is assumed to be a project directory, as well as the user's home directory (#711).
- Changed the ``root`` parameter to ``path`` in the ``signac.get_project`` and ``signac.init_project`` functions and corresponding ``Project`` methods (#757, #758).
- The prefix argument to ``$ signac view`` is now optional and can be provided with ``-p/--prefix`` (#653, #774).
- Tests are run with ``xfail_strict = True`` (#850).
- Detection of an invalid config will raise an error rather than a debug log (#855).
- The package namespace has been flattened so that most functionality is directly available in the ``signac`` namespace (#756, #868).
- The ``calc_id`` function has been moved from the ``hashing`` module to the ``job`` module (#873).
- Tests are run with all warnings treated as errors (#871).
- The contents of the ``syncutil`` module have been moved into ``sync`` (#869).
- The project now leverages a fully pyproject.toml-driven build (#878).
Removed
+++++++
- The pre-SyncedCollection synchronized dictionary classes, including SyncedDict, SyncedAttrDict, and JSONDict (#577).
- The old custom JSON encoder and dumps wrapper (#577).
- The MPIPool and the filesystems.py module (#575).
- The following Project methods: ``get_id``, ``build_job_search_index``, ``build_job_statepoint_index``, ``find_job_ids``, ``reset_statepoint``, ``update_statepoint``, ``create_access_module``, ``index``, ``dump_statepoints``, ``get_statepoint``, ``read_statepoints``, ``write_statepoints``, ``groupbydoc``, ``root_directory``, ``num_jobs`` (#574, #593, #599, #601, #752).
- The following Job methods: ``get_id``, ``workspace``, ``ws``, ``reset_statepoint`` (#578, #752).
- The ``syncutil.copytree`` method (#581).
- All Crawlers, including ``RegexFileCrawler``, ``MainCrawler``, ``MasterCrawler``, ``SignacProjectCrawler``, and ``BaseCrawler``, in addition to all associated functionality in indexing.py (#580).
- The cite.py module (#594).
- The following types of exceptions: ``AuthenticationError``, ``ExportError`, ``FetchError``, ``PermissionsError`` (#595, #596).
- All MongoDB database functionality, including the ``db`` subpackage, the ``connection``, ``crypt``, and ``host`` submodules in the ``common`` subpackage, and all associated configuration information (#576, #596).
- The ``JobSearchIndex`` class (#587).
- The ability to pass indexes to various ``Project`` methods (#599).
- The following ``JobsCursor`` methods: ``groupbydoc``, ``next`` (#601, #604).
- The ``Project.config`` property is no longer mutable. Use the command line ``$ signac config`` to modify configuration (#608, #246, #244).
- The config module and all its functions, all of which have been made private (#674, #753, #789, #847, #877).
- ``Project`` subclasses can no longer define a ``Job`` subclass to use (#588, #693).
- The ``Collection`` class (#664, #667, #683).
- The ``project`` CLI subcommand (#752).
- The ``--workspace`` option for the command line ``job`` subcommand (#752).
- The ability to call ``Project.workspace``, it is strictly a property now (#752).
- ``ProjectSchema.__call__``, ``ProjectSchema.detect`` (#752).
- The ``--doc-filter`` option for several command line subcommands (#613, #795).
- The public API of the ``synced_collection`` subpackage (#807, #790).
- The warnings module (#815).
- The signac.common.validate module (#853).
- The ``Config`` class (#860).
- The ``DictManager`` class (#858).
- The ``signac.testing`` module (#863).
- Zenodo is no longer updated (#903).
Version 1
=========
[1.8.0] -- 2022-10-05
---------------------
Added
+++++
- Official support for Python 3.10 (#631).
- Benchmarks can be run using the ``asv`` (airspeed velocity) tool (#629).
- Continuous integration tests run in parallel with ``pytest-xdist`` (#705).
- The ``Project.path`` and ``Job.path`` properties (#685).
Changed
+++++++
- Schema migration is now performed on directories rather than signac projects and supports a wider range of schemas (#654).
- Deprecated features now use ``FutureWarning`` instead of ``DeprecationWarning``, which is hidden by default (#687, #691, #692).
- Project names have a default in anticipation of removing names entirely. Project names will be removed in signac 2.0 (#644).
- ``Project.workspace`` is now a property, not a method (#685).
- Continuous integration uses GitHub Actions instead of CircleCI (#776, #788).
- Raise errors in testing when ``DeprecatedWarning`` or ``FutureWarning`` is raised (#713).
- Change GitHub PR to check for uncompleted tasks (i.e. unchecked checkboxes) (#686).
Deprecated
++++++++++
- ``Project`` methods ``read_statepoints``, ``write_statepoints``, and ``dump_statepoints`` are deprecated (#579, #197).
- ``Project.index`` method is deprecated (#591, #588).
- ``JobSearchIndex`` class is deprecated (#600).
- ``index`` argument is deprecated in ``Project`` methods (#602, #588).
- ``signac.cite`` module is deprecated (#611, #592).
- The ``config`` module and all its methods are deprecated (#675, #753, #814).
- Accessing ``Project.workspace`` as a method, it should be accessed as a property (#685).
- ``Project.num_jobs`` (#685).
- ``ProjectSchema.__call__``, ``ProjectSchema.detect`` (#685).
Fixed
+++++
- ``H5Store.mode`` returns the file mode (#607).
- User-provided path functions now raise an error if not unique (#666).
- ``Collection`` class no longer raises an error when searching by a primary key that does not exist (#676).
- Relative paths on Windows are not used if the current directory has no common prefix (#777).
- ``get_project()`` now raises an error if provided a root directory that does not exist (#779, #792).
- Catch internally raised warnings on use of deprecated password cache (#754).
- Catch ``KeyError`` from multithreading error (#710).
- Tests now properly show raised warnings (#603).
Removed
+++++++
- Removed upper bound of Python 4 on ``python_requires`` (#780, #781).
- Dropped support for Python 3.6 and Python 3.7 (#715) following the recommended support schedules of `NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`__.
- Dropped dependency on ``deprecation`` package (#687, #718).
- Removed unused ``_extract`` utility function to avoid CVE-2007-4559 (#829).
[1.7.0] -- 2021-06-08
---------------------
Added
+++++
- New ``SyncedCollection`` class and subclasses to replace ``JSONDict`` with more general support for different types of resources (such as MongoDB collections or Redis databases) and more complete support for different data types synchronized with files (#196, #234, #249, #316, #383, #397, #465, #484, #529, #530). This change introduces a minor-backwards incompatible change; for users making direct use of signac buffering, the ``force_write`` parameter is no longer respected. If the argument is passed, a warning will now be raised to indicate that it is ignored and will be removed in signac 2.0.
- Unified querying for state point and document filters using 'sp' and 'doc' as prefixes (#332, #514). This change introduces a minor backwards-incompatible change to the ``Collection`` index schema ('statepoint'->'sp'), but this does not affect any APIs, only indexes saved to file using a previous version of signac. Indexing APIs will be removed in signac 2.0.
Changed
+++++++
- Optimized internal path joins to speed up project iteration (#515).
Deprecated
++++++++++
- ``doc_filter`` arguments, which are replaced by namespaced filters.
- The modules ``signac.core.attrdict``, ``signac.core.json``, ``signac.core.jsondict``, and ``signac.core.synceddict.py`` are deprecated in favor of the new ``SyncedCollection`` classes and will be removed in signac 2.0 (#483).
Fixed
+++++
- Corrected docstrings for ``Job.update_statepoint`` and ``Project.update_statepoint`` (#506, #563).
[1.6.0] -- 2021-01-24
---------------------
Added
+++++
- Implemented ``JobsCursor.__contains__`` check (#449).
- Added documentation for ``JobsCursor`` class (#475).
Changed
+++++++
- Optimized job hash and equality checks (#442, #455).
- Optimized ``H5Store`` initialization (#443).
- State points are loaded lazily when ``Job`` is opened by id (#238, #239).
- Optimized ``Job`` and ``Project`` classes to cache internal properties and initialize on access (#451).
- Python 3.6 is only tested with oldest dependencies (#474).
- Improved documentation for updating and resetting state points (#444).
Deprecated
++++++++++
- Deprecate ``syncutil.copytree`` method (#439).
Fixed
+++++
- Zero-dimensional NumPy arrays can be used in state points and documents (#449).
[1.5.1] -- 2020-12-19
---------------------
Added
+++++
- Support for h5py version 3 (#411).
- Added pyupgrade to pre-commit hooks (#413).
- Code is formatted with ``black`` and ``isort`` pre-commit hooks (#415).
- Added macOS to CircleCI testing pipeline (#281, #414).
- Official support for Python 3.9 (#417).
Changed
+++++++
- Optimized internal function ``_mkdir_p`` (#421).
- Optimized performance of job initialization (#422).
- Optimized performance of buffer storage (#428).
- Optimized performance of creating/loading synced data structures (#429).
[1.5.0] -- 2020-09-20
---------------------
Added
+++++
- Type annotations are validated during continuous integration (#313).
- Added ``_repr_html_`` method in ``ProjectSchema`` class (#314, #324).
- Allow grouping by variables that are not present in all jobs in the project in ``JobsCursor.groupby`` (#321, #323).
- Added parameters ``usecols`` and ``flatten`` to allow selection of columns and flattening of nested data when converting signac data into a pandas DataFrame (#327, #330).
- Added support for `pre-commit hooks <https://pre-commit.com/>`__ (#355, #358).
- Expanded CLI documentation (#187, #359, #377).
Changed
+++++++
- Docstrings are now written in `numpydoc style <https://numpydoc.readthedocs.io/en/latest/format.html>`__.
Fixed
+++++
- Fix the ``signac config verify`` command (previously broken) (#301, #302).
- Warnings now appear when raised by the ``signac`` CLI (#317, #308).
- Fix dots in synchronization error messages (#375, #376).
Deprecated
++++++++++
- Deprecate the ``create_access_modules`` method in ``Project``, to be removed in 2.0 (#303, #308).
- The ``MainCrawler`` class has replaced the ``MasterCrawler`` class. Both classes are deprecated (#342).
Removed
+++++++
- Dropped support for Python 3.5 (#340). The signac project will follow the `NEP 29 deprecation policy <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_ going forward.
- Removed dependency on ``pytest-subtests`` (#379).
[1.4.0] -- 2020-02-28
---------------------
Added
+++++
- Added Windows to platforms tested with continuous integration (#264, #266).
- Add command line option ``-m/--merge`` for ``signac sync`` (#280, #230).
Changed
+++++++
- Workspace directory is created when ``Project`` is initialized (#267, #271).
- Changed testing framework from ``unittest`` to ``pytest`` (#212, #275).
- Refactored internal use of deprecated ``get_statepoint`` function (#227, #282).
Fixed
+++++
- Fixed issues on Windows with ``H5Store``, project import/export, and operations that move files (#264, #266).
- Calling ``items`` or ``values`` on ``_SyncedDict`` objects does not mutate nested dictionaries (#234, #269).
- Fixed issue with ``project.data`` access from separate instances of ``H5StoreManager`` (#274, #278).
- Fixed error when launching ``signac shell`` if permissions are denied for ``.signac_shell_history`` (#279).
Removed
+++++++
- Removed vendored ``tqdm`` module and replaced it with a requirement (#289).
- Removed support for ``rapidjson`` as an alternative JSON library (#285, #287).
- Removed tuple of keys implementation of nested dictionaries (#272, #296).
[1.3.0] -- 2019-12-20
---------------------
Added
+++++
- Official support for Python 3.8 (#258).
- Add properties ``Project.id`` and ``Job.id`` (#250).
- Add ``signac.diff_jobs`` function and ``$ signac diff`` CLI command to compare two or more state points (#248, #247).
- Add function to initialize a sample data space for testing purposes (#215).
- Add schema version to ensure compatibility and enable migrations in future package versions (#165, #253).
Changed
+++++++
- Implemented ``Project.__contains__`` check in constant time (#231).
Fixed
+++++
- Attempting to create a linked view for a Project on Windows now raises an informative error message (#214, #236).
- Project configuration is initialized using ConfigObj, allowing the configuration to include commas and special characters (#251, #252).
Deprecated
++++++++++
- Deprecate the ``get_id`` method in ``Project`` and ``Job`` classes in favor of the ``id`` property, to be removed in 2.0 (#250).
- In-memory modification of the project configuration, to be removed in 2.0 (#246).
Removed
+++++++
- Dropped support for Python 2.7 (#232).
[1.2.0] -- 2019-07-22
---------------------
Added
+++++
- Keep signac shell command history on a per-project basis (#134, #194).
- Add ``read_json()`` and ``to_json()`` methods to ``Collection`` class (#104, #200).
Fixed
+++++
- Fix issue where shallow copies of instances of ``Job`` would behave incorrectly (#153, #207).
- Fix issue causing a failure of the automatic conversion of valid key types (#168, #205).
- Improve the 'dots in keys' error message to make it easier to fix related issues (#170, #205).
- Update the ``__repr__`` and ``__repr_html__`` implementations of the ``Project``, ``Job``, and ``JobsCursor`` classes (#193).
- Reduce the logging verbosity about a missing default host key in the configuration (#201).
- Fix issue with incorrect detection of dict-like files managed with the ``DictManager`` class (e.g. ``job.stores``) (#203).
- Fix issue with generating views from the command line for projects with only one job (#208, #211).
- Fix issue with heterogeneous types in state point values that are lists (#209, #210).
Deprecated
++++++++++
- Support for Python 2.7 is deprecated with this version and will be removed in the next minor release. See the `Python 3 Statement <https://python3statement.org/>`_.
[1.1.0] -- 2019-05-19
---------------------
Added
+++++
- Add command line options ``--sp`` and ``--doc`` for ``signac find`` that allow users to display key-value pairs of the state point and document in combination with the job id (#97, #146).
- Improve the representation (return value of `repr()`) of instances of ``H5Group`` and ``SyncedAttrDict``.
Fixed
+++++
- Fix: Searches for whole numbers will match all numerically matching integers regardless of whether they are stored as decimals or whole numbers (#169).
- Fix: Passing an instance of dict to `H5Store.setdefault()` will return an instance of `H5Group` instead of a dict (#180).
- Fix error with storing numpy arrays and scalars in a synced dictionary (e.g. `job.statepoint`, `job.document`) (#184).
- Fix issue with ``ResourceWarning`` originating from unclosed instance of ``Collection`` (#186).
- Fix issue with using the ``get_project()`` function with a relative path and ``search=False`` (#191).
Removed
+++++++
- Support for Python version 3.4 (no longer tested).
[1.0.0] -- 2019-02-28
---------------------
Added
+++++
- Official support for Python 3.7.
- The ``H5Store`` and ``H5StoreManager`` classes, which are useful for storing (numerical) array-like data with an HDF5-backend. These classes are exposed within the root namespace.
- The ``job.data`` and ``project.data`` properties which present an instance of ``H5Store`` to access numerical data within the job workspace and project root directory.
- The ``job.stores`` and ``project.stores`` properties, which present an instance of ``H5StoreManager`` to manage multiple instances of ``H5Store`` to store numerical array-like data within the project workspace and project root directory.
- The ``signac.get_job()`` and the ``signac.Project.get_job()`` functions that allow users to get a job handle by switching into or providing the job's workspace directory.
- The ``job`` variable is automatically set when opening a ``signac shell`` from within a job's workspace directory.
- Add the ``signac shell -c`` option which allows the direct specification of Python commands to be executed within the shell.
- Automatic cast of ``numpy`` arrays to lists when storing them within a ``JSONDict``, e.g., a ``job.statepoint`` or ``job.document``.
- Enable ``Collection`` class to manage collections stored in compressed files (gzip, zip, etc.).
- Enable deleting of ``JSONDict`` keys through the attribute interface, e.g., ``del job.doc.foo``.
- Pretty HTML representation of instances of ``Project`` and ``JobsCursor`` targeted at Jupyter Notebooks (requires pandas, automatically enabled when installed).
- The ``to_dataframe()`` function to export the job state point and document data of a ``Project`` or a ``JobsCursor``, e.g., the result of ``Project.find_jobs()``, as a ``pandas.Dataframe`` (requires pandas).
Changed
+++++++
- Dots (`.`) in keys are no longer allowed for ``JSONDict`` and ``Collection`` keys (previously deprecated).
- The ``JSONDict`` module is exposed in the root namespace, which is useful for storing text-serializable data with a JSON-backend similar to the ``job.statepoint`` or ``job.document``, etc.
- The ``Job.init()`` method returns the job to allow one-line job creation and initialization.
- The ``search`` argument was added to the ``signac.get_project()`` function, which when True (the default), will cause signac to search for a project within *and above* a specified root directory, not only within the root directory. The behavior without any arguments remains unchanged.
Fixed
+++++
- Fix ``Collection.update()`` behavior such that existing documents with identical primary key are updated. Previously, a ``KeyError`` would be raised.
- Fix issue where the ``Job.move()`` would trigger a confusing ``DestinationExists`` exception when trying to move jobs across devices / file systems.
- Fix issue that caused failures when the ``python-rapidjson`` package is installed. The ``python-rapidjson`` package is used as the primary JSON-backend when installed.
- Fix issue where schema with multiple keys would subset incorrectly if the list of jobs or statepoints was provided as an iterator rather than a sequence.
Removed
+++++++
- Removes the obsolete and deprecated ``core.search_engine`` module.
- The previously deprecated ``Project.find_statepoints()`` and ``Project.find_job_documents()`` functions have been removed.
- The ``Project.find_jobs()`` no longer accepts the obsolete ``index`` argument.
Version 0.9
===========
[0.9.5] -- 2019-01-31
---------------------
Fixed
+++++
- Ensure that the next() function can be called for a JobsIterator, e.g., `project.find()`.
- Pickling issue that occurs when a _SyncedDict (job.statepoint, job.document, etc.) contains a list.
- Issue with the readline module that would cause signac shell to fail on Windows operating systems.
[0.9.4] -- 2018-10-24
---------------------
Added
+++++
- Adds the ``$ signac import`` command and the ``Project.import_from()`` method for the import of data spaces into a project workspace, such as a directory, a tarball, or a zip file.
- Adds the ``$ signac export`` command and the ``Project.export_to()`` method for the export of project workspaces to an external location, such as a directory, a tarball, or a zip file.
- Adds functionality for the rapid initialization of temporary projects with the ``signac.TemporaryProject`` context manager.
- Adds the ``signac.Project.temporary_project()`` context manager which creates a temporary project within the root project workspace.
- Add ``signac`` to the default namespace when invoking ``signac shell``.
- Add option to specify a custom view path for the ``signac view``/ ``Project.create_linked_view()`` function.
- Iterables of documents used to construct a Collection no longer require an ``_id`` field.
Changed
+++++++
- The default path for linked views has been adjusted to match the one used for data exports.
Fixed
+++++
- Fix issue where differently typed integer values stored within a Collection under the same key would not be indexed correctly. This issue affected the correct function of the `$type` operator for aforementioned cases and would lead to incorrect types in the Project schema detection algorithm for integer values.
- Fix issue where jobs that are migrated (state point change), but are not initialized, were not properly updated.
- Fix issue where changes to lists as part of synchronized dictionary, for example a state point or document would not be saved.
- Fix non-deterministic issue occuring on network file systems when trying to open jobs where the user has no write access to the job workspace directory.
[0.9.3] -- 2018-06-14
---------------------
Added
+++++
- Add `$near` operator to express queries for numerical values that match up to a certain precision.
- Add the `$ signac shell` sub command to directly launch a Python interpreter within a project directory.
Fixed
+++++
- Fix issue where a job instance would not be properly updated after more than one state point reset.
[0.9.2] -- 2017-12-18
---------------------
Added
+++++
- Add provisional feature (persistent state point caching); calling the ``Project.update_cache()`` method will generate and store a persistent state point cache in the project root directory, which will increase the speed of many project iteration, search, and selection operations.
- Add ``Project.check()`` method which checks for workspace corruption, but does not make any attempt to repair it.
- The ``Project.repair()`` method will attempt to repair jobs, that have been corrupted by manually renaming the job's workspace directory.
Changed
+++++++
- Enable the *write_concern* flag for the ``job.document``.
- Allow to omit the specification of an authentication mechanism in the MongoDB host configuration.
Fixed
+++++
- Fix critical issue in the ``JSONDict`` implementation that would previously overwrite the underlying file when attempting to store values that are not JSON serializable.
- Fix issue where the ``Project.export()`` function would ignore the update argument when the index to export to would be a MongoDB collection.
[0.9.1] -- 2017-11-07
---------------------
Fixed
+++++
- Fix critical issue in the ``SyncedAttrDict`` implementation that would previously overwrite the underlying file if the first operation was a ``__setitem__()`` operation.
[0.9.0] -- 2017-10-28
---------------------
Added
+++++
- Introduction of ``$ signac sync``, ``Project.sync()``, and ``Job.sync()`` for the simplified and fine-grained synchronization of multiple project data spaces.
- Introduction of ``$ signac schema`` and ``Project.detect_schema()`` for the automatic detection of the implicit and semi-structured state point schema of a project data space.
- Simplified aggregation of jobs over projects and ``Project.find_jobs()`` results with the ``Project.groupby()`` function.
- Support for project-centralized data with the ``Project.document`` attribute and the ``Project.fn()`` method for the wrapping of filenames within the project root directory.
- Added the ``Job.clear()`` and the ``Job.reset()`` methods to clear or reset a job's workspace data.
Changed
+++++++
- Both ``Job.statepoint`` and ``Job.document`` now use the same underlying data structure and provide the exact same API (copy with ``()`` and access of keys as attributes).
- The ``Collection`` class uses an internal counter instead of UUIDs for the generation of primary keys (resulting in improved performance).
- Major performance improvements (faster Collection, improved caching)
- Overhaul of the reference documentation.
Version 0.8
===========
[0.8.7] -- 2017-10-05
---------------------
Fixed
+++++
- Fix an issue where the creation of linked views was non-deterministic in some cases.
- Fix an issue where the creation of linked views would fail when the project contains job with state points that have lists as values.
[0.8.6] -- 2017-08-25
---------------------
Fixed
+++++
- Fix Collection append truncation issue (see issue #66).
[0.8.5] -- 2017-06-07
---------------------
Changed
+++++++
- The signac ids in the `signac find --show` view are no longer enclosed by quotation marks.
Fixed
+++++
- Fix compatibility issue that broke the `signac find --view` and all `--pretty` commands on Python 2.7.
- Fix issue where view directories would be incomplete in combination with heterogeneous state point schemas.
[0.8.4] -- 2017-05-19
---------------------
Added
+++++
- All search queries on project and collection objects support various operators including: *$and*, *$or*, *$gt*, *$gte*, *$lt*, *$lte*, *$eq*, *$ne*, *$exists*, *$regex*, *$where*, *$in*, *$nin*, and *$type*.
- The ``$ signac find`` command supports a simple filter syntax, where key value pairs can be provided as individual arguments.
- The ``$ signac find`` command is extended by a *--show* option, to display the state point and the document contents directly. The contents are truncated to an adjustable depth to reduce output noise.
- The ``$ signac view`` command has an additional filter option to select a sub data space directly without needing to pipe job ids.
- The new ``$ signac document`` command can be used to display a job's document directly.
Changed
+++++++
- Minor performance improvements.
[0.8.3] -- 2017-05-10
---------------------
Changed
+++++++
- Raise ``ExportError`` when updating with an empty index.
Fixed
+++++
- Fix command line logic issue with ``$signac config host``.
- Fix bug, where ``Collection.replace_one()`` would ignore the upsert argument under specific conditions.
[0.8.2] -- 2017-04-19
---------------------
Fixed
+++++
- Fixes a ``TypeError`` which occurred under specific conditions when calling ``Collection.find()`` with nested filter arguments.
[0.8.1] -- 2017-04-17
---------------------
Fixed
+++++
- Fixes wide-spread typo (*indeces* -> *indexes*).
[0.8.0] -- 2017-04-16
---------------------
Overall major simplification of the generation of indexes and the management and searching of index collections without external database.
Added
+++++
- Introduction of the ``Collection`` class for the management of document collections, such as indexes in memory and on disk.
- Generation of file indexes directly via the ``signac.index_files()`` function.
- Generation of main indexes directly via the ``signac.index()`` function and the ``$ signac index`` command.
- The API of ``signac_access.py`` files has been simplified, including the possibility to use a blank file for a minimal configuration.
- Use the ``$ signac project --access`` command to create a minimal access module in addition to ``Project.create_access_module()``.
- The update of existing index collections has been simplified by using the ``export()`` function with the `update=True` argument, which means that stale documents (the associated file or state point no longer exists) are automatically identified and removed.
- Added the ``Job.ws`` attribute, as short-cut for ``Job.workspace()``.
- The ``Job.sp`` interface has a ``get()`` function which can be used to specify a default value in case that the requested key is not part of the state point.
Changed (breaking API)
++++++++++++++++++++++
- The ``$ signac index`` command generates a main index instead of a project index. To generate a project index from the command line use ``$ signac project --index`` instead.
- The ``SignacProjectCrawler`` class expects the project's root directory as first argument, not the workspace directory.
- The ``get_crawlers()`` function defined within a ``signac_access.py`` access module is expected to yield crawler instances directly, not a mapping of crawler ids and instances.
- The simplification of the ``signac_access.py`` module API is reflected in a reduction of arguments to the ``Project.create_access_module()`` method.
Changed (non-breaking)
++++++++++++++++++++++
- The ``RegexFileCrawler``, ``SignacProjectCrawler`` and ``MainCrawler`` classes were moved into the root namespace.
- If a ``MainCrawler`` object is instantiated with the ``raise_on_error`` argument set to True, any errors encountered during crawling are raised instead of ignored and skipped; this simplifies the debugging of erroneous access modules.
- Improved error message for invalid configuration files.
- Better error messages for invalid ``$ signac find`` queries.
- Check a host configuration on the command line via ``$ signac host --test``.
- A MongoDB database host configuration defaults to *none* when no authentication method is explicitly specified.
- Using the ``--debug`` option in combination with ``$ signac index`` will show the traceback of errors encountered during indexing instead of ignoring them.
- Instances of ``Job`` are hashable, making it possible to use them as dict keys for instance.
- The representation of ``Job`` instances via ``repr()`` can actually serves as copy constructor command.
- The project interface implementation performs all non-trivial search operations on an internally management index collection, which improves performance and simplifies the code base.
Deprecated
++++++++++
- The ``DocumentSearchEngine`` class has been deprecated, its functionality is now provided by the ``Collection`` class.
Fixed
+++++
- An issue related to exporting documents to MongoDB collections via pymongo in combination with Python 2.7 has been fixed.
Version 0.7
===========
[0.7.1] -- 2017-01-09
---------------------
Added
+++++
- When the ``python-rapidjson`` package is installed, it will be used for JSON encoding/decoding (experimental).
Changed
+++++++
- All job move-related methods raise ``DestinationExistsError`` in case of destination conflicts.
- Optimized ``$ signac find`` command.
Fixed
+++++
- Fixed bug in ``$ signac statepoint``.
- Suppress 'broken pipe error' message when using ``$ signac find`` for example in combination with ``$ head``.
[0.7.0] -- 2017-01-04
---------------------
Added
+++++
- Add support for Python version 3.6.
- Add support for PyPy and PyPy3.
- Simplified iteration over project data spaces.
- An existing linked view can be updated by executing the view command again.
- Add attribute interface for the access and modification of job state points: ``Job.sp``.
- Add function for moving and copying of jobs between projects.
- All project related iterators support the ``len``-operator.
- Enable iteration over all jobs with: ``for job in project:``.
- Make ``len(project)`` an alias for ``project.num_jobs()``.
- Add ``in``-operator to determine whether a job is initialized within a project.
- Add ``Job.sp`` attribute to access and modify a job's state point.
- The ``Project.open_job()`` method accepts abbreviated job ids.
- Add ``Project.min_len_unique_id()`` method to determine the minimum length of job ids to be unique within the project's data space.
- Add ``Job.move()`` method to move jobs between projects.
- Add ``Project.clone()`` method to copy jobs between projects.
- Add ``$ signac move`` and ``$ signac clone`` command line functions.
- Add ``Job.reset_statepoint()`` method to reset a job's state point.
- Add ``Job.update_statepoint()`` method to update a job's state point.
- Add a ``Job.FN_DOCUMENT`` constant which defines the default filename of the job document file
- The ``$ signac find`` command accepts a ``-d/--doc-filter`` option to filter by job document contents.
- Add the ``Project.create_linked_view()`` method as replacement for the previously deprecated ``Project.create_view()`` method.
Changed
+++++++
- Linked views use relative paths.
- The *Guide* documentation chapter has been renamed to *Reference* and generally overhauled.
- The *Quick Reference* documentation chapter has been extended.
Fixed
+++++
- Fix error when using an instance of ``Job`` after calling ``Job.remove()``.
- A project created in one the standard config directories (such as the home directory) does not take prevalence over project configurations in or above the current working directory.
Removed
+++++++
- The *signac-gui* component has been removed.
- The ``Project.create_linked_view()`` ``force`` argument is removed.
- The ``Project.find_variable_parameters()`` method has been removed
Version 0.6
===========
[0.6.2] -- 2017-12-15
---------------------
Added
+++++
- Add instructions on how to acknowledge **signac** in publications to documentation.
- Add cite module for the auto-generation of formatted references and BibTeX entries.
Removed
+++++++
- Remove SSL authentication support.
[0.6.1] -- 2017-11-26
---------------------
Changed
+++++++
- The ``Project.create_view()`` method triggers a ``DeprecationWarning`` instead of a ``PendingDeprecationWarning``.
- The ``Project.find_variable_parameters()`` method triggers a ``DeprecationWarning`` instead of a ``PendingDeprecationWarning``.
Fixed
+++++
- Make package more robust against PySide import errors.
- Fix ``Project.__repr__`` method.
- Fix critical bug in ``fs.GridFS`` class, which rendered it unusuable.
- Fix issue in ``indexing.fetch()`` previously resulting in local paths being ignored.
- Fix error ``signac.__all__`` namespace directive.
[0.6.0] -- 2016-11-18
---------------------
Added
+++++
- Add the ``export_to_mirror()`` function for mirroring files.
- Introduction of the ``signac.fs`` namespace to simplify the configuration of mirror filesystems.
- Add *errors* module to root namespace. Many exceptions raised inherit from the base exception types defined within that module, making it easier to catch signac related errors.
- Add the ``export_one()`` function for the export of a single index document; simplifies the implementation of custom export functions.
- Opening an instance of ``Job`` with the ``open_job()`` method multiple times and entering a job context recursively is now well-defined behavior: Opening a job now adds the current working directory onto a stack, closing it switches into the directory on top of the stack.
- The return type of ``Project.open_job()`` can be configured to make it easier to specialize projects with custom job types.
Changed
+++++++
- The MainCrawler logic has been simplified; their primary function is the compilation of index documents from subcrawlers, all export logic, including data mirroring is now provided by the ``signac.export()`` function.
- Each index document is now uniquely coupled with only one file or data object, which is why ``signac.fetch()`` replaces ``signac.fetch_one()`` and the latter one has been deprecated and is currently an alias of the former one.
- The ``signac.fetch()`` function always returns a file-like object, regardless of format definition.
- The format argument in the crawler ``define()`` function is now optional and has now very well defined behavior for str types. It is encouraged to define a format with a str constant rather than a file-like object type.
- The ``TextFile`` file-like object class definition in the formats module has been replaced with a constant of type ``str``.
- The ``signac.export()`` function automatically delegates to specialized implementations such as ``export_pymongo()`` and is more robust against errors, such as broken connections.
- The ``export_pymongo()`` function makes multiple automatic restart attempts when encountering errors.
- Documentation: The tutorial is now based on signac-examples jupyter notebooks.
- The ``contrib.crawler`` module has been renamed to contrib.indexing to better reflect the semantic context.
- The ``signac.export()`` function now implements the logic for data linking and mirroring.
- Provide default argument for '--indent' option for ``$ signac statepoint`` command.
- Log, but do not reraise exceptions during ``MainCrawler`` execution, making the compilation of main indexes more robust against errors.
- The object representation of ``Job`` and ``Project`` instances is simplified.
- The warning verbosity has been reduced when importing modules with optional dependencies.
Removed
+++++++
- All modules related to the stale *conversion framework* feature have been removed resulting in a removal of the optional networkx dependency.
- Multiple modules related to the *conversion framework* feature have been removed, including: ``contrib.formats_network``, ``contrib.conversion``, and ``contrib.adapters``.
Fixed
+++++
- Opening instances of ``Job`` with the ``Job.open()`` method multiple times, equivalently entering the job context recursively, does not cause an error anymore, but instead the behavior is well-defined.
Version 0.5
===========
[0.5.0] -- 2016-08-31
---------------------
Added
+++++
- New function: ``signac.init_project()`` simplifies project initialization within Python
- Added optional ``root`` argument to ``signac.get_project()`` to simplify getting a project handle outside of the current working directory
- Added optional argument to ``signac.get_project()``, to allow fetching of projects outside of the current working directory.
- Added two class factory methods to `Project`: ``get_project()`` and ``init_project()``.
Changed
+++++++
- The performance of project indexing and crawling has been improved.
Version 0.4
===========
[0.4.0] -- 2016-08-05
---------------------
Added
+++++
- The performance of find operations can be greatly improved by using pre-generated job indexes.
- New top-level commands: ``$ signac find``, ``$ signac index``, ``$ signac statepoint``, and ``$ signac view``.
- New method: ``Project.create_linked_view()``
- New method: ``Project.build_job_statepoint_index()``
- New method: ``Project.build_job_search_index()``
- The ``Project.find_jobs()`` method allows to filter by job document.
Changed
+++++++
- The ``SignacProjectCrawler`` indexes all jobs, not only those with non-empty job documents.
- The ``signac.fetch_one()`` function returns ``None`` if no associated object can be fetched.
- The tutorial is restructured into multiple parts.
- Instructions for installation are separated from the guide.
Removed
+++++++
- Remove previously deprecated *crawl* keyword argument in index export functions.
- Remove previously deprecated function common.config.write_config().
Version 0.3
===========
[0.3.0] -- 2016-06-23
---------------------
Added
+++++
- Add contributing agreement and guidelines.
Changed
+++++++
- Change license from MIT to BSD 3-clause license.
Version 0.2
===========
[0.2.9] -- 2016-06-06
---------------------
Added
+++++
- Addition of the ``signac config`` command line API.
- Password updates are encrypted with ``bcrypt`` when ``passlib`` is installed.
- The user is prompted to enter missing credentials (username/password) in case that they are not stored in the configuration.
- The ``$ signac confg`` tool provides the ``--update-pw`` argument, which allows users to update their own password.
- Added MIT license, in addition, all source code files contain a short licensing header.
Changed
+++++++
- Improved documentation on how to configure signac.
- The OSI classifiers are updated, including an upgrade of the development status to '4 - beta'.
Fixed
+++++
- Nested job state points can no longer get corrupted. This bug occurred when trying to operate on nested state point mappings.
Deprecated
++++++++++
- Deprecated pymongo versions 2.x are no longer supported.
[0.2.8] -- 2016-04-18
---------------------
Added
+++++
- ``Project`` is now in the root namespace.
- Add ``index()`` method to `Project`.
- Add ``create_access_module()`` method to ``Project``.
- Add ``find_variable_parameters()`` method to ``Project``.
- Add ``fn()`` method to ``Job``, which prepends the job's workspace path to a filename.
- The documentation contains a comprehensive tutorial
Changed
+++++++
- The ``crawl()`` function yields only the index documents and not a tuple of (_id, doc).
- ``export()`` and ``export_pymongo()`` expect the index documents as first argument, not a crawler instance. The old API is still supported, but will trigger a DeprecationWarning.
[0.2.7] -- 2016-02-29
---------------------
Added
+++++
- Add ``job.isfile()`` method
Changed
+++++++
- Optimize ``project.find_statepoints()`` and ``project.repair()`` functions.
[0.2.6] -- 2016-02-20
---------------------
Added
+++++
- Add ``job.reset_statepoint()`` and `job.update_statepoint()`
- Add ``job.remove()`` function
Changed
+++++++
- Sanitize filter argument in all ``project.find_*()`` methods.
- The ``job.statepoint()`` function accurately represents saved statepoints, e.g. tuples are represented as lists, as there is no difference between tuples and lists in JSON.
- signac-gui does not block on database operations.
- signac-gui allows reload of databases and collections of connected hosts.
Fixed
+++++
- ``RegexFileCrawler`` ``define()`` class function only acts upon the actual specialization and not globally on all ``RegexFileCrawler`` classes.
- signac-gui does not crash when replica sets are configured.
[0.2.5] -- 2016-02-10
---------------------
Added
+++++
- Added ``signac.get_project()``, ``signac.get_database()``, ``signac.fetch()`` and ``signac.fetch_one()`` to top-level namespace.
- Added basic shell commands, see ``$ signac --help``.
- Allow opening of jobs by id: ``project.open_job(id='abc123...')``.
- Mirror data while crawling.
- Use extra sources for ``fetch()`` and ``fetch_one()``.
- Add file system handler: ``LocalFS``, handler for local file system.
- Add file system handler: ``GridFS``, handler for MongoDB GridFS file system.
- Crawler tags, to control which crawlers are used for a specific index.
- Allow explicit job workspace creation with ``job.init()``.
- Forwarding of pymongo host configuration via **signac** configuration.
Changed
+++++++
- Major reorganization of the documentation, split into: Overview, Guide, Quick Reference and API.
- Documentation: Add notes for system administrators about advanced indexing.
- Warn about outdated pymongo versions.