-
Notifications
You must be signed in to change notification settings - Fork 185
/
ChangeLog
1725 lines (1413 loc) · 78.4 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
3.7.0:
New features:
- New package promise implementation.
The syntax is much simpler, to try it out, check out the syntax:
packages:
"mypackage"
policy => "absent/present",
# Optional, default taken from common control
package_module => apt_get,
# Optional, will only match exact version. May be
# "latest".
version => "32.0",
# Optional.
architecture => "x86_64";
- Full systemd support for all relevant platforms
- New classes to determine whether certain features are enabled:
* feature_yaml
* feature_xml
For the official CFEngine packages, these are always enabled, but
packages from other sources may be built without the support.
- New readdata() support for generic data input (CSV, YAML, JSON, or auto)
- YAML support: new readyaml() function and in readdata()
- CSV support: new readcsv() function and in readdata()
- New string_mustache() function
- New data_regextract() function
- eval() can now be called with "class" as the "mode" argument, which
will cause it to return true ("any") if the calculated result is
non-zero, and false ("!any") if it is zero.
- New list_ifelse() function
- New mapjson() function as well as JSON support in maparray().
- filestat() function now supports "xattr" argument for extended
attributes.
- "ifvarclass" now has "if" as an alias, and "unless" as an inverse
alias.
- Ability to expand JSON variables directory in Mustache templates:
Prefix the name with '%' for multiline expansion, '$' for compact
expansion.
- Ability to expand the iteration *key* in Mustache templates with @
- Canonical JSON output: JSON output has reliably sorted keys so the
same data structure will produce the same JSON every time.
- New "@if minimum_version(x.x)" syntax in order to hide future language
improvements from versions that don't understand them.
- compile time option (--with-statedir) to
override the default state/ directory path.
- Fix error messages/ handling in process signalling which no longer
allowed any signals to fail silently
- Also enable shortcut keyword for cf-serverd classic protocol, eg to
simplify the bootstrap process for clients that have different
sys.masterdir settings (Redmine #3697)
- methods promises now accepts the bundle name in the promiser string,
as long as it doesn't have any parameters.
- In a services promise, if the service_method bundle is not specified,
it defaults to the promiser string (canonified) with "service_" as a
prefix. The bundle must be in the same namespace as the promise.
- inline JSON in policy files: surrounding with parsejson() is now
optional *when creating a new data container*.
- New data_expand() function to interpolate variables in a data container.
- Add configurable network bandwidth limit for all outgoing
connections ("bwlimit" attribute in "body common control") . To
enforce it in both directions, make sure the attribute is set on both
sides of the connection.
- Secure bootstrap has been facilitated by use of
"cf-agent --boostrap HUB_ADDRESS --trust-server=no"
- Implement new TLS-relevant options (Redmine #6883):
- body common control: tls_min_version
- body server control: allowtlsversion
- body common control: tls_ciphers
- body server control: allowciphers (preexisting)
Changes:
- Improved output format, less verbose, and messages are grouped.
- cf-execd: agent_expireafter default was changed to 120 minutes
(Redmine #7113)
- All embedded databases are now rooted in the state/ directory.
- TLS used as default for all outgoing connections.
- process promise now reports kept status instead of repaired if a
signal is not sent, even if the restart_class is set. The old
behavior was to set the repaired status whenever the process was not
running. (Redmine#7216).
- Bootstrapping requires keys to be generated in advance using cf-key.
- Disable class set on reverse lookup of interfaces IP addresses.
(Redmine #3993, Redmine #6870)
- Define a hard class with just the OS major version on FreeBSD.
- Abort cf-agent if OpenSSL's random number generator can't
be seeded securely.
- Masterfiles source tarball now installs using the usual commands
"./configure; make install".
- Updated Emacs syntax highlighting template to support the latest
syntax enhancements in 3.7.
Deprecations:
- Arbitrary arguments to cfruncommand (using "cf-runagent -o") are
not acceptable any more. (Redmine #6978)
- 3.4 is no longer supported in masterfiles.
Bug fixes:
- Fix server common bundles evaluation order (Redmine#7211).
- Limit LMDB disk usage by preserving sparse areas in LMDB files
(Redmine#7242).
- Fixed LMDB corruption on HP-UX 11.23. (Redmine #6994)
- Fixed insert_lines failing to converge if preserve_block was used.
(Redmine #7094)
- Fixed init script failing to stop/restart daemons on openvz/lxc
hosts. (Redmine #3394)
- rm_rf_depth now deletes base directory as advertised. (Redmine #7009)
- Refactored cf-agent's connection cache to properly differentiate
hosts using all needed attributes like host and port.
(Redmine #4646)
- Refactored lastseen database handling to avoid inconsistencies.
(Redmine #6660)
- cf-key --trust-key now supports new syntax to also update the
lastseen database, so that clients using old protocol will trust
the server correctly.
- Fixed a bug which sometimes caused an agent or daemon to kill or stop
itself. (Redmine #7075, #7244)
- Fixed a bug which made it difficult to kill CFEngine daemons,
particularly cf-execd. (Redmine #6659, #7193)
- Fixed a bug causing systemd not to be detected correctly on Debian.
(Redmine #7297)
- "cf-promises -T" will now correctly report the checked out commit,
even if you haven't checked out a Git branch. (Redmine #7332)
- Reduce verbosity of harmless errors related to socket timeouts and
missing thermal zone files. (Redmine #6486 and #7238)
Masterfiles:
Added:
- Support for user specified overring of framework defaults without
modifying policy supplied by the framework itself (see
example_def.json)
- Support for def.json class augmentation in update policy
- Run vacuum operation on postgresql every night as a part of
maintenance.
- Add measure_promise_time action body to lib (3.5, 3.6, 3.7, 3.8)
- New negative class guard `cfengine_internal_disable_agent_email` so
that agent email can be easily disabled by augmenting def.json
Changed:
- Relocate def.cf to controls/VER/
- Relocate update_def to controls/VER
- Relocate all controls to controls/VER
- Only load cf_hub and reports.cf on CFEngine Enterprise installs
- Relocate acls related to report collection from bundle server
access_rules to controls/VER/reports.cf into bundle server
report_access_rules
- Re-organize cfe_internal splitting core from enterprise specific
policies and loading the appropriate inputs only when necessary
- Moved update directory into cfe_internal as it is not generally
intended to be modified
- services/autorun.cf moved to lib/VER/ as it is not generally intended
to be modified
- To improve predictibility autorun bundles are activated in
lexicographical order
- Relocate services/file_change.cf to cfe_internal/enterprise. This
policy is most useful for a good OOTB experience with CFEngine
Enterprise Mission Portal.
- Relocate service_catalogue from promsies.cf to services/main.cf. It is
intended to be a user entry. This name change correlates with the main
bundle being activated by default if there is no bundlesequence
specified.
- Reduce benchmarks sample history to 1 day.
- Update policy no longer generates a keypair if one is not found.
(Redmine: #7167)
- Relocate cfe_internal_postgresql_maintenance bundle to lib/VER/
- Set postgresql_monitoring_maintenance only for versions 3.6.0 and
3.6.1
- Move hub specific bundles from lib/VER/cfe_internal.cf into
lib/VER/cfe_internal_hub.cf and load them only if policy_server policy
if set.
- Re-organize lib/VER/stdlib.cf from lists into classic array for use
with getvalues
Removed:
- Diff reporting on /etc/shadow (Enterprise)
- Update policy from promise.cf inputs. There is no reason to include
the update policy into promsies.cf, update.cf is the entry for the
update policy
- _not_repaired outcome from classes_generic and scoped_classes generic
(Redmine: # 7022)
Fixes:
- standard_services now restarts the service if it was not already
running when using service_policy => restart with chkconfig (Redmine
#7258)
3.6.5:
Features:
- Introduced "systemd" hard class. (Redmine #6995)
- Added paths to dtrace, zfs and zpool on FreeBSD in masterfiles.
Bug fixes:
- Fixed build error on certain RHEL5 and SLES10 setups. (Redmine #6841)
- Fixed a bug which caused dangling symlinks not to be removed.
(Redmine #6582)
- Fixed data_readstringarrayidx function not preserving the order of the
array it's producing. (Redmine #6920)
- Fixed a bug which sometimes caused CFEngine to kill the wrong daemon
if both the host and a container inside the host were running
CFEngine. (Redmine #6906)
- Made sure the rm_rf_depth bundle also deletes the base directory.
(Redmine #7009)
- Fixed monitord reporting wrongly on open ports. (Redmine #6926)
- Skip adding the class when its name is longer than 1024 characters.
Fixed core dump when the name is too large. (Redmine #7013)
- Fixed detection of stopped process on Solaris. (Redmine #6946)
- Fixed infinite loop (Redmine #6992) plus a couple more minor
bugs in edit_xml promises.
3.6.4:
Features:
- Introduced users promises support on HP-UX platform.
- Introduced process promises support on HP-UX platform.
Bug fixes:
- Fixed bug on FreeBSD which sometimes led to the wrong process being
killed (Redmine #2330)
- Fixed package version comparison sometimes failing with rpm package
manager (Redmine #6807)
- Fixed a bug in users promises which would sometimes set the wrong
password hash if the user would also be unlocked at the same time.
- Fixed a bug on AIX which would occasionally kill the wrong process.
- Improved error message for functions that require an absolute path.
(Redmine #6877)
- Fixed some spelling errors in examples.
- Fixed error in out-of-tree builds when building cf-upgrade.
- Fixed a bug which would make cf-agent exit with an error if it was
built with a custom log directory, and that directory did not exist.
- Fixed ordering of evaluating promises when depends_on is used.
(Redmine #6484, Redmine #5462)
- Skip non-empty directories silently when recursively deleting.
(Redmine #6331)
- Fix memory exhaustion with list larger than 4994 items.
(Redmine # 6672)
- Fix cf-execd segfault on IP address detection (Redmine #6905).
- Fix hard class detection of RHEL6 ComputeNode (Redmine #3148).
3.6.3
New features:
- support for HP-UX 11.23 and later
- experimental support for Red Hat Enterprise Linux 7
Bug fixes:
- fix getindices on multi-dimensional arrays (Redmine #6779)
- fix mustache template method to run in dryrun mode (Redmine #6739)
- set mailto and mailfrom settings for execd in def.cf (Redmine #6702)
- fix conflation of multi-index entries in arrays (Redmine #6674)
- fix promise locking when transferring using update.cf (Redmine #6623)
- update JSON parser to return an error on truncation (Redmine #6608)
- fix sys.hardware_addresses not expanded (Redmine #6603)
- fix opening database txn /var/cfengine/cf_lastseen.lmdb:
MDB_READERS_FULL when running cf-keys --show-hosts (Redmine #6602)
- fix segfault (Null pointer dereference) when select_end in
delete_lines never matches (Redmine #6589)
- fix max_file_size => "0" not disabling or allowing any size
(Redmine #6588)
- fix ifvarclass, with iteration over list, failing when deleting
files with time condition (Redmine #6577)
- fix classes defined with "or" constraint are never set if any value
doesn't evaluate to a scalar (Redmine #6569)
- update "mailfrom" default in default policy (Redmine #6567)
- fix logrotate ambiguity of filename (Redmine #6563)
- fix parsing JSON files (Redmine #6549)
- reduce write count activity to /var partition (Redmine #6523)
- fix files delete attribute incorrectly triggering promise_kept
(Redmine #6509)
- update services bundle output related to chkconfig when run in
inform mode. (Redmine #6492)
- fix Solaris serverd tests (Redmine #6406)
- fix broken bechaviour of merging arrays with readstringarray
(Redmine #6369)
- fix ifelapsed bug with bundle nesting (Redmine #6334)
- fix handling cf_null in bundlesequence (Redmine #6119)
- fix maparray reading whole input array when using subarray
(Redmine #6033)
- fix directories being randomly changed to files (Redmine #6027)
- update defaults promise type to work with classes (Redmine #5748)
- systemd integration in services promises (Redmine #5415)
- fix touch attribute ignoring action = warn_only (Redmine #3172)
- fix 4KB string limit in functions readfile, string_downcase,
string_head, string_reverse, string_length, string_tail,
string_upcase (Redmine #2912)
3.6.2
Bug fixes:
- don't regenerate software_packages.csv every time (Redmine #6441)
- improve verbose message for package_list_command
- fix missing log output on AIX (Redmine #6434)
- assorted fixes to dirname() esp on Windows (Redmine #4716)
- fix package manager detection
- fix build issues on FreeBSD
- allow copying of dead symbolic links (Redmine #6175)
- preserve order in readstringarrayidx (Redmine #6466)
- fix passing of unexpanded variable references to arrays
(Redmine #5893)
- use entries for new {admin,deny}_{ips,hostnames} constraints in
the relevant legacy lists (Redmine #6542)
- cope with ps's numeric fields overflowing to the right
- interpret failing function calls in ifvarclass as class not set
(Redmine #6327)
- remove unexpanded lists when extending lists (Redmine #6541)
- infer start-time of a process from elapsed when needed
(Redmine #4094)
- fix input range definition for laterthan() function (Redmine #6530)
- don't add trailing delimiter when join()'ing lists ending with a
null-value (Redmine #6552)
- 9999999999 (ten 9s) or higher has been historically used as an upper
bound in CFEngine code and policy but because of overflow on 32-bit
platforms it caused problems with big numbers. Fixed in two ways:
first change all existing policy uses to 999999999 (nine 9s instead
of eleven 9s), second fix the C code to not wrap-around in case of
overflow, but use the LONG_MAX value (Redmine #6531).
- cf-serverd and other daemons no longer reload their configuration
every minute if CFEngine is built with an inputs directory outside
of the work directory (not the default). (Redmine #6551)
3.6.1
New features:
- Introduced Solaris and AIX support into the 3.6 series, with many associated build and
bug fixes.
Changes:
- Short-circuit evaluation of classes promises if class is already set (Redmine #5241)
- fix to assume all non-specified return codes are failed in commands promises (Redmine #5986)
- cf-serverd logs reconfiguration message to NOTICE (was INFO) so that it's always logged in syslog
Bug fixes:
- File monitoring has been completely rewritten (changes attribute in files promise), which
eliminates many bugs, particularly regarding files that are deleted. Upgrading will keep
all monitoring data, but downgrading again will reinitialize the DB, so all files will be
reported as if they were new. (Redmine #2917)
- $(this.promiser) expands in files promises for 'transformer', 'edit_template',
'copy_from.source', 'file_select.exec_program', 'classes' and 'action' bodies
(Redmine #1554, #1496, #3530, #1563)
- 'body changes' notifies about disappeared files in file monitoring (Redmine #2917)
- Fixed CFEngine template producing a zero sized file (Redmine #6088)
- Add 0-9 A-Z _ to allowed context of module protocol (Redmine #6063)
- Extend ps command column width and prepend zone name on Solaris
- Fixed strftime() function on Solaris when called with certain specifiers.
- Fixed users promise bug regarding password hashes in a NIS/NSS setup.
- Fixed $(sys.uptime), $(sys.systime) and $(sys.sysday) in AIX. (Redmine #5148, #5206)
- Fixed processes_select complaining about "Unacceptable model uncertainty examining processes" (Redmine #6337)
- ps command for linux has been changed to cope with big rss values (Redmine #6337)
- Address ps -axo shift on FreeBSD 10 and later (Redmine #5667)
- methods and services promises respect action_policy => "warn" (Redmine #5924)
- LMDB should no longer deadlock if an agent is killed on the hub while holding the DB lock.
Note that the change only affects binary packages shipped by CFEngine, since the upstream
LMDB project has not yet integrated the change. (Redmine #6013)
3.6.0
Changes:
- Changes to logging output
- add process name and pid in syslog message (GitHub #789)
- cf-serverd logging levels are now more standardised:
- INFO logs only failures
- VERBOSE logs successful requests as well
- DEBUG logs actual protocol traffic.
- cf-serverd now logs the relevant client IP address on
each message.
- Logging contexts to local database (cf_classes.tcdb) has been deprecated.
- 'usebundle' promisees are logged for all the bundle promises
- output from 'reports' promises has nothing prefixed except 'R: '
- a log line with stack path is generated when the promise type evaluated changes
- LMDB (symas.com/mdb) is the default database for local data storage : use version 0.9.9 or later
cf-agent --self-diagnostics (-x) is only implemented for TCDB, not for LMDB
- port argument in readtcp() and selectservers() may be a
service name (e.g. "http", "pop3").
- Enable source file in agent copy_from promises to be a relative path.
- file "changes" reporting now reports with log level "notice", instead of "error".
- process_results default to AND'ing of set attributes if not specified (Redmine #3224)
- interface is now canonified in sys.hardware_mac[interface] to align with
sys.ipv4[interface] (Redmine #3418)
- cf-promises no longer errors on missing bodies when run without --full-check (-c)
- Linux flavor "SUSE" now correctly spelled with all uppercase in variables and class names
(Redmine #3734). The "suse" lowercase version is also provided for convenience (Redmine #5417).
- $(this.promise_filename) and $(..._dirname) variables are now absolute paths. (Redmine #3839)
- including the same file multiple times in 'body control inputs' is not an error
- portnumber in body copy_from now supports service names like
"cfengine", "pop3" etc, check /etc/services for more.
- The failsafe.cf policy, run on bootstrap and in some other
unusual cases, has been extracted from C code into libpromises/failsafe.cf
- masterfiles
- cf_promises_validated is now in JSON format
- timestamp key is timestamp (sec since unix epoch) of last time validated
- the masterfiles now come from https://github.com/cfengine/masterfiles and are
not in the core repository
- cf-serverd calls cf-agent with -Dcfruncommand when executing cf-runagent requests
- Mark as removed: promise_notkept_log_include, promise_notkept_log_exclude, promise_repaired_log_include,
promise_repaired_log_exclude, classes_include, classes_exclude, variables_include,
variables_exclude attributes from report_data_select body (syntax is valid but not functional).
They have been replaced by the following attributes: promise_handle_include,
promise_handle_exclude, metatags_include, metatags_exclude.
New features:
- New promise type "users" for managing local user accounts.
- TLS authentication and fully encrypted network protocol.
Additions specific to the new type of connections:
- New attribute "allowlegacyconnects" in body server control,
which enables serving policy via non-latest cfengine protocol,
to the given list of hosts. If the option is absent, it
defaults to allow all hosts. To refuse non-TLS connections,
specify an empty list.
- New attribute "protocol_version" in body copy_from, and body
common control, which defines the preferred protocol for
outgoing connections.. Allowed values at the moment: "0" or
"undefined", "classic" or "1", "latest" or "2". By leaving the
copy_from option as undefined the common control option is
used, and if both are undefined then classic protocol is used
by default.
- The new networking protocol uses TLS for authentication,
after which all dialog is encrypted within the established
TLS session. cf-serverd is still able to speak the legacy
protocol with old agents.
- The 'skipverify' option in 'body server control' is
deprecated and only left for compatibility; it does
nothing
- cf-serverd does not hang up the connection if some request
fails, so that the client can add more requests.
- For the connections using the new protocol, all of the
paths in bundle server access_rules now differentiate
between a directory and a file using the trailing
slash. If the path exists then this is auto-detected and
trailing slash appended automatically. You have to append
a trailing slash manually to an inexistent or symbolic
path (e.g. "/path/to/$(connection.ip)/") to force
recursive access.
- New in 'access' promises for 'bundle server access_rules'
- Attributes "admit_ips", "admit_hostnames", "admit_keys",
"deny_ips", "deny_hostnames", "deny_keys"
- "admit_keys" and "deny_keys" add the new functionality
of controlling access according to host identity,
regardless of the connecting IP.
- For these new attributes, regular expressions
are not allowed, only CIDR notation for "admit/deny_ips", exact
"SHA=..." strings for "admit/deny_keys", and exact hostnames
(e.g. "cfengine.com") or subdomains (starting with dot,
e.g. ".cfengine.com") for "admit/deny"_hostnames. Same rules
apply to 'deny_*' attributes.
- These new constraints and the paths in access_rules, can contain
special variables "$(connection.ip)", "$(connection.hostname)",
"$(connection.key)", which are expanded dynamically for every
received connection.
- For connections using the new protocol, "admit" and "deny"
constraints in bundle server access_rules are being phased
out, preferred attributes are now "admit_ips", "deny_ips",
"admit_hostnames", "deny_hostnames", "admit_keys",
"deny_keys".
- New "shortcut" attribute in bundle server access_rules used to
dynamically expand non-absolute request paths.
- masterfiles
- standard library split: lib/3.5 (compatibility) and lib/3.6 (mainline)
- many standard library bundles and bodies, especially packages- and file-related,
were revised and fixed
- supports both Community and Enterprise
- new 'inventory/' structure to provide OS, dmidecode, LSB, etc. system inventory
(configured mainly in def.cf)
- cf_promises_release_id contains the policy release ID which is the GIT HEAD SHA
if available or hash of tree
- a bunch'o'bundles to make starting with CFEngine easier:
- file-related: file_mustache, file_mustache_jsonstring, file_tidy, dir_sync, file_copy,
file_link, file_hardlink, file_empty, file_make
- packages-related: package_absent, package_present, package_latest,
package_specific_present, package_specific_absent, package_specific_latest, package_specific
- XML-related: xml_insert_tree_nopath, xml_insert_tree, xml_set_value, xml_set_attribute
- VCS-related: git_init, git_add, git_checkout, git_checkout_new_branch,
git_clean, git_stash, git_stash_and_clean, git_commit, git
- process-related: process_kill
- other: cmerge, url_ping, logrotate, prunedir
- New command line options for agent binaries
- New options to cf-promises
- '--show-classes' and '--show-vars'
- '--eval-functions' controls whether cf-promises should evaluate functions
- Colorized output for agent binaries with command line option '--color'
(auto-enabled if you set CFENGINE_COLOR=1)
- New language features
- New variable type 'data' for handling of structured data (ie JSON),
including supporting functions:
- 'data_readstringarray' - read a delimited file into a data map
- 'data_readstringarrayidx' - read a delimited file into a data array
- 'datastate' - create a data variable with currently set classes and variables
- 'datatype' - determine the type of the top element of a container
- 'format' - %S can be used to serialize 'data' containers into a string
- 'mergedata' - merge two data containers, slists/ilists/rlists, or "classic"
arrays into a data container
- 'parsejson' - create a data container from a JSON string
- 'readjson' - create a data container from a file that contains JSON
- 'storejson' - serialize a data container into a string
- Most functions operating on lists can also operate on data containers
- pass a data container to a bundle with the @(container) notation
- the module protocol accepts JSON for data containers with the '%' sigil
- Tagging of classes and variables allows annotating of language construct with
meta data; supporting functionality:
- The module protocol in 'commands' promises has been extended to allow setting
of tags of created variables and classes, and the context of created variables
- 'getclassmetatags' - returns list of meta tags for a class
- 'getvariablemetatags' - returns list of meta tags for a variable
- 'body file control' has an 'inputs' attribute to include library files and other
dependencies
- bundlesequences can be built with bundlesmatching() based on bundle name and tags
- New attributes in existing promise types and bodies
- New option 'preserve_all_lines' for insert_type in insert_lines promises
- Caching of expensive system functions to avoid multiple executions of
execresult() etc, can be controlled via cache_system_functions attribute in
body common control
- New option 'mailsubject' in body executor control allows defining the subject
in emails sent by CFEngine
- Support for Mustache templates in 'files' promises; use 'template_method' and
'template_data' attributes. Without 'template_data' specified, uses datastate().
- New and improved functions
- 'bundlesmatching' - returns list of defined bundles matching a regex and tags
- 'canonifyuniquely' - converts a string into a unique, legal class name
- 'classesmatching' - returns list of set classes matching a regex and tags
- 'eval' - evaluates mathematical expressions; knows SI k, m, g quantifiers, e.g. "100k"
- 'findfiles' - list files matching a search pattern; use "**" for recursive searches
- 'makerule' - evaluates whether a target file needs to be rebuilt from sources
- 'max', 'min' - returns maximum and minimum of the numbers in a container or list
(sorted by a 'sort' method)
- 'mean' - returns the mean of the numbers in a container or list
- 'nth' - learned to look up by key in a data container holding a map
- 'packagesmatching' - returns a filtered list of installed packages.
- 'readfile' - learned to read system files of unknown size like those in /proc
- 'sort' - can sort lexicographically, numerically (int or real), by IP, or by MAC
- 'string_downcase', 'string_upcase' - returns the lower-/upper-case version of a
string
- 'string_head', 'string_tail' - returns the beginning/end of a string
- 'string_length' - returns the length of a string
- 'string_reverse' - reverses a string
- 'string_split' - improved implementation, deprecates 'splitstring'
- 'variablesmatching' - returns a list of variables matching a regex and tags
- 'variance' - returns the variance of numbers in a list or container
- New hard classes
- Introduced alias 'policy_server' for context 'am_policy_hub' (the latter will
be deprecated)
- all the time-based classes have GMT equivalents
- New variables
- 'sys.bindir' - the location of the CFEngine binaries
- 'sys.failsafe_policy_path' - the location of the failsafe policy file
- 'sys.inputdir' - the directory where CFEngine searches for policy files
- 'sys.key_digest' - the digest of the host's cryptographic key
- 'sys.libdir', 'sys.local_libdir' - the location of the CFEngine libraries
- 'sys.logdir' - the directory where the CFEngine log files are saved
- 'sys.masterdir' - the location of masterfiles on the policy server
- 'sys.piddir' - the directory where the daemon pid files are saved
- 'sys.sysday' - the number of days since the beginning of the UNIX epoch
- 'sys.systime' - the number of seconds since the beginning of the UNIX epoch
- 'sys.update_policy_path' - the name of the update policy file
- 'sys.uptime' - the number of minutes the host has been online
- 'this.promise_dirname' - the name of the file in which the current promise
is defined
- 'this.promiser_uid' - the ID of the user running cf-agent
- 'this.promiser_gid' - the group ID of the user running cf-agent
- 'this.promiser_ppid' - the ID of the parent process running cf-agent
Deprecations:
- 'splitstring' - deprecated by 'string_split'
- 'track_value'
- 'skipverify'
Bug fixes: for a complete list of fixed bugs, see Redmine at https://cfengine.com/dev
- various fixes in evaluation and variable resolution
- Improve performance of list iteration (Redmine #1875)
- Removed limitation of input length to internal buffer sizes
- directories ending with "/" are not ignored
- lsdir() always return a list now, never a scalar
- 'abortclasses' fixed to work in common bundles and other cases
- namespaced 'edit_line' bundles now work (Redmine#3781)
- lists are interpolated in correct order (Redmine#3122)
- cf-serverd reloads policies properly when they change
- lots of leaks (memory and file descriptor) fixed
3.5.3
Changes:
- Improved security checks of symlink ownership. A symlink created by a user pointing
to resources owned by a different user will no longer be followed.
- Changed the way package versions are compared in package promises. (Redmine #3314)
In previous versions the comparison was inconsistent. This has been fixed, but may
also lead to behavior changes in certain cases. In CFEngine 3.5.3, the comparison
works as follows:
<package-being-considered> <package_select> <package_version>
For instance:
apache-2.2.31 ">=" "2.2.0"
will result in the package being installed.
Bug fixes:
- fix cf-monitord crash due to incorrect array initialization (Redmine #3180)
- fix cf-serverd stat()'ing the file tree every second (Redmine #3479)
- correctly populate sys.hardware_addresses variable (Redmine #2936)
- add support for Debian's GNU/kfreebsd to build system (Redmine #3500)
- fix possible stack corruption in guest_environments promises (Redmine #3552)
- work-around hostname trunctation in HP-UX's uname (Redmine #3517)
- fix body copy purging of empty directories (Redmine #3429)
- make discovery and loading of avahi libraries more robust
- compile and packaging fixes for HP-UX, AIX and Solaris
- fix fatal error in lsdir() when directory doesn't exist (Redmine #3273)
- fix epoch calculation for stime inrange calculation (Redmine #2921)
3.5.2
Bug fixes:
- fix delayed abortclasses checking (Redmine #2316, #3114, #3003)
- fix maplist arguments bug (Redmine #3256)
- fix segfaults in cf-pomises (Redmine #3173, 3194)
- fix build on Solaris 10/SmartOS (Redmine #3097)
- sanitize characters from /etc/issue in sys.flavor for Debian (Redmine #2988)
- Fix segfault when dealing with files or data > 4K (Redmine #2912, 2698)
- Don't truncate keys to 126 characters in getindices (Redmine #2626)
- files created via log_* actions now have mode 600 (Redmine #1578)
- fix wrong log message when a promise is ignored due to 'ifvarclass' not matching
- fix lifetime of persistent classes (Redmine #3259)
- fix segfault when process_select body had no process_result attribute
Default to AND'ed expression of all specified attributes (Redmine #3224)
- include system message in output when acl promises fail
- fix invocation of standard_services bundle and corresponding promise compliance (Redmine #2869)
3.5.1
Changes:
- file changes are logged with log level Notice, not Error
- the CFEngine Standard Library in masterfiles/libraries is now split into
promise-type specific policy files, and lives in a version-specific directory.
This should have no impact on current code, but allows more granular include of
needed stdlib elements (Redmine #3044)
Bug fixes:
- fix recursive copying of files (Redmine #2965)
- respect classes in templates (Redmine ##2928)
- fix timestamps on Windows (Redmine #2933)
- fix non-root cf-agent flooding syslog (Redmine #2980)
- fix email flood from cf-execd due to timestamps in agent output (Redmine #3011)
- Preserve security context when editing or copying local files (Redmine #2728)
- fix path for sys.crontab on redhat systems (Redmine #2553)
- prevent incorrect "insert_lines promise uses the same select_line_matching anchor" warning (Redmine #2778)
- Fix regression of setting VIPADDRESS to 127.0.0.1 (Redmine #3010)
- Fix "changes" promise not receiving status when file is missing (Redmine #2820)
- Fix symlinks being destroyed when editing them (Redmine #2363)
- Fix missing "promise kept" status for the last line in a file (Redmine #2943)
3.5.0
New features:
- classes promises now take an optional scope constraint.
- new built-in functions: every, none, some, nth, sublist, uniq, filter
- every
- none
- some
- nth
- sublist
- uniq
- filter
- classesmatching
- strftime
- filestat
- ifelse
- maparray
- format
- cf-promises flag --parse-tree is replaced by --policy-output-format=, requiring the
user to specify the output format (none, cf, json)
- cf-promises allows partial check of policy (without body common control) without integrity check;
--full-check enforces integrity check
- agent binaries support JSON input format (.json file as generated by cf-promises)
- cf-key: new options --trust-key/-t and --print-digest/-p
- Class "failsafe_fallback" is defined in failsafe.cf when main policy contains errors and
failsafe is run because of this
- add scope attribute for body classes (Redmine #2013)
- Better diagnostics of parsing errors
- Error messages from parser now show the context of error
- new cf-agent option: --self-diagnostics
- new output format, and --legacy-output
- warnings for cf-promises.
- Enable zeroconf-discovery of policy hubs for automatic bootstrapping
if Avahi is present
- Support for sys.cpus on more platforms than Linux & HPUX
Changes:
- parser no longer allows ',' after promiser or promisee. must be either ';' or lval
- Make parser output in GCC compatible format the only supported format
(remove --gcc-brief-format flag)
- Silence license warnings in Enterprise Free25 installations
- action_policy => "warn" causes not_kept classes to be set on promise needing repair.
- command line option version (-V) now prints a shorter parsable version without graphic
- implicit execution of server and common bundles taking arguments is skipped in cf-serverd.
- WARNING: option --policy-server removed, require option to --bootstrap instead
- process promises don't log if processes are out of range unless you
run in verbose mode
- reports promises are now allowed in any context (Redmine #2005)
- cf-report has been removed
- cf-execd: --once implies --no-fork
- Version info removed from mail subject in the emails sent by cf-execd.
The subject will only contain "[fqname/ipaddress]" instead of "communnity/nova [fqname/ipaddress]"
Please change your email filters accordingly if necessary.
- "outputs" promise type is retired. Their semantics was not clear, and the functionality
is better suited for control body setting, not a promise.
- Tokyo Cabinet databases are now automatically checked for
correctness during opening. It should prevent a number of issues
with corrupted TC databases causing binaries to hang.
- Improved ACL handling on Windows, which led to some syntax changes. We now consistently
use the term "default" to describe ACLs that can be inherited by child objects. These
keywords have received new names:
acl_directory_inherit -> acl_default
specify_inherit_aces -> specify_default_aces
The old keywords are deprecated, but still valid. In addition, a new keyword
"acl_inherit" controls inheritance behavior on Windows. This feature does not exist on
Unix platforms. (Redmine #1832)
- Networking code is moved from libpromises to its own library,
libcfnet. Work has begun on making the API more sane and thread-safe.
Lots of legacy code was removed.
- Add getaddrinfo() replacement in libcompat (borrowed from PostgreSQL).
- Replace old deprecated and non thread-safe resolver calls with
getaddrinfo() and getnameinfo().
- Hostname2IPString(), IPString2Hostname() are now thread-safe, and are
returning error when resolution fails.
- Running cf-execd --once now implies --no-fork, and also does not wait
for splaytime to pass.
- execresult(), returnszero() and commands promises no longer requires the first word
word to be an absolute path when using the shell. (Part of Redmine #2143)
- commands promises useshell attribute now accepts "noshell" and "useshell" values. Boolean
values are accepted but deprecated. (Part of Redmine #2143)
- returnszero() now correctly sets the class name in this scenario (Part of
Redmine #2143):
classes:
"commandfailed" not => returnszero("/bin/nosuchcommand", "noshell");
Bugfixes:
- bundles are allowed to be empty (Redmine #2411)
- Fixed '.' and '-' not being accepted by a commands module. (Redmine #2384)
- Correct parsing of list variables by a command module. (Redmine #2239)
- Fixed issue with package management and warn. (Redmine #1831)
- Fixed JSON crash. (Redmine #2151)
- Improved error checking when using fgets(). (Redmine #2451)
- Fixed error message when deleting nonexistent files. (Redmine #2448)
- Honor warn-only when purging from local directory. (Redmine #2162)
- Make sure "restart" and "reload" are recognized keywords in packages. (Redmine #2468)
- Allocate memory dynamically to avoid out-of-buffer or out-of-hash
situations
- fix edit_xml update of existing attributes (Redmine #2034)
- use failsafe policy from compile-time specified workdir (Redmine #1991)
- ifvarclass checked from classes promises in common bundles
- do not wait for splaytime when executing only once
- disable xml editing functionality when libxml2 doesn't provide necessary APIs (Redmine #1937)
- Out-of-tree builds should work again, fixed a bunch of related bugs.
- Fixed race condition in file editing. (Redmine #2545)
- Fixed memory leak in cf-serverd and others (Redmine #1758)
3.4.5 (Bugfix and Stability release)
Bugfixes:
- make qualified arrays expand correcty (Redmine #1998, Mantis #1128)
- correct possible errors in tcdb files when opening
- avoid possible db corruption when mixing read/write and cursor operations
- Allow umask value of 002 (Redmine #2496)
3.4.4 (Bugfix and Stability release)
Bugfixes:
- prevent possible crash when archiving files (GitHub #316)
- don't create symlinks to cf-know in update policy
- don't enable xml support if libxml2 is too old (Redmine #1937)
3.4.3 (Bugfix and Stability release)
Bugfixes:
- Don't flood error messages when processes are out of defined range
- prevent segmentation fault in cf-monitord -x (Redmine #2021)
- when copying files, use same file mode as source file, rather than 0600 (Redmine #1804)
- include xpath in messages generated by edit_xml operations (Redmine #2057)
3.4.2 (Bugfix and Stability release)
Bugfixes:
- Fixes to policies in masterfiles (see masterfiles/Changelog for details)
- Fixes for OpenBSD (GitHub #278)
- Do not canonify values specified in abortbundleclasses/abortclasses (Redmine #1786)
- Fix build issues on NetBSD, SLES 12.2
- Improve error message when libxml2 support is not compiled (Redmine #1799)
- fix potential segmentation fault when trimming network socket data (GitHub #233)
- fix potential segmentation fault when address-lookups in lastseen db failed (GitHub #233)
- execute background promise serially when max_children was reached, rather
than skipping them (GitHub #233)
- fix segmentation fault in cf-promises when invoked with --reports (Redmine #1931)
- fix compilation with Sun Studio 12 (Redmine #1901)
- silence type-pun warning when building on HP-UX (GitHub #287)
3.4.1 (Bugfix and Stability release)
New feature/behavior:
- cf-execd terminates agent processes that are not responsive
for a configurable amount of time (see agent_expireafter in body
executor control), defaulting to 1 week
Bugfixes:
- fix regression of classmatch() failing with hard classes (Redmine #1834)
- create promise-defined and persistent classes in correct
namespace (Redmine #1836)
- several fixes to namespace support
- fix several crash bugs caused by buffer overflow and race
conditions in cf-serverd
- regenerate time classes in cf-execd for each run (Redmine #1838)
- edit_xml: fix select_xpath implementation and update documentation
NOTE: code that uses select_xpath_region needs to be changed to
select_xpath
- edit_xml: make sure that text-modification functions don't overwrite
child nodes
- edit_xml: improve error logging
3.4.0
New features:
- Added rpmvercmp utility to compare versions of RPM packages for
accurate sorting of RPM packages for packages promises.
- Implement network timeout on server side to avoid keeping stale
connections for hours.
- XML editing capabilities. See the documentation for edit_xml
body. Note the new dependency: libxml2.
- Implement inheritance of local classes by bundles called using
"usebundle". By default classes are not inherited. See the
examples/unit_inherit.cf for an example.
- Moved from Nova/Enterprise:
- POSIX ACL support,
- "outputs" promise type,
- remote syslog support.
- packages_default_arch_command hook in packages promises, to
specify default architecture of the packages on the system.
- packages_version_less_command / packages_version_equal_command hooks
in packages promises, to specify external command for native package
manager versions comparison
- agent_expireafter in body executor control allows you to set a
timeout on all cf-agent runs, to enforce a threshold on the
number of concurrent agents
- Running in Solaris zone is now detected and classes "zone" and
"zone_<name>" are created in this case.
- VirtualBox support added to guest_environment promises.
- guest_environment promises are supported under OS X.
- The "depends_on" attribute is now active, for the partal ordering
of promises. If a promise depends on another (referred by handle)
it will only be considered if the depends_on list is either kept
or repaired already.
** WARNING: When upgrading, make sure that any existing use
of depends_on does not make some promises being
unintentionally ignored. This can happen if you are
currently referring to non-existent or never-run handles
in depends_on attributes.
- methods return values, initial implementation
- New format for cf-key -s, includes timestamp of last connection
- cf-promises --parse-tree option to parse policy file and dump it
in JSON format
- Namespaces support for bundles and bodies. See the
examples/unit_namespace*.cf for the usage.
- Default arguments for bundles. See the examples/unit_defaults.cf
- Metadata promise type. See the examples/unit_meta.cf
New semantics:
- Methods promises now return the status of promises
kept within them. If any promise was not kept, the method is not
kept, else if any promise is repaired, the method was repaired
else it was kept.
- Remote variable access in namespaces by $(namespace:bundle.variable)
Changed functionality:
- cf-execd -F switch no longer implies 'run once'. New -O/--once
option is added to achieve this behaviour. This makes cf-execd
easier to run from systemd, launchd and other supervision
systems.
Misc:
- Support for the following outdated platforms and corresponding
classes has been removed. De facto those platforms were
unsupported for a long time, as CFEngine codebase uses C99
language features unavailable on old platforms:
- SunOS 3.x (sun3)
- SunOS 4.x (sun4)
- Ultrix (ultrix)
- DEC OSF/1 AXP (osf)
- Digital UNIX (digital)
- Sony NEWS (newsos)
- 4.3BSD (bsd4_3)
- IRIX (irix, irix4, irix64)
- IBM Academic Operating System (aos)
- BSD/OS / BSDi / BSD/386 (bsdos)
- NeXTSTEP (nextstep)
- GNU Hurd (gnu)
- NEC UX/4800 (ux4800)
- (Old news) Since 3.3.0 the layout of CFEngine Community packages
has changed slightly.
cf-* binaries have been moved to /var/cfengine/bin, due to the
following reasons:
- cf-* binaries are linked to libraries installed to
/var/cfengine/lib, so placing binaries in /usr/local/sbin does not
increase reliability of the CFEngine,
- keeping whole CFEngine under single prefix (/var/cfengine)
makes packaging simpler,
- it matches the layout of CFEngine Enterprise packages.
Please adjust your policies (the recommended ways to deal with
the move are either to adjust $PATH to include /var/cfengine or to
create symlinks in /usr/local/sbin in case you are relying on
binaries to be available in $PATH).
- Workdir location is properly changed if --prefix or --enable-fhs
options are supplied to configure (Mantis #1195).
- Added check for broken libmysqlclient implementations (Mantis #1217).
- Standard library is updated from COPBL repository.
- cf-know is no longer built in Community releases. The only
functionality useful in Community, namely the reference manual
generation, is provided by new compile-time cf-gendoc tool.
- Filename (for storing filechanges) changed
from file_change.log -> file_changes.log (in /var/cfengine/state)
New format for storing file changes introduced:
[timestamp,filename,<N/C/S/R>,Message]
N = New file found
C = Content Changed
S = Stats changed
R = File removed
- Acceptance test suite passes on Mac OS X.
- Changed some port numbers to replace old services with imap(s)
- archlinux hard class on Arch Linux.
- Detect BSD Make and automatically switch to GNU Make during build.
Bugfixes:
- cfruncommand for cf-execd is an arbitrary shell command now (Mantis #1268).
- Fixed broken "daily" splayclasses (Mantis #1307).
- Allow filenames up to 4096 bytes in network transfers (Redmine #1199).
- Fix stale state preserved during cf-serverd reload (Redmine #1487).
- Free disk space calculation is fixed (Mantis #1120).
- Numerous portability bugfixes (especially OpenBSD, Solaris, AIX-related).
- Compatibility fixes for AIX, HP-UX, Solaris (Mantis #1185, Mantis #1177, Mantis #1109).