remove skeleton-suite files triggering malware warnings by adding to .gitignore; reported by Dave Rice
release built with Go version 11, which includes a fix for a CIFS error that caused files to be skipped during file walk; reported by Maarten Savels
save defaults in a configuration file: use the -setconf flag to record any other flags used into a config file. These defaults will be loaded each time you run sf. E.g. sf -multi 16 -setconf then sf DIR (loads the new multi default)
use -conf filename to save or load from a named config file. E.g. sf -multi 16 -serve :5138 -conf srv.conf -setconf and then sf -conf srv.conf
added -yaml flag so, if you set json/csv in default config :(, you can override with YAML instead. Choose the YAML!
the roy compare -join options that join on filepath now work better when comparing results with mixed windows and unix paths
exported decompress package to give more functionality for users of the golang API; requested by Byron Ruth
update LOC signatures to 2018-06-14
update freedesktop.org signatures to v1.10
update tika-mimetype signatures to v1.18
misidentifications of some files e.g. ODF presentation due to sf quitting early on strong matches. Have adjusted this algorithm to make sf wait longer if there is evidence (e.g. from filename) that the file might be something else. Reported by Jean-Séverin Lair
some matches result in extremely verbose basis fields; reported by Nick Krabbenhoeft. Partly fixed: basis field now reports a single basis for a match but work remains to speed up matching for these cases.
update LOC signatures to 2017-09-28
update PRONOM signatures to v93
version information for MIME-info signatures (freedesktop.org and tika-mimetypes) now recorded in mime-info.json file and presented in results
new sets file for PRONOM extensions. This creates sets like @.doc and @.txt (i.e. all PUIDs with those extensions). Allows you to do commands like roy build -limit @.doc,@.docx, roy inspect @.txt and sf -log @.pdf,o DIR
update freedesktop.org signatures to v1.9
out of memory error when using sf -z on compressed files that contain very large files; reported by Terry Jolliffe
report errors that occur during file decompression. Previously, only fatal errors encountered when a compressed file is first opened were reported. Now errors that are encountered while attempting to walk the contents of a compressed file are also reported.
report errors for 'roy inspect' when roy can't find anything to inspect; reported by Ross Spencer
continue on error flag (-coe) can now be used to continue scans despite fatal file errors that would normally cause scanning to halt. This may be useful e.g. for big directory scans over unreliable networks. Usage: sf -coe DIR
update PRONOM signatures to v92
file scanning is now restricted to regular files (i.e. not symlinks, sockets, devices etc.). Reported by Henk Vanstappen.
windows longpath fix now works for paths that appear short
sf -update flag can now be used to download/update non-PRONOM signatures. Options are "loc", "tika", "freedesktop", "pronom-tika-loc", "deluxe" and "archivematica". To update a non-PRONOM signature, include the signature name as an argument after the flags e.g. sf -update freedesktop. This command will overwrite 'default.sig' (the default signature file that sf loads). You can preserve your default signature file by providing an alternative -sig target e.g. sf -sig notdefault.sig -update loc. If you use one of the signature options as a filename (with or without a .sig extension), you can omit the signature argument i.e. sf -update -sig loc.sig is equivalent to sf -sig loc.sig -update loc. Feature requested by Ross Spencer.
sf -update now does SHA-256 hash verification of updates and communication with the update server is via HTTPS.
update PRONOM signatures to v91
fixes to config package where global variables are polluted with subsquent calls to the Add(Identifier) function
fix to reader package where panic triggered by illegal slice access in some cases
roy build and roy add now take a -nobyte flag to omit byte signatures from the identifier; requested by Nick Krabbenhoeft
update Tika MIMEInfo signatures to 1.16
update LOC to 2017-06-10
no changes since v1.7.3, repairing Travis-CI auto-deploy of Debian packages
sf now accepts multiple files or directories as input e.g. sf myfile1.doc mydir myfile3.txt
LOC signature update
code re-organisation to export reader and writer packages
sf -replay can now take lists of results files with -f flag e.g. sf -replay -f list-of-results.txt
the command sf -replay - now works on Windows as expected e.g. sf myfiles | sf -replay -json -
text matcher not allocating hits to correct identifiers; fixes #101
unescaped YAML field contains quote; reported by Ross Spencer
PRONOM v90 update
the -home flag was being overriden for roy subcommands due to interaction other flags
signature updates for PRONOM, LOC and tika-mimetypes
roy inspect accepts space as well as comma-separated lists of formats e.g. roy inspect fmt/1 fmt/2
log files that match particular formats with -log fmt/1,@set2 (comma separated list of format IDs/format sets). These can be mixed with regular log options e.g. -log unknown,fmt/1,chart
generate a summary view of formats matched during a scan with -log chart (or just -log c)
replay scans from results files with sf -replay: load one or more results files to replay logging or to convert to a different output format e.g. sf -replay -csv results.yaml or sf -replay -log unknown,chart,stdout results1.yaml results2.csv
compare results with roy compare subcommand: view the difference between two or more results e.g. roy compare results1.yaml results2.csv droid.csv ...
roy sets subcommand: roy sets creates pronom-all.json, pronom-families.json, and pronom-types.json sets files;
roy sets -changes creates a pronom-changes.json sets file from a PRONOM release-notes.xml file; roy sets -list @set1,@set2 lists contents of a comma-separated list of format sets
roy inspect releases provides a summary view of a PRONOM release-notes.xml file
the sf - command now scans stdin e.g. cat mypdf.pdf | sf -. You can pass a filename in to supplement the analysis with the -name flag. E.g. cat myfile.pdf | sf -name myfile.pdf -. In previous versions of sf, the dash argument signified treating stdin as a newline separated list of filenames for scanning. Use the new -f flag for this e.g. sf -f myfiles.txt or cat myfiles.txt | sf -f -; change requested by pm64
some files cause endless scanning due to large numbers of signature hits; reported by workflowsguy
null bytes can be written to output due to bad zip filename decoding; reported by Tim Walsh
enable -hash, -z, and -log flags for -serve and -multi modes
new hash, z, and sig params for -serve mode (to control per-request)
enable droid output in -serve mode
GET requests in -serve mode now just percent encoded (with base64 option as a param)
-serve mode landing page now includes example forms
code re-organisation using /internal directory to hide internal packages
Identify method now returns a slice rather than channel of IDs (siegfried pkg change)
graph implicit and missing priorities with roy inspect implicit-priorities and roy inspect missing-priorities
error parsing mimeinfo signatures with double backslashes (e.g. rtf signatures)
new sets files (pronom-families.json and pronom-types) automatically created from PRONOM classficiations. Removed redundant sets (database, audio, etc.).
debbuilder.sh fix: debian packages were copying roy data to wrong directory
roy inspect priorities command now includes "orphan" fmts in graphs
-doubleup flag preventing signature extensions loading: since v1.3.0 signature extensions included with the -extend flag haven't been loading properly due to interaction with the doubles filter (which prevents byte signatures loading for formats that also have container signatures defined)
use fwac rather than wac package for performance
roy inspect FMT command speed up by building without reports and without the doubles filter
-reports flag removed for roy harvest and roy build commands
-reports flag changed for roy inspect command, now a boolean that, if set, will cause the signature(s) to be built from the PRONOM report(s), rather than the DROID XML file. This is slower but can be a more accurate representation.