loquat v2.0.0-RC4

• Made LoquatOps public
• Made other components public too

loquat v2.0.0-RC3

This release is brings a lot of improvements with minimum impact on the usage interface. It drops old and unnecessary dependencies, updates the rest, brings some new functionality and improves internals in preparation for the final release.

• #70: Dropped upickle dependency (it was deprecated)
• #73: Dropped better-files dependency
• #77: Local manager functionality:
  • You can run loquat manager locally at your computer, useful for small tasks and testing:

    loquat.launchLocally(user)

  • You can also run progress monitor (a.k.a terminator) manually:

    loquat.monitorProgress(interval = 3.minutes)

• #79, #81: Updated all dependencies and cross-compiled with Scala 2.11/2.12
• Other minor improvements:
  • Added initial SQS visibility timeout as a config parameter
  • Improved worker logging

loquat v2.0.0-RC2

The version 2.0.0-RC1-10-g9abe18a was in use for quite some time, so this is just a release to fix it as the next release candidate. Mostly it's about some minor improvements and bugfixes to the 2.0.0-RC1 version:

• Updated better-files
• #75: Fixed the problem with empty file check
• #74: Logging-related improvements:
  • Fixed worker fatal failure notification subject
  • Improved worker non-fatal error message
  • Made transfer manager down/uploads silent again
  • Removed logging on each log upload

loquat v2.0.0-RC1

• Updated to Statika v2.0

loquat v2.0.0-M9

• #58: Moved data mappings list from config to loquat (to simplify chaining several loquats in a pipeline)
• #57: Fixed termination bugs related to the SQS polling (replaced it with a faster and better checks)
• #61: Manger now sends input tasks in parallel which makes it orders of magnitude faster
• #66: Updated aws-scala-tools to 0.18.1, which led to a lot of internal improvements related to the usage of AWS
• #72: Updated other dependencies
• #42, #60: Added a check that all input/output data-keys have distinct labels
• #65: Loquat config now uses an S3Folder for logs instead of just a bucket name (renamed to logsS3Prefix)
• #46: Added an email notification in case of manager installation failure
• #21: Added fatal worker failures email notifications

---

See the full list of pull requests merged in this release in the v2.0-M9 milestone.

loquat v2.0.0-M8

• #56: Some improvements to the resource handling and a temporary solution for the "Too many open files" issue.
• #59, #62: Added a configuration option to switch skipping empty output files on upload (on by default)

loquat v2.0.0-M7

• Data mappings:
  • #45: Changed DataMappings remoteInput/Output from records to simple Maps
  • Added an option to the config to switch on/off input objects existence check
  • #44: Renamed DataMapping's id to label
  • #47: Added a logger message and some kind of progress output for the input objects existence check
  • #50: Instead of checking input objects object metadata, listing objects with the given prefix
  • #52: Added a check for missing/non-needed input/output data keys in the datamappings
• Data processing:
  • #51: Fixed input objects handling by passing the map of (dataKey.label -> fileLocation) from worker to the processing context; this way context.inputFile lookup doesn't rely on conventions and return the actual file location
• Self-termination:
  • #48: Added a time deadline for SQS polling
  • Added some logging and exception handling to the polling
• Other fixes:
  • #49: Removed any worker instance state tagging for now; was causing RequestLimitExceeded on a big autoscaling group with quick tasks
  • #43: Changed log messages for downloads/uploads

loquat v2.0.0-M6

Milestone release. See notes for 2.0.0.

• #35 add artifact org to S3 object metadata
• #22 support different AWS regions
• update to datasets 0.3.0
• #28 Strings as input data
• #40 check input S3 object existence
• #17 simplify configuration

loquat v2.0.0-M5

Milestone release. See notes for 2.0.0.

• This milestone updates dependencies to use stable cosas v0.8.0 and aws-scala-tools v0.16.0.

loquat v2.0.0-M4

Milestone release. See notes for 2.0.0.

• This milestone updates dependencies to use aws-scala-tools v0.15.0.