v1.19.0

Highlights

• http sends forms and uploads files, with form and multipart.
• http keeps cookies across the steps of a job, with keep_cookies, cookies and res.cookies.
• file() and template() read request bodies and expected responses from files.
• trace_header sends a header that names the run, job and step of each request.
• A {{ }} in a map key is now evaluated. This can change a workflow that had one.

Forms and file uploads

The http action can now send a form and upload files. form sends an application/x-www-form-urlencoded body, and multipart sends text fields and files as multipart/form-data:

- name: Upload an avatar
  uses: http
  with:
    post: /api/images
    multipart:
      title: avatar
      image:
        file: ./fixtures/logo.png
      attachments:
        - file: ./fixtures/terms.pdf
        - content: "a,b\n1,2\n"
          filename: data.csv
          content_type: text/csv
  test: res.code == 201

• In multipart, a string, number or boolean is a text field, and a map with file or content is a file. A list sends the field once per value. file paths are relative to the working directory.
• filename defaults to the base name of file, or the field name for content. content_type defaults to the type the extension names, or application/octet-stream.
• Text fields are sent before files, each sorted by name, since some servers (such as S3 for a POST upload) read only the fields before the file. The body is sent with a Content-Length.
• Either option sets the Content-Type header, replacing one from headers or defaults. Giving form, multipart and body in any combination is an error before the request is sent.
• A multipart body is not shown: req.body is empty and req.multipart shows what was sent. A form body is shown in req.body.

See Sending a Form and Uploading Files.

Cookies across steps

Every http request now has a cookie jar, and keep_cookies: true keeps the cookies a server sets for the following steps of the job:

- name: Logged-in pages
  defaults:
    http:
      url: "{{vars.api_url}}"
      keep_cookies: true
  steps:
    - name: Log in
      uses: http
      with:
        post: /login
        form:
          user: "{{vars.user}}"
          password: "{{vars.password}}"
      test: res.code == 200 && res.cookies.session != ""
    - name: My page
      uses: http
      with:
        get: /me
      test: res.code == 200

• A cookie set on a redirect is now sent on to where the redirect leads. It used to be dropped. A cookie header in headers is still sent.
• res.cookies holds the cookies the server set, by name, including those set on redirects.
• Kept cookies are sent according to their domain, path, expiry and Secure flag. A cookie the server removes or that expires is no longer sent. Each job keeps its own cookies, and each run of a repeated job, and a job run by embedded, starts with none.
• cookies sends cookies by name, with or without keep_cookies, in place of a kept cookie of the same name. They go only to the host of the request, not to another host a redirect leads to.
• The values of cookies and res.cookies are hidden in the output. Their names are shown.

See Cookies.

Bodies and expected responses kept in files

Two new expression functions read a file and expand templates in it. file(path) returns a file's content as it is, and template(s) expands the {{ }} templates in a string:

- name: Create a user
  uses: http
  with:
    post: /users
    headers:
      content-type: application/json
    body: "{{ parse_json(template(file('fixtures/user.json.tmpl'))) }}"
  test: match_json(res.body, parse_json(file('expected/user.json')))

• file paths are relative to the working directory. A missing file, a directory or another non-regular file, and a file larger than 1000000 bytes are errors.
• file does not expand templates, so a file that holds {{ as plain text can be sent as is. Pass it through template to expand them, against the same vars, outputs and other fields as the calling expression.
• A template is replaced by its value as text, so a string that may hold a quote is better written with toJSON, as in {"name": {{ toJSON(vars.name) }}}.
• template calls nest at most 10 levels deep, so a file that reads itself fails at once.
• A var that calls template is evaluated after the vars that do not.

See file and template.

Trace header

The http action can send a header that names the run, job and step of each request, so that the request can be found in the server's access log. trace_header: true sends X-Probe-Trace, and a header name sends the same value under that name:

X-Probe-Trace: run=7f3a9c21e4b05d68; job=login; step=auth; repeat=0; attempt=1

• step is the step's id, or step_<index> for a step without one. repeat counts from 0 and attempt from 1. Values are escaped as in a URL query.
• The run ID is also written to the JSON report as run_id. A job run by embedded has the run ID of the step that embeds it.
• The header shows in req.headers. Giving a header of the same name in headers as well is an error.
• The option is off by default. It is not named trace, which stays the shorthand for the TRACE method.

See Tracing Requests.

Templates in map keys

A map key that holds {{ }} is now evaluated as a template, in with, vars and a step's vars, so a key can come from a value known only at run time:

    body:
      type: status
      "{{ outputs.tenant.id }}":
        active: true

• A key is evaluated as text. Two keys that come to the same key are an error, as is a key that cannot be evaluated; the error names the key as written.
• This changes workflows that had {{ in a key: it used to be sent as written.
• A var whose key reads another var is evaluated after it.

See Expressions and Templates.

Output and logging

• The percent-encoded form of a secret or credential is now hidden too, so a password sent in a form body does not show as p%40ss.
• With -v, the http action logs the method, URL and headers of each request and the status and headers of each response, with credentials hidden. It used to log only a warning that these could not be encoded.

For library users

Two narrow breaks, both in what was added since v1.18.0:

• (*Job).RunStandalone takes ...StandaloneOption after its other parameters. Calls compile as before, but the method as a value has a new type, so code that assigns it to the old function type or an interface with the old signature no longer compiles.
• expr.Expr has a function-typed field, BeforeTemplate, so it is no longer comparable with == or usable as a map key.

New public API:

• actionrpc.StepAction lets an action know the step it runs for and keep state across the steps of a job. RunStep(call Call) receives With, State and Step (run ID, job, step index, ID and name, repeat, attempt). actionrpc.RunStep and actionrpc.RunExecutableStep are the entry points, and StepActionRunner and MockActionRunner.Calls / States are the runner side. Action, Run, RunExecutable and ActionRunner are unchanged, and the new step and state fields of the RPC messages are ignored by existing external actions.
• Workflow.RunID() returns the run ID, and WithRunID passes one to RunStandalone.
• http.RequestStep is the http request entry point that takes a Call.

See Knowing the Step and Keeping State and the CHANGELOG.

Changelog

• 1cbc9a6 Bound file reads and evaluate template vars after the others
• a7da753 Bound what template expands and evaluate the vars it reads first
• e587abb Escape line breaks in multipart names and keep the caller's headers
• 8109f18 Evaluate templates in the keys of maps
• d1a367d Hide errors under keys that become credentials, count every key
• bf477c7 Keep action state in the form it takes when it is sent
• ac03aa3 Keep cookies as they came and state as it would be sent
• fb80dbe Keep cookies in a job through state that actions keep
• 9120ac0 Key a list var by a name of its own and describe EvalTemplateMap
• 484c320 Log the http request and response with credentials hidden
• 17d322f Merge pull request #317 from linyows/feat/http-multipart-form
• cefe878 Merge pull request #318 from linyows/feat/action-state-cookies
• 1a6cc0d Merge pull request #319 from linyows/feat/file-template-functions
• b88e83b Merge pull request #320 from linyows/feat/trace-header
• 251756a Merge pull request #321 from linyows/feat/template-map-keys
• ce14bac Merge pull request #322 from linyows/release/v1.19.0
• db7e92b Move the state tests next to the tests of the code they test
• 080caec Name a step without an id as step_ in the trace header
• 155d094 Name colliding keys as written, hide only template errors
• e64c8e9 Prepare the changelog for v1.19.0
• 15a327b Read files and expand templates in them with file and template
• 2a49bab Run the form and upload example in the end-to-end tests
• c778743 Send forms and file uploads from form and multipart
• b72b207 Stop template at its limit, refuse a FIFO before opening it
• bbffb44 Tell actions about their step and send a trace header from http
• bb41156 Tell domain cookies apart by domain and copy the state kept