Skip to content

@snopoke snopoke released this Jul 10, 2019

Upgrade notes
If you are exporting to SQL and using config files with multiple tables (Excel sheets) it is recommended that you do a full re-export of your data to ensure that there is no missing data. You could do this in one of the following ways:

  1. Use the --start-over flag to ignore any existing checkpoints.
  2. Edit the file to change the checkpoint key.
  3. Change the checkpoint key manually using the commcare-export-utils set-checkpoint-key command.

New features
(#123) add sha1 function to allow creating hashes of output values. Mostly useful when exporting nested objects and the jsonpath ID becomes too long for the SQL primary key column.

Improvements / fixes
(#118) Don't set checkpoint to current time. Always use time from batch if available, otherwise don't log a checkpoint.

(#121) Use a separate checkpoint for each table being exported. This impacts any export configurations that have multiple tables configured in one file. Fixes #119

Other Changes

  • improvements to console logging
  • fix deprecation warnings
Assets 2

@snopoke snopoke released this Feb 5, 2019 · 43 commits to master since this release

(#117) upgrade openpyxl

Assets 2
Dec 24, 2018
Release 1.1.0

@nickpell nickpell released this Dec 24, 2018 · 54 commits to master since this release

(#114) For string columns, in MSSQL use NVARCHAR(MAX) and in PostgreSQL use TEXT.

Assets 2

@snopoke snopoke released this Oct 11, 2018 · 68 commits to master since this release

(#108) catch all parsing errors from jsonpath

Assets 2

@snopoke snopoke released this Oct 8, 2018 · 70 commits to master since this release

Maintenance release:

(#106) Correctly handle unicode characters in headings when writing data to CSV

Assets 2

@snopoke snopoke released this Sep 4, 2018 · 75 commits to master since this release

(#85) Improve performance when exporting multiple tables from the same datasource e.g. form + repeats

  • Instead of fetching the same data for each table only fetch the data once and have each table process it directly

(#97) Allow specifying alternate source field names

  • Each source field is considered in order and the first one to appear in the data (even if it has an empty value) will be used.
  • Additional columns may be created for specifying alternate field names. There are two options that may be used:
    • Alternate fields listed as CSV
      • Column title: "Alternate Source Fields"
      • Column contents: A list of comma-separated field names to look for
    • Alternate fields listed in individual columns
      • One column added for each alternate field name
      • Columns named "Alternate Source Field [N]" (N is a number)

(#99) Allow specifying a checkpoint key via the command line to prevent having to re-start export after editing the query file

  • This allows users to make changes to the query file without having to re-export existing data.
  • This change also includes a new tool for interrogating the checkpoint data and for converting non-keyed checkpoints to keyed checkpoints. See commcare-export-utils --help.

(#98) Don't reuse checkpoints when querying different sources
(#105) Handle errors encountered when fetching data and retry with backoff
(#101) Unwrap 'object' values from CommCare HQ if possible
(#104) Prevent usage of --since and --until from affecting checkpoints
(#92) Quote special characters in source field names
(#91) Allow specifying the SQL table name in the query sheet in addition to the sheet name to get around the limitations on sheet name length

Assets 2

@snopoke snopoke released this Jul 12, 2018 · 211 commits to master since this release

  • fix pagination bug for forms with multiple sorting
Assets 2
Jul 11, 2018

@snopoke snopoke released this Jul 2, 2018 · 216 commits to master since this release

  • Fix bug in checkpointing when exporting to SQL
Assets 2
You can’t perform that action at this time.