v3.3.0

v3.3.0 (2024-01-23)

Highlights

• Some workflows should see increased performance thanks to #8343. Thanks to @TyShkan for reporting, investigating and sharing the profiling results!
• Warnings are now logged whenever a selected stream or property is not present in the tap's catalog.

✨ New

• #8351 Warn if select rules are not used on catalog -- Thanks @haleemur!
• #8176 Add new --update flag to meltano add to re-add plugins to a project -- Thanks @ReubenFrankel!
• #7874 Add setting sensitive flag -- Thanks @ReubenFrankel!

🐛 Fixes

• #8365 Retain stderr in meltano config <plugin> test -- Thanks @ReubenFrankel!
• #8354 Emit pip installation errors

⚡ Performance Improvements

• #8343 Cache Project.dotenv_env

📚 Documentation Improvements

• #8306 Add changelog updates

v3.3.0rc1

First release candidate for Meltano v3.3.0

v3.3.0a2

Full Changelog: v3.3.0a1...v3.3.0a2

v3.3.0a1

Full Changelog: v3.2.0...v3.3.0a1

v3.2.0

v3.2.0 (2023-12-05)

Highlights

• The new --from-file option lets you pass setting values to meltano config from files and STDIN, thanks to @XshubhamX!

• The new option --merge-state lets you run partial full-table pipelines, for example when you want to refresh a single stream without losing the state of other streams.

• Environment variables are now expanded in array settings, for example MY_BUCKET will now expanded in the example below:

  plugins:
    extractors:
    - name: tap-spreadsheets-anywhere
      variant: ets
      pip_url: git+https://github.com/ets/tap-spreadsheets-anywhere.git smart_open[s3]
      config:
        tables:
        - name: customers
          path: s3://${MY_BUCKET}
          pattern: files/customers\.csv\.*gz
          start_date: '2017-05-01T00:00:00Z'
          key_properties: [id]

✨ New

• #8184 Log Meltano version and Operating System at invocation of CLI -- Thanks @ashu565!
• #8228 Support setting configuration from files and STDIN -- Thanks @XshubhamX!
• #8194 Allow escaping stream names with periods in select rules
• #8241 Support Python 3.12
• #8215 Support authenticating to Azure state backend without a connection string -- Thanks @XshubhamX!
• #8258 Add a --merge-state flag to meltano run to merge the current pipeline state with that of the latest run
• #8268 Expand environment variables in array setting values

🐛 Fixes

• #8249 Display special characters in column names in the output of meltano select --list
• #8227 Validate input to settings with kind: options
• #8232 Interval validation for meltano schedule set -- Thanks @XshubhamX!
• #8225 Redirect tap's stderr to /dev/null during a configuration test -- Thanks @raiatul14!
• #8301 Interactive config now indicates that you're typing a redacted value -- Thanks @abastola0!

📚 Documentation Improvements

• #8267 Add Q4 Changelog
• #8260 Add information about database schema destination -- Thanks @EChaffraix!
• #8259 Fix links in v3 migration guide
• #8251 Fix dbt model in part3.mdx -- Thanks @diegoquintanav!
• #8214 Document other supported ways of configuring an S3 endpoint URL
• #8211 Sync sidebar order with index page link-lists, remove broken /index links -- Thanks @mjsqu!
• #8209 Remove broken link (reference/index) and order links to match sidebar -- Thanks @mjsqu!
• #8210 Add tabs for all remaining env/cli/config settings -- Thanks @mjsqu!
• #8201 Use tabs for examples on settings page -- Thanks @mjsqu!
• #8200 Update contribution guidelines -- Thanks @mjsqu!
• #8155 Add september changelog

Full Changelog: v3.1.0...v3.2.0

v3.2.0rc1

v3.2.0rc1 (2023-12-05)

✨ New

• #8258 Add a --merge-state flag to meltano run to merge the current pipeline state with that of the latest run
• #8268 Expand environment variables in array setting values

🐛 Fixes

• #8301 Interactive config now indicates that you're typing a redacted value -- Thanks @abastola0!

v3.2.0b1

v3.2.0b1 (2023-11-21)

✨ New

• #8184 Log Meltano version and Operating System at invocation of CLI -- Thanks @ashu565!
• #8228 Support setting configuration from files and STDIN -- Thanks @XshubhamX!
• #8194 Allow escaping stream names with periods in select rules
• #8241 Support Python 3.12
• #8215 Support authenticating to Azure state backend without a connection string -- Thanks @XshubhamX!

🐛 Fixes

• #8249 Display special characters in column names in the output of meltano select --list
• #8227 Validate input to settings with kind: options
• #8232 Interval validation for meltano schedule set -- Thanks @XshubhamX!
• #8225 Redirect tap's stderr to /dev/null during a configuration test -- Thanks @raiatul14!

📚 Documentation Improvements

• #8267 Add Q4 Changelog
• #8260 Add information about database schema destination -- Thanks @EChaffraix!
• #8259 Fix links in v3 migration guide
• #8251 Fix dbt model in part3.mdx -- Thanks @diegoquintanav!
• #8214 Document other supported ways of configuring an S3 endpoint URL
• #8211 Sync sidebar order with index page link-lists, remove broken /index links -- Thanks @mjsqu!
• #8209 Remove broken link (reference/index) and order links to match sidebar -- Thanks @mjsqu!
• #8210 Add tabs for all remaining env/cli/config settings -- Thanks @mjsqu!
• #8201 Use tabs for examples on settings page -- Thanks @mjsqu!
• #8200 Update contribution guidelines -- Thanks @mjsqu!
• #8155 Add september changelog

v3.1.0

v3.1.0 (2023-09-26)

✨ New

• #8169 Added --force-install CLI flag for meltano add command. -- Thanks @XshubhamX!
• #8145 Save pip logs to a file in the system directory
• #7907 Add plugin from ref -- Thanks @ReubenFrankel!

🐛 Fixes

• #8179 Removed the --verbose option from CLI, which no longer had any effect -- Thanks @arorarohan981!
• #8044 Better error message when Azure connection string is missing
• #8115 Do not perform destructive state ops when answering 'no' in prompt

📚 Documentation Improvements

• #8178 Top-level ordered lists are now rendered correctly in docs
• #8156 Added docs for --cwd CLI option -- Thanks @XshubhamX!
• #8143 Remove Cloud announcement banner
• #8113 Update Cloud "Getting Started" guide to include the lock command
• #8111 Add example of database URI for SQL Server -- Thanks @wesseljt!
• #8087 Add cloud state command

v3.0.0

v3.0.0 (2023-09-05)

BREAKING CHANGES

• #8041 Remove target_schema loader extra
• #8007 Always require plugin lock files
• #7499 Remove API and web UI
• #7656 Remove the meltano discover command

✨ New

• #8020 Add python setting
• #7997 Add el command as an alias of elt and deprecate both elt and the --transform option
• #7992 Add support for mappers in the invoke command
• #7989 Add meltano hub ping command
• #7984 Add meltano config --unsafe flag -- Thanks @ReubenFrankel!
• #7981 Support psycopg[binary] under a new postgres extra
• #7963 Bump SQLAlchemy to 2.0
• #7932 Add "did you mean" CLI command name suggestions
• #7846 Add aliases for the @once schedule interval

🐛 Fixes

• #8042 Ensure migration lock file is closed after read
• #8043 Ensure elt/run log file is closed
• #8031 Redact DB password from logs
• #8015 Display a better error message when database URI is null
• #7982 Use hyphens consistently for CLI options
• #7947 Support PEP 440 direct references in pip_url

⚙️ Under the Hood

• #7964 Replace deprecated locale.getdefaultlocale with locale.getlocale -- Thanks @AmirAflak!
• #7960 Compatibility with SQLAlchemy 2

📚 Documentation Improvements

• #8011 Add JSON Schema information -- Thanks @anden-akkio!
• #7991 Fix cloud install git URL
• #7975 Link mssql additional component to system database explanation
• #7979 Correct repo_ext to ext_repo for plugin definition syntax -- Thanks @ReubenFrankel!
• #7953 Update tagline in readme
• #7951 Update changelog
• #7950 Fix logging example -- Thanks @aminebeh!
• #7948 Update docusaurus.config.js -- Thanks @gridig!
• #7945 Douwe fixes
• #7944 Quick doc fixes
• #7936 Migrate Meltano docs to Docusaurus
• #7902 Remove ELT messaging, move things around
• #7931 Cloud docs ssh key validate tip
• #7930 Meltano cloud to meltano-cloud
• #7926 Fix state backend env vars in examples

v3.0.0rc0

First release candidate for Meltano v3