Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for mappers in the invoke command #7992

Merged
merged 23 commits into from
Aug 10, 2023

Conversation

pnadolny13
Copy link
Contributor

@pnadolny13 pnadolny13 commented Aug 4, 2023

Closes #3232

I'm not sure if theres a better way to get the mapping_name from a plugin object but this properly finds the mapper/mapping from my testing. I was able to run cat github_out.json | meltano invoke hide-github-mails using the tutorial mapper example to hash github emails and everything worked as expected.

UPDATE: it looks like we do a similar lookup for mapping_name in

if plugin.extra_config.get("_mapping_name") == mapping_name:

@netlify
Copy link

netlify bot commented Aug 4, 2023

Deploy Preview for meltano ready!

Name Link
🔨 Latest commit d2f48cd
🔍 Latest deploy log https://app.netlify.com/sites/meltano/deploys/64d521c8b597cd00085a3ea5
😎 Deploy Preview https://deploy-preview-7992--meltano.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@pnadolny13 pnadolny13 marked this pull request as ready for review August 7, 2023 19:54
@pnadolny13
Copy link
Contributor Author

It looks like at this point the only check failing is precommit which is saying the method is too complex. I can quickly refactor that method but it will ballon the diff of this PR a bit. Let me know what you would like me to do.

@codecov
Copy link

codecov bot commented Aug 8, 2023

Codecov Report

Merging #7992 (d2f48cd) into main (a704ec0) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #7992   +/-   ##
=======================================
  Coverage   89.53%   89.53%           
=======================================
  Files         297      297           
  Lines       21897    21903    +6     
  Branches     2441     2443    +2     
=======================================
+ Hits        19605    19611    +6     
  Misses       1936     1936           
  Partials      356      356           
Files Changed Coverage Δ
src/meltano/core/block/parser.py 86.23% <100.00%> (-1.27%) ⬇️
src/meltano/core/project_plugins_service.py 93.00% <100.00%> (+0.48%) ⬆️
tests/meltano/cli/test_run.py 98.36% <100.00%> (-0.01%) ⬇️
tests/meltano/core/test_project_plugins_service.py 100.00% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@edgarrmondragon edgarrmondragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@edgarrmondragon edgarrmondragon changed the title feat: add support for mappers in the invoke command feat: Add support for mappers in the invoke command Aug 9, 2023
@edgarrmondragon
Copy link
Collaborator

@pnadolny13 Should the docs in https://docs.meltano.com/guide/mappers/ be updated? I think at least the FAQ section should, wdyt?

@pnadolny13
Copy link
Contributor Author

@edgarrmondragon good point, I'll update that and see if theres other docs that can be updated too.

@pnadolny13
Copy link
Contributor Author

@edgarrmondragon I updated the docs. One final review then I think this should be good to merge!

docs/docs/guide/mappers.md Outdated Show resolved Hide resolved
@edgarrmondragon
Copy link
Collaborator

:shipit:

@pnadolny13 pnadolny13 merged commit 0d1ec4c into main Aug 10, 2023
43 checks passed
@pnadolny13 pnadolny13 deleted the support_invoking_mappers branch August 10, 2023 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to invoke mappings directly with meltano invoke
2 participants