A project template for DataOps CI/CD on GitHub.
To test, first use the Open in VS Code
link above to open the project in VS Code using Docker containers.
Note: The only prerequisites to get started is to install VS Code and Docker. As needed, VS Code will prompt you to install any additional extensions needed to open the project.
Once you've opened the project, test that meltano is running executing the following in a new VS Code terminal. These commands add a sample tap and target and then run a simple pipeline test:
meltano add extractor tap-carbon-intensity
meltano add loader target-jsonl
meltano elt tap-carbon-intensity target-jsonl
Debugging:
- If you get a message that the plugin is not yet installed, you might need to wait until all installs have completed. These should automatically run in a new terminal the first time you open the repo.
- If the plugins are still not available and you don't see an open terminal with the install progress, you
can create a new terminal and manually run
meltano install
to install any missing plugins.
meltano ui start
After running the above command, you should see an option from VS Code to "Open in Browser". If not, you can manually navigate to http://localhost:5000.