Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Mention --from-ref in the custom extractor guide #8378

Merged
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions docs/docs/tutorials/custom-extractor.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,39 @@ Installing extractor 'tap-jsonplaceholder'...
Installed extractor 'tap-jsonplaceholder'
```

Alternatively, you can create a [plugin definition](/concepts/project#custom-plugin-definitions) YAML file locally and add to your project using the `--from-ref` option:

<details>
<summary><code>tap-jsonplaceholder.yml</code></summary>

```yml
name: tap-jsonplaceholder
namespace: tap_jsonplaceholder
pip_url: -e ../tap-jsonplaceholder
variant: test
```

</details>

```bash
meltano add --from-ref tap-jsonplaceholder.yml extractor tap-jsonplaceholder
```

:::info
<p>The plugin name must be present in the YAML file to constitute a valid plugin definition - supplying it as a command argument is a no-op in this case</p>
:::

As you develop your custom extractor, it is possible that its settings will change:
- New functionality is added, requiring new setting(s)
- Existing functionality is modified, requiring setting(s) to be renamed
- Existing functionality is removed, requiring setting(s) to be removed

In this case, you will need to update the extractor in your project - by maintaining your plugin definiton YAML file in line with changes to the tap as you go, this is a simple process of running the previous command along with the `--update` flag:

```bash
meltano add --update --from-ref tap-jsonplaceholder.yml extractor tap-jsonplaceholder
```

### Add a JSONL target

Run the command below to add the JSONL loader that will contain the extracted data stream:
Expand Down