KNOTS is a quick and intuitive visual ETL tool that allows you to do complex data replication with ease. Using the visual interface, you can now bring your data together with the power of Singer taps and targets, without the complexity.
Using KNOTS, you can import data from a number of datastores on an ad-hoc basis or you can download knots and run with a job scheduler of your choice to make sure your data is always up to date. With the intuitive interface, you can configure robust data replication processes in minutes, and KNOTS is always free.
Download and install the latest release. For Windows, you'll want to use the
.exe installer. For Mac, you'll want the
Please note that for Windows, you'll receive a popup that states "Windows protected your PC". Click on
More info, followed by
Run anyway. This is an issue that's actively being worked on and won't be around for much longer.
KNOTS depends on Docker being installed and running. Docker is a tool designed to make it easier to create, deploy, and run containers. Containers allow us to package up an application or library with all of its dependencies. Each of the individual taps and targets are packaged into containers as they may have widely-different sets of dependencies.
The installers and instructions for Docker are available here:
Windows: The first time that KNOTS attempts to set up a tap, Docker will ask the user to share the
C drive. Once you accept, you may be asked for your password.
Mac: Check Docker file sharing preferences and
make sure that
/Users is a shared directory.
Running the app
- From the home screen, click on
Get Started, or
New knoton the upper right-hand corner.
- Select a tap to use from the list of available taps.
- Provide the configuration values required by the tap.
- Click on
Continueto run the tap in Discovery mode.
- Select the tables/streams that you would like to sync.
- Like with the tap, select a target from the list and provide its configuration information.
- Enter a name for the new knot, and click on
Save & Runto execute it.
Once the process has finished, click on
Done to return to the home screen. You should now see a list of your
saved knots and the various actions (
Sync new data,
Sync all data,
Delete) that can
be taken on them.
Run with a scheduler
As of today, KNOTS allows you to update the data by manually clicking on the
Sync new data action. By making use
Export action though, and the resources that it provides, it's possible to setup a job that will update
the data automatically on a schedule.
The exported package is a ZIP file that includes the tap and target for your knot, as well as their configurations, and a Makefile. You can read more about Makefiles here, but the gist of it is that we've specified different shell commands in there that can be executed to sync the data.
As an example, if you're on a Mac or Linux computer, you can
cd into the directory with the Makefile,
make sync to update the data from the point of last run. You can take it a step further by
automating the process using crontab.
This requires that your PC be on, and a better approach would be to use a server that you maintain or
some sort of cloud service.
KNOTS has been released as an open-source project. Community participation is encouraged and highly appreciated. If you'd like to contribute, please follow the Contributing Guidelines.
If you run into any problems, please create a new issue.