Skip to content

Commit

Permalink
docs: Add twitterparser example to use lbsntransform as a package
Browse files Browse the repository at this point in the history
  • Loading branch information
Sieboldianus committed Apr 4, 2022
1 parent 8ee3932 commit 2a68f08
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions docs/package.md
Expand Up @@ -17,6 +17,19 @@ which includes the code that is invoked on command line use.

We plan to update this section with a Jupyter Lab example notebook.

### Example tool "twitterparser"

An example program `twitterparser` that uses lbsntransform as a package is available
[here](https://gitlab.vgiscience.de/lbsn/tools/twitterparser).

The tool demonstrates an (imaginary) custom processing pipeline for Twitter data,
stored in ZIP files on a remote server. The `twitterparser` tool will `(1)` connect to
this remote server via SSH, `(2)` pull zip files and extract the json files locally and `(3)`
transfer json files to a remote [lbsn-raw](https://gitlab.vgiscience.de/lbsn/databases/rawdb)
or [lbsn-hll](https://gitlab.vgiscience.de/lbsn/databases/hlldb) database using lbsntransform.

Use this tool as a template or starting point and modify `__main__` to your needs.

## Contribute

lbsntransform is in an early stage of development.
Expand Down

0 comments on commit 2a68f08

Please sign in to comment.