River-Net

A peformant distributed .NET service for scheduling Elasticsearch ETLs. It uses Quartz.Net for scheduling and runs in IIS (for the time being).

##Documentation

• Setup
• API
• River Sources
• Deployment

### Setup Under River.Quartz there is a [quartz.sql](https://github.com/CompassPHS/River-Net/blob/master/River.Quartz/quartz.sql) file. This will create a database called ```RiverNet```, a schema called ```quartz```, and the necessary objects to support [Quartz.NET](https://github.com/quartznet/quartznet ""). The SQL Server instance is specified in the [connectionString](https://github.com/CompassPHS/River-Net/blob/master/River.Api/Web.config#L58) in the quartz section of the web.config under River.Api. ### API Jobs are created through ```PUT``` requests to ```/api/river/{job_name}```

A typical job looks like this:

{
  "name":"job_name", 
  "cron":"0 */2 * * *"
  "suppressNulls":true,
  "source":{
    "type":"database",
	"connectionString":"Server=server_name;Database=db_name;Connection Timeout=30;Trusted_Connection=True;",
     "command":"exec river_SomeJob"      
  },
  "destination":{
    "url":"http://localhost:9200",
    "index":"index",
    "type":"type"
  }
}

"cron" is optional. Excluding a cron will run the job immediately and once only.

"suppressNulls" will exclude any fields with the value NULL from the output document.

### River Sources There are two kinds of sources: Database and FlatFile

Here's a Database source:

{
  "type":"database",
  "connectionString":"Server=server_name;Database=db_name;Connection Timeout=30;Trusted_Connection=True;",
  "command":"exec river_SomeJob"      
}

Here's a FlatFile source:

{
  "type":"flatFile",
  "location":"c:\path\to\file",
  "delimiters":[',']
}

### Deployment This app runs in IIS and is subject to the idle timeout and recycling any other IIS site would have.