Skip to content

RestConnector

dovydasm edited this page Dec 17, 2018 · 35 revisions

[WORK IN PROGRESS]

The RestConnector is the most common connector type. It is used for retrieving and parsing data via HTTP requests.

RestConnector

A RestConnector has the following anatomy:

<RestConnector Id="MyConnector" Title="My Connector" HelpUrl="" HelpText="" Hidden="false">
  <Parameters>
  ...
  </Parameters>
  <Fetch>
  ...
  </Fetch>
  <Parse>
  ...
  </Parse>
  <Fail>
  ...
  </Fail>
</RestConnector>

RestConnector Atributes

  • Id (required) - A unique connector identifier. Can be the same as Title.
  • Title - A title that will show up in the connector menu. If title is not included, Id will be used instead.
  • HelpUrl - URL to a help page for the particular connector. If possible, use a link to a particular endpoint in official API documentation.
  • HelpText - Short description of the connector. It's displayed at the top of the connector panel in Excel.
  • Hidden - If set to True, the connector won't appear in the list of connectors in Excel. Used in conjunction with Prepare and Lookup connectors

Parameters

Parameters are sent to APIs via requests and they specify which records we want to retrieve. Use Parameters element and its child elements to build user friendly graphical interfaces where users can enter and choose parameter values.

The values can be inserted into URLs inside Fetch.Url, as well as RequestBody for POST requests. Use @(Model.ParameterId) to insert a value from a specific parameter. For example, if the following parameter is defined:

<Text Id="Email" Title="Filter by email" Required="true" HelpText="A filter on the list based on the customer’s email field"/>

We can use @(Model.Email) inside Fetch.Url:

<Fetch>
 <Fetch.Url>
  <![CDATA[
   https://api.stripe.com/v1/customers?email=@(Model.Email)
  ]]>
 </Fetch.Url>
</Fetch>

If user enters johndoe@gmail.com, the connector will fetch data from https://api.stripe.com/v1/customers?email=johndoe@gmail.com.

All parameters support the following attributes:

  • Id (required) - A unique identifier.
  • Title - A title that is shown above the parameter in the connector's graphical interface. Can be different from Id. Doesn't have to be unique.
  • Required - Default is False. When set to True, user is stopped and warned when clicking the Insert button. This should always be set to true for parameters that are mandatory for specific API endpoints.
  • DefaultValue - When this is defined, it shows up as a default value for the specific parameter in the graphical interface (the user can change it).
  • Debug.DefaultValue - Same as DefaultValue, but only works when Debug Mode is enabled in SeoTools.
  • HelpText - A short description for the parameter. When this is defined, a question mark icon appears next to the title and when user hovers the mouse over it, the HelpText is shown in a tooltip.
  • HelpUrl - A URL that contains help for this specific field.

Text

A simple textbox.

Example:

<Text Id="Email" Title="Filter by email" Required="true" HelpText="A filter on the list based on the customer’s email field"/>

Result:

Text parameter

Attributes:

  • Multiline - When set to True, allows user to enter multiple text lines, instead of one.

Number

Similar to Text, but allows only numbers to be entered.

Example:

<Number Id="Year" Title="Year" Required="true" DefaultValue="2016"/>

Result:

Number parameter

Checkbox

A simple checkbox with True/False value.

Example:

<Checkbox Id="Active" Title="Only return active products" DefaultValue="False" HelpText=""/>

Result:

Checkbox parameter

Radio

A parameter where several values are predefined and user has an option to choose one of them. Useful when there are few possible options (for example "ascending" and "descending" sorting options). Radio must have a DataSource element which itself should contain at least one Item element. Each Item is displayed as a choice in the user interface. The Id attribute is the value and Title is the text that shows up in the interface.

Example:

<Radio Id="Order" Title="Order by" DefaultValue="relevance" Required="false" HelpText="">
  <DataSource>
    <Item Id="relevance" Title="Relevance"/>
    <Item Id="population" Title="Population"/>
    <Item Id="elevation" Title="Elevation"/>
  </DataSource>
</Radio>

Result:

Radio parameter

Select

Similar to Radio, but the values are displayed in a dropdown box. Useful when there are many possible values to choose from. The syntax for inner elements DataSource and Item is exactly the same as with Radio.

Example:

<Select Id="TxType" Title="Transaction Type" Required="false" DefaultValue="all">
  <DataSource>
    <Item Id="all" Title="All"/>
    <Item Id="charge" Title="Charge"/>
    <Item Id="refund" Title="Refund"/>
    <Item Id="payment" Title="Payment"/>
  </DataSource>
</Select>

Result:

Select parameter

MultiSelect

Similar to Select, but multiple values can be selected. Values can be grouped into expandable groups. Currently in development.

Date

A parameter that allows user to choose a date.

Attributes:

  • Nullable - If set to true, the value evaluates to null when no value is selected. We can then use Utils.IsNullDate(Model.DateParameterId) to test if user hasn't chosen a date.

Example:

<Date Id="DisplayDate" Title="Date" Required="false" Nullable="true" HelpText="Local Time of the From Time Zone."/>

Result:

Date parameter

DateInterval

Allows user to choose a time period between two dates.

Attributes:

  • Nullable - If set to true, the value evaluates to null when no value is selected.

Example:

<DateInterval Id="DateInterval" Title="Interval" Required="false" Nullabe="true"/>

Result:

DateInterval parameter

Filter

Allows creating custom queries. Currently in development.

Reusing Items in multiple parameters

It's possible to write a single set of items and reuse it in multiple Select and Radio parameters. This is useful when multiple connectors/API endpoints require the same parameter, such as sorting direction (asc/desc) or languages. To achieve this, a Resource element containing Items has to be added to Resources, which itself is on the same level as RestConnectors.

Example:

<?xml version="1.0" encoding="utf-8" ?>
<Suite Category="..." Title="..." Id="..." RequireVersion="7.0.0" SourceUrl="..." HelpUrl="...">
  <Resources>
    <Resource Id="Languages">
      <Item Id="en" Title="English">
      <Item Id="fr" Title="French">
    </Resource>
  </Resources>
  
  <RestConnector Id="Connector1">
    <Parameters>
        <Select Id="Language">
          <DataSource>
            <Resource Id="Languages"/>
          </DataSource>
        </Select>
    </Parameters>
  </RestConnector>

  <RestConnector Id="Connector2">
    <Parameters>
        <Radio Id="Language">
          <DataSource>
            <Resource Id="Languages"/>
          </DataSource>
        </Radio>
    </Parameters>
  </RestConnector>
</Suite>

Both the Select in Connector1 and Radio in Connector2 will have the same two choices.

Fetch

HttpSettings

The HttpSettings element contains elements that modify the connector's HTTP request. Use these when a request requires specific headers, authentication, method different than GET, or a specific body.

RequestMethod

Used to specify an HTTP Request Method (aka HTTP verb). Mostly used with POST value, but other values are supported too. Default value is GET.

Example:

<HttpSettings>
  <RequestMethod>POST</RequestMethod>
</HttpSettings>

RequestContentType

Sets the request Content Type to one specified inside the element.

Example:

<RequestContentType>application/json</RequestContentType>

RequestHeaders

Generates HTTP request headers. Useful for authentication, CSRF tokens, and other headers that the server might require. Add Header element inside this element for each additional header. Set the attribute Name to header name and value to header value.

Example:

<RequestHeaders>
  <Header Name="User-Agent">SeoToolsForExcel</Header>
  <Header Name='Authorization'>Bearer @(Model.ApiKey)</Header>
</RequestHeaders>

RequestForm

Generates proper field values for a form (such as a login form). Use it with RequestMethod set to POST and appropriate RequestContentType.

Use Param elements inside RequestForm for form fields. The Name attribute specifies the form field name

Example: If we wanted to submit the following login form:

<form action="/login" method="POST">
  <input id="loginUsername" type="text" name="username"/>
  <input id="loginPassword" type="password" name="password"/>
</form>

The HttpSettings would look like this:

<HttpSettings>
  <RequestContentType>application/x-www-form-urlencoded</RequestContentType>
  <RequestMethod>POST</RequestMethod>
  <RequestForm>
    <Param Name='username'>@(Model.Username)</Param>
    <Param Name='password'>@(Model.Password)</Param>
  </RequestForm>
</HttpSettings>

RequestBody

Creates a request body. Useful when the server expects a specific body (for example a JSON object) Example;

<HttpSettings>
  <RequestBody>
    <![CDATA[
    {
      "key": "@(Model.ApiKey)",
      "tag": "@(Model.Tag)"
    }
    ]]>
  </RequestBody>
</HttpSettings>

Parse

Parsers

JsonPath

Used for parsing JSON data. The majority of all APIs return data in JSON format.

XPath

XPath (XML Path Language) is a language for querying XML data. We use XPath parsers to parse XML API responses, as well as HTML (regular webpages).

Parsing Json with XPath parser

Starting with SeoTools version 8.0.15, it's possible to use XPath parser (with XPath expressions) to query JSON data. JSONPath is very limited and sometimes it's impossible to query data the way we want to (or when API creators don't adhere to best practices). In such cases, it can be advantageous to use XPath.

Compute

Header

The Header parser can be used to output HTTP response users. With the attribute name we specify what response header.

<Parse>
   <Header Name="Server" Id="Server" Title="Server"/>
</Parse>

Converters

Converter is an attribute used in parser tags (JsonPath, XPath, etc). It sets the format of the sheet column that the parser corresponds to. If not specifically set, Auto value is used and SeoTools tries to guess the format based on the value of the data. We recommend setting specific converters for rows with known data type.

  • String - Converts value to a text String. Use this for regular text.
  • Bool - Converts value to Boolean (true/false).
  • Int - Converts number into Integer. Supports values from -2,147,483,648 to -2,147,483,648.
  • Double - Converts number into Double. Use this for large numbers and decimals.
  • DateTime - Converts date and time into Excel date type. If the value is in Unix timestamp format, add a Converter.SourceFormat attribute
  • Auto - Detects data type automatically and converts it. It's preferrable to use type-specific converters, if you know the data type, because they're faster.

Converter.SourceFormat attribute

Used to set DateTime format for source data. Use this along with DateTime converter, only when DateTime can't properly parse a row by itself.

  • If source data is a Unix timestamp (aka POSIX, Unix Epoch Time), use Converter.SourceFormat="Unix"
<JsonPath Title="Created" Id="Created" Expr="created_time" Converter="DateTime" Converter.SourceFormat="Unix" HelpText=""/>
  • If the DateTime can't guess the source format automatically, use Converter.SourceFormat="" with a custom Standard date format
<JsonPath Expr="created_at" Id="Created" Title="Created" Converter="DateTime" Converter.SourceFormat="ddd MMM dd HH:mm:ss +ffff yyyy"/>

Converter.ResultFormat attribute

Fail

Under the Fail tag we can list parsers that is used to detect if request failed.

This fail parser will check the HTTP response header Server and produce an exception if this value is not equal to Microsoft-IIS/10.0.

<Fail>
  <Header Name="Server" Fail.If="IsNotEqual" Fail.ComparedTo="Microsoft-IIS/10.0" Fail.Action="Exception" Fail.Message="Failed '{0}'."/>
</Fail>

Fail.If can have the following values:

  • IsNotEmpty (default)
  • IsEmpty
  • IsEqual
  • IsNotEqual
  • RegexMatches

Use Fail.ComparedTo to compare values.

Fail.Action can have following values:

  • EmptyResult
    • Produces an empty result.
  • Retry
    • Use with Fail.RetryTimes and Fail.RetryAfter (ms).
  • Message (default)
    • Use with Fail.Message

Prepare

Miscellaneous

You can’t perform that action at this time.