Setting up a Webhook

Overview

Rivery  Webhooks allow you to connect HTTP endpoints on the web to your cloud data warehouse. Using Webhooks, you can stream real-time events or notifications as they happen and record them in cloud RDBMS or Storage type. In this guide we show you how to enable Webhooks and configure them to work with your chosen HTTP endpoint.

Webhook Data Source to Target

Step 1:

On the Rivery Console, select the
 option on the top right.

Step 2:

Select "Data Source to Target" from the menu: 

Step 3: 

Give the River a name, description and assign it to a group if you desire. On the "Source" tab, scroll down to the bottom and select:  from the options.

Step 4: 

Toggle the Webhook to true:This will bring up the following submenu:

Hit the "Got it" button and go to the "Target" tab.

Step 5:

Select your desired data warehouse destination from the options available:

 and configure the settings for your destination including your connection, database, schema and table name (Rivery will create the table if a table with the specified table name does not exist) :

Step 6:

Go back to the "Source" tab and in the "JSON example" field, paste a sample JSON response representing the structure of the data that will be sent to the webhook endpoint. If you do not have this sample, you can either obtain a sample using a dummy webhook endpoint or manually map the columns of the data in the next step.

Step 7:

Go to the "Columns Mapping" tab and (if you pasted an example JSON in Step 6) hit the  button to generate destination column mapping automatically* (it will be based on the fields of the uploaded JSON). If you did not paste the example JSON in Step 6, you will need to manually create your destination columns.

*NOTE: As of the Alpha version of the Webhooks feature, nested JSON columns need to be manually mapped on the Columns Mapping tab, even if the example JSON is provided. Nested column auto-mapping is a feature that will be developed in a later version.

Step 8:

Once your columns have been mapped, toggle to true:  If everything has been configured correctly, you will be shown a screen with your unique Webhook rivery endpoint. Copy this endpoint, and hit "Got it".

Step 9:

Navigate to your web service that you desire to use Webhooks with and paste the Rivery endpoint into the "Payload URL" or related option, this establishes the link between your web service and the Rivery console. In this example, we are using GitHub:

Step 10:

Go back to your Webhooks river and under the schedule tab, set the cadence you desire for your data warehouse to be updated with new data listened in from your desired Webhook endpoint. The default is 1 hr, but it can be set to as low as 5 minutes.