Action River - General Overview

Introduction

This document provides a step-by-step guide on utilizing Rivery's Action River feature to establish a connection with any REST endpoint and incorporate it as a new data source. While Rivery frequently includes built-in support for various data sources, there might be situations where the specific data source you require isn't included in Rivery's list of supported options. In such cases, this document can be useful and help you avoid unnecessary coding.

Action Rivers are designed for situations where a native connector is unavailable. By referring to the API documentation of the Source you want to connect to, you can create an Action River to establish the connection and retrieve data effectively.

Types of Action Rivers

There are 2 types of Action Rivers in Rivery:

• REST Action: Used to perform single actions such as fetching data from or sending data to a REST API endpoint.
• Multi Actions: Allows orchestration of complex processes by combining multiple actions in a sequential order, including REST Actions and other types of actions.

Creating a Custom Connector

Once a connection to a REST endpoint has been established, it can be saved as a custom connector using the REST API Source. To do this, create a new Source to Target River, select the REST API Source you just created, and then proceed to configure your Target in the same way you would for any other Source to Target River.

REST Action

Used to perform single actions such as fetching data from or sending data to a REST API endpoint, REST Action in Rivery simplifies the process of interacting with web services. It allows users to make API calls to retrieve, update, or delete data, making it a versatile tool for integrating external data sources into your data workflows. By configuring REST Actions, users can seamlessly connect to various APIs, handle authentication, and manage parameters to efficiently automate data retrieval and updates.

Authentication

To create a connection to an API, the user needs to provide the API URL. If the user is creating a Source, they will make a GET call to the API. A username and password, OAuth2, or Multi-Bearer authentication can all be used for authentication. This API endpoint is a public endpoint, so authentication is not required.

Testing the Connection

In this example, the user has access to an API endpoint that returns a list of users. By selecting the "Test REST Action" button, the user can verify the connection. This action provides a success code of 200 (connection succeeded) and a sample dataset.

Automating Table Structure Mapping

In this case, the user chooses a Source to Target River and selects the REST API as the Source and Snowflake as the Target. Rivery handles the task of using the JSON payload from the API endpoint to define the target schema. The user can easily use the "Auto Mapping" button and Rivery will automatically map the table structure and data types based on the metadata retrieved from a sample test.

Ingesting Data

After the table structure has been generated, data can be ingested by running the River. Data will be retrieved from the API based on the specified parameters (such as last modified date) and pushed into the target (in this case, Snowflake) using the specified primary key for merging.

Multi-Actions

Multi-Actions in Rivery are a way to orchestrate complex processes by combining multiple actions in a sequential order. This document provides an overview of Multi-Actions, including their creation, configuration, and execution within Rivery.

Cases Using Multi-Actions

Many REST APIs require multiple steps in order to connect to the service, and GET/POST requests. For example, first, submit a request to generate a token and then send a request to the service using that token. In this example, the first action will be a REST Action to generate the token and the second action will be the request to the service to make a GET or POST call.

Action Variables

Action variables in Rivery serve as an interface between actions and other rivers, enabling the use of dynamic values within Rest Actions and Multi Actions.

Variable Looping in Action Rivers

The Variable Looping feature transforms how variables are managed in the REST Action River, providing a smooth and effective method of iteration. Instead of iterating through Source to Target Rivers for each endpoint separately, this capability allows direct looping within the Action, iterating over a list of parameter values. This flexibility is invaluable for scenarios with varying parameter sets. For example, when an API endpoint accepts diverse values for a specific parameter, Variable Looping enables users to efficiently handle all potential cases. This enhancement significantly enhances load time efficiency and minimizes resource consumption.