Logic Step

Overview

Logic Rivers are so-called because a logical data model defines the structure of data elements as well as the relationships between the Rivers.

Logic Rivers are used for data transformations and workflow orchestration, and they can contain other Rivers within them.

For data transformations, you can use SQL for in-warehouse transformations or Python for more complex scenarios.

Rivery's Orchestration supports branching, multi-step processes, conditional logic, loops, and more, making it simple to create complex workflows.

Logic Step Types

You can select your Logic stp type using the drop-down list in the logic step's upper left corner.

Each logic step type is a different task type.

Logic Step type overview:

• SQL / DB Transformation: Run an in-database query or a custom SQL script using the syntax compatible with your cloud database, and then save the results into a table, file, DataFrame, or variable.
• River: Trigger existing River within your account. This could be a Source to Target River that you wish to coordinate alongside other Source to Target Rivers and transformation steps.
• Action: Make any custom REST call.
• Python: Allows for quick and easy data manipulation using Python scripts.

SQL / DB Tranformations

The 'SQL / DB Tranformations' option allows for in-database transformations using a query or SQL statement (the SQL syntax to be used is that of your cloud data warehouse).

First, select your desired cloud database and set the connection.

Next, enter your query into the 'SQL' box. You can click the magnifying glass to enlarge the space where you can view the query.

Notes -

• It is recommended that you build the query itself in the query editor of your cloud database, so that testing and debugging can be most efficient.
• If comments are nessecary, use the comments syntax /* comment*/. The alternative option: --comment syntax make all the next query to be remarked as comment. Therefore, please try to avoid using this syntax in Logic in Rivery.

After the query has been pasted into the Source section, it's time to decide on the desired target. In 'SQL / Script' logic step types, Rivery can store the results of the source query into a database table, variable, or file storage.

When storing data into a database table, you can choose between three loading modes: Overwrite, Upsert-Merge, and Append Only.

• Overwrite: Replace the target table completely with the data from this run. If the target table does not yet exist, it will be created automatically.
• Upsert-Merge: Only add new or changed records to the target table based on a defined key in the Column Mapping.
• Append Only: Union the data from this load onto the existing target table.

If you choose to use Upsert-Merge functionality, you can define your merge key by clicking into the Column Mapping (at the very bottom of the logic step) and highlighting the key indicator for the desired field. You may need to first select 'Auto Mapping' to fetch your table schema.

The next option for storing data into a target is file storage. The type of file storage allowed will depend on what cloud connection you are using. The supported file store options are Amazon S3, Google Cloud Storage and Azure Blob Storage.

For this option, enter the bucket you wish to save the results as well as the desired file name. By scrolling down, you'll see more options to customize by file type, delimiter (for CSV) and compression.

Lastly, the 'SQL / Script' logic step type provides the ability to store data values into variables which can be called downstream in the Logic River.

In a SQL/Script logic step, variables can be stored either at the River level or the Environment level.
A River variable allows for dynamic value passing between steps within the same River session.
On the other hand, an Environment variable is an account-level variable, accessible via the 'Variables' page in the left-hand menu, and can be used across multiple Rivers.

River

The 'River' option of a logic step will trigger the selected river to run.

For example, the logic step above will trigger an existing river in our account called 'FB ads' which pulls ad data from the Facebook Ads connector.

Action

The 'Action' logic step will trigger an existing Action river in your Rivery account. Action Rivers allow for custom REST calls to any API on the web.

Once the desired Action River is selected, any inputs corresponding with the river will appear for enhanced customization.

For a more in-depth walkthrough of Action Steps in Logic Rivers, see the Action Step documentation.

Python

This feature is a part of the Logic River type, which enables you to easily build a data transformation process using steps, variables, containers, conditions, loops, actions, and River runs.

Please refer to our documentation on Python for additional details.

Logic Step Container

Condition

The Condition logic step type allows for enhanced conditional capabilities in a Logic River. Upon selecting this option, a user interface for applying a condition to a logic step will appear.

Here you can enter a static value or existing variable for which your condition to meet. Based on the results of the condition as its processed, you can choose to run a step, pass over it, stop it, or fail the river.

If the condition is not true, there is a second conditional container where the 'ELSE' of your condition is processed.