Snowflake as a Target
  • 7 Minutes to read
  • Dark
    Light
  • PDF

Snowflake as a Target

  • Dark
    Light
  • PDF

Preface

This guide will show you how to create an S3 bucket for Snowflake staging while also setting the required credentials for using Snowflake with Rivery.

Before you use this guide, Please make sure you have a valid Snowflake account.

If you don't have any valid Snowflake account, please create one. If you or your company already has a Snowflake account, you can connect/create a Rivery user.


Setting up your Snowflake Environment

Rivery's Snowflake environment can be configured in two ways, but only one method for granting access must be chosen:

The first method will grant an existing user the SYSADMIN role, which will allow rivery access to all object in the database and will be able to extract from, and ingest to, new and existing tables.

The second method will either create or use existing user, role, database and warehouse depending on the settings, and will either give Rivery the ability to ingest data to and/or extract data from (depending on the settings) said objects.

First Method - Global Permission With Sysadmin Role

Snowflake allows a single user to have multiple roles. Rivery will need to execute various commands on Snowflake, so you must assign a SYSADMIN role to the Rivery user you’ll be leveraging:

  1. Access your Snowflake console. Utiliza a user that has access to the ACCOUNTADMIN role

  2. Select the worksheet tab.

  3. Run the following commands:

    begin;
    
     /* Set variables for script, select an existing user or create a new one, replace the value after var_user with 
     the name of the user, do not forget to replace {password} with the password of your choice 
     if you are creating a new user. */
    
       set var_user = 'RIVERY_USER';
       set var_password = '{password}';
       
     /* switch to ACCOUNTADMIN role: only an ACCOUNTADMIN can set or unset a user’s role */
        USE ROLE accountadmin;
        
        /* Create a user for Rivery or use an existing one */
       create user if not exists identifier($var_user)
       password = $var_password;
       
        GRANT ROLE SYSADMIN TO USER identifier($var_user);
        ALTER USER identifier($var_user) SET DEFAULT_ROLE = SYSADMIN;
    commit; 
    

Second Method - Specific Permission With Custom Role

Copy the following script to your Snowflake console, make sure to change the variables to your desired variable and only run necessary steps, pay attention to the instructions on the script.

begin;
 
   /* Set variables for script, You can choose to work with the default suggested values in the script or use your own. 
   Do not forget to replace {password} with the password of your choice if you are creating a new user. */
   set var_user = 'RIVERY_USER';
   set var_password = '{password}';
   set var_role = 'RIVERY_ROLE';
   set var_database = 'RIVERY_DATABASE';
   set var_warehouse = 'RIVERY_WAREHOUSE';

   
   /* Switch to securityadmin role:  
Role that can manage any object grant globally, as well as create, monitor, and manage users and roles */
   use role securityadmin;
 
   /* Create role for Rivery */
   create role if not exists identifier($var_role);
   grant role identifier($var_role) to role SYSADMIN;
 
   /* Create a user for Rivery */
   create user if not exists identifier($var_user)
   password = $var_password
   default_role = $var_role
   default_warehouse = $var_warehouse;
 
   grant role identifier($var_role) to user identifier($var_user);
 
   /* switch to sysadmin role: 
Role that has privileges to create warehouses and databases (and other objects) in an account. */
   use role sysadmin;
 
   /* Create a warehouse for Rivery, this step is optional */
   create warehouse if not exists identifier($var_warehouse)
   warehouse_size = xsmall
   warehouse_type = standard
   auto_suspend = 300
   auto_resume = true
   initially_suspended = true;
 
   /* Create database for Rivery, this step is optional */
   create database if not exists identifier($var_database);
   
 
   /* Grant Rivery role access to warehouse */
   grant USAGE
   on warehouse identifier($var_warehouse)
   to role identifier($var_role);
 
   /* grant Rivery access to database */
   grant CREATE SCHEMA, MONITOR, USAGE
   on database identifier($var_database)
   to role identifier($var_role);
   
   /* ATTENTION! if you are looking to ingest and extract data from objects created within Rivery 
   and not objects that already exist you can stop the flow here */
   
   /* Grant access to all existing sachems on the database */
   grant ALL on all SCHEMAS IN DATABASE identifier($var_database) to ROLE identifier($var_role);

   /* Grant access to all  existing tables on the database, might take several minutes if there are many tables */
  grant ALL on ALL TABLES IN DATABASE identifier($var_database) to ROLE identifier($var_role);
   
 
 commit; 
 

Please Note:
If you wish to add Masking Policy permissions for this user, please refer to the 'Enforce Masking Policy' section.


Create Network Policy for Rivery IPs [Optional]

In some cases, your Snowflake account may be access restricted by IPs or domains. In such cases, you must add Rivery IPs to your Snowflake Network Policy in order to connect successfully.

Note: Open Rivery IPs in Snowflake Network Policies may block any other unspecified IPs in the network policy. Make sure you've whitelisted all of your IPs in Snowflake’s network policies before creating Rivery's. Read more about network policies here .

In order to create a Network Policy for Rivery IPs:

  1. Log into your Snowflake account.

  2. Make sure the user is set to a ACCOUNTADMIN or SYSADMIN role

  3. In the worksheet, run the following command:

CREATE OR REPLACE NETWORK POLICY RiveryIPs ALLOWED_IP_LIST = (Copy our most recent whitelisted IPs here)

Connection Procedure

You can connect to Snowflake using one of two methods:

Basic Authentication

To connect to your Snowflake account using Basic Authentication, follow the steps below:

  1. Go to the Connections menu in Rivery:
    image.png

  2. In Snowflake Connection form type in your Connection Name.

  3. Select the Basic Authentication Type.

  4. Enter your Username and Password.

  5. Enter the name of your Warehouse (mandatory).

  6. Enter the name of your Role (optional).
    Note:
    If you leave it blank, the account's default one will be used, and in that case, the default Role must have access to the selected Warehouse, or the connection will fail.

  7. Input your Locator.

  8. Use the Test Connection function to see if your connection is up to the task.
    If the connection succeeded, you can now use this connection in Rivery.

Please Note:

The Locator is a combination of your Region and Account Name, which can be found in your Snowflake URL: https://account-name.eu-central-1.snowflakecomputing.com
Your Locator is: account-name.eu-central-1

If you’re using Snowsight- https://app.snowflake.com/eu-central-1/account-name
Your Locator is: account-name.eu-central-1

  1. Set your Custom FileZone to save the data in your own staging area (Optional).
  2. Use the Test Connection function to see if your connection is up to the task. If the connection succeeded, you can now use this connection in Rivery.
    You can now pipe data to Snowflake by leveraging this connection in any river!

image.png


Key-Pair Authentication

To connect to your Snowflake account using Key-Pair Authentication, follow the steps below:

  1. Open a Terminal window to generate the private key.
    (Mac / Linux)
Please Note:

Windows is not supported.

  1. Enter the following command into your Terminal window to download the Key file to your Documents folder:
cd Documents

Please Note:
To confirm the command, click the Enter key.

  1. Run the following command to generate an unencrypted version of the Key:
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt

image.png

  1. Create the Public key by referencing the Private key. The following command assumes the private key is encrypted and stored in the file rsa key.p8.
    Simply copy and run it:
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
  1. This will allow you to generate a private key in PEM (Privacy Enhanced Mail) format:
cat rsa_key.pub

image.png


This is the PEM format:

image.png

  1. To assign the public key to a Snowflake user, run the ALTER USER command in your Snowflake console:
    Note:
    Replace < Username > with your snowflake Username.
alter user < Username > set rsa_public_key='';

image.png

  1. Copy the PEM format from step 5 and paste it between the apostrophes in the Snowflake console code:

image.png

  1. Go to the Connections menu in Rivery:
    image.png

  2. Fill out the Snowflake Connection form with the following information:

a. Type in Connection Name.
b. Select the Key-Pair Authentication Type.
c. Enter your Username.
d. Enter the name of your Warehouse (mandatory).
e. Enter the name of your Role (optional).
Note:
If you leave it blank, the account's default one will be used, and in that case, the default Role must have access to the selected Warehouse, or the connection will fail.

f. Input your Locator.

Please Note:

The Locator is a combination of your Region and Account Name, which can be found in your Snowflake URL: https://account-name.eu-central-1.snowflakecomputing.com
Your Locator is: account-name.eu-central-1

If you’re using Snowsight- https://app.snowflake.com/eu-central-1/account-name
Your Locator is: account-name.eu-central-1

  1. Drag the rsa_key.p8 file from your Documents folder to the label, or simply browse for it.

  2. Set your Custom FileZone to save the data in your own staging area (Optional).

  3. Use the Test Connection function to see if your connection is up to the task.
    If the connection succeeded, you can now use this connection in Rivery.

image.png

Consult the Snowflake documentation to generate an encrypted Key or for more information on the Key-Pair configuration process.


Was this article helpful?