Custom FileZone for Snowflake
  • 3 Minutes to read
  • Dark
    Light
  • PDF

Custom FileZone for Snowflake

  • Dark
    Light
  • PDF

This is a step-by-step tutorial to setting up a Custom FileZone for Snowflake.

Overview

Rivery allows you to create a custom FileZone, to manage your data in your own S3 or Azure service. It means you're in charge of your own stage in Snowflake terms. For up to 24 hours, your data will be stored in your bucket. A bucket is a container that holds objects. An object is a file and any metadata that describes that file.


Prerequisites


Amazon S3 Buckets

Create Your S3 Bucket

  1. Sign in to the AWS Management Console and open the Amazon S3 console

  2. Click the Create Bucket button.

  3. Fill in a Bucket name.
    The bucket's name must include:

  • Be unique across all of Amazon S3.
  • The length should be between 3 and 63 characters.
  • Uppercase characters are not allowed.
  • Begin with a lowercase letter or a numerical value.

Note:
You can't change the bucket's name once it's been created. See Bucket Naming Rules for more information on bucket naming.

  1. Choose an AWS Region , the same region where your Redshift resides.

Note:
To reduce latency and costs while meeting regulatory requirements, choose a Region close to you.

  1. Select Next.
    image.png

  2. Choose your preferred settings by following the on-screen instructions on the console.

  3. Click Create Bucket.


Create an AWS User for Rivery

Create a user for Rivery and grant it permission to manage and read the FileZone bucket by following these steps:

  1. Sign in to the AWS Management Console and open the Amazon S3 console.

  2. Navigate to Users - > Add User.

image.png

  1. In the console, set your username, and check the Access type to Programmatic access.

image.png

  1. Click Next: Permissions.

  2. In the Set Permissions form select Attach Existing Policies Directly.

  3. Click Create Policy.

image.png

  1. Navigate to the JSON tab.

image.png

  1. Copy and paste the following policy:
    Note:
    Replace <RiveryFileZoneBucket> with the name of your S3 bucket.
{
 "Version":"2012-10-17",
 "Statement":[
   {
    "Sid":"RiveryManageFZBucket",
    "Effect":"Allow",
    "Action":[
    "s3:GetBucketCORS",
    "s3:ListBucket",
    "s3:GetBucketLocation"
     ],
    "Resource":"arn:aws:s3:::<RiveryFileZoneBucket>"
   },
   {
    "Sid":"RiveryManageFZObjects",
    "Effect":"Allow",
    "Action":[
      "s3:ReplicateObject",
      "s3:PutObject",
      "s3:GetObjectAcl",
      "s3:GetObject",
      "s3:ListObjects*",
      "s3:PutObjectVersionAcl",
      "s3:PutObjectAcl",
      "s3:ListMultipartUploadParts"],
    "Resource":"arn:aws:s3:::<RiveryFileZoneBucket>/*"
  },
  {
     "Sid":"RiveryHeadBucketsAndGetLists",
     "Effect":"Allow",
     "Action":"s3:ListAllMyBuckets",
     "Resource":"*"
  }
 ]
}
  1. Select Review Policy.
  2. Give the Policy a name and click Create Policy.
  1. Using the rounded arrows on the upper right, refresh the list of policies, check the policy you just created, and select Next: Tags.

image.png

  1. Click Next: Tags, Next: Review and then Create User to complete the process.

  2. In the summary screen, you'll find the user's AWS credentials (Access key id and Secret access key), which you can download as a CSV file (this is the only time you'll be able to do so).

image.png

  1. The User should now be able to manage and read the FileZone bucket that was created for Rivery. Please double-check that the policy you created is linked to the user you created.

create-an-aws-iam-user_mceclip63.png

Connect to Amazon S3

To connect to Amazon S3, see our Amazon S3 Connection documentation.


Azure Blob Storage Container

Please refer to our Azure Blob Storage Connection documentation to create a Standard Azure account and container.
Please Note: For Azure Blob Storage containers to work properly with Snowflake, a SAS token must be entered in the connection form.
To create a SAS Token, consult the Microsoft documentation.
image.png

Once you've finished creating the container and connecting to Azure Blob storage in Rivery, continue to 'Configure Custom FileZone for Snowflake in Rivery' down below.


Configure Custom FileZone for Snowflake in Rivery

  1. Go to Connections -> +New Connection and search for Snowflake.

  2. Type in your Connection Details and Credentials.
    image.png

  3. Toggle the Custom File Zone to true.

  4. Pick your cloud storage option.

  5. By clicking on FileZone Connection, you can select the previously configured FileZone Connection.

  6. Choose a Default Bucket (Container) from the drop-down list.

  7. Use the Test Connection function to see if your connection is up to the task.
    If the connection was successful, click Save.

image.png


Was this article helpful?