MongoDB Walkthrough

Introduction

This document aims to guide users through the process of creating a MongoDB instance, and performing basic database operations.

How to Work with MongoDB?

Step 1: Establish a Connection

Make sure you possess appropriate credentials for establishing a connection with a MongoDB database server.

Step 2: Choose the River Mode

Once you have successfully established a connection to MongoDB, select the River mode to load data into a Target. There are 3 available options for you to choose from:

• Multi-Tables : Simultaneously load multiple tables from MySQL to your desired Target. In this process, Rivery conducts a full ELT process which will extract the full structure of all selected tables before filtering and writing only selected columns to your target. This can result in data transferred being of a larger volume than data written to target.

  • Standard Extraction - This River mode maps, transforms, and loads data from multiple tables into a unified schema. It uses SQL queries for transformations and can be scheduled or manually triggered.
  • Change Streams - This mode monitors Source database logs, capturing and transforming changes in real-time. This ensures minimal data loss and low latency transfer when loading the transformed data into the target database, keeping it in sync with the Source.

• Legacy River: Select a sole Source table for loading into a Target. Legacy rivers allow you to replicate fields in single or array mode to your target with a high level of granular control over data-type mapping, and filtering of nested objects.

Please Note:

• Incrementing columns with the '$' operator is not supported in Multi-Tables and Legacy River modes
• In all River Mode except Change Streams, when you choose a Collection and go to the Table Settings tab in the Schema section, you'll find a feature that improves adaptability and simplifies recovery in situations where mapping gaps occur due to the default sampling method.
  
  
  By default, the mapping process samples the last 100 documents based on user preference, with a maximum sample size of 10,000. Users also have the option to sample either the last or first records.
  
  
  To apply changes to the mapping configuration, it's necessary to reload the mapping. Click on Apply Mapping Changes to proceed.

• The Timestamp datatype in MongoDB will return the timestamp value without milliseconds, displaying as 'YYYY-MM-DD HH:MM:SS'

Step 3: Run the River

Once you have successfully configured the River mode, selected the extraction method, and set up the scheduling, you are now ready to initiate the River and start its operation.