Date Range Settings

How to Use Date Range Variables in Your API Configuration

This guide will help you understand how to effectively use date range variables in your API configurations, particularly when working with date ranges in loops. These variables allow you to manage and control the date range for your API requests dynamically.

Step 1: Understand the Date Range Variable Syntax

To use a date range variable, you need to enclose it within double curly brackets and percentage signs. This syntax helps define the start and end of the date range within your loop. Here’s how to structure it:

• Start of the Date Range: {{%variable_name_START%}}
• End of the Date Range: {{%variable_name_END%}}

The variable_name is the name of the variable you declare in the loop step. The START and END specify the start and end points of the current iteration in the loop.

Step 2: Access Specific Parts of the Date Object

You can extract specific components of the datetime object using dot notation. This allows you to access particular parts of the date, such as seconds, minutes, hours, etc. Here are some examples:

• Seconds: {{%variable_name_START.SECOND%}} - Returns the seconds of the current date.
  • Example: For the datetime 2000-10-01 10:00:45, it would return 45.

Remember, variables are case-sensitive, so you must use uppercase letters for the date parts like START and END.

Step 3: Configure Date Range Settings

In your configuration, you can define the date range settings, including the start date, end date, date format, and chunk size. Here’s what you need to configure:

• Start Date: The beginning of the date range.
• End Date: The end of the date range.
• Date Format: The format in which the dates are represented.
• Chunk Size: The size of each date range increment, in seconds.

Example:

date_range_settings:
  start_date: "2024-08-01T00:00:00Z"
  end_date: "2024-08-15T00:00:00Z"
  date_format: "%Y-%m-%dT%H:%M:%SZ"
  chunk_size_seconds: 172800

In this example, the date range loop will send requests between 2024-08-01T00:00:00Z and 2024-08-15T00:00:00Z, with each increment being 172800 seconds (or two days).

Step 4: Break Down the Date Range Increments

Once your settings are configured, the date range will automatically increment according to the specified chunk size. For example:

• First Range: 2024-08-01T00:00:00Z to 2024-08-03T00:00:00Z
• Second Range: 2024-08-03T00:00:00Z to 2024-08-05T00:00:00Z
• And so on...

Available Breakdown Options for Datetime Objects

When accessing specific parts of the datetime object, the following breakdowns are available:

• .SECOND
• .MINUTE
• .HOUR
• .DAY
• .MONTH
• .YEAR

These components allow you to customize how you handle datetime objects in your API requests.

Example

"""
connector:
  name: Github connector
  type: rest
  base_url: https://api.github.com/repos
  variables_storages:
    - name: results storage
      type: file_system
      path: storage/results/filesystem
  variables_metadata:
    current_date:
      storage_name: results storage
      format: txt
    final_output_file:
      storage_name: results storage
      format: json

steps:
  - name: Loop_Over_Commits
    description: Retrieve the commits from a given repository
    type: loop
    loop:
      type: date_range
      variable_name: current_date
      date_range_settings:
        start_date: "2024-08-01T00:00:00Z"
        end_date: "2024-08-15T00:00:00Z"
        date_format: "%Y-%m-%dT%H:%M:%SZ"
        chunk_size_seconds: 172800
    steps:
      - name: Get_Commits
        description: Retrieve commits that were created between given dates
        endpoint: "{{%BASE_URL%}}/tensorflow/tensorflow/commits"
        http_method: GET
        type: rest
        query_params:
          since: "{{%current_date_START%}}"
          until: "{{%current_date_END%}}"
        variables_output:
          - response_location: data
            variable_name: final_output_file
            variable_format: json
            overwrite_storage: false
            transformation_layers:
              - type: extract_json
                json_path: $[*].commit
                from_type: json
"""