Inter-Cloud Data Transfer

Stream data from one object store to another without intermediate storage.

The Inter-Cloud Data Transfer integration allows users to transfer data to, from, and between major private and public cloud providers like AWS, Google Cloud, and Microsoft Azure. It also supports the transfer of data to and from a Hadoop Distributed File System (HDFS) and to major cloud applications like OneDrive and SharePoint. Integrations within this solution package include:

• AWS S3
• Google Cloud
• SharePoint
• Dropbox
• OneDrive
• Hadoop HDFS

Key Features:

• Transfer data to, from, and between any cloud provider
• Transfer between any major storage applications like SharePoint, Dropbox, and others
• Transfer data to and from a Hadoop File System (HDFS)
• Download a URL's content and copy it to the destination without saving it in temporary storage
• Data is streamed from one object store to another (no intermediate storage)
• Very fast, if the object stores are in the same region
• Preserves always timestamps and verifies checksums
• Supports encryption, caching, compression, chunking
• Perform dry-runs
• Dynamic token updates for SharePoint connections
• Regular expression-based include/exclude filter rules
• Supported actions are:

• List objects, list directory
• Copy/move
• Remove object/object-store
• Perform dry-runs
• Monitor object
• Copy URL

What's New in v1.2.0

New choice field for write options

• Do not overwrite existing objects
• Replace existing objects
• Create a new timestamp if it exists
• Use custom parameters

 New text field to set the recursion depth

• The default value is “1” — no recursion allowed

 New checkbox “Error on no transfer”

• The task will fail if no transfer has been performed

 File Monitor incl. Trigger functionality (similar as the agent file monitor in the Universal Controller)

• Skip if Active
• Trigger on Exists
• Use server modtime — time when the file was uploaded to the object store (S3 bucket, Azure Container, etc. ) 
• Task(s) to launch (CSV-List). You can launch one or multiple tasks, if the monitor goes to success.

 New Remote: SFTP client 

• Support for SFTP streaming
• Task acts as SFTP client

 Windows agent support

• Task can execute rclone installed on Windows

Note:

* SharePoint Online will only work with the latest Universal Agent 7.2