Technical Brief: Stonebranch Universal Data Mover

By Michael Boon

In many companies, file transfer processes represent more than 50 percent of the entire batch workload. Typical file transfer tools such as FTP and SFTP do not meet the requirements for safe data transfer. Likewise, the margin of error in data transfer through a process such as FTP is not appropriately managed in a company-wide WAN. As a text-based protocol, FTP transfers authentication information as well as the data as unencrypted text.

Topics:
Tagged as:

“We're an IT service provider for banks that transmit over 3,000 files a day on a wide variety of system platforms. We have been doing this successfully for over a decade with Universal Data Mover. The mature scripting language in Universal Automation Center ensures that we satisfy our many customers.”

Klaus Martin, Head of Administration Services, GAD eG

Background

Additionally, companies often need to audit and report on the timing, nature and results of their file transfer operations. Audits may range from simple questions such has “When was a certain file transfer with a certain result executed?” to the highly complex inquiries of the history of data transfers in a business process. 

Stonebranch Universal Data Mover closes the existing gaps in file transfer and processing solutions and has been proven to meet the highest security standards. Various SSL encryption algorithms are supported, and user authentication takes place through an internal security concept, which also supports LDAP/AD. The architecture is based on fault-tolerant agents, which trap connection terminations caused by network errors and reestablish network sessions to ensure the completion of data transfer processes.

File transfer processes are usually triggered by events or dependencies. These dependencies can be configured in Universal Controller, the web-based user interface of Stonebranch Universal Automation Center; the connection to the actual file transfer process will be established through various triggers. If the condition arises, the specified process will be executed and monitored. 

The integrated file transfer language supports very complex file transfer processes. As of this publication, applied FTP/sFTP shell based scripts can be replaced with unified and centrally administrated Universal Data Mover scripts.

Universal Data Mover

Universal Data Mover (UDM) is a synchronous data transfer solution. It allows complete control over and view of data transfers from your existing Stonebranch Universal Controller or any other scheduler. 

In addition to providing secure and reliable file transfer across platforms, Universal Data Mover includes numerous automation and integration functions. These include dynamic triggering and application event coordination, as well as advanced scripting—using popular scripting languages—which allows you to automate complex business involving file transfer and processing.

Universal Data Mover makes all of this functionality available 24×7, with a fault tolerant protocol that ensures uninterrupted file transfer without manual intervention.

Installation

Installation routines are provided to allow Universal Data Mover to be installed on distributed servers in just a few minutes.

In an SMP/E environment on the z/OS operating system, Universal Data Mover can be ready for production in less than one hour, with a few prepared jobs.

With Stonebranch Universal Agent, it is possible to execute automated file transfer between agents immediately, without any installation. 

If a new server is integrated, only one Universal Agent needs to be installed on the new server. The Universal Agent connects automatically to the Universal Controller (or more precisely, to the Universal Automation Center Middleware OMS) and is immediately ready for use.

Core Functions

Universal Data Mover provides a number of core functions that deliver key benefits to IT management and the businesses it serves: 

  • Integration in Existing Workflows
  • Security
  • Reliability
  • Central Control
  • Automatic Notifications
  • Reporting
  • Automated File Transfer

Integration in Existing Workflows

File transfers are easily added to an existing workflow using the Universal Controller WEB GUI. 

In the following example, the file transfer task “Copy Files SAP Appl. Server”  is dropped into an SAP billing workflow.

Security

The efficacy of the solution’s security has been verified by numerous penetration tests performed by customers and by security experts. Most recently, BND-certified Secuvera GmbH performed a penetration test at the offices of German insurer LVM  . Security features in Universal Data Mover include

  • Authentication, encryption (TLS), and compression (HASP, zLIB).
  • A single channel for data and control.
  • Single port activation on the firewall.
  • Secure connection of the agents with the controller through access control. 
  • Every change will be audit proofed and automatically recorded in the system.

Reliability Features

  • Automated failover in case a host is unavailable.
  • Automated restart.
  • Network error tolerance: automatically attempts to restore network connection with the server within a configurable interval. 
  • 3-party file transfer.
  • Very powerful integrated file transfer script language.

Central Control 

All file transfer scripts in Universal Data Mover are administered centrally with version control in Universal Controller. All file transfers can be monitored centrally through the Universal Controller dashboard and interventions can be applied on demand.

Automatic Notifications 

An SNMP trap or a custom definable e-mail notification can be sent out automatically with every status change (such as success or failure) of the file transfer. 

Reporting

At any time, you can use the integrated reporting generator to obtain self-configured reports in CSV, Excel, or PDF formats.

Automated File Transfer

There are several options for automation of a file transfer: 

  • Time trigger: Start a file transfer at a specific time/date.
  • Task monitor trigger: Start a file transfer if one or several tasks are finished, (For example, “Start a file transfer if the SAP job from the night before is finished.”) 
  • File monitor trigger: Start a file transfer with the arrival of a file or event.
  • Composite trigger: A combination of all the above triggers.

Various transfer scenarios will be described in the following section.

Universal Data Mover Architecture

The architecture of the Universal Data Mover allows for a traditional two-party transfer between a local and a remote site, as well as specific three-party transfer between two remote sites, controlled from a local agent through any scheduler. 

With the help of this architecture, Universal Data Mover can exchange data securely through the firewall. Data transport is encrypted and executed through the TCP/IP report. Using the Execute function in Universal Data Mover, the user can execute scripts and commands on the remote site before and after the file transfer (for example, start processing the data sent).

TCP/IP Connection Buildup 

The connection to the partner will be built up via the TCP/IP report. IP addresses as well as host names can be used as addressing for the partner. The entrance port at the recipient of the data recipient (Default: 7887) can be modified and thereby secured within the firewall. 

UDM Manager, Server, and Broker

Universal Data Mover is comprised of the UDM Manager, the UDM Server, and the UDM Broker.

The UDM Manager is the local, initiated component and is authorized at the remote site by the UDM Broker. After successful authorization, the UDM Broker will start an appropriate UDM Server. These connect with the UDM Manager to execute the data transfer. The governance and control lies at the local site on the UDM Manager. 

Depending on the operating system, the UDM Broker runs as a started task, a daemon or a service; the server is a sub-component of the broker. The UDM Manager can be called up in batch job, a script or from the command line.   

Login Data

A user ID and an appropriate password at the remote site are required for the authorized setup of the connection. These login data will be encrypted by the owner at the remote site and will be made available for the submitter as authorization data at the local site. This ensures no unauthorized access to the system or to the data. 

Encryption

Data is secured through SSL encryption routines during data transfer.

Available encryption routines: 

Universal Data Mover - Extended Functions

UDM provides a very extensive scripting language, which directs the various UDM functions.

Following is a sampling of the available functions:

Training

Stonebranch offers standardized training packages to enable users to rapidly acquire proficiency in Universal Data Mover. The following two classes are recommended for users who want to get started as quickly as possible:   

  • Managed File Transfer Basic (SB-UDM-BA-01)
  • Automation Center MFT advanced scripting (SB-UDM-AD-01)

Training sessions are limited to eight participants and can be delivered in German or in English. 

References

Selected Stonebranch customers with Universal Data Mover Business Partner licenses: