Technical Brief: Stonebranch Universal Agent

By Michael Boon

Topics:
Tagged as:

Our Universal Agent technology allows you to schedule any new application running on a UNIX, Linux, Windows, or cloud platform from your existing TWS/z mainframe scheduler (or any other scheduler). This enables your mainframe scheduler to automate cross-platform business processes such as triggering and controlling your enterprise-wide SAP batch process from your mainframe.

Summary

At Stonebranch, we are fully committed to leverage all z/OS advancements that are crucial to support your mission-critical business processing. While mainframes are still very relevant to many businesses, we will strive to give you modern ways to continue to support those workload processes in conjunction with newer technology.

The following paper describes how the Stonebranch Universal Agent can help you, as a mainframe customer, to 

  • enable your mainframe scheduler for cross-platform business processing; 
  • simplify your architecture by introducing a single universal agent technology for all your platforms; 
  • perform a fast and low risk transition towards a distributed scheduler; and 
  • reduce your license cost and operational cost by introducing the 100% web-based Universal Automation Center workload automation tool.

Enable your Mainframe Scheduler for Cross-platform Business Processing

Our Universal Agent technology allows you to schedule any new application running on a UNIX, Linux, Windows, or cloud platform from your existing TWS/z mainframe scheduler (or any other scheduler). This enables your mainframe scheduler to automate cross-platform business processes such as triggering and controlling your enterprise-wide SAP batch process from your mainframe.

Schedule Any Application on Any Platform

Universal Agents are capable of working with any application or tool that can access a command line interface. In addition, Universal Agents enable direct integration with your applications using a variety of web services protocols, initiating and monitoring application processes using SOAP, HTTP(s), JMS, and MQSeries protocols. 

For SAP-based business processes, our Universal Agents have a built-in SAP-certified connector, allowing customers to directly define and integrate SAP processes with any other type of processing throughout the enterprise.

Consolidate all your Scheduling Agents to a Single agent for All Platforms

With our Universal Agent technology, you can consolidate all your currently used IBM agents like FTA’s, standard agents and x-agents (or any other non-IBM agent) using our single automation agent.  This allows you to consolidate to a single standard for all of your scheduling agents, greatly simplifying the task of deploying and maintaining job-scheduling agents on all of your servers.

Perform Managed File Transfers Between all Platforms

Universal Agents include a highly secure, very powerful managed file transfer solution (Universal Data Mover) supporting transfer between any platforms. Our file transfer solution can be fully managed by your existing TWS/z mainframe scheduler (or any other scheduler) rendering third-party file transfer products obsolete.

In summary, the Stonebranch Universal Agent simplifies your infrastructure with a single automation agent for multiple automation tasks including managed file transfer.

Fast and Low-risk Transition to a Distributed Scheduler

The Universal Agent is a vendor-independent scheduling agent, which works with any job scheduler(s) on all major computing platforms, both mainframe and distributed. All schedulers that run on z/Series, i/Series, UNIX, Linux, and Windows are supported. This means, if you are planning to control all or only parts of your jobs from a distributed scheduler, you do not need to make any changes to your Universal Agent architecture. No extra investments need to be made, as you are taking full advantage of the existing network of Universal Agents. Our Universal Agents can even manage workloads from more than one scheduler at the same time. For example, using the same Universal Agent you can run a job triggered from the mainframe TWS/z scheduler as well as a job executed from a connected distributed scheduler such as TWS/d, Universal Automation Center, or any other scheduler. This functionality allows you to lower your risk by transitioning gradually (for example, application by application) from a mainframe scheduler towards a distributed scheduler. You can still run parts of your jobs scheduled by your mainframe scheduler, while a distributed scheduler such as Universal Automation Center already controls other jobs.

Benefits of Using Universal Automation Center with the Universal Agent

When working with containers, security is key. Containers are more or less isolated from the host OS system, but they use the same kernel. Due to potential security risks, a container should be executed in using a non-privileged user account. In some cases, it may be necessary to use some kernel functionalities, when mounting a USB drive, for examplThe Universal Agent is scheduler independent. However, if you are planning to replace your existing distributed scheduler or planning to move towards a single scheduling engine for your entire enterprise, the Stonebranch Universal Automation Center solution will provide you a state-of-the-art, 100% web-based scheduler that reduces your licensing and operation costs.

Lower Licensing Cost

  • No payment based on MIPS capacity
  • No extra charge for z/OS Rerun/Restart, rerun any instance
  • Integrated “highly secure” managed file transfer

Lower Operational Cost

  • Single-agent technology proven over more than 15 years of operations
  • Incredible ease-of-use, including a video classroom for self-paced learning
  • Automatic version control on all changes
  • Effective lifecycle management, from development through test and-production
  • Elimination of IT audit violations by tracking every change and user action
  • Query and control tasks started from within Workload Automation Center Web GUI

Improved Business Support

  • Self-service dashboards from any web-enabled device
  • Built-in graphical reporting with security access by group or usere. Universal Automation Center allows one to centrally maintain individual credentials for each container. 
  • This security concept has been consistently validated by external BSI certified security companies, including detack and Secuvera.

5 Key Reasons to Adopt Universal Automation Server, 4 Customer Successes

How it Works

The Universal Agent consists of a Manager component and a Server component. The Manager executes commands on any computer running the Server component.

You indicate to the Manager what commands to execute and how the standard input, output and error data should be processed. 

The Manager connects to the Server and processes your request. The Manager executes remote commands, as they would be if you entered them directly on the remote command line. Standard input and output and error files are supplied using JCL DD statements. The Manager for z/OS executes as a batch job.

The following illustration shows how the Manager triggers the Server agent to execute a remote UNIX script. The Manager receives the remote UNIX command through the JCL triggered by a Job in TWS/z.

Secure and Reliable

The Stonebranch Universal Agent uses the latest security standards verified by external security companies.  

Secure Network Communication

Communication across the network is implemented using SSL standards. This provides a secure and reliable network infrastructure for job scheduling across all of your platforms.

All credentials used in a command file (or even the whole command) can be encrypted using the Universal Encrypt utility.  

Fault Tolerance 

Our network fault-tolerant protocol prevents loss of visibility to your active workloads. If the network connection is lost, the Universal Agent traps the connection termination caused by the network error and re-establishes the network connections. When connections have been re-established, processing resumes automatically from the location of the last successful message exchange. No program restarts are required, and no data is lost.

Manager fault tolerance provides an execution environment in which the Manager agent (Source Agent) is not required in order for the user process to continue execution on the remote system. The user process can execute to completion with or without a Manager Agent connected.

Deploying the Universal Agent

Universal Agent for z/OS is provided as an easily installed SMP/E package. For all other platforms, such as UNIX, Linux, and Windows, there are OS packages. No hooks or APIs are required.

Scenarios

The following chapter provides an overview of how to simplify your architecture by consolidating all your currently used agents towards a single universal agent technology for all your platforms. We will also explain how the Universal Agent, in conjunction with the Universal Automation Center, enables you to perform a fast and low-risk transition towards a distributed scheduler.

Sample Case – TWS/z Workload Controlled by Multiple Agents

The following assumes that you are running a TWS/z scheduler on your mainframe, which uses different IBM agent technologies to remotely start jobs in your distributed “Server” based environment. 

Note: In order to make this example more generic, and to show that the Stonebranch Universal Agent can replace any type of agent, we have assumed that multiple agent technologies are in use. If you have already migrated everything to z-centric agents, the approach remains the same. 

Transition Approach 

The following illustration outlines the transition from IBM Tivoli z-centric, x-agents (or FTA, SA) to Universal Agent technology.

Transition steps:

Extract all TWS/z definitions for every TWS/z instance via Batch Loader statements retrieved via the batch command interface tool EQQYCAIN.

  1. Extract workstation and variables definitions via TWS/z batch job. 
  2. Scan JCL and Script Lib members from EQQJBLIB and EQQSCLIB datasets as a base to create new JCL members.
  3. Retrieve parameters such as TPGLINFO member from EQQPARM LIB. 
  4. Load the extracts into the Stonebranch transition tool.
  5. The Stonebranch transition tool identifies all relevant options and/or parameters, which are required to execute the job:
  6. A new JCL member will be generated with the same name as the existing one.
  7. All required parameters will be transferred to the related parameter definitions of the Universal Agent.
  8. Current parameter definitions that are no longer required will not be used.

Example

The transition tool reads the Batch Loader file and the related SCLIB member and creates a new JCL member with the SAP job call in the Universal Agent SAP-compliant syntax. The SAP Syntax used by the Universal Automation Center is readable by any SAP consultant. No vendor-proprietary SAP Syntax is needed.

Possible project approach 

In this scenario, the Stonebranch Transition Team would execute the transition for a selected sample environment. After a successful transition, you could perform the transition of any other environment using the Stonebranch transition tool. Stonebranch would provide support, if needed.

Sample Case – Migration from TWS/z to Universal Automation Center

Companies today are migrating away from mainframe infrastructure in favor of x86-based distributed applications for a number of reasons: 

  • Corporate consolidations or acquisitions: The acquired company must often abandon its mainframe to accommodate data center integration with a new owner or parent company.
  • Time: Mainframes and their applications can be challenging to maintain, as it is increasingly difficult to find qualified personnel to keep older systems and software running in a timely fashion. 
  • New applications, such as SAP HANA or Oracle EXADATA are designed for open systems. 
  • The major cloud services such as AMAZON EC2 and Microsoft Azure are based on X86 architecture. 
  • Money: High one-off costs for a new top-tier mainframe can exceed $1 million. There are also high annual operational costs due to hardware upgrades, hardware maintenance, and software licensing.

To manage workloads in such a transitory environment, you would need a scheduler, such as Universal Automation Center, to support both your new or future business processes (DevOps, Cloud, SOA, SAP HANA) while continuing to serve your current mainframe-based workload processes.

When making the transition from TWS/z to Universal Automation Center, only the TWS/z controller (or other scheduler) instances need to be transitioned. The Universal Agent infrastructure remains the same.

Another important benefit of the approach is the possibility of parallel operations of TWS/z and Universal Automation Center. The Universal Agents can be connected to multiple schedulers at the same time. It is therefore possible to continue to execute some workflows using a TWS/z scheduler, while others are controlled by the Universal Automation Center. This phased transition is less risky than a “big-bang” approach.

Transition Steps

1. Extract transition data of each TWS/z instance: 

  • Definitions 
  • Workstations 
  • Special resources 
  • Calendars 
  • Variables 
  • ETT 
  • Periods/rules
  • JCL
  • Parameters

2. Load the extracts into the Stonebranch transition tool.

3. The Stonebranch transition tool identifies all relevant options and/or parameters required to execute the job and uses them to create a Universal Automation Center loadable XML file.

4. The XML file will be loaded via the Universal Automation Center GUI and the result validated. Manual adjustments can be made where needed.

Transition Approach

Stonebranch provides a proven methodology for the transition of workload management systems to the 21st-century workload automation engine, Universal Automation Center. This methodology has been developed and enhanced over the last decade and has been used by more than 200 clients worldwide for successful transitions from any scheduling engine to any scheduling engine. Over the years, the methodology has been optimized for the needs of Universal Automation Center and has been proven successful with transitions at ING Belgium and Achmea in the Netherlands, as well as with several customers in the U.S. such as Lands’ End, United Supermarkets and OPERS. 

The methodology, shown in Figure 10 focuses on the mitigation of risks and ensures adherence to the budget and time guidelines of the transition project. One of the cornerstones is the permanent transfer of knowledge during the complete project to ensure the client’s ability of to manage workloads independently once the transition is finalized and accepted.

Although from a time and budget perspective, a turnkey transition is usually preferred, this methodology allows for the adoption of the process to any customer’s specific needs and requirements.

STEP 1: Initiate & Setup

Environment setup

At the end of this phase, the project has been defined from an organizational perspective, and the required target architecture for the Universal Automation Center has been defined and set up. The Initiate & Setup step is divided in two phases. Phase 1 ensures that the environment is set up properly to cover the needs and requirements of the client; Phase 2 ensures that project team members are trained on the solution and are capable of using the Solution in all project relevant aspects. In addition, the security definitions have been defined, and the project team has developed a user- role- and permission concept.

Project team member training

In the second phase of Initiate & Setup, project team members are trained to ensure that they are proficient in all aspects of the software. Training is staggered to ensure that users are ready to assume their roles when their part of the project is operational.

STEP 2: Analyze

Although the environment is analyzed and the project scope is defined in the pre-sales stage, this phase is used to ensure that a specific, detailed analysis is done together with customer experts. External interfaces must be identified; change processes must be defined to make sure that changes are synchronized between the environments during the transition project. In addition, special effort will be made to identify all external interfaces, and solutions will be put in place to keep these interfaces up and running.

STEPS 3 and 4: Sample / Production Workload transition

In these phases, the final transition of the data is done, and all elements from the current scheduling solution are provided as objects in the Universal Automation Center Database. Depending on customer needs and requirements, a special “sample workload transition” can be executed, where a representative set of applications is run through the transition program, and the results are validates together with the customer. At the end of this phase, the Universal Automation Center Database is built to reflect a “1:1” mirror of the current scheduler database, executing all jobs at the same time, depending on the same events, with the same dependencies as they are scheduled today. Of course, certain adaptations are required to reflect different technologies, but the results of the execution are identical.

STEP 5: Validation of results

During the validation, the current schedule definitions and the newly built Universal Automation Center definitions are compared to make sure that all jobs or schedules are executed properly. Forecasting reports for daily, weekly monthly or yearly executions are created and compared, and potential application-specific requirements are verified.

STEP 6: Cut over & Go-live

Once the data has been validated and approved by the customer, and a date has been defined for the final cut over, this project phase makes sure that all requirements for an error-free and smooth go-live are met. Fallback scenarios are defined and implemented; the actual go-live process is defined and communicated; resources are defined and tasks assigned for the go-live. For the defined period, resources will be onsite and available 24×7 to make sure that questions are responded to quickly and support is available for operations and application.

STEP 7: Project Closure

In the Project Closure phase, all parties involved will finalize their relevant project documentation and go through the “lessons learned” sessions to make sure that future projects or project phases are adopted accordingly.

Proof Points and Key Benefits

Following are the main benefits of transitioning to the Universal Agent, as well as a list of several of our enterprise customers who use Stonebranch Universal Agent technology and Universal Automation Center.