- Data Pipeline Automation
Real-world IT teams are using IT automation and orchestration platforms to break down organizational silos, improve productivity and observability, and empower end-users with self-service automation capabilities.
Fortunately, we were lucky enough to have a few leading IT automation and data-focused teams join Stonebranch Online 2022 to share their stories. Keep reading to hear practical advice from Stonebranch customers with ING Group, Seattle Children’s Hospital, and Vermont Information Processing.
Data Pipeline Orchestration with Seattle Children’s Hospital and Vermont Information Processing
Seattle Children’s Hospital
Headquartered in Washington, Seattle Children’s Hospital (SCH) is repeatedly recognized as a top-ranked pediatric hospital in North America. They provide hope, care, and cures to help every child live the healthiest and most fulfilling life possible.
Three SCH team members — Robert Lamb, Grace Li, and Kishore Uppada — joined Stonebranch Online to discuss their experiences using UAC for Data Pipeline orchestration.
The UAC was brought into SCH by the big data platform group, which is responsible for ETL, BI, and developer functions. This group closely aligns with the platform services group and the big data team.
A primary goal of the project was to orchestrate the jobs that automate IBM DataStage, their ETL solution. Additionally, they are focused on automating additional parts of their data pipeline, which includes tools like Tableau, PowerBI, and Cloudify.
Self-service enablement is a big deal at Seattle Children’s Hospital. They have multiple data-focused teams that align with various hospital service groups. Each team manages various subject matter areas. As a result, making it easy for each team to see and access only what they need for their respective group role is critical. With over 150 unique groups and growing, SCH required an easy and scalable way to deliver access to the centralized UAC platform for each group.
For that, SCH takes advantage of Business Services within the UAC. Business Services is an access control feature that allows administrators to provide role-based security and permissions.
During their Stonebranch Online session, SCH shared a clever automated workflow, where they use the UAC to fully automate requests for access to the UAC for new groups. The process starts when an engineer submits a ticket in Jira. Once the Platform Services team checks and approves the request, automation picks up the ticket and creates the new business service, assigns the right roles and permissions, promotes it to production, and collects the data in a backend database for tracking.
Automating business services for SCH helps save the team time and allows them to focus on other initiatives that improve employee and patient experience.
Vermont Information Processing (VIP)
VIP creates software and processes data to help their customers in the beverage industry manage inventory, distribution, and sales. With a business model built on a foundation of data, it’s critical to have the right tools and processes in place to manage that data flow — and scale as the company continues to grow.
Helen Pampinella, chief data architect with VIP, shared her perspective on how Stonebranch helps them do just that.
Similar to Seattle Children’s data pipeline automation journey, Helen’s team started using UAC for DataStage to support ETL orchestration in 2020. Because UAC makes it easy to identify and root-cause issues, her team has re-created every job in DataStage within the UAC. In fact, during her session, Helen shared a beautiful workflow that maps directly to every single automated process in DataStage. Doing so offers observability to see what failed and where it failed, makes it easy to get to the right log data, and allows them to re-run or skip tasks at the touch of a button.
In addition to DataStage, Helen’s team focuses on orchestrating several other data tools along the data pipeline. VIP uses the UAC to automate processes in AWS EC2, IBM iSeries, Microsoft SQL Server, Snowflake DB, and a whole lot more.
For AWS EC2, she illustrated a workflow that makes it easy to quickly spin up cloud computing power on-demand from the UAC. Then the workflow spins AWS EC2 down when the developers are finished with whatever data modeling or processing they’re working on.
Today, VIP has added 19 teams to the UAC platform to automate and orchestrate processes throughout the organization.
Click here to watch the Stonebranch Online session on-demand: Customer Panel: How to Orchestrate a Data Pipeline with Seattle Children's Hospital and Vermont Information Processing (VIP)
ING Group: Self-Service Automation, Jobs-as-Code, and Infrastructure Orchestration
ING Group is a multinational banking and financial services corporation that serves around 38 million customers. ING's Patryk Urbańczyk and Mariusz Dybowski offered their strategy to provide self-service automation to over 500 internal ING groups, representing thousands of end-users across the global business.
ING uses UAC for workload automation across their entire global business. As a focus for their Stonebranch Online session, Patryk and Mariusz shared how they automate the onboarding process for all stakeholders, apply appropriate access controls and security, and enable developers with self-service access to the UAC.
ING has many groups in vastly different geographies that require access to the Universal Automation Center. Because different developers prefer different ways to create and manage workflows, ING provides a choice. Patryk and his team empower their end-users to either use the browser-based GUI within the UAC or to create jobs-as-code using an IDE.
Additionally, ING business users go to a self-service portal when they want a Universal Agent installed on their machines. From there, UAC automates the processes involved in setting up a unique domain with specific agents and objects that associate only with that domain.
In addition to self-service support, ING uses UAC as their infrastructure orchestration tool to provision a host system.
ING uses UAC’s broad integration capabilities to connect to multiple tools in the provisioning process, including VMware vCenter, Infoblox, Ansible, SCCM, and ServiceNow.
Click here to watch the Stonebranch Online session on-demand: How ING Empowers End-Users with Self-Service Automation
Curious to see how IT automation can drive business success for you? Check out the full Stonebranch Online virtual education series, an annual online event designed to help you evolve your IT automation and orchestration program.
Start Your Automation Initiative Now
Schedule a Live Demo with a Stonebranch Solution Expert