Blog Post Multi-Cloud Data Transfer: How to Solve the Challenge

Multi-cloud data transfer within a hybrid IT environment is, in some ways, like a complex math problem: You need to solve for any number of variables before the multi-cloud equation can work the way your business needs it to. In that sense, a key variable in the multi-cloud equation is how efficiently and securely data moves between cloud service providers like AWS, Google Cloud and Azure.  

Many organizations have applications and their corresponding data in different cloud service providers. A real-world example is when an organization has its ERP application installed in Azure, while the data warehouse this application accesses is in Amazon S3. This scenario, or many others like it, are becoming common practice. 

Of course, multi-cloud environments create complexity. When workloads are deployed across private cloud/data centers, public clouds and multi-cloud Kubernetes, IT teams struggle to ensure security and operational governance. In addition, it is challenging to get a unified view into the overall health of this hybrid cloud setup.

The Historic Solve

Organizations typically use intermediate storage systems to connect applications to their data. That is, before the data goes from one cloud provider to another, it goes through an on-prem or cloud system that an organization must maintain.  

The problem is that intermediate storage does not provide a direct connection between applications in one provider and data in another provider. To return to the equation analogy, you can certainly solve the equation by slotting in intermediate storage as a variable, but it’s not an elegant (or efficient, or secure) solution. And as we all know, the elegance of a solution—whether in mathematics or IT infrastructure—counts for a lot.

A More Elegant Approach: Eliminate the Middle Man

At Stonebranch, we believe in breaking down silos, which is the genesis for our Inter-Cloud Data Transfer solution package, part of Universal Automation Center (UAC). 

The Inter-Cloud Data Transfer solution package eliminates the need for intermediate storage systems. With organizations running more and more applications in the public cloud, and storing more and more data there, bypassing intermediate storage can deliver significant efficiency gains. First and foremost, you no longer need to copy the data two times and double the number of data transfers needed.  

By allowing you to transfer data directly from one cloud bucket to another, you can connect the dots directly between cloud service providers. Thus, you elimiate redundant storage systems used in historic multi-cloud data transfer scenarios. Of course, this approach means less system management work for the IT team. But it also means increased data-transfer speeds, enhanced security and improved control and centralized views into your full data pipeline.

For Hybrid IT Environments, Including Multi-Cloud

The Inter-Cloud Data Transfer solution package was created by working side-by-side with our customers. These organizations approached us to discuss how they could use our embedded managed file transfer capabilities to move data between different cloud providers. This short-term need was to move to a 100% multi-cloud environment. The long-term strategic goal for these customers is not to own or operate any local servers. 

This new solution package can, however, support your company’s IT-environment strategy even if you’re not planning on moving to a 100% multi-cloud environment in the near or medium term. 

Most organizations today operate hybrid IT environments, with cloud, container and on-prem systems, and they will continue to do so for the foreseeable future. Therefore, if your workflows spread over multiple cloud and on-prem environments, this solution package connects them wherever they are. This approach eliminates the need for different on-prem automation and cloud automation solutions. We should note here that one of Universal Automation Center’s strongest differentiators is its ability to connect workflows and data across on-prem, private cloud and public cloud environments; unlike other solutions, UAC functions seamlessly wherever your data and applications reside.

There is also a strong future-proof aspect to Universal Automation Center. Chances are, your organization plans to move more of its applications and data to the cloud. Wherever your organization stands in its cloud journey, and whatever your roadmap to a multi-cloud environment, this UAC integration approach is designed to rapidly create strong and secure integrations with any application or platform, for whatever comes next. 

Now is a good time to see the solution package in action. Watch the nine-minute product demonstration video, below, and learn how to easily manage multi-cloud data transfers to, from and between any of the major cloud providers.

Features and Benefits

The Inter-Cloud Data Transfer solution package offers numerous advantages over cloud vendors’ web-storage interfaces. As a command-line program to manage files on cloud storage, this solution package supports numerous cloud storage products, including S3 object stores, business and consumer file storage services, as well as standard transfer protocols.

Among the solution package’s key features and benefits are: 

• Improved speed. As noted above, by eliminating the need to copy and transfer data twice, data is transferred directly from bucket to bucket. When transferring data within the same provider, speeds can approach those of a LAN. 
• Enhanced security. Web-storage interfaces provided by cloud vendors can’t match the security features, including user and credential management, built into the UAC. Moreover, in certain instances, the solution package allows you to bypass the Internet when transferring data between buckets in the same provider (at a high level, such transfers bypass the Internet when the buckets are in the same region).
• Simulated Workflows and Data Transfers. For scheduling teams, this new solution package offers compelling new features. By simulating file transfers before you run them, you can determine available bandwidth and available capacity on the target system. And by testing workflows before running them, you can be sure they will be executed properly and as expected.
• Preserving time stamps and verifying checksums. This solution package is especially valuable to the DataOps team for the visibility it gives into data transfers. The protocols in the Universal Data Mover already confirm that the data sent and the data received are the same. Additionally, you receive time stamps and have a dashboard that verifies checksums all in one place.
• Improved performance of business applications. This solution allows you to integrate multi-cloud data transfer tasks into any workflow. As an example, one of Stonebranch’s customers is already using this solution package directly in a consumer-facing application, which runs in OpenShift, to improve the speed and security of the application’s file-transfer functionality.

An Elegant, Efficient Solution

As demand for new digital products and services continues to grow, organizations expect innovation and efficiency from IT (Infrastructure and Operations in particular). When it comes to connecting applications and data located in different cloud service providers with a multi-cloud data transfer approach, Universal Automation Center's Inter-Cloud Data solution package delivers both efficiency and elegance. Click these links to find the Inter-cloud Data Transfer and Inter-cloud File Monitor extensions, which pair together for a complete cloud monitoring and transfer solution pack.  

In terms of efficiency, the solution package obviates the need for intermediate storage systems, which lightens the I&O team’s maintenance and oversight responsibilities while also streamlining the flow of data in multi-cloud environments. In terms of innovation—which we can equate with elegance in this case—the solution package helps to reduce the complexity of a multi-cloud environment while greatly improving the way data is transferred.