Do enterprises have to use ‘TruckerNet’ to migrate their data to the cloud?
The shift of enterprise IT to cloud based services continues to grow unabated. Enterprises are moving to the cloud not just due to lower costs but due to flexibility. The ability to scale up IT infrastructure during periods of peak demand and/or growth and pay for it on an on-demand basis is very attractive.
Public cloud services such as AWS, Microsoft Azure, and Google Cloud are growing collectively at a 43% CAGR. AWS continues to be the market leader, but Microsoft has emerged as a clear #2 and Google is also investing to expand its cloud service beyond SMB into the enterprise market.
Once an enterprise makes the decision to migrate their IT infrastructure to a public cloud service, they are faced with the task of migrating their base data to a public cloud provider’s datacenter. This initial migration can be a massive undertaking. Typically, only 10% of an enterprise’s data changes per day and that 10% needs to be backed up daily. But for the initial establishment of an enterprise into the cloud, 100% of their data must be migrated into the cloud service’s datacenter (or 10x the amount of a daily backup). This initial migration to the cloud can be described like a python snake trying to digest an alligator.
Figure 1: Migration of 80 TB of data to the cloud using Snowballs over ‘TruckerNet’
Several years ago, AWS introduced a large scale data transport solution called Snowball to facilitate the migration of this initial dataset to their datacenters. Each Snowball storage device contains between 50 and 80 TB of storage capacity. The customer uploads their data to one or more Snowballs via their local network and then ships the Snowballs via truck – let’s call this ‘TruckerNet’ – to an AWS datacenter where their data is then uploaded from the Snowballs to the AWS cloud (Figure 1).
Snowball was introduced to speed up the transfer time for initial migrations to the cloud. Several years ago, a 1 Gbps connection was typical for an enterprise WAN, and at that speed it would take 7.4 days to upload an 80 TB dataset to the cloud (as shown in Figure 2). With this length of time to migrate one’s data to the cloud, the Snowball approach made sense.
Figure 2: Transfer Time for Migration of 80 TB of data to the cloud using legacy 1 Gbps WAN
Now what cloud customers have not considered is the impact of their service provider delivering a Liquid Spectrum powered WAN service to the cloud.
Ciena’s Liquid Spectrum network solution changes the way optical WAN networks to the cloud are engineered, operated, and monetized to help operators realize the benefits of the Adaptive Network. Ciena’s Adaptive Network™ is built on three key foundational elements:
1. Programmable Infrastructure
2. Analytics and Intelligence
3. Software Control and Automation
Liquid Spectrum software applications allow network owners to get more value from their networks, so they can scale up WAN capacity dynamically to speed up applications such as initial data migration to the cloud.
Ciena was also the first to deliver a coherent modem that can tune optical capacity from 100G to 400G in 50G increments, enabling users to optimize capacity across any link in the network. Ciena’s Liquid Spectrum apps combine highly instrumented, programmable hardware with advanced software applications, allowing users to crank up bandwidth as they need it wherever system margin is available. By enabling the Adaptive Network with Liquid Spectrum, enterprises will have the ability to securely increase their WAN connections from 1 Gbps to say 100 Gbps for the amount of time they need to make their migration to the cloud. Think of the python expanding in width to swallow his lunch without much effort.
Figure 3: Enabling the Adaptive Network with Liquid Spectrum
As service providers evolve towards the Adaptive Network, with increased levels of automation and fewer manual processes, security is paramount to protect the exponential growth in data traffic over the network and to ensure that the right security measures are in place to detect and mitigate security breaches. Designed to ensure all in-flight data remains private and secure, Ciena’s WaveLogic Encryption solutions bulk encrypt all in-flight data, across high-capacity wavelengths, end-to-end with no increase in latency.
Figure 4: Inflight Optical Encryption
So, let’s go back to our cloud migration example. With the ability to expand WAN connection speed to the cloud by an additional 100 Gbps, our 80 TB data set can be transferred to the cloud in less than 2 hours and can be securely delivered with in flight optical encryption ensuring that an enterprise’s data is protected during the transfer. This clearly exceeds what limited security is available while one or more Snowballs are manually shipped to the cloud provider.
Figure 5: Transfer Time for Migration of 80 TB of data to the cloud using Liquid Spectrum Applications
Since AWS introduced their Snowball device a few years ago, the amount of data that enterprises are transferring to their cloud has increased by leaps and bounds, requiring many Snowballs for data migration. One of AWS’ customers requested a way to migrate several Petabytes (1,000 TB) of data to the cloud. AWS’s response was to step it up to a solution they call the “Snowmobile”. This is actually an 18-wheeler truck that is designed to transport up to 125 Snowballs at a time or 100 Petabytes of data.
Figure 6: Transfer Time for Migration of 10 Petabytes of data to the cloud using a 10 Gbps WAN
Let’s look at an example to transfer 10 Petabytes of data using a common WAN interface @ 10 Gbps. At this speed, it would take over 3 months of continuous transfer time to migrate this much data.
Figure 7: Transfer Time for Migration of 10 Petabytes of data to the cloud using a “Snowmobile”
With a 3 month plus migration time using a 10 Gbps WAN, it’s understandable why AWS introduced the Snowmobile approach. Note the need for one or more security guards on the Snowmobile to ensure that the data is secure. AWS even stated that their Snowmobile is “unbranded and painted white” to ensure that it remains inconspicuous and enhance its security.
Figure 8: Transfer Time for Migration of 10 Petabytes of data to the cloud using Liquid Spectrum Applications
The advantage of this approach is that WAN capacity can scale and increases as new coherent technologies become available. Now let’s revisit our 10 Petabyte example and apply Liquid Spectrum apps to next gen coherent technology such as WaveLogic 5 running at 800 Gbps. While a data transfer of this size can take months at 10 Gbps and many days using an unbranded 18-wheeler truck, it can be migrated in roughly one day using Liquid Spectrum applications with 800G wavelengths. No security guards required here with in flight optical encryption on this Adaptive Network.
In summary, the shift to the cloud is accelerating and the amount of data that enterprises must migrate is only increasing. As service providers power their WAN services with an Adaptive Network, they can enable their customers to scale their WAN connections to the cloud to speed up these cloud data migrations and eliminate the need for TruckerNet. Saves a lot of hard drives, windshield time and security guards.
Thank you. Your comment has been received and should appear on the blog shortly.