Scale Across: The Network Behind Distributed Training

1 00:00:00,467 --> 00:00:02,102 Hi, I’m Spencer Giacalone, 2 00:00:02,102 --> 00:00:05,939 Senior Director, Cloud AI Architecture at Ciena. 3 00:00:06,506 --> 00:00:10,844 Today we’re talking about Scale Across networking in AI clusters. 4 00:00:11,644 --> 00:00:15,048 Scale Across is the extension of lossless connectivity 5 00:00:15,048 --> 00:00:20,220 between campuses or regions for the purposes of training an AI model. 6 00:00:20,820 --> 00:00:22,122 In Scale Across environments, 7 00:00:22,122 --> 00:00:25,392 customers connect multiple training datacenter campuses 8 00:00:25,392 --> 00:00:28,895 over a massive coherent optical network and then run 9 00:00:28,895 --> 00:00:32,132 all the GPUs as a single distributed cluster. 10 00:00:32,799 --> 00:00:36,503 Scale Across exists because many modern GPU clusters 11 00:00:36,503 --> 00:00:40,673 need more power than there exists in a single datacenter campus. 12 00:00:41,107 --> 00:00:44,210 Because of this, customers are being forced to distribute 13 00:00:44,210 --> 00:00:47,080 their GPU clusters across different datacenter 14 00:00:47,080 --> 00:00:49,616 campuses or geographic regions. 15 00:00:50,183 --> 00:00:54,621 Scale Across networks are built to transport large, bursty workloads 16 00:00:54,621 --> 00:01:00,060 with no packet drops or jitter because both of these slow or stall training. 17 00:01:00,794 --> 00:01:04,764 Scale Across has two main components to prevent traffic loss. 18 00:01:04,964 --> 00:01:08,435 The first part deals with congestion management and load balancing 19 00:01:08,435 --> 00:01:10,336 at the IP layer and above. 20 00:01:10,837 --> 00:01:13,339 The second part is the coherent optical layer 21 00:01:13,339 --> 00:01:16,576 which provides enough bandwidth to contain the large, 22 00:01:16,576 --> 00:01:19,913 loss-sensitive bursts that AI training creates. 23 00:01:20,613 --> 00:01:23,516 The coherent optical layer is the foundation on which 24 00:01:23,516 --> 00:01:26,920 all other congestion management features are built, 25 00:01:27,020 --> 00:01:30,590 and that makes Ciena is an integral part of distributed training clusters. 26 00:01:30,890 --> 00:01:34,094 Scale Across has two distinct design patterns. 27 00:01:34,294 --> 00:01:36,529 The first is called “Near Scale Across”. 28 00:01:36,529 --> 00:01:40,633 In Near Scale Across, customers deploy as much capacity as possible. 29 00:01:40,800 --> 00:01:43,036 But they place campuses near enough 30 00:01:43,036 --> 00:01:46,773 that intermediate line amplification or ILAs aren’t needed. 31 00:01:47,073 --> 00:01:49,609 Deployment consists of hundreds of fiber pairs 32 00:01:49,609 --> 00:01:53,179 using coherent plugs and line systems at the terminal sites. 33 00:01:53,580 --> 00:01:56,483 Unlike traditional networks that grow incrementally, 34 00:01:56,483 --> 00:02:00,487 each fiber is fully filled with point-to-point C+L band capacity 35 00:02:00,487 --> 00:02:01,855 on day one. 36 00:02:02,055 --> 00:02:04,958 The second design pattern is “Far Scale Across”. 37 00:02:04,958 --> 00:02:08,361 In Far Scale Across, training data centers are distributed 38 00:02:08,361 --> 00:02:12,332 across a larger area to access more sites and more power. 39 00:02:12,832 --> 00:02:16,269 Far Scale Across designs use massive optical capacity, 40 00:02:16,269 --> 00:02:19,572 but at distances where ILAs are required. 41 00:02:20,507 --> 00:02:23,076 Because ILA huts are space and power limited, 42 00:02:23,176 --> 00:02:27,447 Far Scale Across makes multi-rail line systems a necessity. 43 00:02:28,414 --> 00:02:31,284 Without multi-rail, ILA hut constraints 44 00:02:31,284 --> 00:02:34,287 would block Far Scale Across projects. 45 00:02:34,687 --> 00:02:37,757 Why are clusters getting so big that Scale Across is needed? 46 00:02:38,324 --> 00:02:42,362 To improve frontier model performance, customers have several options: 47 00:02:42,896 --> 00:02:44,564 increase model efficiency, 48 00:02:44,564 --> 00:02:46,399 which has just led to more demand. 49 00:02:46,399 --> 00:02:49,068 Build larger, more complex models. 50 00:02:49,269 --> 00:02:51,971 Deploy more or more powerful hardware, 51 00:02:52,405 --> 00:02:53,806 or train longer. 52 00:02:54,607 --> 00:02:58,077 In practice, they’re doing all of these, and the result has been a sharp 53 00:02:58,077 --> 00:03:02,682 increase in power demand—forcing training infrastructure to be geo-distributed. 54 00:03:02,849 --> 00:03:05,552 Distributed training is more complex than training 55 00:03:05,552 --> 00:03:07,887 in a single campus or datacenter. 56 00:03:08,688 --> 00:03:11,024 Some complexity is due to the normal operational 57 00:03:11,024 --> 00:03:12,959 challenges of long-haul links. 58 00:03:13,660 --> 00:03:14,928 And some complexity is due 59 00:03:14,928 --> 00:03:18,231 to managing the loss sensitive bursty workload itself. 60 00:03:18,698 --> 00:03:23,036 Packet level congestion control is particularly challenging at distance. 61 00:03:23,036 --> 00:03:26,806 Over-tuning congestion management protocols can cause the network to stop 62 00:03:26,806 --> 00:03:28,208 sending traffic. 63 00:03:28,208 --> 00:03:31,477 Under-tuning, on the other hand, leads to packets drops. 64 00:03:31,477 --> 00:03:34,714 And either of these will stall or slow training. 65 00:03:35,582 --> 00:03:38,084 However, a simple way to derisk distributed 66 00:03:38,084 --> 00:03:41,354 training is through highly parallel coherent optical capacity. 67 00:03:41,754 --> 00:03:45,258 In summary, models are growing so large that clusters can no longer 68 00:03:45,291 --> 00:03:47,393 be contained in a single campus. 69 00:03:47,560 --> 00:03:52,298 Multi-campus clusters are now being deployed and connected with lossless connectivity. 70 00:03:52,632 --> 00:03:56,836 Highly parallel coherent capacity is a foundational part of Scale Across, 71 00:03:56,836 --> 00:03:59,305 and that’s what we here at Ciena do best. 72 00:03:59,839 --> 00:04:01,507 Ciena addresses Scale Across 73 00:04:01,507 --> 00:04:03,610 with WaveLogic coherent transceivers 74 00:04:03,610 --> 00:04:05,712 and RLS open line systems. 75 00:04:06,279 --> 00:04:09,048 And our new RLS Hyper-Rail is specifically designed 76 00:04:09,048 --> 00:04:11,150 for Far Scale Across applications. 77 00:04:11,618 --> 00:04:13,486 Contact Ciena to find our more.