Network Cabling Guidelines - The Must-know For Every Low-Voltage Engineer

Let's first understand some common terminology used in data centers, although we often encounter these terms, we rarely delve into their meanings.

 

Internal network access switch, a 1U box-type network device equipped with 48 10G optical ports and 4 40G optical ports; 10G optical ports connect to server 10G ports using AOC cables, and 40G optical ports connect to the internal network core in the data center using MPO fiber; each TOR switch provides server access for two racks.

Server iLO access switch, also known as the management network access switch, a 1U box-type network device equipped with 48 10/100/1000M auto-sensing electrical ports and 1 1/10G optical port; 48 electrical ports connect to server Gigabit ports and network device MGMT ports; the optical port connects to the data center management network core using 10G multimode fiber; each iLO switch provides access for servers and network devices in two racks.

 

Core network device, a frame-type device with multiple 40G and 10G optical ports (configuration varies), 40G optical ports connect to the data center TOR using MPO fiber, and 40G optical ports connect to the data center core device using MPO fiber.

 

Core network device, a frame-type device with multiple 40G and 10G optical ports (configuration varies), 40G optical ports connect to the internal network core using MPO fiber, 10G optical ports connect to transmission (dedicated line) using 10G single-mode fiber, and 10G optical ports connect to unified access and exit devices using 10G multimode fiber.

 

Similar to NAT conversion devices, used for translating between public and private IP addresses, a single device is equipped with multiple 10G optical ports, 10G optical ports connect to the data center core using multimode fiber, and connect to the external network core; additionally, it connects to the session synchronization switch using 10G multimode fiber for session synchronization between different devices.

 

Core network device in the external network area, equipped with 40G and 10G optical ports, 10G optical ports connect to the unified access and exit, independent external network area access, and security devices using multimode fiber, and 40G optical ports connect to the external network boundary using MPO fiber.

 

Network device interconnected with ISPs, configured with 40G and 10G optical ports, 10G optical ports connect to ISP devices using single-mode fiber, and another 10G optical port connects to security devices using multimode fiber, 40G optical ports connect to the external network core using MPO fiber.

 

A fiber-like cable with integrated optical modules at both ends, used for connecting TOR switches to server 10G ports.

 

Cat6 cable or  Cat 6a copper cable with RJ45 connectors, used for connecting iLO access switches to server Gigabit ports and network device MGMT ports.

 

Within the same IDC and the same specification racks, the cabling method should be generally consistent, facilitating daily operations and maintenance. Core network devices, internal network access devices, management network access devices, and other network devices with different roles should have neat cabling, with optical fibers and network cables not blocking the air intake and exhaust of network devices, and should not be too long to the bottom of the rack. Labels on optical fibers and network cables should be clear. The method of inserting optical fibers and network cables on the front of network devices should be as consistent as possible, with optical fibers and network cables bound, and the power and network cables on the back of network devices should be neat and not cluttered, with strong and weak cables separated, giving a sense of overall neatness.

 

Core network devices in IDC, especially internal network core devices, have a large number of interconnections with TOR, so there will be many optical fibers. The optical fiber layout for core network devices must be neat, not crossing horizontally or vertically, and the aesthetic cabling inside the rack is one of the key points in network device cabling standards. Optical fiber layout diagram:

Optical fiber layout diagram

 

Wiring standards inside the core cabinets

 

Optical fibers are routed vertically from either the left or right side of the device, and the routing location should not block the air intake and exhaust of network devices. Optical fibers are bound with Velcro straps at intervals (note: white fixed straps should not be used), but not too tightly and with a bending radius between 100 to 130 degrees, typically around 110 degrees, allowing for easy loosening and tightening, with clear labels, and after plugging into the device port, the label should not be blocked, leaving some length for easy plugging and unplugging of optical fibers.

 

Internal network access switches are located at the top of the rack, with internal network TOR switches connecting to the internal network core using 40G MPO optical fibers and to servers using 10G-AOC cables. iLO switches connect to the management network core using 1G or 10G ports and to servers using Gigabit ports. Two racks in the data center share one internal network TOR and one iLO access switch (5-meter cables within the same rack, 8-meter cables across adjacent racks), with approximately 18 servers placed in each rack. Optical fiber and network cable layout diagram:

Optical fiber and network cable layout diagram:

 

Cables laid by internal network access switches are organized using cable managers, with every 4 network cables bound with straps on the cable manager, each cable labeled with a unique label, and the other end placed in the rack tray position. The length of optical fibers laid does not need to be too long, with a bending radius not too large, clear labels, and optical fibers connecting to different cores should preferably use different colored labels to distinguish.

 

Interconnection cables between network devices in the data center need to pass through cable trays (except within a single rack). Optical fiber layout diagram in cable trays:

Optical fiber layout diagram in cable trays:

 

As shown in the diagram, optical fibers and network cables are laid separately and neatly in the cable tray, bound with straps at intervals, maintaining aesthetics.

 

Inside the rack (ordinary server rack), there are mainly internal network AOC cables and iLO management network cables, as well as power cables. Since there will be many cables in a single rack, and each type of cable needs to be bound and laid on the inner side of the rack. As shown in the diagram:

 

Different cables are distinguished by different colors, such as internal network cables: blue, iLO cables: gray, power cables: black. The length of the reserved cables should not be too long, just enough to reach from one PDU to the other side, and too long cables will affect cable management. The cabling and binding requirements between two servers are as shown in the diagram: (due to electromagnetic interference from network cables, strong and weak cables should be separated, and power cables and network cables should be bound separately)

 

 

Since a single TOR has 4 MPO cables connecting to 4 internal network cores, labels are used to distinguish connections to different internal network cores, such as red, yellow, blue, and green corresponding to internal network cores 1, 2, 3, and 4; the label format is consistently "Rack ARack B#Which One", such as XX Room-02-01XX Room-01-08#1. By convention, Rack A is designated as the core rack, while Rack B is the non-core rack.

Whether it is network cables or optical fibers interconnecting special network devices, the labels are uniformly marked as "Rack ARack B#Which One", such as XX Room-02-08XX Room-01-08#1.

The front reflects the rack position and port information of the internal network access switch, and the back reflects the rack position information of the server; for example, front A1-4-J-17-24-Gi1/1, back J-17-01 (where A1-4 represents the room name, J refers to the J column, 017 refers to 17 racks, 24 represents the 24th tray position, Gi1/1 refers to the 1/1 port of the switch. The back indicates the rack position of the server).

A single rack has a height of 48U, with a server on every 2U space, i.e., every 2U has a tray, corresponding from bottom to top as 01, 02, 03...24, etc.

Since a single rack has 2 PDU power supplies, the 2 PDU are named as Path A and Path B, with power labels formatted as numbers followed by letters. For instance, 01A-24A, 01B-24B.