Photo by Fikri Rasyid on Unsplash

Photo by Fikri Rasyid on Unsplash

Network overlay with VXLAN

Safety Net

Article from ADMIN 71/2022
VXLAN addresses the need for overlay networks within virtualized data centers accommodating multiple tenants.

If high availability or load balancing is required on servers across geographically separated locations, many of these services require direct access over Layer 2. However, if the Layer 2 link, which is based on classic 802.1Q VLAN technology, is interrupted by a routed link in Layer 3, the required transparency is lost. A virtual extensible local area network (VXLAN) solves this problem by extending Layer 2 accessibility over the existing Layer 3 structure with an overlay network.


System administrators face the challenge of planning scalable networks while maintaining appropriate security and availability requirements. Some server systems need to be located in a redundant infrastructure on a subnet. Additionally, virtualized systems need to be capable of migration between multiple sites on the same subnet, whether in live operation or in a disaster recovery scenario. Moreover, today's networks need be able to meet the increasing demand for bandwidth.

In classic scenarios, the required security can usually be achieved by implementing virtual local area networks (VLANs) to reduce the size of broadcast domains in combination with firewall rules or static packet filters (access control lists, ACLs) on routers or Layer 3 switches. In some cases, routing virtualization implemented by virtual routing and forwarding (VRF) is used at the routing level to enable multiclient capability.

Layer 2 Limitations

Routed networks are typically used to ensure scalability and avoid Layer 2 loops. Layer 2-only networks use methods such as Spanning Tree defined in IEEE 802.1d, its extension Rapid Spanning Tree (IEEE 802.1w), or Multiple Spanning Tree (IEEE 802.1s). All three have the advantage of ensuring loop-free operation. Layer 2 loops overload the switches and can be complex to troubleshoot.

Freedom from loops is ensured for redundant links by path blocking, but this comes at the cost of compromising maximum throughput, because the maximum combined or load-distributed data rates cannot be fully leveraged in this way. Additionally, some switches impose restrictions on the number of spanning tree instances, which, in turn, limits scalability for methods that run one dedicated instance per VLAN, which is the case with Rapid Per-VLAN Spanning Tree.

Classical Network Architecture Limits

A routed design (Figure 1) addresses these issues – again, with different variants, such as a hierarchical three-layer model with core, distribution, and access switches. Routed access, in which VLANs are only implemented locally on the respective access switch or access switch stack, is increasingly being used on classic campus networks.

Figure 1: In the classic routed three-layer design, host A and host B must be on different subnets.

An alternative in data centers is the spine-leaf model, where central switches, the spines, are connected to the leaves by routed links, and the leaves are connected to the servers. These leaves are usually designed as top-of-rack switches. Normally, IT managers would limit the number of VLANs required to one top-of-rack switch at any time and route them on their own uplink levels to leverage the routing protocols' properties for redundancy and load balancing across multiple routed links. However, this setup results in VLANs being restricted to the rack only, which contradicts the requirement for flexibility in redundancy and virtualization solutions referred to earlier.

Many data center services still require direct Layer 2 connectivity. One example is VMware vMotion, which enables live migration of virtual machines from one hardware setup to another. This configuration would not be possible in routed designs in different racks because IP-based re-addressing is impractical in most cases.

On local networks with end users, this connectivity does not play a major role. After all, the host usually doesn't care about the IP subnet it resides on, as long as it can access the required services over its gateway without the firewalls blocking the attempt. However, data centers need more, so how can you ensure – despite routed designs – that redundant services in distributed data centers can communicate with each other over Layer 2 while allowing virtual machines to move between racks without IP address changes?

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • GENEVE network tunneling protocol
    LAN data transmission has evolved from the original IEEE 802.3 standard to virtual extensible LAN (VXLAN) technology and finally to today's Generic Network Virtualization Encapsulation (GENEVE) tunneling protocol, which offers improved flexibility and scalability, although it still faces some issues. We look at the three technologies and their areas of application.
  • Successful protocol analysis in modern network structures
    Virtual networks and server structures require additional mechanisms to ensure visibility of data streams. We show how to monitor and analyze network functions, even when virtualization is involved.
  • Software-defined networking in OpenStack with the Neutron module
    In classical network settings, software-defined networking (SDN) is a nice add-on, but in clouds, virtual networks are an essential part of the environment. OpenStack integrates SDN technology through the Neutron module.
  • Virtual networks with Hyper-V in Windows Server 2016
    Microsoft provides some interesting virtualization features in current and future versions of Windows Server. You can connect or isolate virtual machines, and Windows Server 2016 even supports virtual switches.
  • Layer 3 SDN
    Calico chooses an unusual approach for software-defined networking, relying on open standards like BGP. We look at the distinctions and advantages of Calico.
comments powered by Disqus
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs

Support Our Work

ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.

Learn More”>


		<div class=