19.4 Overlay Networks

Introduction

Overlay networks let you build a “virtual” network on top of an existing “underlay” network, without changing the physical or IP infrastructure underneath. In modern data centers, virtualization platforms, and clouds, overlays are essential for scaling, isolating tenants, and making networks more flexible than traditional VLAN based designs.

This chapter focuses on what is specific to overlay networks themselves. Concepts such as virtual switches, SDN control, and cloud VPCs are discussed in their own chapters, so here they will only be mentioned when they are directly relevant to understanding overlays.

Underlay vs Overlay

To understand overlays, it is helpful to separate two layers of networking:

The underlay is the real, physical or IP network that actually forwards packets. It includes physical links, physical routers, and switches, and the IP routing between them. The underlay knows about real IP addresses assigned to physical interfaces and usually runs standard routing protocols.

The overlay is a virtual network that uses the underlay as a transport. Endpoints in the overlay see each other as if they were directly connected at Layer 2 or Layer 3, even if they are physically scattered across different racks, buildings, or data centers. The overlay encapsulates its own traffic inside packets that the underlay can understand and forward.

You can picture the underlay as a road system between cities and the overlay as private tunnels that run between specific buildings in those cities. Cars move on the roads using normal traffic rules, but some companies build private tunnels that carry their own vehicles between offices. The roads do not need to know anything about the internal rules of those tunnels.

Encapsulation in Overlays

Overlay networks rely on encapsulation. A frame or packet from a virtual network is wrapped inside another header that belongs to the underlay. The outer header ensures the underlay can deliver traffic between overlay nodes, while the inner header preserves the original addresses and identifiers of the overlay.

A generic sequence looks like this:

1. A virtual machine or container sends traffic toward its default gateway or another endpoint in its virtual network.
2. A device that participates in the overlay, often called a tunnel endpoint, receives this traffic.
3. The tunnel endpoint adds an outer header, such as an IP header and sometimes a UDP header, and an overlay header that identifies which virtual network this traffic belongs to.
4. The underlay network forwards the packet based on the outer IP header.
5. At the remote tunnel endpoint, the outer headers are removed. The inner frame or packet is delivered to the destination as if it had been sent over a local network.

In textual form, an overlay packet looks like this:

The inner addresses can belong to entirely separate private networks, even if the underlay uses overlapping or different addressing. This separation is the basis of multi tenant isolation in many environments.

Tunnel Endpoints

The critical role in an overlay is played by tunnel endpoints. These are the devices that encapsulate and decapsulate traffic. You will meet various names for them in different technologies, such as VTEP, tunnel router, or NVGRE endpoint, but the basic idea is the same.

A tunnel endpoint connects one or more local segments of a virtual network to the wider overlay. On the local side, it might be a virtual switch port that connects to a VM, or a physical access port that connects to a server. On the overlay side, it has an underlay IP address which other endpoints use as a destination for encapsulated packets.

The tunnel endpoint must keep some kind of mapping between overlay addresses and the underlay locations. For example, to send traffic to a particular virtual machine, it needs to know which remote tunnel endpoint currently hosts that virtual machine. How these mappings are learned and distributed is a key difference between various overlay solutions and is often influenced by SDN controllers or orchestration systems.

Overlay Address Spaces and Isolation

Overlays allow you to create separate address spaces that coexist on the same underlay. Multiple tenants or applications can use the same private IP ranges such as 10.0.0.0/8 without conflict, because the overlay encapsulation keeps their traffic separate.

To distinguish one overlay network from another, an identifying field is used in the overlay header. In older VLAN based designs this role was played by the VLAN ID. Overlays use a similar principle but often with a much larger space of identifiers, which allows more virtual networks than traditional VLANs.

Some common properties of overlay identifiers are worth noting:

They uniquely identify a virtual network or segment.
They are carried inside the encapsulation header, not in the underlay header.
They can support large numbers of separate networks, often up to millions.

This capacity for large scale isolation is one of the main reasons overlays became popular in large virtualized data centers and public clouds.

Common Overlay Encapsulation Families

Several encapsulation formats are frequently used for overlay networks. They differ in header structure and sometimes in their assumptions about the environment, but from a beginner’s point of view they all follow the same basic idea: take an Ethernet or IP packet from a virtual network and send it across an IP underlay.

VXLAN

Virtual Extensible LAN, or VXLAN, is one of the most widely used overlay technologies in data centers and is also referenced by later topics like VXLAN in advanced networking. VXLAN encapsulates Layer 2 Ethernet frames inside UDP over IP.

A simplified view of a VXLAN packet is:

The key piece is the VXLAN Network Identifier, often simply called the VNI. It identifies the virtual network. The space of VNIs is much larger than the VLAN ID space, which makes VXLAN suitable for large multi tenant environments.

Because VXLAN uses UDP, it can benefit from existing IP and routing features in the underlay without requiring special protocol support. Hardware and software switches act as VXLAN tunnel endpoints and perform the necessary encapsulation and decapsulation.

Generic Routing Encapsulation (GRE)

GRE is a more general tunneling protocol that can carry many different types of payloads inside IP. It is often used to build simple point to point tunnels between routers. In an overlay context, GRE can be used to connect separate networks across an IP underlay while preserving original headers.

A GRE packet typically has the following structure:

GRE does not provide its own concept of a large virtual network identifier like VXLAN’s VNI. It is more generic and can be used for many purposes, including basic VPN style tunnels or links between virtual routers.

Other Encapsulations

Other encapsulation types such as NVGRE and GENEVE exist and are often associated with particular vendors or ecosystems. They follow the same basic pattern. They add specific fields in the overlay header that can carry identifiers and metadata for virtual networks and services, but the core idea remains encapsulation of one packet inside another.

Overlay Networks and Mobility

One practical advantage of overlays is the ability to move workloads without changing their IP or MAC addresses from the perspective of the workload. This is particularly valuable in environments with virtual machines or containers that can be migrated between physical hosts.

When a virtual machine moves to a different physical host, the tunnel endpoint hosting it changes. From the virtual machine’s point of view, it is still in the same virtual network and still uses the same IP address. From the overlay’s point of view, the mapping from that IP and MAC address to the underlay location is updated.

If the underlay is properly routed and any necessary overlay control information is updated quickly, the virtual machine can continue to communicate with minimal interruption. This decoupling of logical addressing from physical location is a core strength of overlay designs.

Scale and Design Considerations

Overlay networks affect how you design and operate large environments. Several points are especially relevant.

First, overlays shift complexity from the underlay to the edge, at the tunnel endpoints. The underlay can be kept relatively simple and focused on IP connectivity and equal cost multipath routing, while intelligence and policy live at the overlay edge.

Second, overlays depend on accurate and up to date mappings between overlay endpoints and their underlay locations. Older designs might rely on flooding and learning inside the overlay, which can limit scale. Newer approaches often use controllers or databases that distribute endpoint information more efficiently.

Third, overlays must coexist with the physical underlay in terms of performance. Encapsulation adds extra headers, which increases packet size. If the maximum transmission unit, or MTU, of the underlay is too small, this can lead to fragmentation or dropped packets. You will see MTU and fragmentation in more detail elsewhere, but in overlay design this is a particularly important practical issue.

Finally, overlays introduce new troubleshooting layers. Problems can occur in the underlay, in the overlay encapsulation, or in the virtual networks themselves. A clear separation between these layers, along with good monitoring, is critical for reliable operation.

Overlays in Cloud and Data Center Environments

Public clouds and modern data centers use overlays extensively. When you create a virtual network in a cloud platform, you are often creating a logical construct that is implemented as an overlay on top of the provider’s physical infrastructure.

From your perspective, you see isolated subnets, routing tables, and security rules between them. Internally, the provider uses overlay encapsulation between its network nodes so that many customers can share the same physical hardware without interfering with each other.

In private data centers, overlays are heavily used in conjunction with virtual switches and SDN controllers. Virtual machines or containers plug into virtual networks that stretch across racks or even across sites. Overlays provide the connectivity fabric that supports this illusion of a single, flat, flexible network, even though the underlying topology may be large, complex, and highly routed.

Summary

Overlay networks create virtual networks by encapsulating traffic inside an underlay network. Tunnel endpoints perform encapsulation and decapsulation, and an overlay identifier in the header separates one virtual network from another and allows many isolated networks to coexist. Common encapsulations such as VXLAN and GRE implement these ideas with different formats and capabilities.

By decoupling logical topology and addressing from the physical infrastructure, overlays enable large scale multi tenancy, workload mobility, and flexible design in data centers and clouds. At the same time, they introduce specific requirements for underlay design, endpoint mapping, and troubleshooting, which you must keep in mind when working with virtualized and cloud networking environments.