Week 3
Virtual Networking and Linux Networking Fundamentals
OPS3 - Virtualization and Cloud Infrastructure
Welcome to Week 3!
What You'll Learn This Week
1. Runtime vs. Persistent Networking (ip vs nmcli)
1.1 The Kernel Layer: ip (Runtime)
The ip command is part of the iproute2 suite. It communicates directly with the Linux Kernel via Netlink sockets. When you run an ip command, you are modifying the direct, in-memory state of the network stack.
- Speed: Changes happen instantly (microseconds).
- Scope: Changes affect the running system only.
- Risk: Changes are lost on reboot. If you restart the server, the kernel reloads, and your manual ip changes disappear.
Key Commands:
- Link Status: ip link show (Displays MTU, MAC addresses, and UP/DOWN state).
- IP Addresses: ip addr show (Displays IPv4/IPv6 addresses attached to links).
- Routing Table: ip route get 8.8.8.8 (Simulates a packet to 8.8.8.8 and tells you which interface and gateway would be used).
1.2 The Configuration Layer: nmcli (Persistent)
To make settings permanent, we need a daemon that reads config files and applies them at boot. In modern Linux (including RHEL/CentOS and many Debian setups), this is NetworkManager. The command-line tool for this is nmcli.
- Profiles: NetworkManager saves settings as "Connection Profiles" (saved in /etc/NetworkManager/system-connections/).
- Logic: It handles complex logic, like "connect to Wi-Fi X when available, otherwise fall back to Ethernet Y."
- Persistence: Changes made here are written to disk and survive reboots.
Hands-On Example:
Use nmcli to set a static IP, ensuring the server always boots with the same address.
Section 1 Checkpoint
Summary:
- ip commands are runtime only (lost on reboot).
- nmcli commands are persistent (saved to disk).
- Correct networking requires understanding both layers.
Reflection:
- Why did Linux move away from ifconfig?
- If you set an IP with ip addr add and then reboot, what happens?
Resources:
- Red Hat: ip Command Cheat Sheet
2. Network Namespaces (The "Containers" of Networking)
2.1 The "Tenant Problem"
Imagine you are hosting Amazon Web Services. Customer A wants a private network using the IP range 192.168.1.0/24. Customer B also wants to use 192.168.1.0/24. On a normal OS, you cannot have the same IP address twice. It would cause an IP conflict.
- To solve this, we use Namespaces.
- A Network Namespace is like a parallel universe for networking.
- It partitions the kernel's network structures.
- Inside a namespace, you have a completely independent set of interfaces, routes, and firewall rules.
- Customer A lives in Namespace A, and Customer B lives in Namespace B.
Figure 2.1: Visualizing Isolation of Network Resources within a Single Linux Kernel.
2.2 Hands-On Example
We will create a "sandbox" namespace to demonstrate this total isolation.
Section 2 Checkpoint
Summary:
- Namespaces isolate network stacks (interfaces, routes, firewalls).
- They solve the "Multi-Tenant" problem (overlapping IPs).
- They are the foundation of all Container technology.
Reflection:
- Can a process in Namespace A see the network traffic of Namespace B?
- How does this isolation improve security in a cloud environment?
Resources:
- Introduction to Linux Namespaces
3. Connecting the Dots: Veth Pairs and Bridges
3.1 The Virtual Cable (veth)
- You cannot plug a physical Ethernet cable into a software namespace.
- Instead, Linux provides the Virtual Ethernet (veth) device.
- A veth pair is always created as two connected interfaces—a "pipe." Packets sent into Interface A instantly arrive at Interface B, and vice versa.
- This allows us to tunnel traffic from the Global Host namespace into the isolated "sandbox" namespace.
Hands-On Example:
Connecting the Sandbox to the Host.
3.2 The Virtual Switch (bridge)
- Connecting two points with a veth is easy.
- But what if you have 50 Containers?
- You cannot create a mesh of cables between every single one.
- You need a device to connect them all to a central point.
- A Linux Bridge is a software implementation of a standard Ethernet Switch.
- MAC Learning: The bridge listens to incoming frames. If it sees a packet from MAC AA:BB:CC coming from Port 1, it records this in a table.
- Forwarding: If it later receives a packet destined for AA:BB:CC, it looks up the table and sends it only to Port 1. This reduces traffic congestion compared to a "Hub" which broadcasts everything.
How devices plug in:
- Containers: Use veth pairs (as discussed in 3.1).
- Virtual Machines: Use TAP Interfaces. A tap device simulates an Ethernet device at Layer 2. QEMU/KVM creates a tap interface (e.g., tap100i0) for the VM, and the other end connects to the Bridge. This allows the VM to send raw Ethernet frames onto the switch.
Example: Manually creating a TAP interface (What Proxmox does behind the scenes)
Hands-On Example: Connecting Two Namespaces (Red & Blue)
We will act as a "Virtual Switch" administrator. We want to connect two isolated namespaces so they can talk to each other.
Section 3 Checkpoint
Summary:
- veth pairs act as virtual cables connecting namespaces.
- Linux Bridges act as virtual switches, forwarding frames based on MAC addresses.
- We can build complex topologies entirely in software.
Reflection:
- If you forget to "plug" the veth into the bridge (master br0), what happens?
- Why do we need two ends to a veth pair?
Resources:
- Linux Bridge Command Help
4. Container Networking in Practice (Docker, Podman, LXC)
4.1 Docker (The Standard Bridge)
When you install Docker, it creates a Linux Bridge named docker0.
- The Bridge: docker0 (Usually 172.17.0.1/16).
- The Veth: Every time you run docker run, Docker creates a veth pair. One end sits on the host (plugged into docker0), the other sits inside the container.
- The Namespace: The container is just a process running inside a network namespace.
CLI Comparison:
| Action |
Manual Linux Command |
Docker Command |
| Create Net |
ip link add br0 type bridge |
docker network create mynet |
| Create NS |
ip netns add container1 |
docker run --name container1 ... |
| Connect |
ip link set veth master br0 |
Auto-connected to docker0 or custom net |
4.2 Podman (Rootless Networking)
Podman often manages containers without root privileges. Standard bridges require root. How does it work?
- slirp4netns: Podman uses a User-Space networking stack. It "slurps" traffic from the host interface and pipes it into the container namespace without touching the kernel's actual bridge settings.
- Performance: Slightly slower than Docker's bridge, but much more secure.
4.3 LXC (System Containers)
LXC is closer to Proxmox's approach. It typically uses lxcbr0.
- Unlike Docker, LXC networking is defined in a config file. By default, it connects to lxcbr0 and requests an IP via DHCP, just like a physical computer would.
Section 4 Checkpoint
Summary:
- Docker automates the creation of Bridges (docker0) and Veth pairs.
- Podman uses slirp4netns for rootless, secure networking.
- LXC behaves more like traditional VMs on the network.
Reflection:
- Why is "Rootless" networking considered more secure?
- How does docker network create simplify what we did in Section 3?
Resources:
- Docker Networking Overview
5. Advanced Linux Operations
5.1 Link Aggregation (Bonding/LACP)
Hardware fails. Cables get cut, SFP modules burn out, and switch ports die. If your Hypervisor is connected via a single cable, you have a Single Point of Failure (SPOF).
Bonding allows you to combine multiple physical interfaces (e.g., eno1 and eno2) into a single virtual interface (bond0).
- Mode 1 (Active-Backup): Only one cable is used. If it fails, the traffic flips to the second cable. Simple, but wastes 50% bandwidth.
- Mode 4 (802.3ad LACP): The Gold Standard. The server and switch communicate to balance traffic across both cables simultaneously. You get double the speed and redundancy.
Configuration:
5.2 IP Forwarding & Routing
A Linux server usually behaves like an End Host—it consumes packets sent to it and ignores the rest. However, in virtualization (specifically NAT Mode), the Linux host must act as a Router. It needs to accept packets from VMs and forward them to the Internet.
To do this, we must toggle a specialized kernel parameter: ip_forward. We also need to manipulate the Routing Table to tell the kernel where networks live.
Commands:
5.3 The Linux Firewall (iptables)
- Security in the cloud isn't just about an external firewall appliance.
- Every Linux kernel has a built-in firewall called Netfilter.
- We manipulate it using iptables.
- It processes packets in "Chains":
1.
- INPUT: Filters traffic destined for the localhost.
Hands-On Example: securing a host.
Modern Frontends: firewalld vs iptables
While iptables (and its successor nftables) is the low-level tool, managing thousands of rules manually is hard. Modern distros use "Frontend Controllers":
- RHEL/CentOS/Fedora: Use firewalld. It organizes rules into "Zones" (Public, Home, Work).
- Debian/Ubuntu: Often use ufw (Uncomplicated Firewall).
These tools write the underlying iptables/nftables rules for you.
5.4 Network Reconnaissance (ss & nmap)
Before you can analyze traffic, you often need to know what ports are open or who is listening.
Socket Statistics (ss)
ss is the modern replacement for the deprecated netstat. It allows you to see which processes on your server are listening for connections.
Network Mapper (nmap)
nmap is the industry standard for network discovery. Unlike ss (which looks locally), nmap allows you to scan remote servers to see what they are exposing.
5.5 Traffic Analysis (tcpdump)
When networking breaks, it often fails silently. A firewall drops a packet without an error message. A route sends traffic into a black hole. To fix this, we need Packet Capture.
tcpdump is the CLI version of Wireshark. It puts the network card into "Promiscuous Mode," allowing it to see every packet on the wire, not just those addressed to it.
Commands:
Section 5 Checkpoint
Summary:
- Bonding aggregates links for redundancy (Active/Backup) or speed (LACP).
- Routing allows a Linux host to act as a Gateway.
- IPTables controls traffic flow (Input, Output, Forward).
- SS/Nmap allow you to discover open ports and listening services.
- Tcpdump allows us to see the actual packets on the wire.
Reflection:
- If you have a Bond with 2 cables and one is cut, does the server go offline?
- Why do we need ip_forward=1 for NAT?
Resources:
Now that we have dismantled the Linux kernel concepts, we can look at how Proxmox Virtual Environment (PVE) uses them to manage VM networking. Proxmox does not invent its own networking stack; it "orchestrates" standard Linux tools (Bridges, OVS, IPTables) via a web interface.
6. Open vSwitch (OVS): The Cloud Switch
6.1 Architecture: Control Plane vs. Data Plane
- Unlike a physical dumb switch, OVS is split into two parts:
1.
- Userspace (Control Plane): ovs-vswitchd and ovsdb-server handle the database and logic.
- This is where you configure "Flows" and "Ports".
- 2.
- Kernel Module (Data Plane): This handles the actual packet switching at wire speed.
6.2 Installing OVS
Unlike the standard bridge, OVS is not always installed by default.
6.3 The Power of Flows
Standard bridges forward based on MAC addresses. OVS forwards based on Flow Rules. A flow rule matches a packet's header fields and performs an action.
Example Scenario:
You want to block a specific "Bad Neighbor" VM (IP 10.0.0.66) from sending traffic, but allow everything else.
- Linux Bridge: Cannot do this natively (needs complex iptables).
- OVS: One simple command.
6.4 SDN Controller Integration
OVS is designed to be controlled remotely by an SDN Controller (like OpenDaylight or ONOS). This allows a central brain to program the switches across the entire datacenter.
Section 6 Checkpoint
Summary:
- OVS is a programmable, carrier-grade virtual switch.
- It separates the Control Plane (Logic) from the Data Plane (Forwarding).
- It uses "Flows" instead of just MAC tables, allowing for complex logic (Drop IP X, Redirect Port Y).
Reflection:
- How does OVS differ from a standard Linux Bridge?
- What is the role of an SDN Controller?
Resources:
- Open vSwitch Crash Course
7. Proxmox Networking
7.1 Networking Modes
When configuring a VM's hardware, you choose how it connects to the bridge. We primarily see three patterns.
This is the standard configuration for 90% of deployments. The VM becomes a full peer on the physical network.
Packet Flow:
[VM eth0] -> [Host tap100i0] -> [Linux Bridge vmbr0] -> [Physical NIC eno1] -> [Physical Switch]
Detailed Configuration:
In /etc/network/interfaces, a Bridge looks like this:
Implication:
- DHCP: The VM sends a broadcast DISCOVER packet. It traverses the bridge, hits the physical wire, and reaches your office router. The router assigns an IP directly to the VM.
- Visibility: Every device on your LAN can ping the VM.
Sometimes you cannot get extra public IPs (e.g., in a data center giving you only 1 IP). You need to create a private network inside the host and share the single public IP.
Packet Flow:
[VM eth0] -> [Host tap100i0] -> [Private Bridge vmbr1] -> [Host Routing/NAT] -> [Physical NIC eno1] -> Internet
Detailed Configuration:
This mode requires Masquerading (Source NAT) in iptables.
VLANs allow multiple logical networks to share one physical cable. To do this, we insert a 4-byte "Tag" into the Ethernet Frame Header.
Processing Flow:
You can create VLAN interfaces on standard Linux without a bridge using the ip command. This creates a virtual interface that automatically tags/untags packets.
Egress (Host -> Switch): The packet leaves eno1 with the tag attached. The physical switch reads "20" and puts it in the correct broadcast domain.
VLAN Aware Bridge:
Modern Proxmox uses "VLAN Aware" bridges. Instead of creating vmbr0.10, vmbr0.20, etc., you toggle bridge-vlan-aware yes. The bridge then acts like a "Trunk Port", capable of carrying all VLANs simultaneously.
Section 7 Checkpoint
Summary:
- Proxmox uses standard Linux Bridges and IPTables underneath the GUI.
- Bridged Mode: VM is on the LAN. Simplest.
- NAT Mode: VM is hidden behind the Host. Requires Masquerading.
- VLANs: Isolate traffic on the same physical wire using Tags.
Reflection:
- If you are in a Data Center with only 1 Public IP, which mode MUST you use?
- How does a switch know which VLAN a packet belongs to?
Resources:
8. Additional Resources
9. Lab Exercises
Summary
Review the key concepts covered in this week's material
Questions?