SDPT Lab 8

De la WikiLabs
Jump to navigationJump to search

Week 8 Lab Activity: Networking, Security, and VPNs

Objective

Bring up two VMs that you fully control --- one acting as the server, one as an ARM-based edge device --- and use them to practice the techniques from this week's lecture: a raw TCP echo, an HTTP/REST exchange, a TLS-protected version of the same, and a three-peer WireGuard mesh that stitches your laptop and both VMs into one private encrypted overlay network.

This lab is allowed to run longer than your usual 1h50min. Plan for roughly 2h30min if you have not used QEMU before.

Background

Why two VMs

Your capstone has two physical machines (a Pi and a server VM). For this lab you do not have a Pi, so you will emulate one with QEMU running an ARM64 Debian image. This has two pedagogical benefits: you practice exactly the cross-architecture work that production embedded developers do, and you get a self-contained environment that does not depend on lab hardware or shared infrastructure.

Choosing your virtualization stack

You have several options for running VMs on your laptop. Each has tradeoffs that matter for this lab.

VirtualBox + QEMU mixed. The most common student instinct is to use VirtualBox (familiar GUI) for the x86 VM and QEMU for the ARM VM. This is the option to avoid. On Windows, VirtualBox 7.x technically coexists with QEMU's WHPX accelerator, but the two compete for the same hypervisor primitives and you will see strange failures (slow boots, network glitches, occasional VM crashes) that are extremely hard to diagnose mid-lab.

KVM (Linux native). If your laptop runs Linux, KVM is built into the kernel and gives you near-native VM speed for x86 guests. ARM guests still run under TCG (software emulation) since you are emulating a different architecture. This is the cleanest option if it is available to you.

QEMU on Windows with WHPX. QEMU runs natively on Windows. For x86 guests it can use Microsoft's WHPX accelerator, giving acceptable performance. But WHPX only accelerates same-architecture virtualization, so your ARM VM will boot in software emulation: functional, but slow (5--10 minutes to first login).

WSL2 on Windows. Run everything inside WSL2 and treat your machine as a Linux host. This works well, requires a one-time WSL2 setup, and lets you follow the Linux instructions verbatim. Best Windows option if you are willing to set up WSL2.

QEMU on macOS with HVF. QEMU on macOS can use the Hypervisor.framework accelerator (`-accel hvf`) for x86 guests. ARM guests use TCG. Apple Silicon Macs are an interesting case: the ARM VM is actually faster than the x86 VM, because ARM is the native architecture. Adjust the architecture flags accordingly.

Recommendation for this lab: QEMU for both VMs, on whatever host OS you have. One tool, one set of quirks, one mental model. The setup tutorial below uses QEMU exclusively.

Networking the two VMs

QEMU offers two relevant networking modes. User-mode networking (`-net user` or `-netdev user`) is the default: simple, requires no permissions, but the VMs cannot directly address each other --- they each see a private NAT'd network. Useful for outbound-only traffic (apt-get, ssh into the VM via port-forwarding), useless for "VM A talks to VM B".

TAP-based bridged networking (`-netdev tap`) gives each VM a real-looking interface on a software bridge on the host. The VMs can address each other directly using their bridge IPs. This is what we need. On Linux it requires a `bridge-utils` setup (15 minutes once); on Windows it requires installing the OpenVPN TAP driver and is fiddlier; on macOS it is well-supported.

For the lab, we will use a third hybrid option that works the same on all platforms: each VM uses user-mode networking to reach the internet, plus a second `socket` interface that connects the two VMs directly. This is QEMU's `-netdev socket` feature: one VM listens on a TCP port, the other connects to it, and to both guests it looks like a normal network interface. No bridge, no admin permissions, no platform-specific drivers.

The cpp-httplib library

Single-header HTTP server and client at https://github.com/yhirose/cpp-httplib. Drop `httplib.h` next to your code, `#include` it, you have HTTP. With `-DCPPHTTPLIB_OPENSSL_SUPPORT` and OpenSSL installed, the same library gives you HTTPS via `httplib::SSLServer` and `httplib::SSLClient`. For JSON, use https://github.com/nlohmann/json (also single-header).

WireGuard

WireGuard is in the Linux kernel since 5.6. Configuration is one short text file per peer. Each peer has a public/private keypair generated with `wg genkey | tee privatekey | wg pubkey > publickey`. Each peer's config lists the other peers' public keys and which IPs to expect from each. Useful references: `man wg`, `man wg-quick`, https://www.wireguard.com/quickstart/.

Wireshark

The standard packet inspection tool. Useful filters for this lab: `tcp.port == 9000`, `http`, `tls`, `udp.port == 51820`. Tip: to capture traffic between two QEMU VMs that talk over a `socket` interface, capture on the loopback interface of your host (since QEMU's socket netdev tunnels through host TCP).

Setup tutorial: bringing up the two VMs

Allocate roughly 20--30 minutes for this section. Read through it once before typing.

Step S1: install QEMU

  • Linux (Debian/Ubuntu): `sudo apt install qemu-system-x86 qemu-system-arm qemu-utils ovmf qemu-efi-aarch64`
  • Linux (Fedora/RHEL): `sudo dnf install qemu-system-x86 qemu-system-aarch64 qemu-img edk2-ovmf edk2-aarch64`
  • macOS (Homebrew): `brew install qemu`
  • Windows: download the installer from https://www.qemu.org/download/#windows and add the install directory to your PATH. Verify with `qemu-system-x86_64 --version` in a fresh terminal.

You also need OpenSSL (for the TLS exercise) and Wireshark on your host. Install via your package manager or https://www.wireshark.org/.

Step S2: download the two cloud images

We use Debian's generic cloud images because they are designed for virtualization and boot quickly. Both are ~600 MB.

Save both to a working directory, e.g. `~/lab8/`.

Step S3: create a cloud-init seed image (one for each VM)

The cloud images expect a tiny "seed" disk on first boot that tells them the hostname, the user account, and an SSH key. Create two text files:

`user-data-server`: 

#cloud-config
hostname: lab8-server
users:
  - name: lab
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    lock_passwd: false
    plain_text_passwd: lab
    ssh_authorized_keys:
      - ssh-ed25519 AAAA... your-public-key-here
ssh_pwauth: true
chpasswd:
  expire: false

`user-data-edge` is identical except `hostname: lab8-edge`.

Also create a one-line `meta-data` file: 

instance-id: iid-local-01

Build the seed images with `cloud-localds` (Linux, comes with `cloud-image-utils`) or with `genisoimage`/`mkisofs`: 

cloud-localds seed-server.iso user-data-server meta-data
cloud-localds seed-edge.iso user-data-edge meta-data

If you do not have `cloud-localds`, the equivalent `genisoimage` invocation is: 

genisoimage -output seed-server.iso -volid cidata -joliet -rock \
            user-data-server meta-data

Step S4: resize the disk images

Default cloud images are tiny (~2 GB) and will fill up fast. Grow them to 8 GB each: 

qemu-img resize debian-12-genericcloud-amd64.qcow2 8G
qemu-img resize debian-12-genericcloud-arm64.qcow2 8G

Step S5: boot the x86 server VM

Open a terminal and start the server: 

qemu-system-x86_64 \
  -name lab8-server \
  -machine q35 -accel kvm -accel whpx -accel hvf -accel tcg \
  -cpu max -m 1024 -smp 2 \
  -drive file=debian-12-genericcloud-amd64.qcow2,if=virtio \
  -drive file=seed-server.iso,if=virtio,format=raw \
  -netdev user,id=net0,hostfwd=tcp::2222-:22 \
  -device virtio-net-pci,netdev=net0 \
  -netdev socket,id=link,listen=:10000 \
  -device virtio-net-pci,netdev=link,mac=52:54:00:12:34:56 \
  -nographic

Notes on this command line:

  • The `-accel kvm -accel whpx -accel hvf -accel tcg` chain tells QEMU to try the fastest accelerator available on whatever host OS you are running, falling back to TCG. You only get one of them; QEMU picks whichever works.
  • The first `-netdev user` is for outbound traffic (apt, etc.) and forwards host port 2222 to the guest's SSH.
  • The second `-netdev socket,listen=:10000` is the inter-VM link --- this VM listens for the other VM to connect.
  • `-nographic` disables the QEMU window and uses your terminal as the console. To exit QEMU: `Ctrl-A` then `X`.

First boot takes 30--90 seconds. Wait for the login prompt and log in with user `lab`, password `lab`.

In a second terminal, you can also SSH in: 

ssh -p 2222 lab@localhost

Step S6: boot the ARM edge VM

In another terminal: 

qemu-system-aarch64 \
  -name lab8-edge \
  -machine virt -cpu max -m 1024 -smp 2 \
  -bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd \
  -drive file=debian-12-genericcloud-arm64.qcow2,if=virtio \
  -drive file=seed-edge.iso,if=virtio,format=raw \
  -netdev user,id=net0,hostfwd=tcp::2223-:22 \
  -device virtio-net-pci,netdev=net0 \
  -netdev socket,id=link,connect=:10000 \
  -device virtio-net-pci,netdev=link,mac=52:54:00:12:34:57 \
  -nographic

Notes:

  • `-machine virt` is QEMU's generic ARM platform.
  • `-bios` points to the UEFI firmware. The path varies by distro: on Fedora/RHEL it is `/usr/share/edk2/aarch64/QEMU_EFI.fd`; on macOS Homebrew it is `$(brew --prefix qemu)/share/qemu/edk2-aarch64-code.fd`; on Windows it ships in the QEMU install directory.
  • The second `-netdev socket,connect=:10000` connects into the server VM's listening socket. Start the server first, then start the edge.
  • SSH in via `ssh -p 2223 lab@localhost`.

ARM-on-x86 emulation is slow. Boot may take 5--10 minutes on first run. Be patient. On Apple Silicon hosts, ARM is native and boot is fast.

Step S7: configure the inter-VM link

The `socket` netdev gives each VM a second NIC, but the kernel does not auto-configure it. Inside each VM, find the second interface (probably `enp0s3` or `ens4`) and assign it a static IP on a private subnet:

On the server VM: 

sudo ip link set ens4 up
sudo ip addr add 172.30.0.1/24 dev ens4

On the edge VM: 

sudo ip link set ens4 up
sudo ip addr add 172.30.0.2/24 dev ens4

(Substitute the actual interface name. `ip a` shows it.)

Verify with `ping 172.30.0.2` from the server. If you see replies, the inter-VM link is up. If not, double-check that you started the server before the edge, and that both `socket` netdevs use the same port number.

Step S8: install lab dependencies on both VMs

Run on each VM: 

sudo apt update
sudo apt install -y build-essential cmake git \
                    libssl-dev nlohmann-json3-dev \
                    wireguard-tools tcpdump curl

For cpp-httplib, just download the single header into your working directory: 

mkdir -p ~/lab8 && cd ~/lab8
wget https://raw.githubusercontent.com/yhirose/cpp-httplib/master/httplib.h

You are now ready for the lab proper.

Tasks

Work in your `~/lab8` directory. Keep all the small programs you write --- you will submit them.

  1. TCP echo, raw sockets. Write two C++ programs: a server that listens on TCP port 9000 and echoes back whatever it receives, and a client that connects, sends a short message read from `argv[1]`, prints the reply, and disconnects. Use POSIX sockets directly (`socket`, `bind`, `listen`, `accept`, `read`, `write`). Run the server on the server VM (binding to `172.30.0.1`); run the client from the edge VM, sending to `172.30.0.1:9000`. Verify the round trip works. Skim the data with `sudo tcpdump -i ens4 -A port 9000` on either VM during the exchange.
  1. HTTP/REST with cpp-httplib. Replace your TCP echo with an HTTP equivalent. The server exposes one POST endpoint, `/access`, which expects a JSON body of the form `{"card_id": "..."}` and replies with `{"granted": true}` if the card ID is in a hardcoded allow-list of your choice (at least three IDs), otherwise `{"granted": false}`. The client constructs the JSON, POSTs it, parses the reply, and prints `GRANTED` or `DENIED`. Use `nlohmann/json` for both encoding and decoding. Test from the command line first using `curl` against your server, before moving to the C++ client.
  1. Inspect the wire. Open Wireshark on the host, capturing on the loopback interface (since the QEMU socket link tunnels through host loopback). Trigger an HTTP request from your client. Find the request and the response in the capture. Confirm that you can read the card ID, the JSON body, and the headers in plaintext. Save a screenshot or the raw capture for your writeup.
  1. Add TLS. Generate a self-signed server certificate and matching key with `openssl`. Recompile your server program against the SSL-enabled cpp-httplib by defining `CPPHTTPLIB_OPENSSL_SUPPORT` and linking `-lssl -lcrypto`. Switch `httplib::Server` to `httplib::SSLServer`, passing your cert and key paths. Switch the client to `httplib::SSLClient` and configure it to trust your self-signed cert (or, for testing only, disable verification). Get the same `/access` endpoint working over HTTPS on port 9443. Note: search the cpp-httplib README for the exact constructor signatures --- they are stable but worth confirming against the version you downloaded.
  1. Inspect the wire again. Capture the HTTPS exchange in Wireshark. Confirm that the JSON body and headers are no longer visible: you should see the TLS handshake (the `Client Hello`, `Server Hello`, `Certificate` messages should be readable as TLS metadata) followed by encrypted application data records. Note in your writeup what is still visible to an eavesdropper despite TLS (hint: at least the destination IP, port, and the SNI hostname in the handshake).
  1. Add a simple authentication step. Extend your HTTPS server to require an `Authorization: Bearer <token>` header. Reject any request without a valid token with HTTP 401. Choose any token value, and pass it from the client. Verify with `curl --cacert ./cert.pem -H "Authorization: Bearer ..."` that wrong tokens get 401 and the right one gets through.
  1. Generate the WireGuard keys. On each of the three peers --- your laptop (host), the server VM, the edge VM --- generate a private/public keypair with `wg genkey | tee privatekey | wg pubkey > publickey`. Collect all three public keys somewhere you can copy from. Treat the private keys as you would treat passwords.
  1. Design the WireGuard topology. Pick virtual IPs for each peer on a private subnet (e.g. `10.66.0.1` for the host, `10.66.0.2` for the server VM, `10.66.0.3` for the edge VM). Decide which peer will act as the rendezvous --- the one with a stable, reachable address that the others connect to. The host is the natural choice here because both VMs can reach it. Pick a UDP port for WireGuard (51820 is conventional).
  1. Write the three configs and bring them up. Create `/etc/wireguard/wg0.conf` on each peer with the appropriate `[Interface]` and one `[Peer]` block per other peer. The host's config has two `[Peer]` entries (one for each VM). Each VM's config has one `[Peer]` entry pointing at the host. Bring each interface up with `sudo wg-quick up wg0`, verify with `sudo wg show`. From inside the server VM, ping `10.66.0.3` (the edge); from the edge VM, ping `10.66.0.2` (the server). Both should succeed even though no direct socket netdev connects them.
  1. Re-run the HTTPS exercise over WireGuard. Stop your HTTPS server, restart it bound to the WireGuard IP `10.66.0.2` instead of `172.30.0.1`. Point the client at `https://10.66.0.2:9443/access`. Verify it still works. Capture the host's WireGuard UDP port in Wireshark this time --- you should see only encrypted UDP packets, with no visible TLS handshake or application protocol. Two layers of encryption, one capture: this is defense in depth.
  1. Short writeup. In a `WRITEUP.md` file at the root of your submission, answer:
    1. Which transport (raw TCP, HTTP, MQTT) would you use for the capstone, and why?
    2. What is still visible to an eavesdropper of HTTPS-without-VPN traffic?
    3. What is still visible to an eavesdropper of HTTPS-over-WireGuard traffic?
    4. Was the inter-VM connection for this lab actually NAT'd? Why or why not? (Look at your QEMU command lines and think carefully.)
    5. What would change if you replaced the QEMU edge VM with a real Raspberry Pi on your home network?

Bonus tasks (optional)

Pick at most one. These earn no extra points but make a real difference to your project.

  • MQTT pub/sub. Install `mosquitto` (broker) on the server VM and `mosquitto-clients` on the edge. Use `mosquitto_pub` and `mosquitto_sub` to send a "card scanned" event from the edge to the server over topic `lab8/door/cards`. Then write a tiny C++ subscriber using `paho-mqtt-cpp`. Hint: the broker speaks plaintext on port 1883 by default; for TLS use 8883.
  • Mutual TLS. Extend your HTTPS exercise so the client also presents a certificate that the server verifies. This eliminates the need for the bearer token entirely, because possession of the private key proves identity.
  • Replay protection. Add a nonce or timestamp to each `/access` request and have the server reject replays. Demonstrate that replaying a captured request with `curl` gets rejected.
  • Persistent WireGuard. Configure `wg-quick` to start at boot on both VMs, so your tunnel survives a reboot.

Deliverables

Submit a single archive (zip or tar) containing:

  1. Source code for all programs you wrote: TCP echo client/server, HTTP server/client, HTTPS variant, plus any bonus code.
  2. A `Makefile` or `CMakeLists.txt` that builds them. Pick whichever you prefer.
  3. All three WireGuard config files (with the private keys redacted), one per peer, named clearly.
  4. Two Wireshark capture screenshots: one of plaintext HTTP showing the JSON, one of HTTPS-over-WireGuard showing only encrypted UDP.
  5. `WRITEUP.md` answering the questions in Task 11.

Common pitfalls

The ARM VM takes forever to boot. That is normal on x86 hosts without same-arch acceleration. Do not kill it; first boot is the slowest because cloud-init runs.

The `socket` netdev does not connect the two VMs. Most common cause: starting the edge VM before the server VM, so there is nothing for the edge's `connect=:10000` to connect to. Start the server first.

`ping 172.30.0.2` works but my TCP server is not reachable. Almost always a firewall on the server VM. Either disable the firewall (`sudo ufw disable` if active) or open the port.

Wireshark shows no traffic on loopback. On Linux you may need to configure permissions: `sudo dpkg-reconfigure wireshark-common` and add yourself to the `wireshark` group, then log out and back in. On macOS, ChmodBPF must be installed (the Wireshark installer prompts for this).

cpp-httplib does not compile. Make sure you are using a C++17-or-later compiler (`-std=c++17`). The header is large and the first compilation takes 10+ seconds.

TLS verification fails on the client. Self-signed certificates are not trusted by default. You either pass the cert path explicitly (`set_ca_cert_path`) or, for testing only, disable verification (`enable_server_certificate_verification(false)`). The lab grading does not require a properly trusted cert.

WireGuard handshake never completes. The most common cause is a typo in a public key. Public keys must match exactly between peers. Second most common: the rendezvous peer's UDP port is not actually reachable --- check host firewall settings and any `Endpoint` line that points at `localhost` vs an actual reachable IP.

My VM lost the IP address on the inter-VM link after a reboot. Expected --- `ip addr add` is not persistent. Either re-run it after each boot, or write a `/etc/systemd/network/` config to make it permanent.

Looking ahead

Next week we return to the CI pipeline with the third quality gate: dynamic analysis. You will run Valgrind, AddressSanitizer, and UndefinedBehaviorSanitizer against the Oven Controller, find bugs that static analysis missed, and add a sanitizer build to your CI pipeline.

Adus de la „http://wiki.dcae.pub.ro/index.php?title=SDPT_Lab_8&oldid=8342”