Networking
22 posts on Networking: design decisions and how-to guides from small infrastructure estates run on a very low budget.
- OPNsense: announce your own prefix with FRR BGP
The FRR plugin on an OPNsense pair is enough to announce a portable /24 to a transit provider. The settings that matter, the blackhole route without which nothing is advertised, strict filters both ways, and how to confirm the announcement from outside.
Announce your own /24 from an OPNsense pair: BGP without a router budgetProvider-assigned addresses lock you to a contract and turn every service IP into a migration project. A portable prefix, an ASN and FRR on OPNsense are enough. The decisions that made the first announcement boring.
- OPNsense: retire a public IP address without a blackhole
An alias nothing referenced was deleted during a cleanup and a routed /28 went dark for about 26 hours. The procedure that came out of it, from the inventory of references to the provider's written confirmation, dual-running the replacement, deleting on both nodes and probing from outside.
- OPNsense as the routing peer for a NetBird mesh
Install the NetBird agent on both firewalls of an HA pair, advertise one route per VLAN with the pair as routing peers, add the outbound NAT the agent's masquerade does not cover, filter on the overlay interface, and avoid the endpoint trap that drops large packets.
- Catalyst 3850: console access from a Linux jumphost and config backups
A USB serial adapter on the stack's console port, the operator in the dialout group, screen at 9600 8N1, and a small expect script that captures the running configuration into the repository. Plus the break-glass path and why management egress never goes over anyone else's WiFi.
Self-hosted NetBird as the operator plane: firewalls as routing peers, policies as transportOne small VM runs the control plane, with its database shipped to a warm standby every thirty seconds. The firewalls are the routing peers. NetBird policies are treated as transport; real authorisation lives at the service.
- OPNsense: fix HA config sync that stopped syncing
The backup looks healthy and is a month behind. How to tell, what actually triggers the XMLRPC sync, what never syncs at all, and a drift check you can run from a laptop every fifteen minutes.
OPNsense HA that actually fails over: VHID 1 everywhere, and the sync that is not automaticTwo firewalls are only a pair if the backup holds the master's config. The VHID numbering that scales past VLAN 255, the config sync that only runs when asked, and the drift audit that caught rules living on one node.
- Proxmox: SDN VLAN zones with an external IPAM
Two VLAN zones on the VLAN-aware bridge, a VNet per VLAN, and NetBox as the IPAM behind the tenant zone so the subnet Proxmox knows about is the one the address plan allocated. SDN does the plumbing; the firewall still does policy and NAT.
- Proxmox: a cluster API address that follows a healthy node
One address for the Proxmox API that moves to a node whose pveproxy actually answers, using keepalived VRRP with a track script. A node with a wedged API still answers ARP, so the check has to ask port 8006, not the kernel.
Ex-lease Dell Servers, one Ceph pool, no local-zfsSecond-hand servers are cheap. What costs money later is a storage decision that blocks migration. Here is the cluster layout that keeps HA, migration and backup working.
- OPNsense: split-horizon DNS with Unbound forward zones
Internal zones forwarded to the per-site authoritative server, public names overridden to internal addresses for internal clients, and DNS over TLS to the upstream. Plus the test that proves the public answer is right, which is not the one you run from your laptop.
- OPNsense: manage aliases, rules and NAT from Python
The OPNsense API is good enough to run an estate's firewall policy from a script, as long as the script reads back every write, marks what it owns, and remembers that the HA sync will not run on its own. A working pattern with requests.
- Catalyst 3850: upgrade IOS-XE on a stack in install mode
Confirm install mode, clean the flash, copy and verify the image, install it to every member in one command, reload with a console attached, and keep the old image on flash for the rollback you hope not to need.
- OPNsense: VLANs on a trunk from a Catalyst, with Kea DHCP
Adding a tenant VLAN to an OPNsense pair fed by a Catalyst trunk, from the switch port to the CARP gateway, the firewall rule order that keeps tenants apart, and a Kea DHCP pool for provisioning. Done twice, because there are two nodes.
- Catalyst 3850: port-channel is up but no traffic passes
LACP bundles, the MAC table has entries, and nothing gets through. The port is tagging the wrong way for its host class, and the switch has no reason to tell you.
- Catalyst 3850: cross-stack LACP to a Proxmox bond, both sides
One port on each stack member, a port-channel in LACP active mode, a trunk whose native VLAN does not exist, and an 802.3ad bond under a VLAN-aware bridge on the Proxmox node. Both halves, and how to check they agree.
- Catalyst 3850: trunks with a native VLAN that does not exist
Set every trunk's native VLAN to one you never create, so untagged frames vanish instead of landing in VLAN 1. Then carry the provider's WAN handoff through the stack as a tagged VLAN to both firewalls.
- Catalyst 3850: stack two switches and add a member safely
Cable the stack ring, set priorities in EXEC mode (not config mode), match the software before the new member joins, and power on one switch at a time. The traps are the priority command and auto-upgrade.
Second-hand Catalyst, first-class conventionsA used two-member switch stack gives you cross-stack LACP for the price of a prosumer switch. The value is in the conventions, not the hardware.
- OPNsense: port forward to a VM behind a CARP pair
A port forward on an HA pair has to land on the shared WAN address, not a node's own, and it has to exist on both nodes. The forward, its filter rule, why NAT reflection is the wrong fix for the inside test, and how to test from outside properly.
A VLAN scheme you can derive in your headSegmentation fails when nobody can remember it. Encode site, VLAN and role into the address so a firewall rule, a DHCP reservation and a MAC can all be derived from a tenant number.