Physical and Logical Architecture

This post continues from Home Networking 1.

Brief VLAN legend

The vlans we’re using for now are established like so:

Logical Layout

Recall from earlier we mentioned Router On a Stick, implemented thusly:

  ---
title: Logical VLAN connections
---
graph LR;
sfp((ISP/WAN));
switch;
ap(access point);
hypervisor;
firewall;
sfp-->|vlan/900|switch;
subgraph vlan/all;
switch<-->|vlan100,vlan/all|firewall;
end
subgraph vlan/other;
switch<-->|vlan100,vlanall|hypervisor
end
subgraph WiFi Networks;
ap<-->|vlan/1016,
vlan/1032,
vlan/100
|switch
end

ISP modem goes into the switch, which in turn goes to the firewall. The ISP modem is an SFP unit, so it goes into one of the switches SFP+ slots. From there, a different switch-inserted SFP+ module connects to an SFP+ module on the firewall, over 10GB optical cables. The optical cables we’re using are “LC UPC Duplex”, meaning there’s one dedicated line for outgoing, and one dedicated line for incoming. Things can, in theory, go very fast on the internal network.

Once network packets pass through the firewall, they go back out into the switch to their destination vlan device.

ISP packets come in on a port marked for vlan 900. This is a standard marking for ISP / inbound-WAN traffic. Once it gets to the firewall, it will be sent back through the switch with whatever vlan is appropriate, defined by rules we will discuss later.

Physical Layout

Because our vlans are traveling over a physical switch with physical ports, we can’t just define the vlans in the opnsense firewall, we also have to define them on the switch.

On our 24 Ethernet / 4 SFP port switch, we have roughly the following layout:

• Ports 1-12 (1/1 - 1/12 in FS switch parlance) (ethrenet) - untagged*, meaning that anything plugged into them is capable of being routed anywhere. This is done for management purposes, therefore anything plugged into this set of items should be for device and infrastructure management.
  • * “Untagged” in a vlan-aware system implies vlan/1. Don’t ask me why.
• Port 1 (1/1) - dedicated to the Managemnet interface of the Firewall. This provides us with direct access to the OPNSense control panel / CLI.
• Port 24 (reserved) - for future expansion, this is marked as being a Trunk (aka, capable of carrying multiple vlans) carrying other, yet to be defined vlans, which is meant to be connected to a yet-to-be-installed second switch. This is meant to allow us to drop in another switch onto th network if we run out of ports on our current one.
• Port 25 (SFP+/fiber) - the actual switch<->firewall traffic. This port is makred as being a trunk for every for every single vlan (EXCEPT vlan/100 management traffic) we define on the network. This is necessary because we want all traffic to flow through the firewall.
• Port 28 (SFP+/fiber) (1/28) - vlan900 for ISP traffic, and is where the ISP’s SFP+ WAN modem unit is plugged into.

Or, in table form:

The other ports will be filled in later

*the Access Point requires some “““flexibility””” in our system because it only has one port, meaning we have to combine our Management and Client traffic over one port using multiple VLANs on one port. Ideally we would have had two ports, one dedicated for management, and the other for client traffic. It gets a little tricky, and we will discuss this later.

The actual physical port diagram looks like this:

  ---
title: Physical Port Diagram
---
graph LR;
switch;
isp((ISP/WAN));
ap(access point);
hypervisor;
firewall;
switch-->|1|firewall;
switch-->|2|hypervisor;
switch-->|24|ap;
isp-->|28|switch;
switch-->|25|firewall;
switch-->|26|hypervisor;

As you can see, the AP only having one port breaks the pattern a little bit, but we can work around it.