Forward traffic between two libvirt NAT bridges (firewalld nftables)

On RHEL with firewalld’s nftables backend, allowing one libvirt VM network to reach another is not a classic firewall-cmd --direct problem anymore. Libvirt installs its own ip libvirt_network table, and that is what rejects NEW traffic between bridges.

The problem

Two isolated libvirt NAT networks, for example:

• br-network-1 → 172.17.1.0/24
• br-network-2 → 172.17.2.0/24

Each VM can reach the host and the outside world (via NAT). Ping from one network to the other fails with Destination Port Unreachable from the host bridge IP.

firewalld may already show forward: yes on the libvirt zone. That is not enough.

Libvirt creates:

1
2
3
4

table ip libvirt_network
  chain guest_cross   # only accepts iif==oif (same bridge)
  chain guest_input   # NEW inbound to a bridge is REJECT
  chain guest_nat     # masquerade guest → elsewhere

Cross-bridge packets never match guest_cross, then hit guest_input and die. firewalld runs later (filter + 10), so policies / zone forward cannot save a packet already rejected in libvirt_network.

The approach

Keep the two bridges in the libvirt zone (libvirt will put them back anyway). Insert ACCEPT rules at the head of guest_cross. That alone is enough for packets to be forwarded.

Optionally, also skip masquerade between the two guest subnets in guest_nat. That is not required for connectivity: without it, ping still works, but libvirt’s NAT rules rewrite the source to the host IP on the egress bridge (172.17.2.1 instead of 172.17.1.3). Conntrack fixes the return path, so you get a working but SNAT’d conversation. The guest_nat bypass keeps real guest source IPs.

To make that durable and editable, four files (also packaged at github.com/tinsjourney/libvirt-bridge-forward):

1. Policy file — bridge-forward.conf

1
2

# <from_bridge>  <to_bridge>  <oneway|both>
br-network-1  br-network-2  both

• oneway — only from → to
• both — both directions

Edit this file, then restart (or reload) the systemd unit. Subnets for the NAT bypass are discovered from each bridge’s IPv4 kernel route on the host (172.17.1.0/24, 172.17.2.0/24, …).

2. Apply script — libvirt-bridge-forward

Installed as /usr/local/sbin/libvirt-bridge-forward.

Commands:

1
2
3

libvirt-bridge-forward apply    # flush managed rules, then apply conf
libvirt-bridge-forward flush    # remove managed rules only
libvirt-bridge-forward status   # show conf + current nft rules

On apply it roughly does:

1. Wait until ip libvirt_network chains guest_cross and guest_nat exist.

2. Delete previous rules tagged with comment prefix lbf-.

3. For each conf line, insert at the head of guest_cross:

   1	iif <from> oif <to> accept

4. And insert in guest_nat (optional for reachability, useful for real source IPs):

   1	ip saddr <from_cidr> ip daddr <to_cidr> return

   Without this, libvirt still masquerades 172.17.1.0/24 → !172.17.1.0/24, so traffic to the other guest net is SNATed via the host.

5. If direction is both, do the reverse pair as well.

Managed rules are identified by comments such as lbf-c-br-network-1-to-br-network-2, so re-apply is idempotent.

3. Systemd unit — libvirt-bridge-forward.service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

[Unit]
Description=Apply libvirt bridge-forward.conf (cross-bridge forwarding)
After=virtnetworkd.service libvirtd.service firewalld.service
Wants=virtnetworkd.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=/bin/sleep 1
ExecStart=/usr/local/sbin/libvirt-bridge-forward apply
ExecReload=/usr/local/sbin/libvirt-bridge-forward apply
ExecStop=/usr/local/sbin/libvirt-bridge-forward flush

[Install]
WantedBy=multi-user.target

Enable and use:

1
2
3
4
5
6

systemctl enable --now libvirt-bridge-forward.service

# after editing the conf:
systemctl restart libvirt-bridge-forward.service
# or:
systemctl reload libvirt-bridge-forward.service

RemainAfterExit=yes makes restart/reload meaningful for a oneshot service.

4. Libvirt network hook — /etc/libvirt/hooks/network

Whenever libvirt starts or restarts a virtual network, it rebuilds ip libvirt_network and your inserts disappear. The hook puts them back:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#!/bin/bash
NETWORK="$1"
OPERATION="$2"
SUBOP="$3"

case "${OPERATION}/${SUBOP}" in
  started/end|restarted/end) ;;
  *) exit 0 ;;
esac

/usr/local/sbin/libvirt-bridge-forward apply || true
exit 0

Make it executable (chmod +x) and give it the right SELinux type:

1
2

chmod +x /etc/libvirt/hooks/network
restorecon -vF /etc/libvirt/hooks/network

On RHEL with SELinux Enforcing, virtnetworkd still cannot run the hook until this boolean is on (otherwise virsh net-start fails with exit 126 / Permission denied, and autostart networks never come up after reboot):

1

setsebool -P virt_hooks_unconfined on

The hook runs the same apply path as the systemd unit, so the conf stays the single source of truth.

How a packet is allowed

Example: VM on br-network-1 (172.17.1.3) pings VM on br-network-2 (172.17.2.5).

1. Guest routes via 172.17.1.1 (host on the bridge).
2. Host forwards (net.ipv4.ip_forward=1).
3. Packet hits libvirt_network forward hook:
   • guest_cross: match iif br-network-1 oif br-network-2 → accept (our rule).
4. guest_nat: match saddr 172.17.1.0/24 daddr 172.17.2.0/24 → return (skip masquerade; omit this and the packet is still forwarded, but SNATed).
5. firewalld forward path for the libvirt zone already accepts intra-zone forward.

Without step 3, guest_input would REJECT the NEW flow toward br-network-2. Step 4 only preserves the real source address.

Also keep firewalld zone forward enabled:

1
2

firewall-cmd --permanent --zone=libvirt --add-forward
firewall-cmd --reload

Quick real example — one nft rule (one way)

To allow only br-network-1 → br-network-2 by hand (no conf, no service), insert this at the head of libvirt’s guest_cross chain:

1
2

nft insert rule ip libvirt_network guest_cross \
  iif "br-network-1" oif "br-network-2" counter accept

That single rule is enough for one-way forwarding. It is wiped when the libvirt network is restarted unless you re-add it (or use the four files above).

If you also want the destination VM to see the real source IP (no SNAT via the host), add:

1
2

nft insert rule ip libvirt_network guest_nat \
  ip saddr 172.17.1.0/24 ip daddr 172.17.2.0/24 counter return