Wireguard debugging

Notes

• even without a configured endpoint, wireguard remembers your ip.
  ping from remote end, then try from this end again
  (note that a stateful firewall will quickly discard this memory)

Connection Troubleshooting

# 1. confirm wireguard UDP port is accessible 
nc -uvz {remote-host} {wireguard-port} && echo "connection succeeded"

    # 1.a [fail] check source/dest firewalls (and any in between) for blocked packets
    sudo tcpdump udp -n -e -ttt -i pflog  # pf
    sudo journalctl --dmesg -f            # nftables

# 2. confirm wg-*.conf addresses/ports are defined, and correct

# 3. confirm dest's wg-*.conf has src listed as a peer

# 4. confirm src/dst firewalls not blocking ICMP traffic for your ping test

# 5. many programs (salt, for example) communicate failures using ICMP. 
#    accept icmp on your host and watch firewall logs.

# 6. check your input AND OUTPUT firewall rules.
#    Are you allowlisting the port you're trying to communicate on?

# 7. Is this the first time you're connecting to server, 
#    and do you not have 'Endpoint' configured on the host you are connecting to?
#    If not, you'll need to initiate the connection from your side for the first time.
#    Try rebooting non-host with firewall disabled, followed by a ping-test.
#    If this succeeds, restarting with firewall enabled afterwards should also work.

# 8. check wireguard logs for rejected connections (??)

# 9. restart both sides. sometimes everything is configured properly and it this just magically resolves it

Multiple wg Interfaces

jim salter blogged about having multiple wireguard interfaces on one host
his setup included PostUp/PostDown instructions that created routing table entries, and solidified connections with a ping
If you're having trouble, try this.

# NOTE: linux specific

[Interface]
   Address = 10.0.0.2/24
   PrivateKey = PRIVATE_KEY_FROM_CLIENT1
   # set up routing from server/wg0 to server/wg1
   PostUp = route add -net 10.0.1.0/24 gw 10.0.0.1 ; ping -c1 10.0.0.1  # <-- (this)
   PostDown = route delete -net 10.0.1.0/24 gw 10.0.0.1                 # <-- (this to)
   SaveConfig = false

While I suspect my issue had to do with routing (perhaps changed routes?)
after several hours of debugging, restarting both sides magically fixed it.
I have now encountered this problem twice, and solved it the same way both times.
I tend to run into this when creating new jails.