Running pfsense, I was able to route my entire LAN subnet through a VPN. I have firewall and NAT rules that use an alias to filter outgoing connections to specific domains outside of the VPN gateway.
This works great. But here’s the problem. Wildcards are not supported within pfsense aliases, and therefore unless you know the specific subdomain for a service, there’s no way to reroute services that use rotating or load balancing subdomains.
Surely this is a big problem in large companies. I’m sure they utilize a paid solution to solve this problem.
Are there any solutions for self hosting that are FOSS or within pfsense?
Your first suggestion is a clever one.
I can imagine writing a small script on the host machine to listen for subdomains, forward them to pfsense to update the aliases, and possibly set them to expire after a few days for security reasons.
Surely something like this exists. How to find it…?
If you search for pfsense alias script, you’ll find some examples on updating aliases from a script, so you’ll only need to write the part that gets the hostnames. Since it sounds like the hostnames are unpredictable, it might be hard as the only way to get them on the fly is to listen for what hostnames are being resolved by clients on the LAN, probably by hooking into unbound or whatever. If you can share what the service is it would make it easier to determine if there’s a shortcut, like the example I gave where all the subdomains are always in the same CIDR and if one of the hostnames is predictable (or if the subdomains are always in the same CIDR as the main domain for example, then you can have the script just look up the main domain’s cidr). Another possibly easier alternative would be to find an API that lets you search the certificate transparency logs for the main domain which would reveal all subdomains that have SSL certificates. You could then just load all those subdomains into the alias and let pfsense look up the IPs.
I would investigate whether the IPs of each subdomain follow a pattern of a particular CIDR or unique ASN because reacting to DNS lookups in realtime will probably mean some lag between first request and the routing being updated, compared to a solution that’s able to proactively route all relevant CIDRs or all CIDRs assigned to an ASN.