Interesting NAT Issues When Running a Node in a Corporate Firewall System

This diatribe below might help some who are leveraging a more complex network infrastructure than your typical home network deployment.

Setting up an Allstar node is relatively simplistic but registration issues can be problematic.

As I was leveraging a small component of my company’s test infrastructure, complexity of the networking allows for flexibility. I was piggybacking off one of our VMware clusters to host my radio less node. Later this will be using DV Switch for linking into a DMR network.

This is an insight into my journey of registration.

Current system setup:
An Allstar ISO image was loaded into a VMware virtual machine with 10 GB disk / 2 GB RAM and one VMXnet3 network adapter.

The internet link is out via a pair of clustered Fortigate firewalls out through a dual homed separate ISP with BGP peering to both the ISPs. We have a /24 subnet.

The issues began when I couldn’t get the node registered up on the node status page.
No matter what I tried, I could not get the thing online.

All the usual things like the iax.conf and the rpt.conf files are checked with special attention to the registration statement for the node in the iax.conf.

Going into the Asterisk CLI by doing "asterisk -rvvv’ will allow you to do some debugs.
“iax2 show registry” within the CLI will hopefully show that the system is registered as below:

zl1sww-aslgwCLI> iax2 show registry
Host dnsmgr Username Perceived Refresh State
44.72.21.13:4569 Y <Public_IP of your ISP Address>:4569 60 Registered
== Refreshing DNS lookups.
[Apr 22 11:23:12] NOTICE[2720]: dnsmgr.c:178 dnsmgr_refresh: host ‘register.allstarlink.org’ changed from 44.72.21.13 to 178.32.160.85
zl1sww-aslgw
CLI>

So it shows it’s registered.

I tried to link to it from my home node…

[Apr 22 15:14:56] WARNING[983]: app_rpt.c:22875 rpt_exec: Reported node 47563 cannot be found!!
== Spawn extension (radio-secure, 51553, 1) exited non-zero on ‘IAX2/125.237.53.155:4569-5360’
– Hungup ‘IAX2/125.237.53.155:4569-5360’
zl1sww-aslgw*CLI>

This shows that this node doesn’t know how to get back to the incoming call.
This is likely caused by the /var/lib/asterisk/rpt_extnodes file not existing.
This file contains a list of node numbers and IP mappings for all the nodes registered on the system.
There is a process that uses wget to pull down the latest rpt_extnodes file every 5 minutes.

I was very perplexed as to why I couldn’t get registered on the status page or get the periodic updates of the rpt_extnodes file.

I loaded the TCP Dump utility on the node to see what traffic was going in and out to the registration site.

“sudo apt-get install tcpdump” will get the tcpdump program in the node.

Then run “tcpdump -vvv -i host 'register.allstarlink.org

What I saw was that I was getting forbidden off the registration site evident by the following:
“register-sea.allstarlink.org.http > 10.20.30.150.37128: Flags [P.], cksum 0x1dcc (correct), seq 1:204, ack 164, win 508, options [nop,nop,TS val 3829999163 ecr 43816047], length 203: HTTP, length: 203
HTTP/1.1 403 Forbidden
Server: nginx/1.17.5”

The key part here was being forbidden.

At this point I logged a support request and Marshall KE6PCV kindly responded to the request. At the same time we were talking together over Allstar from our own nodes.

Having mentioned what I was experiencing, we did a TeamViewer session so he could look for himself. All looked good.

What I gleaned from talking with Marshall, was that there are security mechanisms in place such like the expected IP address from which the registration request and the VOIP data stream on the selected port 4569/udp in this case, have to be coming from the same IP address. One stream source IP has to match the other stream. This is to stop spoofing.

This got me thinking, as we have a whole /24 internet presence, we have many public virtual IP addresses handling various inbound services.

By default, a firewall will NAT (Network Address Translate) out its base external public IP address.

What I had to do to fix the abovementioned issues was to create a NAT pool with a single egress IP address the same as the incoming Part Address Translation (PAT).

This was added to the policy and it all started working. Without the NAT pool, we were sending registration requests out the base IP address of the firewall and receiving inbound traffic via the inbound PAT rule. Allstar does not like asymmetric data paths.

Without knowledge of the security mechanisms in place use in the Allstar server farms, it would have been very tricky to find this issue.

Thanks to Marshall KE6PCV for explaining the security mechanisms at the registration site. It helped to understand what could be happening.

Most home users hosting nodes will not experience this as they will generally only get one public IP address which handles in and out traffic.

Cheers,
Simon - ZL1SWW.

This is not unique to Allstar. If you run an SMTP server and have reverse DNS configured, you better be coming outbound from your firewall on the same IP address as you have defined for the inbound connections. Many other services expect this as well.

I have a pool of 5 static IP’s at home and using a Ubiquiti Edgerouter, I have to map internal services to use specific outbound IP’s as well.

Yes agreed on SMTP and reverse lookups. Have to have the in and out coming from the same IP. A lot of things however are forgiving of having asymmetrical comms paths.

You are lucky having 5 static IPs at home! Can’t get that easily here in ZL, we only get one IP unless you are a corporate and want to pay the $$$ and even some ISPs here are doing CGNAT as well to make matters worse…

Cheers!