K2CB
July 31, 2026, 2:36am
1
For the past few weeks, my hub node is failing. Symptoms are:
Allmon show other previously connected nodes now in a "CONNECTING" status
No new nodes can connect.
Numerous error messages appear in the messages.log file:
[2026-07-26 00:08:52.291] Asterisk 22.9.0+asl3-3.9.3-1.deb13 built by builder @ allstarlink.org on a x86_64 running Linux on 2026-05-28 18:41:49 UTC
[2026-07-26 00:08:52.792] WARNING[56266][C-00000093] channel.c: Exceptionally long voice queue length (97 voice / 99 total) queuing to Announcer/IAXLink-0000125e;1
[2026-07-26 00:08:53.092] WARNING[15613][C-0000004d] channel.c: Exceptionally long voice queue length (97 voice / 99 total) queuing to Announcer/IAXLink-00000351;1
[2026-07-26 00:08:54.072] WARNING[56266][C-00000093] channel.c: Exceptionally long voice queue length (97 voice / 99 total) queuing to Announcer/IAXLink-0000125e;1
[2026-07-26 00:08:54.372] WARNING[15613][C-0000004d] channel.c: Exceptionally long voice queue length (97 voice / 99 total) queuing to Announcer/IAXLink-00000351;1
[
If I reboot the node, sometimes it will be fine for a day, other times only a few hours.
Any suggestions as to what may be causing this? I have not made any configuration changes in a long time. This seemed to start after updating to the latest version.
Current version info:
OS : Debian GNU/Linux 13 (trixie)
│ OS Kernel : 6.12.96+deb13-amd64
│
│ Asterisk : 22.9.0+asl3-3.9.3-1.deb13
│ ASL [app_rpt] : 3.9.3
│
│ Installed ASL packages :
│
│ Package Version
│ ============================== ==============================
│ allmon3 1.9.1-1.deb13
│ asl3 3.18.2-2.deb13
│ asl3-asterisk 2:22.9.0+asl3-3.9.3-1.deb13
│ asl3-asterisk-config 2:22.9.0+asl3-3.9.3-1.deb13
│ asl3-asterisk-modules 2:22.9.0+asl3-3.9.3-1.deb13
│ asl3-menu 1.18-1.deb13
│
Eric
K2CB
N8EI
July 31, 2026, 2:40pm
2
This is a known bug caused by network conditions relating to either misbehaving peer clients or internet port scans. We're hopeful the next release will fix this.
opened 05:59AM - 14 Aug 25 UTC
app_rpt
So, my ASL3 went off the rails tonight.
This is ASL3 (versions below), running … on a Pi 4. It has three nodes, two are `chan_voter` (435451/435452) and one is `chan_simpleusb` (1999).
Received an email from a member that the repeater had been stuck on transmit. "for a very long time". I checked Allmon3, and it appeared to not be able to connect to the manager, as nothing was being shown for the nodes.
Logged in via SSH, Asterisk was running. Connected to the console, and started seeing warnings from `channel.c`:
```
Connected to Asterisk 22.4.1+asl3-3.5.4-1.deb12 currently running on ve7rpt (pid = 2001)
[2025-08-13 22:18:32.866] WARNING[2033]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 398 total) queuing to IAX2/44.32.40.212:4569-4215
[2025-08-13 22:18:42.866] WARNING[2040]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 399 total) queuing to IAX2/44.32.40.212:4569-4215
[2025-08-13 22:18:52.866] WARNING[2039]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 400 total) queuing to IAX2/44.32.40.212:4569-4215
```
Did a `module reload` from the CLI, and got dumped back to my shell prompt. Reconnected to the Asterisk CLI to catch some of the messages:
```
Connected to Asterisk 22.4.1+asl3-3.5.4-1.deb12 currently running on ve7rpt (pid = 2001)
[2025-08-13 22:18:32.866] WARNING[2033]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 398 total) queuing to IAX2/44.32.40.212:4569-4215
[2025-08-13 22:18:42.866] WARNING[2040]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 399 total) queuing to IAX2/44.32.40.212:4569-4215
[2025-08-13 22:18:52.866] WARNING[2039]: channel.c:1105 __ast_queue_frame: Exceptionally long queue length (0 voice / 400 total) queuing to IAX2/44.32.40.212:4569-4215
```
Repeater stopped transmitting, and normal operations appeared to have resumed.
Start of event appears to be `[2025-08-13 21:27:04.109]`. I did the `module reload` apparently at `Aug 13 22:20:59 `.
Versions:
```
OS : Debian GNU/Linux 12 (bookworm)
OS Kernel : 6.12.34+rpt-rpi-v8
Asterisk : 22.4.1+asl3-3.5.4-1.deb12
ASL [app_rpt] : 3.5.4
Installed ASL packages :
Package Version
============================== ==============================
allmon3 1.5.1-1.deb12
asl3 3.8-1.deb
asl3-asterisk 2:22.4.1+asl3-3.5.4-1.deb12
asl3-asterisk-config 2:22.4.1+asl3-3.5.4-1.deb12
asl3-asterisk-modules 2:22.4.1+asl3-3.5.4-1.deb12
asl3-menu 1.14-1.deb12
```
Attached are the Asterisk log and journalctl log from the event period. No debugging symbols in the installed packages, so it looks like I can't get a backtrace.
[messages.log](https://github.com/user-attachments/files/21766841/messages.log)
[journalctl.log](https://github.com/user-attachments/files/21766840/journalctl.log)
opened 11:36AM - 30 Jul 26 UTC
I believe still this guy: https://github.com/AllStarLink/app_rpt/issues/307
Los… s of internet connection with an outbound connection:
```
[2026-07-26 12:48:17.358] WARNING[195171] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785084362&seqno=10374&nodes=T2324&apprptvers=3.10.0&apprptuptime=620914&totalkerchunks=1&totalkeyups=833&totaltxtime=12404&timeouts=1&totalexecdcommands=117&keyed=0&keytime=489722' failed with error: Failed to connect to stats.allstarlink.org port 80 after 134495 ms: Could not connect to server
[2026-07-26 12:51:09.009] WARNING[25175] chan_iax2.c: Resyncing the jb. last_delay 231, this delay 6203, threshold 1148, new offset -6266
[2026-07-26 12:51:32.859] WARNING[25179] chan_iax2.c: Resyncing the jb. last_delay 0, this delay -6150, threshold 1272, new offset -116
[2026-07-26 12:52:15.890] WARNING[25173] chan_iax2.c: Resyncing the jb. last_delay 0, this delay 13032, threshold 1272, new offset -13148
[2026-07-26 12:52:22.862] WARNING[195256] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785084722&seqno=10380&nodes=T2324&apprptvers=3.10.0&apprptuptime=621274&totalkerchunks=1&totalkeyups=835&totaltxtime=12418&timeouts=1&totalexecdcommands=117&keyed=0&keytime=490082' failed with error: Could not resolve host: stats.allstarlink.org
[2026-07-26 12:52:32.842] WARNING[25179] chan_iax2.c: Resyncing the jb. last_delay 0, this delay -13049, threshold 1272, new offset -99
[2026-07-26 12:54:01.192] WARNING[25182] chan_iax2.c: Max retries exceeded to host 44.15.66.4 on IAX2/44.15.66.4:4569-5561 (type = 6, subclass = 11, ts=70777733, seqno=216)
[2026-07-26 12:54:12.366] WARNING[195284] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785084842&seqno=10382&nodes=C2324&apprptvers=3.10.0&apprptuptime=621394&totalkerchunks=1&totalkeyups=835&totaltxtime=12418&timeouts=1&totalexecdcommands=117&keyed=0&keytime=490202' failed with error: Could not resolve host: stats.allstarlink.org
[2026-07-26 12:54:13.120] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:14.400] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:15.680] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:16.960] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:18.240] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:19.520] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:20.800] WARNING[25418] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-26 12:54:21.586] WARNING[25417] app_rpt.c: attempt_reconnect: cannot find node 2324
```
There are many of these:
```
[2026-07-26 21:31:25.772] WARNING[25177] chan_iax2.c: Max retries exceeded to host 44.15.66.4 on IAX2/44.15.66.4:4569-11995 (type = 6, subclass = 11, ts=469994, seqno=154)
```
"Maybe" we didn't actually hang up on the port? even though we are trying to reconnect (many of these as well):
```
[2026-07-28 01:08:08.661] WARNING[25417] app_rpt.c: attempt_reconnect: cannot find node 2324
[2026-07-28 01:08:18.754] NOTICE[25417] app_rpt.c: Reconnect Attempt to 2324 in progress
```
Finally, it all goes away when the internet returns + call succeeds.
A bit more concise version on a flaky internet link:
```
[2026-07-29 12:14:04.084] WARNING[247451] chan_iax2.c: Max retries exceeded to host 44.15.66.4 on IAX2/44.15.66.4:4569-207 (type = 7, subclass = 0, ts=58013327, seqno=249)
[2026-07-29 12:14:16.007] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:14:17.287] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:14:18.567] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:14:19.219] NOTICE[247534] app_rpt.c: Reconnect Attempt to 2324 in progress
[2026-07-29 12:15:40.224] WARNING[263407] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785341720&seqno=972&nodes=T2324&apprptvers=3.10.1&apprptuptime=58125&totalkerchunks=0&totalkeyups=152&totaltxtime=3588&timeouts=0&totalexecdcommands=9&keyed=0&keytime=58135' failed with error: Could not resolve host: stats.allstarlink.org
[2026-07-29 12:16:50.296] ERROR[247431] netsock2.c: getaddrinfo("register.allstarlink.org", "(null)", ...): Temporary failure in name resolution
[2026-07-29 12:16:50.296] WARNING[247431] acl.c: Unable to lookup 'register.allstarlink.org'
[2026-07-29 12:20:58.542] WARNING[247452] chan_iax2.c: Resyncing the jb. last_delay -428, this delay 7917, threshold 1168, new offset -7917
[2026-07-29 12:21:30.458] WARNING[263473] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785342080&seqno=978&nodes=T2324&apprptvers=3.10.1&apprptuptime=58485&totalkerchunks=0&totalkeyups=153&totaltxtime=3591&timeouts=0&totalexecdcommands=9&keyed=0&keytime=58495' failed with error: Could not resolve host: stats.allstarlink.org
[2026-07-29 12:21:49.933] WARNING[247451] chan_iax2.c: Resyncing the jb. last_delay 729, this delay -8609, threshold 1270, new offset 692
[2026-07-29 12:22:22.279] WARNING[247454] chan_iax2.c: Resyncing the jb. last_delay 0, this delay 2346, threshold 1270, new offset -1654
[2026-07-29 12:23:00.738] WARNING[247456] chan_iax2.c: Max retries exceeded to host 44.15.66.4 on IAX2/44.15.66.4:4569-5401 (type = 6, subclass = 11, ts=500000, seqno=163)
[2026-07-29 12:23:07.606] WARNING[247452] chan_iax2.c: Silently dropping frame without existent call number: 2063
[2026-07-29 12:23:12.667] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:13.947] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:15.227] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:16.507] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:17.787] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:19.067] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:20.347] WARNING[247535] channel.c: Exceptionally long voice queue length (97 voice / 97 total) queuing to Announcer/IAXLink-00000007;1
[2026-07-29 12:23:21.426] WARNING[247534] app_rpt.c: attempt_reconnect: cannot find node 2324
[2026-07-29 12:23:31.494] NOTICE[247534] app_rpt.c: Reconnect Attempt to 2324 in progress
[2026-07-29 12:24:36.752] WARNING[247453] chan_iax2.c: Resyncing the jb. last_delay -53, this delay 4724, threshold 1000, new offset -4724
[2026-07-29 12:25:01.939] WARNING[247457] chan_iax2.c: Resyncing the jb. last_delay 0, this delay -4813, threshold 1000, new offset 89
[2026-07-29 12:25:51.962] WARNING[263501] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785342331&seqno=983&nodes=T2324&apprptvers=3.10.1&apprptuptime=58736&totalkerchunks=0&totalkeyups=153&totaltxtime=3591&timeouts=0&totalexecdcommands=9&keyed=0&keytime=58746' failed with error: Could not resolve host: stats.allstarlink.org
[2026-07-29 12:26:11.552] WARNING[247448] chan_iax2.c: Resyncing the jb. last_delay 9, this delay 9613, threshold 1000, new offset -9524
[2026-07-29 12:26:31.952] WARNING[247453] chan_iax2.c: Resyncing the jb. last_delay 0, this delay -9600, threshold 1000, new offset 76
[2026-07-29 12:27:11.668] WARNING[247451] chan_iax2.c: Resyncing the jb. last_delay 0, this delay 9735, threshold 1000, new offset -9659
[2026-07-29 12:27:43.675] WARNING[247451] chan_iax2.c: Resyncing the jb. last_delay 0, this delay 2032, threshold 1000, new offset -11691
[2026-07-29 12:27:51.955] WARNING[263507] app_rpt.c: statpost to URL 'http://stats.allstarlink.org/uhandler?node=572921&time=1785342451&seqno=985&nodes=T2324&apprptvers=3.10.1&apprptuptime=58856&totalkerchunks=0&totalkeyups=153&totaltxtime=3591&timeouts=0&totalexecdcommands=9&keyed=0&keytime=58866' failed with error: Could not resolve host: stats.allstarlink.org
```
I suspect we stop servicing the "maintained" local channel connected to the bridge. Any audio going into the bridge will cause this error until we reconnect.
N8RAW
July 31, 2026, 4:23pm
3
It "should" be harmless assuming it's the same issue. When an outbound connection has failed the redial function can take a long time looking up the DNS name which causes these queue errors. Once the DNS is back and/or internet is stable it should just continue to work.
I have something "cooked up" to handle the frames in this condition but it should not be a problem.
That said, there is another bug I've been chasing where we have long queues on the repeater channels. Still hunting for that elusive one, so it happens PLEASE share logs.
K2CB
July 31, 2026, 4:41pm
4
Let me know which logs specifically, and I will be happy to post them.
Thank you.