Conn time column, the times are incrementing at like half speed. Each second of time takes 2 seconds in reality. What the heck is up? Firefox, Chrome, other computers, no difference.
Pi appliance fully up to date.
Conn time column, the times are incrementing at like half speed. Each second of time takes 2 seconds in reality. What the heck is up? Firefox, Chrome, other computers, no difference.
Pi appliance fully up to date.
Sounds like there is either network issue between your AllStarLink node or the allmon3 process on the device is being blocked by other activity on the system. The allmon3 process on the node has to be able to complete a sampling of the system(s) information from Asterisk and push it all back to the client. You may be able to find out what's slow by stopping the allmon3 service and running it by hand (has root) using allmon3 --debug
.
Ok thank you for your feedback.
I can see in the debug output that the "CONN" list from AMI is being probed every one second.
However, the output from AMI showing the times is not "changing" at a one second rate.
I.e.:
Conn: 42060 76.150.151.240 0 IN 08:51:20 ESTABLISHED
if I watch 10 of these float by in 10 seconds, the 08:51:20 only ends up at mabe 08:51:24
Definitely does not seem like an allmon3 issue hmmmm.
That is VERY weird. Almost sounds like there's an asterisk problem then. Do you have any errors in /var/lib/asterisk/messages
?
Is 26 connections too much for a pi3 lol
In the last 30 minutes, just some jitter buffer messages
This is our node:
[http://2495.nodes.allstarlink.org:4438/allmon3/](http://2495.nodes.allstarlink.org:4438/allmon3/
I will also add, we went to ASL3 about a week ago. I've noticed this for the week. Not sudden. Never got to test this pi3 under load though.
Anecdotally, I have a beefier ASL3 VM. Just ran a quick test, and with one connected node, the "CONN Time" column seems to stall out a bit as well. I noticed I lost 6 seconds of CONN TIME over a minute.
To be clear, allmon3 itself is not guaranteed to complete a polling cycle within 1 second. First, the sheer mathematics of the whole thing doesn't lend itself perfectly to a 1000ms loop time all the way up and back. Secondly, a larger node with many items to process or something under heavier load may slow down the AMI polling. This is in the "just how it is" nature of it. In fact, Allmon3 (and Supermon and others) are somewhat abusing the AMI interface by polling it so fast/often. It generally works, but there are conditions where that may not always be perfect.
What I would suggest it turn of Allmon3 and any other motioning system - Supermon, AllScan, etc. The run while true; do asterisk -rx 'rpt lstats YOUR_NODE_HERE'; sleep 1; done
and see Asterisk itself is what isn't "keeping up" with the timers. Again, the timers aren't guaranteed to refresh every second because this isn't a fixed-time system.
OK. yes I just tried polling this over a minute as well:
sudo asterisk -rx "rpt xnode 2495"
The ESTABISHED time for nodes on the pi3 increased by 24 seconds, when I ran that command again.
Does not seem to be Allmon3, but something with asterisk.
I can't be the only one lol??
I also repeated that, but stopping allmon3. Same result, the connected counter increased only 24 seconds over one minute for a sample node.
Were you exec'ing "sudo asterisk ..." each time? If so, can you start with a shell that is already running as root and then exec just asterisk ...
? That would take the time associated with the sudo
command out of the mix.
So far, unfortunately, yes
Lol
Just to clarify, we are seeing this on a VM of Debian Bookworm running on proxmox host, and a pi3 appliance. Both are fully up to date using apt.
So a few other considerations:
outstreamcmd
, a lot of events defined, unusual GPIO, etc.What you are experiencing is very atypical.
The VM in proxmox has one node connected to test, and is basically stock. That one doesn't track connected time out of asterisk as mentioned. Loses 6 seconds of the one connected nodes connected time every minute.
The pi has about 29 clients. That device is nearly stock as well, and is a hub. It doesn't track connected time either. The pi has much less CPU power than the proxmox device above. Loses maybe 36 seconds every minute of connection time per client.
I checked a third device, a friends ASL3 device and his loses about 4 seconds of connected time every minute as well. His device is a pi3 that is recently quite up to date with apt as well. His device is not a hub and just connects to one other node.
None of these have any crazy setups.
Another data point:
A freshly imaged pi3 with the lastest release image, and no other updates.
If I connect it to another node, this pi3 loses 3 seconds of CONN Time every minute.
By loss of CONN TIME, its not that it 'skips' seconds on update, but that the time is foverever gone. So eventually, the true CONN TIME could be off by 1 hour and 12 seconds over a 24 hour period.
Whatever is going on, has to be asterisk related and perhaps CPU load or speed dependent.
I've tried on multiple Pis and a VM and I cannot recreate this behavior. Will you please try updating to the latest code in the beta repository (use asl-repo-switch -l beta
) and see if it persists? That's the one different is all my test systems are using the latest code.
Our pi node with ~29 clients and the promox VM with one client are on "devel" and have been for a few days. I didn't notice any difference from pre or post devel.
When I switch to beta I don't get offered any other packages than what I get from devel.
Meaning:
sudo asl-repo-switch -l devel
sudo apt update
and
sudo asl-repo-switch -l beta
sudo apt update
I am not offered any newer packages.
my pi with 29 clients shows:
sudo asl-show-version
********** AllStarLink [ASL] Version Info **********
OS : Debian GNU/Linux 12 (bookworm)
OS Kernel : 6.12.20+rpt-rpi-v8
Asterisk : 22.2.0+asl3-3.4.3-1.deb12
ASL [app_rpt] : 3.4.3
Installed ASL packages :
Package Version
============================== ==============================
allmon3 1.4.2-1.deb12
asl3 3.7.1-1.deb
asl3-asterisk 2:22.2.0+asl3-3.4.3-1.deb12
asl3-asterisk-config 2:22.2.0+asl3-3.4.3-1.deb12
asl3-asterisk-modules 2:22.2.0+asl3-3.4.3-1.deb12
asl3-menu 1.13-1.deb12
asl3-pi-appliance 1.10.0-1.deb12
asl3-update-nodelist 1.5.1-1.deb12
cockpit 287.1-0+deb12u3
cockpit-bridge 287.1-0+deb12u3
cockpit-networkmanager 287.1-0+deb12u3
cockpit-packagekit 287.1-0+deb12u3
cockpit-sosreport 287.1-0+deb12u3
cockpit-storaged 287.1-0+deb12u3
cockpit-system 287.1-0+deb12u3
cockpit-wifimanager 1.1.1-1.deb12
cockpit-ws 287.1-0+deb12u3
dahdi 1:3.1.0-2
dahdi-dkms 1:3.4.0-6+asl
dahdi-linux 1:3.4.0-6+asl
22.2.0+asl3-3.4.3-1.deb12
is the latest.
Then yes, per the output above, we are latest and have been for a few days. I have rebooted.