Allmon3 is not responding to requests for this node

Allmon
1

Hi…

Every so Often I get the Following Error’s on Some Nodes on Allmon3

Allmon3 is not responding to requests for this node. Check server logs

Where would the Log be so I can take a Look at it…

Running systemctl restart allmon3 temporarily fixes the issue…

Thanks

Rob…

2

Can you provide the relevant logging from /var/log/daemon.log on your system for anything from allmon3 with the label “ERROR”?

3

I get the same errors, intermittently. But it’s only only for some or all of the remote nodes. I ran that command and I’m getting a bind address error, already in use. I am running for test purposes Allmon2, Allmon3, and Supermon, and I wonder if you can’t have all three binding to the same 0.0.0.0 address?
EDIT: I’m thinking that if I want to run multiple monitors simultaneously, I’ll need to run separate AMI ports for each. That makes sense to me.
But I need to change all nodes.

4

@Crigstar I doubt Allmon3 would conflict with Allmon3 or Supermon. When you get the errors, same question - what’s in /var/log/daemon.log when that happens?

5

I appreciate your response.
I get this which seems to be IPv6:

OSError: [Errno 98] error while attempting to bind on address (‘::’, 16700, 0, 0): address already in use

And I repeated it and got this which looks like IPv4:
OSError: [Errno 98] error while attempting to bind on address (‘0.0.0.0’, 16700): address already in use
and a little more info
Sep 12 10:18:14 ASL-melbourne asterisk[761]: AllStar Info: Connected <=IN== 58507 K2IP ClearNode El Segundo CA USA (xxx.yyy.zzz.aaa:4569)
Sep 12 10:40:22 ASL-melbourne asterisk[761]: AllStar Info: Disconnected =v= 58507 K2IP ClearNode El Segundo CA USA (xxx.yyy.zzz.aaa:4569)
Sep 12 11:58:40 ASL-melbourne allmon3[14999]: INFO: (allmon3) starting status ws 58507@ xxx.yyy.zzz.aaa on port 16706
Sep 12 11:58:40 ASL-melbourne allmon3[14999]: INFO: (allmon3) starting cmd ws 58507@ xxx.yyy.zzz.aaa on port 16707
Sep 12 11:58:43 ASL-melbourne allmon3[14999]: INFO: (asl_allmon.allmon3_ws_status) broadcasting status for 58507 on port 16706
Sep 12 12:02:45 ASL-melbourne asterisk[761]: AllStar Info: Connected <=IN== 58507 K2IP ClearNode El Segundo CA USA (xxx.yyy.zzz.aaa:4569)
Sep 12 13:04:08 ASL-melbourne allmon3[14999]: WARNING: (asl_allmon.allmon3_ws_status) ami_conn socket problem for node 58507: socket timeout
Sep 12 13:04:08 ASL-melbourne allmon3[14999]: INFO: (asl_allmon.allmon3_ws_status) node: 58507 - sleeping for RETRY_INTERVAL of 15
Sep 12 13:04:24 ASL-melbourne allmon3[14999]: INFO: (asl_allmon.allmon3_ws_status) node: 58507 - attempting reconnection retry #1
Sep 12 13:04:28 ASL-melbourne allmon3[14999]: INFO: (asl_allmon.allmon3_ws_status) node: 58507 - connection reestablished after 1 retries
Sep 12 14:52:07 ASL-melbourne asterisk[761]: AllStar Info: Connected <=IN== 58507 K2IP ClearNode El Segundo CA USA (xxx.yyy.zzz.aaa:4569)

6

@Crigstar I wonder if you have something local that’s already using that port. WITHOUT allmon3 running at all what’s the output of ss -ltnp?

7

It’s behaving itself now. So I stopped allmon3, ran the command, started Allmon3 and it’s still good. Showing all nodes. The output follows here:
root@ASL-melbourne:~#
root@ASL-melbourne:~# ss -ltnp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 2 127.0.0.1:5038 0.0.0.0:* users:((“asterisk”,pid=775,fd=7))
LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:* users:((“systemd-resolve”,pid=461,fd=17))
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:((“sshd”,pid=793,fd=3))
LISTEN 0 128 0.0.0.0:5355 0.0.0.0:* users:((“systemd-resolve”,pid=461,fd=12))
LISTEN 0 128 :80 : users:((“apache2”,pid=32400,fd=4),(“apache2”,pid=31552,fd=4),(“apache2”,pid=31551,fd=4),(“apache2”,pid=31549,fd=4),(“apache2”,pid=31548,fd=4),(“apache2”,pid=5734,fd=4),(“apache2”,pid=5733,fd=4),(“apache2”,pid=5732,fd=4),(“apache2”,pid=5731,fd=4),(“apache2”,pid=5730,fd=4),(“apache2”,pid=2105,fd=4),(“apache2”,pid=832,fd=4))
LISTEN 0 128 [::]:22 [::]: users:((“sshd”,pid=793,fd=4))
LISTEN 0 128 :443 : users:((“apache2”,pid=32400,fd=6),(“apache2”,pid=31552,fd=6),(“apache2”,pid=31551,fd=6),(“apache2”,pid=31549,fd=6),(“apache2”,pid=31548,fd=6),(“apache2”,pid=5734,fd=6),(“apache2”,pid=5733,fd=6),(“apache2”,pid=5732,fd=6),(“apache2”,pid=5731,fd=6),(“apache2”,pid=5730,fd=6),(“apache2”,pid=2105,fd=6),(“apache2”,pid=832,fd=6))
LISTEN 0 128 [::]:5355 [::]: users:((“systemd-resolve”,pid=461,fd=14))
root@ASL-melbourne:~#