RTCM/Voter stops TXing/RXing

I have a Voter board which stops transmitting and receiving. The board in questions simulcasts in a three site system.

Master site - TX/RX (works fine)
Secondary Site - TX/RX (stops working after a while)
Third Site - RX only (works fine and is on the same IP/VLAN as the above Secondary site)

The system has working fine for a few years until recently when a GPS failed at the secondary site. I replaced it with a like for like replacement and it worked fine for a day or so until it stopped transmitting. I tested and it also was not receiving.

I can ping the voter board, reboot both the master and that board and it still doesn’t bring it back up. Only a reboot of asterisk fixes it. The status also shows PTT as 1 when transmitting on the faulty board.

In the past I have had a similar issue, however a reboot of everything fixed it and I never had reoccurring problems.

The only thing I can think of is trying to increase the buflen to see if it helps.

Regards
Hayden VK7HH

I have not had that issue. If nothing changed other than the GPS I would first re-verify all wires are connected the same way and that there are no opens or shorts. Then I would check the settings in the new GPS to verify that all options are set exactly the same as the working one.
I have had 2 systems at the same site and used 1 GPS to feed 2 voter boards and it has worked fine for several years. Once in a while when setting up a new site with a new GPS I have found that the factory software settings in the GPS needed to be adjusted to get it to work. Most often PPS pulse width was the problem.

Try the debug mode on the voter board to watch the data coming from the GPS. Note that doing this will often prevent even an otherwise good installation from working while the debug is running, but at least you can see the data and make sure that it is the same as the data that a good site gets from the GPS. Don’t forget to disable all debug for normal operation.

Chuck

···

On Sun, Oct 13, 2019 at 10:12 PM Haydenph91 via AllStarLink Discussion Groups noreply@community.allstarlink.org wrote:


Haydenph91

    October 14

I have a Voter board which stops transmitting and receiving. The board in questions simulcasts in a three site system.

Master site - TX/RX (works fine)

Secondary Site - TX/RX (stops working after a while)

Third Site - RX only (works fine and is on the same IP/VLAN as the above Secondary site)

The system has working fine for a few years until recently when a GPS failed at the secondary site. I replaced it with a like for like replacement and it worked fine for a day or so until it stopped transmitting. I tested and it also was not receiving.

I can ping the voter board, reboot both the master and that board and it still doesn’t bring it back up. Only a reboot of asterisk fixes it. The status also shows PTT as 1 when transmitting on the faulty board.

In the past I have had a similar issue, however a reboot of everything fixed it and I never had reoccurring problems.

The only thing I can think of is trying to increase the buflen to see if it helps.

Regards

Hayden VK7HH

Hey Chuck.
Failed again twice yesterday. I increased the buflen to 160 (from 140) and increased the TX buffer values on all the boards.

I have yet to check all the GPS strings to make sure they match.
The GPS I use is the BG7TBL (green) GPSDO. They don’t really have any configuration that needs to be done to them.

I’ve seen those BG7TBL units slip by 1 second. I’d check the time if I were you.

asterisk -r
voter debug level 3

With the repeater not in use you can see the time from each of the GPS and know if one is out of bounds.

If I have any issues this is always the first thing I look at.

Chuck

···

On Tue, Oct 15, 2019 at 4:48 PM Haydenph91 via AllStarLink Discussion Groups noreply@community.allstarlink.org wrote:


Haydenph91

    October 15

Hey Chuck.

Failed again twice yesterday. I increased the buflen to 160 (from 140) and increased the TX buffer values on all the boards.

I have yet to check all the GPS strings to make sure they match.

The GPS I use is the BG7TBL (green) GPSDO. They don’t really have any configuration that needs to be done to them.


Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

Thanks Chuck.

I ran the debug, but the repeater was in the working state. I will have to check it again when it fails. Interestingly I didn’t reboot to get it to work this time. It just came good.

The GPS times were all correct. The Systimes were within a few hundred miilseconds as was the drain time. I’ll check it again when it fails.

Hayden

···

On Wed, Oct 16, 2019 at 4:38 PM Chuck Henderson via AllStarLink Discussion Groups noreply@community.allstarlink.org wrote:


Chuck_Henderson

    October 16

asterisk -r

voter debug level 3

With the repeater not in use you can see the time from each of the GPS and know if one is out of bounds.

If I have any issues this is always the first thing I look at.

Chuck

··· (click for more details)


Visit Topic or reply to this email to respond.


In Reply To


Haydenph91

    October 15

Hey Chuck. Failed again twice yesterday. I increased the buflen to 160 (from 140) and increased the TX buffer values on all the boards. I have yet to check all the GPS strings to make sure they match. The GPS I use is the BG7TBL (green) GPSDO. They don’t really have any configuration that needs t…


Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

Hi Chuck,
Below is the output of the debug. This is in the state where the site “Grey.Mt” is not transmitting or receiving. The site Mt.Wellington is the main site and has TX/RX working fine. Mt.Nelson is a RX only site.

Grey Mt and Mt.Wellington both use BG7TBL units. Mt Nelson uses a garmin puck.
I’m in range of Grey Mt however only Mt.Wellington votes when I try to access it.

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
GPSTime (Mt.Nelson): 10/19/19 16:22:51.000000000
SysTime: 10/19/19 16:22:51.520000
DrainTime: 10/19/19 16:22:51.480
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6
sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352
sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036
GPSTime (Mt.Wellington): 10/19/19 16:22:52.000000000
SysTime: 10/19/19 16:22:52.260000
DrainTime: 10/19/19 16:22:52.240
Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.7
GPSTime (Grey.Mt): 10/19/19 16:22:52.000000000
SysTime: 10/19/19 16:22:52.280000
DrainTime: 10/19/19 16:22:52.260
Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 835.0
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
GPSTime (Mt.Nelson): 10/19/19 16:22:52.000000000
SysTime: 10/19/19 16:22:52.520000
DrainTime: 10/19/19 16:22:52.480
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6
sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352
sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036
GPSTime (Mt.Wellington): 10/19/19 16:22:53.000000000
SysTime: 10/19/19 16:22:53.260000
DrainTime: 10/19/19 16:22:53.240
Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.8
GPSTime (Grey.Mt): 10/19/19 16:22:53.000000000
SysTime: 10/19/19 16:22:53.280000
DrainTime: 10/19/19 16:22:53.260
Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.8
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
GPSTime (Mt.Nelson): 10/19/19 16:22:53.000000000
SysTime: 10/19/19 16:22:53.520000
DrainTime: 10/19/19 16:22:53.500
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6
sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352
sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036
GPSTime (Mt.Wellington): 10/19/19 16:22:54.000000000
SysTime: 10/19/19 16:22:54.280000
DrainTime: 10/19/19 16:22:54.240
Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.9
GPSTime (Grey.Mt): 10/19/19 16:22:54.000000000
SysTime: 10/19/19 16:22:54.300000
DrainTime: 10/19/19 16:22:54.260
Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.7
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
GPSTime (Mt.Nelson): 10/19/19 16:22:54.000000000
SysTime: 10/19/19 16:22:54.500000
DrainTime: 10/19/19 16:22:54.480
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.8
sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352
sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036
GPSTime (Mt.Wellington): 10/19/19 16:22:55.000000000
SysTime: 10/19/19 16:22:55.260000
DrainTime: 10/19/19 16:22:55.240
Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.8
GPSTime (Grey.Mt): 10/19/19 16:22:55.000000000
SysTime: 10/19/19 16:22:55.300000
DrainTime: 10/19/19 16:22:55.260
Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.7
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
Time: 10/19/19 16:22:55.500, (Mt.Wellington) RSSI: 249
DrainTime: 10/19/19 16:22:55.500
SysTime: 10/19/19 16:22:55.523, diff: 20000000,index: 800
GPSTime (Mt.Nelson): 10/19/19 16:22:55.000000000
SysTime: 10/19/19 16:22:55.540000
DrainTime: 10/19/19 16:22:55.500
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 289.0
Time: 10/19/19 16:22:55.520, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.520
SysTime: 10/19/19 16:22:55.543, diff: 20000000,index: 800
Time: 10/19/19 16:22:55.540, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.540
SysTime: 10/19/19 16:22:55.563, diff: 20000000,index: 800
Time: 10/19/19 16:22:55.560, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.560
SysTime: 10/19/19 16:22:55.583, diff: 20000000,index: 800
Time: 10/19/19 16:22:55.580, (Mt.Wellington) RSSI: 252
DrainTime: 10/19/19 16:22:55.580
SysTime: 10/19/19 16:22:55.603, diff: 20000000,index: 800
Time: 10/19/19 16:22:55.600, (Mt.Wellington) RSSI: 252
DrainTime: 10/19/19 16:22:55.600
SysTime: 10/19/19 16:22:55.623, diff: 20000000,index: 800
Voter client Mt.Wellington selected for node 1727
Sending from client Mt.Wellington RSSI 249
Time: 10/19/19 16:22:55.620, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.620
SysTime: 10/19/19 16:22:55.643, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 253
Time: 10/19/19 16:22:55.640, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.640
SysTime: 10/19/19 16:22:55.663, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 253
Time: 10/19/19 16:22:55.660, (Mt.Wellington) RSSI: 253
DrainTime: 10/19/19 16:22:55.660
SysTime: 10/19/19 16:22:55.683, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 253
Time: 10/19/19 16:22:55.680, (Mt.Wellington) RSSI: 252
DrainTime: 10/19/19 16:22:55.680
SysTime: 10/19/19 16:22:55.703, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 252
sending audio packet to client Mt.Wellington digest 5711e352
sending audio packet to client Grey.Mt digest 6e825036
Time: 10/19/19 16:22:55.700, (Mt.Wellington) RSSI: 251
DrainTime: 10/19/19 16:22:55.700
SysTime: 10/19/19 16:22:55.723, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 252
sending audio packet to client Mt.Wellington digest 5711e352
sending audio packet to client Grey.Mt digest 6e825036
Time: 10/19/19 16:22:55.720, (Mt.Wellington) RSSI: 241
DrainTime: 10/19/19 16:22:55.720
SysTime: 10/19/19 16:22:55.743, diff: 20000000,index: 800
Sending from client Mt.Wellington RSSI 253
sending audio packet to client Mt.Wellington digest 5711e352
sending audio packet to client Grey.Mt digest 6e825036

Here is what I noticed right away. But it does not explain why Gray is not receiving. How do you know that Gray has a good signal? I see no reason here for Grey to not be sending audio to Mt Wellington other than it must think that it does not have a qualifying signal level and CTCSS signal. From what I see here i would not suspect a GPS issue with Gray, but would suspect a radio interface issue is more likely. Nelson looks to have a serious network or GPS issue. But I only have tunnel vision. On one of my my systems I have 3 receive sites, one connected via the Internet and one connected via an 18 mile 5ghz link with a 12db s:n ratio and all arrive within less than 20 ms of each other all the time. But 20 ms seems very reasonable, 260 ms seems very unreasonable, long for even a slow DSL line.
Have you tried doing a software reboot via a telnet session to all 3 of the voter boards, not a hardware reboot? This way the GPS is already feeding good data to the voter as it boots up. This might resolve the Nelson issue.
GPSTime (Mt.Wellington): 10/19/19 16:22:52.000000000
SysTime: 10/19/19 16:22:52.260000 ( first to arrive at Mt Wellington ) TX/RX working fine BG7TBLDrainTime: 10/19/19 16:22:52.240
Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.7
GPSTime (Grey.Mt): 10/19/19 16:22:52.000000000
SysTime: 10/19/19 16:22:52.280000 ( 20ms later to arrive at Mt Wellington ) no tx/rx garmin puckDrainTime: 10/19/19 16:22:52.260
Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 835.0
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a
GPSTime (Mt.Nelson): 10/19/19 16:22:52.000000000 This arrived way too late to be any good. If this is network propagation time then the network is broken, otherwise the GPS is brokenSysTime: 10/19/19 16:22:52.520000 ( 260ms later to arrive at Mt Wellington ) RX only site. BG7TBLDrainTime: 10/19/19 16:22:52.480

···

On Sat, Oct 19, 2019 at 12:37 AM Haydenph91 via AllStarLink Discussion Groups noreply@community.allstarlink.org wrote:


Haydenph91

    October 19

Hi Chuck,

Below is the output of the debug. This is in the state where the site “Grey.Mt” is not transmitting or receiving. The site Mt.Wellington is the main site and has TX/RX working fine. Mt.Nelson is a RX only site.

Grey Mt and Mt.Wellington both use BG7TBL units. Mt Nelson uses a garmin puck.

I’m in range of Grey Mt however only Mt.Wellington votes when I try to access it.

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a

GPSTime (Mt.Nelson): 10/19/19 16:22:51.000000000

SysTime: 10/19/19 16:22:51.520000

DrainTime: 10/19/19 16:22:51.480

Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6

sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352

sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036

GPSTime (Mt.Wellington): 10/19/19 16:22:52.000000000

SysTime: 10/19/19 16:22:52.260000

DrainTime: 10/19/19 16:22:52.240

Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.7

GPSTime (Grey.Mt): 10/19/19 16:22:52.000000000

SysTime: 10/19/19 16:22:52.280000

DrainTime: 10/19/19 16:22:52.260

Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 835.0

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a

GPSTime (Mt.Nelson): 10/19/19 16:22:52.000000000

SysTime: 10/19/19 16:22:52.520000

DrainTime: 10/19/19 16:22:52.480

Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6

sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352

sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036

GPSTime (Mt.Wellington): 10/19/19 16:22:53.000000000

SysTime: 10/19/19 16:22:53.260000

DrainTime: 10/19/19 16:22:53.240

Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.8

GPSTime (Grey.Mt): 10/19/19 16:22:53.000000000

SysTime: 10/19/19 16:22:53.280000

DrainTime: 10/19/19 16:22:53.260

Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.8

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a

GPSTime (Mt.Nelson): 10/19/19 16:22:53.000000000

SysTime: 10/19/19 16:22:53.520000

DrainTime: 10/19/19 16:22:53.500

Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.6

sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352

sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036

GPSTime (Mt.Wellington): 10/19/19 16:22:54.000000000

SysTime: 10/19/19 16:22:54.280000

DrainTime: 10/19/19 16:22:54.240

Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.9

GPSTime (Grey.Mt): 10/19/19 16:22:54.000000000

SysTime: 10/19/19 16:22:54.300000

DrainTime: 10/19/19 16:22:54.260

Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.7

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a

GPSTime (Mt.Nelson): 10/19/19 16:22:54.000000000

SysTime: 10/19/19 16:22:54.500000

DrainTime: 10/19/19 16:22:54.480

Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 288.8

sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 5711e352

sending KEEPALIVE (GPS) packet to client Grey.Mt digest 6e825036

GPSTime (Mt.Wellington): 10/19/19 16:22:55.000000000

SysTime: 10/19/19 16:22:55.260000

DrainTime: 10/19/19 16:22:55.240

Got GPS (Mt.Wellington): Lat: 4253.68S, Lon: 14714.19E, Elev: 1246.8

GPSTime (Grey.Mt): 10/19/19 16:22:55.000000000

SysTime: 10/19/19 16:22:55.300000

DrainTime: 10/19/19 16:22:55.260

Got GPS (Grey.Mt): Lat: 4305.48S, Lon: 14706.96E, Elev: 834.7

sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 3e775a5a

Time: 10/19/19 16:22:55.500, (Mt.Wellington) RSSI: 249

DrainTime: 10/19/19 16:22:55.500

SysTime: 10/19/19 16:22:55.523, diff: 20000000,index: 800

GPSTime (Mt.Nelson): 10/19/19 16:22:55.000000000

SysTime: 10/19/19 16:22:55.540000

DrainTime: 10/19/19 16:22:55.500

Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.95E, Elev: 289.0

Time: 10/19/19 16:22:55.520, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.520

SysTime: 10/19/19 16:22:55.543, diff: 20000000,index: 800

Time: 10/19/19 16:22:55.540, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.540

SysTime: 10/19/19 16:22:55.563, diff: 20000000,index: 800

Time: 10/19/19 16:22:55.560, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.560

SysTime: 10/19/19 16:22:55.583, diff: 20000000,index: 800

Time: 10/19/19 16:22:55.580, (Mt.Wellington) RSSI: 252

DrainTime: 10/19/19 16:22:55.580

SysTime: 10/19/19 16:22:55.603, diff: 20000000,index: 800

Time: 10/19/19 16:22:55.600, (Mt.Wellington) RSSI: 252

DrainTime: 10/19/19 16:22:55.600

SysTime: 10/19/19 16:22:55.623, diff: 20000000,index: 800

Voter client Mt.Wellington selected for node 1727

Sending from client Mt.Wellington RSSI 249

Time: 10/19/19 16:22:55.620, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.620

SysTime: 10/19/19 16:22:55.643, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 253

Time: 10/19/19 16:22:55.640, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.640

SysTime: 10/19/19 16:22:55.663, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 253

Time: 10/19/19 16:22:55.660, (Mt.Wellington) RSSI: 253

DrainTime: 10/19/19 16:22:55.660

SysTime: 10/19/19 16:22:55.683, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 253

Time: 10/19/19 16:22:55.680, (Mt.Wellington) RSSI: 252

DrainTime: 10/19/19 16:22:55.680

SysTime: 10/19/19 16:22:55.703, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 252

sending audio packet to client Mt.Wellington digest 5711e352

sending audio packet to client Grey.Mt digest 6e825036

Time: 10/19/19 16:22:55.700, (Mt.Wellington) RSSI: 251

DrainTime: 10/19/19 16:22:55.700

SysTime: 10/19/19 16:22:55.723, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 252

sending audio packet to client Mt.Wellington digest 5711e352

sending audio packet to client Grey.Mt digest 6e825036

Time: 10/19/19 16:22:55.720, (Mt.Wellington) RSSI: 241

DrainTime: 10/19/19 16:22:55.720

SysTime: 10/19/19 16:22:55.743, diff: 20000000,index: 800

Sending from client Mt.Wellington RSSI 253

sending audio packet to client Mt.Wellington digest 5711e352

sending audio packet to client Grey.Mt digest 6e825036


Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

Thanks Chuck.
I just noticed that Grey is back receiving/transmitting again. Strange. I tried rebooting everything earlier and couldn’t get it back. This has happened before and a simple reboot of things makes it work again. Anyhow it’s back working again and I can see it is receiving me. See if you can dig anything out of the voter debug below.

Strange too about Nelson. That has worked fine from what I can tell. Regardless I have rebooted it as you suggested. So the below is in it’s working state. I also haven’t ruled out maybe a crook fuse or something to the radio which could be causing it to work intermittently. Just strange that before I could reboot things and it would work, not something a bad fuse usually can be fixed by.

GPSTime (Mt.Wellington): 10/19/19 19:52:31.000000000
SysTime: 10/19/19 19:52:31.280000
DrainTime: 10/19/19 19:52:31.260
Got GPS (Mt.Wellington): Lat: 4253.69S, Lon: 14714.19E, Elev: 1237.2
GPSTime (Grey.Mt): 10/19/19 19:52:31.000000000
SysTime: 10/19/19 19:52:31.300000
DrainTime: 10/19/19 19:52:31.260
Got GPS (Grey.Mt): Lat: 4305.49S, Lon: 14706.96E, Elev: 841.4
GPSTime (Mt.Nelson): 10/19/19 19:52:31.000000000
SysTime: 10/19/19 19:52:31.520000
DrainTime: 10/19/19 19:52:31.500
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.96E, Elev: 288.6
sending KEEPALIVE (GPS) packet to client Mt.Nelson digest 602596ba
sending KEEPALIVE (GPS) packet to client Mt.Wellington digest 3e10c65b
sending KEEPALIVE (GPS) packet to client Grey.Mt digest f8489a3e
GPSTime (Mt.Wellington): 10/19/19 19:52:32.000000000
SysTime: 10/19/19 19:52:32.280000
DrainTime: 10/19/19 19:52:32.260
Got GPS (Mt.Wellington): Lat: 4253.69S, Lon: 14714.19E, Elev: 1237.0
GPSTime (Grey.Mt): 10/19/19 19:52:32.000000000
SysTime: 10/19/19 19:52:32.300000
DrainTime: 10/19/19 19:52:32.260
Got GPS (Grey.Mt): Lat: 4305.49S, Lon: 14706.96E, Elev: 841.4
GPSTime (Mt.Nelson): 10/19/19 19:52:32.000000000
SysTime: 10/19/19 19:52:32.520000
DrainTime: 10/19/19 19:52:32.500
Got GPS (Mt.Nelson): Lat: 4255.16S, Lon: 14718.96E, Elev: 288.6
Time: 10/19/19 19:52:32.520, (Mt.Wellington) RSSI: 15
DrainTime: 10/19/19 19:52:32.520
SysTime: 10/19/19 19:52:32.541, diff: 20000000,index: 800
Time: 10/19/19 19:52:32.500, (Grey.Mt) RSSI: 251
DrainTime: 10/19/19 19:52:32.520
SysTime: 10/19/19 19:52:32.542, diff: 60000000,index: 480
Time: 10/19/19 19:52:32.540, (Mt.Wellington) RSSI: 15
DrainTime: 10/19/19 19:52:32.540
SysTime: 10/19/19 19:52:32.562, diff: 20000000,index: 800
Time: 10/19/19 19:52:32.520, (Grey.Mt) RSSI: 250
DrainTime: 10/19/19 19:52:32.540

Rebooted all three boards.
Nelson is still out by almost 280ms. I can only assume that the IP link is causing the issue to that site. The latency shouldn’t be that high either.
Nelson is located at the end on a VLAN which linked over IP backhaul. Pinging from the router (which is located at Grey indicates a ping time of 4ms. So the timing can’t be due to latency?

Voter ping results:
PING (Mt.Wellington): Packets tx: 100, rx: 100, oos: 0, Avg.: 3.030 ms
PING (Mt.Wellington): Worst: 4 ms, Best: 3 ms, 100.0% Packets successfully received (0.0% loss)

PING (Grey.Mt): Packets tx: 100, rx: 99, oos: 0, Avg.: 37.242 ms
PING (Grey.Mt): Worst: 62 ms, Best: 33 ms, 99.0% Packets successfully received (1.0% loss)

PING (Mt.Nelson): Packets tx: 100, rx: 98, oos: 0, Avg.: 38.418 ms
PING (Mt.Nelson): Worst: 63 ms, Best: 35 ms, 98.0% Packets successfully received (2.0% loss)

I must be spoiled with near perfect networks. I would be going crazy trying to fix my network if it ever shows any packet loss. But I can remember a time when packet loss was just expected, but not anymore. I am finding it increasingly difficult to maintain my 5ghz links because the number of signals on the band seems to increase every day. Since the FCC gave away our frequencies in the 5ghz band and now anyone and everyone can use them, it takes a lot of effort to maintain the links. If I switch to monitor mode I usually see anywhere from 300 to 900 access points sharing any frequency that I try. I keep making the dishes bigger and that makes the aiming more and more difficult, but I am still maintaining zero packet loss for the links and near zero latency. Sometimes maybe a millisecond. I am currently using 30db gain antennas, I don’t know what my next step up will be and I don’t know if my Rohn 25 tower can support bigger dishes. Even now on very windy days I can see the tower twisting back and forth which has an impact on the signal to noise, but still I am not seeing any packet loss or more than a millisecond delay. Rohn tower is amazing that it can take all that twisting for so many years and not turn to dust. :slight_smile:
By the way, While I have always wanted to maintain our systems independent of the Internet, I recently started using the Internet for one link after I discovered that the Internet could provide a sub millisecond connection between 2 of my sites where one end had lost it’s 5ghz receiver and the weather was not allowing for a tower climb. Both ends have fiber to the house but one end did not have a globally unique IP address so I had to add a raspberry Pi at each end and establish a VPN between them. Even with all that overhead the total round trip time is less than a millisecond and a traceroute shows about 12 routers between us.

Anyway, getting back to your issue, do you know why you have packet loss and so much (38ms) delay?

What does it look like if you ping with a size of 1500? Does your packet delay and loss go way up?

Chuck

···

On Sat, Oct 19, 2019 at 6:28 AM Haydenph91 via AllStarLink Discussion Groups noreply@community.allstarlink.org wrote:


Haydenph91

    October 19

Rebooted all three boards.

Nelson is still out by almost 280ms. I can only assume that the IP link is causing the issue to that site. The latency shouldn’t be that high either.

Nelson is located at the end on a VLAN which linked over IP backhaul. Pinging from the router (which is located at Grey indicates a ping time of 4ms. So the timing can’t be due to latency?

Voter ping results:

PING (Mt.Wellington): Packets tx: 100, rx: 100, oos: 0, Avg.: 3.030 ms

PING (Mt.Wellington): Worst: 4 ms, Best: 3 ms, 100.0% Packets successfully received (0.0% loss)

PING (Grey.Mt): Packets tx: 100, rx: 99, oos: 0, Avg.: 37.242 ms

PING (Grey.Mt): Worst: 62 ms, Best: 33 ms, 99.0% Packets successfully received (1.0% loss)

PING (Mt.Nelson): Packets tx: 100, rx: 98, oos: 0, Avg.: 38.418 ms

PING (Mt.Nelson): Worst: 63 ms, Best: 35 ms, 98.0% Packets successfully received (2.0% loss)


Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

Sorry Chuck,
I did forget to explain that Mt Wellington is on a different connection. The three sites aren’t connected via LAN, but the internet.

Mt Wellington (main site) <— Internet <— via mainland Australia <— back to my state of Tasmania <— Grey Mt (Nelson).

Can’t help the internet going via mainland Australia, that’s how almost every connection has to go, thus the 38ms delay. I don’t think there is any packet loss. I was watching the voter ping and it got to 100 successfully but then hung… and then reported that 1 or 2 packets were lost. In the testing from 1-99 I didn’t see any drop.

It’s not possible for me to not use the internet. The sites don’t have LOS and the carrier I use for the two sites on the same VLAN are not on the main site.

It’s worked perfectly before I had to change out what I thought was a faulty BG7TBL unit. I now suspect that unit had a bad SMA to BNC connector on the GPS antenna. The actual voter board kept disconnecting randomly because it was losing it’s GPS. Now the GPS is solid and the board stays online, but the radio randomly stops transmitting or receiving.

The only thing in common I can think of, if that site is showing correct timing, no Ethernet out of bounds errors and shows PTT in status as 1 when transmitting is that the physical connections as you say to the radio hardware are intermittent (difficult to explain how both PTT and COR would fail)… or power is being lost to the radio hardware. The voter panel works off a different power cable so it’s feasible I guess.

Sigh… another trip to the repeater site!

Hi again Chuck,
I changed buflen to 220 and increased the TX delay as per our discussions yesterday. The site was transmitting and receiving this morning when I left for work.

I just tested again at lunch and it’s not working, so I suspect something in hardware now. I’ll bring it all back down off the mountain to put on the bench and see if I can work out what is going on.

Thanks
Hayden

Hi Chuck,
Went to the repeater site last night. Found the issue!
After much messing around, I had discovered that a BNC lead/BNC tee had gone bad from my GPS to the radio (to GPS lock it). Therefore the radios reference was intermittently disappearing, which explains the lack of RX and TX. I removed the tee and plugged it in directly and it came good. The other side of the tee goes to my voter panel box (which GPS locks the board for simulcasting). For now that’s not locked but running off an OCXO, which should be okay for now. I can hear warble and a bit of phase tearing in the overlap, but I can live with that until I replace the cable next trip.

Site 1 (Main) - Wellington
Site 2 - Grey
Site 3 - Nelson

I’m still getting the 200ms delay to Site 3 though. Still can’t explain that. The ping time is pretty much the same as it is to the Site 2 from Site 1 (main), is on the same network/VLAN as Site 2 and seems to vote fine?? On air it sounds fine too.

The network config for that site (to the best of my knowledge is as follows:

Router (Site 2) --> VLAN over one or possibly several IP links - not quite sure how my ISP set this up --> switch at Site 3 --> Fibre --> Fibre to ethernet converter --> Voter board at Site 3.