if you were to use the same ASL3 configuration (multiple interfaces and all the rest) but instead employ a Raspberry Pi 3B+, i am certain you will find it then behaves as expected.
there is something inherent in the 4B (and probably 5, too) drivers causing the issues you have experienced.
see Multi-node ASL3 server topic, with special attention to the concluding post from March 21.