Trouble Shooting

Overview

This section covers the steps to restore service when remote communication with the Fairhaven system breaks down. The ARMAC protection functions (AR2 and AR3 autonomous control) operate independently of internet and continue running during any communication outage. The troubleshooting steps below are concerned with restoring remote visibility and control.

For RPi-level technical recovery commands (Docker, Tailscale, filesystem, watchdog) refer to the Remote Recovery Run Sheet.

Understanding the Communication Layers

Before troubleshooting it helps to understand what is running and how the pieces connect.

Three remote access paths to the system:

Tailscale and ZeroTier VPNs -- the normal means of SSH access and HA dashboard access. Both provide direct encrypted connections to the RPi. They work well when the FritzBox internet is up but may not function through mobile carrier-grade NAT (CGNAT), so they can fail when the RUT240 mobile failback is active.
Reverse SSH tunnel -- the RPi maintains a permanent outbound SSH connection to the Contabo VPS. This opens a relay port on the VPS through which a shell connection to the RPi can be made from any location. It works through any internet connection including mobile, because the connection is outbound from the RPi and is not affected by CGNAT. With a local port forward it also provides access to the HA browser interface. This is the primary independent backup when Tailscale and ZeroTier are unavailable.
Nabu Casa -- a cloud relay service providing HA dashboard access. If a Nabu Casa subscription is maintained, the RPi holds a permanent outbound HTTPS connection to the Nabu Casa servers, giving dashboard access through any internet connection. Given that the reverse SSH tunnel now provides an independent path to both shell and HA interface, Nabu Casa is an optional additional layer rather than a requirement.

Two internet paths:

FritzBox -- the primary broadband router.
RUT240 mobile router -- 4G mobile failback. Provides outbound internet to the RPi when FritzBox WAN is down, enabling the reverse SSH tunnel (and Nabu Casa if maintained) to continue working. Tailscale and ZeroTier may not work through mobile NAT.

Symptom 1 -- Cannot Reach the HA Dashboard

Try in this order:

Connect via Tailscale or ZeroTier and open HA at the local Tailscale address. If it loads, the system is running normally.
If Tailscale and ZeroTier fail, try the reverse SSH tunnel with a port forward. With the tunnel running, open http://localhost:8123 in a browser for the full HA interface.
If a Nabu Casa subscription is maintained, try the Nabu Casa URL as an additional check.
If all paths fail, the RPi is likely down or has no internet -- see Symptom 2.

Symptom 2 -- FritzBox Internet Appears Down

How to recognise it: The reverse SSH tunnel still works (confirming the RPi has mobile internet). Tailscale and ZeroTier do not. The MyFritz remote URL times out.

The RUT240 mobile router is providing internet in place of the FritzBox. The ARMAC system continues to operate. The reverse SSH tunnel (and Nabu Casa if maintained) remain available. Tailscale and ZeroTier are likely blocked by mobile carrier NAT.

To restore FritzBox internet remotely:

Confirm the reverse SSH tunnel is reachable, confirming the RPi has mobile internet via RUT240.
Power cycle the FritzBox using the Tuya switch connected to the RUT240 WiFi network. Wait two minutes.
If MyFritz responds and Tailscale reconnects, the FritzBox WAN has recovered.
If the FritzBox does not recover after a power cycle, the fault is with the ISP or the FritzBox hardware. The system remains controllable via the reverse SSH tunnel until a site visit.

Symptom 3 -- RPi Appears Completely Offline

How to recognise it: The reverse SSH tunnel does not connect, Tailscale and ZeroTier both fail, and the Tuya switches on both networks are also unreachable.

This means either the RPi has crashed or lost power, or both internet paths are simultaneously down.

Steps:

Power cycle the RPi using the Tuya switch on the RUT240 WiFi network. Wait two minutes.
Try the reverse SSH tunnel again. The RPi and HA Docker containers start automatically on boot.
If the RPi does not respond after a power cycle, a site visit is required. The ARMAC autonomous protection functions (AR2 and AR3) continue operating independently of the RPi.

Symptom 4 -- HA Loads but Shows No Live Data

How to recognise it: The HA dashboard is accessible but sensor readings are stuck at old values, zero, or unavailable.

HA is running but the data pipeline from the Arduino modules has broken. The most likely cause is that one or both Docker containers have stopped, or the Arduino MQTT connection has dropped.

Steps:

SSH to the RPi by any available path.
Run docker ps -a and confirm both the Home Assistant container and the Mosquitto MQTT container show as Up. If either is stopped, restart it with docker start <container-name>.
If both containers are running, check MQTT is receiving data: docker logs mosquitto --tail 50. Look for connection messages from AR1 and AR2.
If AR1 is not connected to MQTT, check AR1 WiFi status. AR3 activates a SoftwareSerial backup path automatically after 10 seconds of RS-485 silence and routes commands via AR1 WiFi, but AR1 must itself be connected to the network for this to work.

Symptom 5 -- HA Responds but Pump Commands Are Not Executed

How to recognise it: The HA dashboard loads, sensors show live values, but sending a pump on or off command produces no result.

The HA to MQTT to Arduino command chain is broken at some point.

Steps:

Confirm MQTT is running: docker ps -a.
Check whether AR3 is receiving MQTT messages. AR3 subscribes to command topics via its WiFi connection. If AR3 has lost WiFi it will not receive commands even if MQTT is healthy.
Check whether AR1 is connected. AR1 is the primary command relay from HA to AR3 via RS-485. If AR1 has lost WiFi or MQTT, commands will not reach AR3 unless the SoftwareSerial backup path is active.
If AR3 is confirmed connected and commands are still not executing, check the relay module wiring and the contactor.

Symptom 6 -- RS-485 Communication Between AR1 and AR3 Has Failed

How to recognise it: AR3 serial log shows no RS-485 frames being received from AR1.

Important: pump command control is not immediately lost. AR3 automatically activates a SoftwareSerial backup path after 10 seconds of RS-485 silence. Commands are then received via AR1 WiFi and MQTT instead of the RS-485 bus. This switchover requires no manual intervention -- provided AR1 remains connected to WiFi, the pump can still be commanded through HA as normal. The steps below are to diagnose and restore the RS-485 bus itself.

Steps:

Check the BUS ISOLATE switch on the RS-485 A wire at the house end is in the ON (closed) position.
Check that the Gravity RS-485 module at the house end is powered and its indicator LEDs are active.
With the BUS ISOLATE switch open, check continuity on each wire -- expect near-zero resistance on each conductor. With the switch closed and the system live, expect approximately 120 ohm across A and B at the shed end (the termination resistor).
If the Gravity module appears dead, power cycle it by disconnecting and reconnecting its supply. If it does not recover, replace it: open the BUS ISOLATE switch first before powering down or connecting the replacement module.

Symptom 7 -- All Internet Connectivity Lost

How to recognise it: Both the FritzBox and the RUT240 mobile router have lost connectivity simultaneously. All remote access paths (Tailscale, ZeroTier, reverse SSH tunnel) are unreachable.

This is an unusual situation requiring simultaneous failure of both internet paths. The ARMAC system continues to run and protect autonomously.

Pump control options when all internet is down:

SMS to the pump control. The RUT240 SIM accepts SMS commands to turn the pump on or off directly. Send an SMS to the RUT240 mobile number with the appropriate command. This path is independent of internet and operates as long as mobile coverage is available and the RUT240 has power.
Autonomous mode. If the RUT240 also fails, or if no SMS action is taken, AR3 enters autonomous mode automatically after 5 minutes without receiving a heartbeat from the RPi. In autonomous mode AR3 runs the full protection programme based on live sensor data -- pressure, temperature, and level sensors -- without any remote input. The pump will be started and stopped as required to maintain protection. No intervention is needed; the system is designed to fail safely in this mode.

Note: once internet is restored, all remote access paths reconnect automatically. The RPi and AR3 will re-synchronise without manual intervention.

Summary -- Which Paths Survive Which Failures

Failure	Tailscale / ZeroTier	SSH Tunnel	Nabu Casa (if maintained)	ARMAC Protection
FritzBox WAN down, RUT240 active	No	Yes	Yes	Unaffected
RPi down	No	No	No	Unaffected
HA Docker stopped	Yes (shell only)	Yes (shell only)	No	Unaffected
MQTT stopped	Yes	Yes	Yes (dashboard only)	AR3 autonomous continues
RS-485 failed	Yes	Yes	Yes	AR3 auto-switches to SoftwareSerial backup
All internet down	No	No	No	SMS pump control; then full autonomous mode

For technical RPi recovery commands (Docker, Tailscale, filesystem, watchdog) refer to Remote Recovery Run Sheet.

<< | | >> |Table of Contents>

Page last modified on June 27, 2026, at 08:01 am