Here are three non-obvious bugs we ran into while building ZPi Controller. They come from different domains, but they have one thing in common: each surfaced while trying to build a system that does not break when the user makes a mistake.

Gotcha 1: typing 5000 as the Modbus port takes down the whole web UI

Symptom

A Modbus TCP port conflict starts with the user typing 5000 as the Modbus slave port on the system page and saving. The web UI stopped opening. Over SSH:

zpi-controller.service: Main process exited, code=killed, status=6/ABRT
Restart=on-failure, restart counter = 27

The service had failed and restarted 27 times. From the log:

ZPi.IO08R.Plc.Modbus.ModbusTcpSlaveService[0]
  Modbus TCP slave listening on port 5000 (unit 1)
Failed to bind to address http://[::]:5000: address already in use.

The Modbus slave binds 5000 first (colliding with the HTTP port), Kestrel cannot take the same port and dies, systemd restarts the service, and around it goes. The user cannot reach the web UI, so they cannot fix the setting either. Fatal.

First attempt: a typed-level guard

public void Load()
{
    _opts = JsonSerializer.Deserialize<ModbusOptions>(...);
    if (_opts.TcpSlave.Port == HttpPort) {
        _opts.TcpSlave.Port = 502;   // auto-correct
        Save(_opts);
    }
}

Deployed. We forced port=5000 into the file artificially and restarted, and it did not recover. Why?

A GET on /api/modbus showed what deserialization actually produced:

{
  "tcpSlave": {
    "enabled": false,     ← the user set true, so why false?
    "port": 502,          ← set to 5000, so why the default 502?
    "unitId": 1,          ← this one is correct
    "map": { "p": 0, "m": 4096, ... }  ← everything inside map is correct
  }
}

Only enabled and port fell back to their defaults. unitId and map came through fine, even though they are properties of the same object. An inconsistent, partial deserialization failure.

We still do not know the exact cause (somewhere in the combination of System.Text.Json, the CamelCase PropertyNamingPolicy and nested objects), but the conclusion is that typed deserialization can silently drop some fields. That meant _opts.TcpSlave.Port was always the default 502, so our typed-level guard would never fire.

The real fix: sanitise at the raw text stage

If deserialization cannot be trusted, block the problem before it. Read the file as a string and substitute with a regular expression:

public void Load()
{
    var raw = File.ReadAllText(_path);

    // Phase 1: raw-text pre-sanitise — bulletproof
    var patched = Regex.Replace(
        raw,
        @"(""port""\s*:\s*)5000(?!\d)",
        $"${{1}}{DefaultModbusPort}",       // ${1} is explicit. Writing $1 is parsed as $15 followed by 02.
        RegexOptions.IgnoreCase);
    if (patched != raw) {
        File.WriteAllText(_path, patched);
        Console.Error.WriteLine("[ModbusStore] raw JSON: \"port\": 5000 → 502 (HTTP port collision)");
    }

    // Phase 2: deserialize as before (now safe)
    _opts = JsonSerializer.Deserialize<ModbusOptions>(patched, JsonOpts);

    // Phase 3: typed-level guards (belt and braces)
    if (_opts.TcpSlave.Port == HttpPort) { ... }
}

Three layers of defence: raw text, typed, and UI-side validation on Replace(). Injecting the port collision by hand and restarting now recovers automatically:

[ModbusStore] raw JSON: "port": 5000 → 502 (HTTP port collision)
Modbus TCP slave listening on port 502 (unit 1)
Now listening on: http://[::]:5000
HTTP: 302

What we took away

  • User input has to be validated on save and on load. Validating only in the UI leaves you exposed to direct SSH edits and to files written by older versions.
  • Do not trust typed deserialization alone. Catch the critical fields once more at the raw string stage. That kind of duplication is cheap compared to the reliability it buys.
  • Always log an automatic recovery. Silent magic ruins debugging.

Gotcha 2: mDNS — after a hostname change, getent returns nothing

Symptom

An avahi hostname change goes through the helper (zpi-set-hostname.sh), which runs hostnamectl, updates /etc/hosts and calls systemctl reload-or-restart avahi-daemon. And yet:

sudo /usr/local/sbin/zpi-set-hostname.sh going-zpi-XXX
hostname changed: going-zpi → going-zpi-XXX

getent hosts going-zpi-XXX.local
   ← nothing!

getent hosts going-zpi.local
192.168.0.x going-zpi.local   ← the old name is still alive

Pinging from a PC on the same LAN failed as well. The hostname had changed, but mDNS was not advertising the new one.

Cause

The avahi-daemon in Debian Trixie does not pick up a hostname change through the “reload” path of systemctl reload-or-restart. The reload returns success while the stale advertisement stays in place. A full restart is required.

Fix

# end of the helper
systemctl restart avahi-daemon 2>/dev/null || \
    systemctl reload-or-restart avahi-daemon 2>/dev/null || true

# self-verification
sleep 1
if getent hosts "${NEW}.local" > /dev/null 2>&1; then
    echo "hostname changed: $OLD -> $NEW (mDNS verified)"
else
    echo "warning: ${NEW}.local not yet resolving — avahi may need another second"
fi

A restart is slightly heavier (one extra second), but the difference in reliability is not close. One line of self-verification makes it obvious whether the change actually took.

What we took away

  • The meaning of reload versus restart differs from daemon to daemon. It is safer for the helper to verify the behaviour after restarting.
  • mDNS is surprisingly fussy about cache and advertisement refresh. When a core attribute such as the hostname changes, just restart.

Gotcha 3: C# Regex.Replace reads $1 as $15

The C# regex replacement bug is a by-product of gotcha 1. We replaced "port": 5000 with "port": 502 using a raw text regex, and the resulting JSON was broken:

{
  "tcpSlave": {
    "port": $1502,    ← what?
    ...
  }
}

The code:

var patched = Regex.Replace(
    raw,
    @"(""port""\s*:\s*)5000(?!\d)",
    $"$1{DefaultModbusPort}",       // $1 + 502 = ???
    RegexOptions.IgnoreCase);

Inside a C# $"..." interpolated string, $1 is just the literal text $1. {DefaultModbusPort} is interpolated to 502. The resulting replacement string is:

"$1502"

When Regex.Replace receives that, it cannot tell whether the capture group number is $1, $15, $150 or $1502. The documentation says:

“Substitutions that use a value greater than the highest number of capturing groups are interpreted as literal text.”

In practice, does it try group 1502, fail, and fall back to literal? Try group 15, fail, and fall back to literal? The exact behaviour is ambiguous, and in our case it did not resolve to $1 as group 1 plus a literal 502 — the whole thing was treated as literal text.

Fix: write ${1} explicitly

The documented way in C# regex:

$"${{1}}{DefaultModbusPort}"
//   ↑ ${1} is an explicit group reference
//   ↑↑↑ in C#, `{{` produces a literal `{`
//        resulting replacement string: "${1}502"

Now ${1} is capture group 1 and 502 is a literal, exactly as intended.

What we took away

  • The $ patterns in a Regex.Replace replacement string are every bit as fussy as PowerShell escaping.
  • When a numeric literal follows a group number, always write ${N} explicitly.
  • The same trap exists in other languages: PowerShell, sed and perl all share it.

Wrapping up — the pattern behind all three gotchas

The common pattern is below. All three of these looked like they would simply work, and none of them did.

  1. Validate at several layers. The Modbus port is checked when the UI saves, at the raw text stage on disk, and again after typed deserialization — three layers of defence.
  2. Verify after restarting. For actions with a delayed effect, such as an mDNS change, the helper should confirm the result itself.
  3. Be explicit where escaping is ambiguous. A form like ${1} looks longer, but it saves debugging time.

Once it is all built, it looks obvious. Before you build it, it is not.

Validate at several layers, verify after every change, and be explicit where escaping is ambiguous — that was the answer to all three gotchas.

What comes next in the series

  • Part 5 — WiFi auto-recovery and boot time from 1 min 31 s down to 33 s
  • Part 6 — what comes next (MQTT, WebSocket, alarms)

Contact