Firmware development

Point MeshBench at a MeshCore checkout. It builds it, loads it, and assigns it to every node of that role. Nothing is added to your checkout and nothing in it is modified.

meshbench dev -from ~/src/MeshCore

building simple_repeater from /home/alex/src/MeshCore
simple_repeater local-fix-relay-suppression (native)   0.6 MB
  in the workbench's firmware library
  assigned to every simple_repeater node

That is the whole loop. Edit, run it again, and the network is running your change. It needs MeshBench installed and a C++ compiler; nothing else.

The build lands in the firmware library as a *native* build: MeshCore compiled for this machine, which is the deterministic arm and the one every measurement uses. Native and emulated is the two-minute version of that choice.

What the command does#

The first run also fetches the Crypto library MeshCore builds against, once, and caches it. There is nothing to install beforehand beyond a C++ compiler.

Watching for changes#

meshbench dev -from ~/src/MeshCore -watch

Rebuilds and reassigns whenever a source file under src/ or examples/ changes. Leave it running in a terminal beside your editor.

Choosing a role#

MeshCore is several applications from one tree. -role selects which:

A build is assigned to the role it was built for. Assigning a repeater build to a companion would make it a different kind of node, not a different version of the same one.

Comparing your change against a baseline#

The reason to run a change on three hundred nodes is to find out whether it is an improvement. That is a sweep: two arms, the same network, the same seeds, only the firmware differing. Load a shipped -strict fixture as the arena, and Time and determinism is why the same seeds make a difference attributable to the firmware.

meshbench dev -from ~/src/MeshCore -name my-change -assign=false

then, in the workbench, define a sweep whose two arms name my-change and the release you branched from. See Experiments.

Three things decide whether such a comparison means anything:

Node storage is isolated per arm. A node keeps its preferences across runs, as hardware does, so a node that has run before loads its stored value rather than your changed default. A sweep runs each arm in its own storage root, so no arm inherits another's state; only runs made by hand outside a sweep need the firmware library's wipe button.

Give every role a build. A network with companions or a room server needs a build for each of them. Roles you did not change take the published release.

Include a control. Two arms built from identical source must produce identical numbers. When they do, a difference between real arms is attributable to the firmware. Every run records the checksum of each binary it attached, so "which build produced this number" is answerable from the results.

Attaching a debugger#

node.debug makes a running node debuggable and says how to attach. The firmware is the real firmware, so the debugger is the ordinary one: gdb, lldb, or VS Code with the C/C++ extension. The node must be running firmware; a node that is not is refused with is not running firmware, so there is nothing to debug.

{"id":1,"method":"node.debug","params":{"node":"West Lomond"}}

print(wb.call("node.debug", {"node": "West Lomond"})["attach"])

ans, err := wb.Call(ctx, "node.debug", map[string]any{"node": "West Lomond"})

The answer carries attach, a debugger command line, and vscode, a configuration to paste into launch.json. From the first call sim.state reports the run as not reproducible, because a debugger stopping a node moves its clock. {"node": ..., "off": true} stops debugging.

A native node#

A native node is an ordinary process on this machine, and the debugger attaches to it by its process ID:

gdb -p 2104750 ~/.cache/meshbench/firmware/native/.../meshcore-simple_repeater-linux-amd64

The answer has pid, binary and symbols: debug when the binary carries DWARF, symbols when it has a symbol table only, stripped when it has neither, and unknown when it cannot be read as ELF. For source-level debugging, build the firmware from a checkout with meshbench dev.

A breakpoint holds the node rather than failing it. A native node normally has to answer every tick within the deadline or it is marked down. While it is being debugged, a node that does not answer within two seconds is held instead: the rest of the mesh carries on without it, and when the debugger lets it go it catches up to the run's clock and rejoins. Its clock jumps by however long it was stopped, as a real board stopped by a JTAG probe does. off gives it its deadline back.

• Nothing listens on the network. No port is opened for a native node; attaching is the operating system's own process debugging, from this machine only. On Linux that needs ptrace permission: kernel.yama.ptrace_scope set to 0, or the debugger run as root.
• Boot cannot be caught. A native node runs setup() as soon as it starts, so by the time anything can attach it has booted. wait is refused for a native node for that reason.

An emulated board#

An emulated board's firmware runs inside QEMU or Renode, and the way in is the emulator's GDB stub. node.debug restarts the board with one.

{"id":2,"method":"node.debug","params":{"node":"Deck","wait":true}}

• gdb_port is where the stub listens. Pass port to choose it, a whole number from 1 to 65535; without it a free port is picked.
• wait holds the CPU at reset until a debugger continues, so boot can be stepped through.
• debugger is the gdb that speaks the board's architecture: xtensa-esp32-elf-gdb or xtensa-esp32s3-elf-gdb for an ESP32 board under QEMU, arm-none-eabi-gdb (or gdb-multiarch) for an nRF52 board under Renode.

The image a board runs is a flashable .bin or .uf2 with no symbols. Point the debugger at the firmware.elf the same build produced, for a PlatformIO build .pio/build/<env>/firmware.elf:

arm-none-eabi-gdb -ex 'target remote 127.0.0.1:3333' .pio/build/RAK_4631_repeater/firmware.elf

off restarts the board without the stub.

Where the stub listens differs by emulator, and the difference matters. A GDB stub is the whole machine: anything connected to it can read and write the board's memory and registers, and stop or change the firmware.

• QEMU binds its stub to 127.0.0.1, so only this machine can reach it.
• Renode's stub listens on every network interface, not only this machine's own. While debugging is on, anyone who can reach the port over the network has full read and write access to the emulated board. Debug a Renode board only on a network you trust, or with the port blocked by a firewall, and turn it off when finished.

Versions are per role#

MeshCore tags one role at a time. repeater-v1.17.0 and companion-v1.17.0 are different releases, and a bare v1.17.0 matches neither.

If you build with PlatformIO#

An optional post-build script hands each build over automatically, so a normal PlatformIO build appears in the library without a separate command:

extra_scripts = post:meshbench.py

The environment name supplies the board and the role, for example Heltec_v3_companion_radio_usb. This is a convenience: meshbench dev needs nothing added to your tree.