A neural keyword spotter on an ESP32-S3: say “go” at the Mac microphone and the chip prints >>> heard: GO. It exists twice, once in C++ and once in Rust, on the same ESP-IDF v6.1 checkout, the same board config, the same trained weights, and every timing in this piece was read off the board.
Why build the comparison at all
Advice about Rust on microcontrollers comes in two directions: the future of embedded firmware, or a masochistic detour around perfectly good C++. Neither camp brings measurements, and a production language choice should not rest on vibes. So I built the same firmware twice and put both builds on the same board, with controls tight enough that a skeptical room cannot dismiss the result.
The constraints dominate the design: an ESP32-S3-WROOM-1 at 240 MHz, 512 KB of internal SRAM, 8 MB of PSRAM, FreeRTOS, nothing else. The firmware analyses the last second of audio every 240 ms, so the per-window budget is a correctness requirement: overrun it and the UART receive buffer silently drops samples. The per-window budgets were computed on paper before any firmware existed; this repo is where a trained model that small met silicon.
The workload is keyword spotting over “yes no up down left right on off stop go” plus silence and unknown, 12 classes: a DS-CNN with 7,212 parameters and about 3.3 M MAC per inference, quantised int8, behind an integer log-mel front-end (FROM REPO: docs/ARCHITECTURE.md). The network runs on one of two backends behind the same function, selected at build time: portable hand-written kernels, the language experiment, or ESP-DL 3.3.13, Espressif’s SIMD library, the production path.
15 commits, September 30 to October 16, 2026 (MEASURED: git log; shallow clone, so a floor). The README calls the repo a worked example and the base for a systems-engineering tech talk, so the bar was that the comparison survive a hostile audience. What follows is the pipeline, four decisions, the numbers, and what broke.
The pipeline both languages implement
flowchart LR
MIC["Mac microphone / WAV"] --> HOST["host/mic_stream.py: 20 ms frames"]
HOST -->|"UART0 921600 baud: A5 5A type len payload sum8"| RING["1 s ring buffer, 16000 x int16"]
RING -->|"every 240 ms"| FE["integer front-end: FFT512, mel, log2"]
FE -->|"49 x 40 int8"| NN["int8 DS-CNN: 7212 params"]
NN -->|"12 int32 logits"| DET["detector: margin >= 3000, 2 windows in a row"]
DET -->|"@DETECT word"| CON["host console"]
subgraph SPEC["model/ - the executable specification"]
TR["train + quantise in PyTorch"] --> REF["Python integer reference + 12 golden clips"]
end
REF -->|"feature CRC + logits checked on device, every boot"| NN
The tested path has no microphone on the board: the Mac streams 16 kHz PCM in 20 ms frames over the console UART at 921600 baud, framed with sync word, length and checksum (FROM REPO: docs/ARCHITECTURE.md). A ring buffer holds the last second; every 240 ms the engine computes 49 x 40 int8 log-mel features, runs the network, and a state machine announces a keyword only when the same class wins by a margin of 3000 in two consecutive windows, then mutes for a second. The wake-word follow-up adds an I2S microphone option that compiles but is untested on hardware (FROM REPO: docs/WAKEWORD.md).
Everything on the device side exists twice, cpp/main/*.cpp and rust/src/*.rs, with the same module names, function names and text output. The SDK underneath is the same checkout; the one deliberate difference is the compiler for application code: GCC for C++, rustc on Espressif’s LLVM Xtensa fork for Rust.
Testing is three layers deep in both languages: a boot self-test comparing feature CRC and all logits of 12 embedded golden clips against the Python reference; tests/e2e_stream.py, which streams the clips through the live UART path and compares every one of the 101 windows, not just the detections; and the model evaluation harness in model/ (FROM REPO: docs/ARCHITECTURE.md). That compare-every-window layer is what caught the failure described at the end.
Decisions I would defend
One integer specification, written once in Python
model/kws/frontend.py and model/kws/net.py::int_forward are the executable specification: NumPy with explicit integer semantics. Both firmwares are ports of it, and the boot self-test proves the ports against it on real hardware, every boot. Floating point was rejected on an argument I would make again: a float FFT differs between GCC and LLVM (FMA contraction, libm), so the two firmwares would agree only to a tolerance, and a regression could hide inside it. With integers, any mismatch is a bug. The first run of both firmwares was 12/12 bit-exact against the reference (FROM REPO: docs/COMPARISON.md).
The cost is hand-written fixed-point DSP: a 512-point FFT with Q15 twiddles and a shift after every butterfly, 40 triangular mel filters with u64 accumulation, and log2 through a 64-entry table. All of it must be identical in three languages, and the Python reference asserts that no intermediate exceeds int32.
Same SDK, really
Fair means setup discipline. C++ builds against ESP-IDF directly with CMake. The Rust build drives the same IDF checkout through esp-idf-sys, so bootloader, components, partition table and the GCC that compiles the SDK are identical in both builds. Diffing the two generated sdkconfig files shows exactly four differences, all task stack sizes (main task 8 vs 16 KiB) (FROM REPO: docs/ARCHITECTURE.md). The rejected alternative is esp-hal, the no_std path without ESP-IDF, as a third column; it was never evaluated, and the repo says so instead of guessing.
“Same SDK” still cost one fight: IDF 6.x defaults to picolibc, esp-idf-sys 0.38.1’s bindgen step resolved newlib’s headers first and died with unknown type name '__FILE', and the fix pins newlib in the shared board config so both languages run identical SDK bits (FROM REPO: docs/COMPARISON.md). That is the integration tax benchmark tables never show.
Two backends behind one function signature
Both firmwares expose the network as run_net(features, workspace, logits): built-in kernels portably, ESP-DL through the vendor’s SIMD code. ESP-DL is C++-only, so both firmwares reach it through the same C shim (41 lines, MEASURED: wc -l on shared/espdl/kws_espdl_shim.cpp), which the Rust build compiles as an extra IDF component and calls over FFI:
impl NetWorkspace {
pub fn new() -> Self {
assert_eq!(unsafe { kws_espdl_init() }, 0, "ESP-DL model failed to load");
Self(())
}
}
/// features: [49][40] i8 -> logits: [12] i32 (raw int8 model outputs, real = value / 8)
pub fn run_net(features: &[i8], _ws: &mut NetWorkspace, logits: &mut [i32; N_CLASSES]) {
assert_eq!(features.len(), N_FRAMES * N_MEL);
assert_eq!(unsafe { kws_espdl_run(features.as_ptr(), logits.as_mut_ptr()) }, 0);
}
From rust/src/nn_espdl.rs; the C++ twin calls the same shim without the unsafe blocks.
Two alternatives got rejected. TFLite Micro, because the built-in kernels exist precisely for bit-exactness across languages and readability; the vendor library is the production path, not the experiment (FROM REPO: docs/TALK_OUTLINE.md). And w8a16 quantisation: 93.0% validation accuracy in emulation, but int16 activations double activation memory and slow the kernels, so it was not deployed (FROM REPO: docs/MODEL.md). ESP-DL’s own price: 1.07 MB more flash (the 1 MB app partition no longer fits; a 3 MB one ships), about 44 KiB less free RAM, and a second quantiser with its own story, below.
Faster, without changing one output bit
The first kernels were the obvious ones, and they were too slow: the network alone took 153.5 ms in C++ and 228.6 ms in Rust against the 240 ms budget (MEASURED: docs/COMPARISON.md). The optimisation ladder that followed is the most instructive table in the repo: 4-channel blocking in the pointwise kernels, uint8 activations and int16 weights (values still int8) so each operand loads in one instruction, and a requantisation that uses only the high word of the product, because Xtensa has no 64-bit shifts and the compiler was emitting a long inline sequence for each of the 144,000 outputs. C++ went 153.5 to 79.5 ms, Rust 228.6 to 139.1 ms, and the boot self-test re-proved bit-exactness after every step. The same pointwise inner loop from both trees shows where the remaining gap lives, first cpp/main/nn.cpp:
for (int oc = 0; oc < CHANNELS; oc += 4) {
const int16_t *w0 = w + oc * CHANNELS, *w1 = w0 + CHANNELS, *w2 = w1 + CHANNELS, *w3 = w2 + CHANNELS;
int32_t a0 = bias[oc], a1 = bias[oc + 1], a2 = bias[oc + 2], a3 = bias[oc + 3];
for (int ic = 0; ic < CHANNELS; ++ic) {
const int16_t x = px[ic]; // 16x16-bit multiplies (MUL16S) are cheaper than 32x32
a0 += w0[ic] * x; a1 += w1[ic] * x; a2 += w2[ic] * x; a3 += w3[ic] * x;
}
Then rust/src/nn.rs:
for i in 0..40 {
let x = win[i] as i32;
a0 += w0[i] as i32 * x;
a1 += w1[i] as i32 * x;
a2 += w2[i] as i32 * x;
a3 += w3[i] as i32 * x;
}
The C++ keeps both operands at 16 bits and GCC selects the cheaper 16x16 multiply; the Rust version widens to i32, the idiomatic safe shape. Over the whole ELF, the C++ build carries 45 mul16s and 104 mull, the Rust build 9 and 668 (FROM REPO: docs/COMPARISON.md). That is a correlation over whole binaries, not a proof, and the docs file it under hypothesis.
Rejected: blaming bounds checks. The hottest Rust kernel was rewritten with raw pointers and no checks: no faster, 240 ms vs 229 ms for the whole network. Safe iterator code was not the tax; code generation was (MEASURED: docs/COMPARISON.md, section 3).
Where the languages actually differ
Speed, size, build time and free RAM all favour C++; the numbers are below. The more interesting answer is what Rust bought, and the docs are precise about it: shape, not speed. The UART frame parser is an exhaustive match on an enum (the C++ twin is a switch, where a forgotten case compiles); the detector state is Option<usize> instead of a -1 sentinel; slices carry their lengths, so a wrong chunk size fails loudly (rust/src/host_link.rs, rust/src/engine.rs and their cpp/main twins). The docs add the counterweight: none of these prevented a bug that actually occurred. The bugs worth remembering were system-level: stride confused with frame size, PSRAM placement, a 115200 vs 921600 baud mismatch (FROM REPO: docs/COMPARISON.md, section 6). On a codebase this small, memory safety bought clean code, not fewer bugs.
Numbers
Every number comes from this repository, labeled by how it was verified.
- Per 1 s window in the live loop: C++ 85.8 ms, Rust 151.1 ms built-in; 14.6 ms and 20.3 ms with ESP-DL (MEASURED: esp_timer, 20 runs, ±0.1 ms; FROM REPO docs/COMPARISON.md)
- Network on ESP-DL: 8.3 ms in both languages, 9.6x the C++ built-in kernels and 17x Rust’s; CPU load at one window per 240 ms: 36% vs 63% built-in (MEASURED: docs/COMPARISON.md)
- App image: 620,208 B vs 858,176 B built-in, 1,691,616 B vs 1,929,040 B with ESP-DL, each embedding 375 KiB of test clips (MEASURED: xtensa-esp32s3-elf-size; FROM REPO docs)
- Internal RAM free once ready: 272 vs 256 KiB built-in, 228 vs 212 KiB with ESP-DL; the model runs entirely from internal SRAM, PSRAM unused (FROM REPO: docs/ARCHITECTURE.md)
- Clean build, warm caches: 37 s vs 66 s; 183 packages in Cargo.lock (MEASURED: docs/COMPARISON.md; lockfile counted in this clone)
- Firmware source: 552 C++ lines vs 601 Rust, built-in backends, no generated code (MEASURED: wc -l; FROM REPO docs/COMPARISON.md)
- Model quality: 92.49% int8 on 4,914 held-out clips (92.71% float); streaming detector: 86% of keyword clips right, 2.9% of non-keyword clips trigger an event;
@DETECTlands at roughly the moment the word ends, ±0.4 s (FROM REPO: docs/MODEL.md, README; demo-grade, labeled so) - Quantisation: ESP-PPQ’s default
klcalibration collapsed ESP-DL validation accuracy to 54.6%;mseplus 300 LSQ steps recovered test accuracy to 92.57% vs 92.49% built-in (FROM REPO: docs/MODEL.md) - UNKNOWN: the root cause of the Rust/C++ kernel gap (hypothesis: back-end code generation), the ESP-DL device-vs-emulation rounding difference, and everything about
esp-hal, which was not evaluated
What broke, and what I would do differently
The first Rust firmware passed its demo and failed real time. The network alone took 228.6 ms of the 240 ms budget, the UART receive buffer overflowed, and the end-to-end test reported 86 windows instead of 101 with shifted contents, while the detections still looked right (FROM REPO: docs/COMPARISON.md, section 1). A test that compares every window caught what a demo would have hidden.
The vendor quantiser nearly shipped a broken model. ESP-PPQ’s default KL calibration produced a model at 54.6% validation accuracy that exported without complaints; the error report showed roughly 40% noise-to-signal power in the last layers, a textbook outlier-histogram failure of power-of-two scales (FROM REPO: docs/MODEL.md). “It exported fine” is not a result; accuracy gets re-measured after every vendor tool now. Likewise, the incremental front-end made the cold path 3 ms slower (25.5 to 28.5 ms), cause never chased; the docs report what was measured, not a tidy story.
Some things I still cannot explain, and the repo says so: on 58 of the 101 live windows the device’s ESP-DL logits differ from ESP-PPQ’s emulation by 1-2 output units, enough to flip the top class on two near-tie windows (detections never differ); all rounding policies were tried, the cause is unknown, and the e2e test documents a ±3-unit tolerance. The Rust optimisation ladder was also only measured at its first and last steps, while C++ was measured at every rung: an asymmetry I created by fixing C++ first.
The production direction got decided without the full second column. The wake-word follow-up, a user-trainable phrase flashed as a 27 KB model partition in 0.3 s, shipped C++-only; a Rust wake firmware is listed as a possible next step, not built (FROM REPO: docs/WAKEWORD.md). The open experiment is esp-hal: if the gap really is back-end code generation, a different compiler target would settle it.
Code: private repository - write to [email protected] to learn more.