WebBLE Dual Board Control - v16

Nano 33 BLE Sense (peripheral, inference-only) + XIAO ESP32S3 (dual role: central to Nano33, peripheral to this page)

Uses Web Bluetooth directly for the boards - Chrome or Edge on Windows/Mac/Linux/ChromeOS/Android only. Phone motion training below uses a separate, plain-web-standards API (DeviceMotionEvent) that also works in mobile Safari.

This page can train a model right here in the browser, using the identical Conv1D+Dense+Dense+Output architecture both boards run, then push the trained weights to either board. The Nano33 never trains - it only ever receives a finished model (from the ESP32, from this page, or baked in at compile time) and runs inference locally, reporting results back over BLE. Quick and full inference both work everywhere.

What's new in v16 - browser-only fix, no firmware changes needed (nano33-v14.ino / esp32-v14.ino still current):

• Fixed phone-motion-trained models coming out entirely NaN. Symptoms were: the live loss chart trailing off partway through training instead of showing all epochs, "Run Browser Inference" showing NaN for every class, and the SAME NaN persisting even after successfully pushing that model to the Nano33 (i.e. this wasn't a display bug - the trained weights themselves were corrupted). Investigated the whole phone data path - window layout, zero-filling the channels a phone doesn't have (magnetometer/mic/all 7 static channels), and the normalization/ clipping in myNormalizeWindow() - and all of that was already correct; zero-filling unavailable channels is the right approach and was never the problem. The actual bug: the hand-rolled SGD in myTrainStep() applied every weight update completely unclipped. Phone datasets tend to be smaller and lower-diversity than board ones (fewer samples, and 10 of the 17 input channels carry zero information for every single sample), which concentrates all the learning signal onto a handful of live weights - making it much easier for one epoch to take an oversized step, overshoot a weight into NaN/Infinity, and then have every future step touching that weight compute to NaN forever after (there was no path back once that happened). Every weight/ bias update is now routed through a new myClipStep() that bounds each individual update to a small fixed range - this makes that first NaN structurally impossible, for phone OR board data, with no effect on runs that were already numerically stable. Added an epoch-snapshot-and-rollback safety net in myTrainInBrowser() as well, in case anything else ever manages to produce a non-finite loss - it rolls back to the last good epoch and halves the learning rate instead of continuing to train on a corrupted model.

What's new in v14.2:

• "Fetch ESP32 Model to Browser" - round 3, real fix. v14.1's much more patient retry loop still stalled at the EXACT same byte count (2968) as before, byte-for-byte, on a second run with different retry timing - which rules out ordinary congestion (that would vary run to run) and points at a hard limit specific to BLE's confirmed INDICATE mechanism rather than something more retrying could fix. This matches a known class of Web Bluetooth / OS Bluetooth stack limitation with sustained indicated transfers. The ESP32's outbound binary channel is switched from INDICATE to NOTIFY (unconfirmed) - it no longer depends on an OS-level confirmation channel that had demonstrated it doesn't reliably complete for this data volume.
• New BLE-free fallback: "Paste Model From Serial". In case NOTIFY still isn't fully reliable on some machine/browser/OS combination, the ESP32 got a new e Serial command that dumps its trained model as plain hex text over USB Serial. Copy it out of the Serial Monitor and paste it into the new box under the ESP32's panel - this goes over USB + clipboard, not BLE, so it's immune to any BLE-side limitation entirely. Same underlying idea as the existing "Save myWeights.h" download, applied to a live model load instead of a compile-time bake-in.

What's new in v14.1:

• Fixed the webpage misreading on-device inference results. Both boards send predIdx,conf,p0,p1,...,pN-1 - predicted index, then a deliberate duplicate of that class's own confidence, then the full per-class array. The page was only skipping the first field, so the duplicate "conf" value got mistaken for class 0's probability and every real value shifted one slot late - the classic symptom was two classes both showing near 100%, even though each board's own Serial Monitor (which never goes through this parsing) was correct the whole time. Now skips both fields correctly.
• "Fetch ESP32 Model to Browser" - round 2 (superseded by v14.2 above). v14's fix still stalled partway through the ~131-chunk model transfer in real testing. A single 30ms retry didn't give the BLE stack close to enough time to recover, so myServerSendBinary() was changed to retry a stuck chunk in a bounded loop (up to 3 seconds) instead of giving up almost immediately. Turned out to help but not fully solve it - see v14.2.

What's new in v14 - requires nano33-v14.ino AND esp32-v14.ino:

• Consistent "On-device result" text everywhere. The Nano33 and ESP32 used to print different Serial Monitor formats for full vs quick inference (and neither board matched the other). Both boards now build the result line through one shared function each, and this page's on-screen text uses the identical format: On-device result: 1normal (0unknown=3%, 1normal=90%, 2issue=7%,)
• Nano33 got a 'd' debug-toggle Serial menu item, matching the ESP32's - it had none before, so its per-timestep sampling debug lines were always on with no way to quiet them.
• "Fetch ESP32 Model to Browser" - first attempt. v13's fix (a blind pacing delay between BLE indicate() calls) reduced but didn't eliminate dropped chunks on the ~131-chunk model transfer, because a delay alone never confirms anything actually arrived. myServerSendBinary() now checks each indicate()'s own return value instead of just hoping (superseded by the more patient retry logic in v14.1 above).
• The webpage no longer gets permanently stuck after one failed/stalled transfer. A binary transfer that stalls partway (e.g. the bug above, before it was fixed, or any other dropped packet) used to leave the browser's receive state stuck forever, silently corrupting every Capture/Quick/Fetch for that board afterwards until the page was reloaded. A stale-transfer watchdog now resets it automatically and logs why.
• "Run Browser Inference" relabeled and given a tooltip explaining what it actually does: runs the CURRENT browser model against the LAST CAPTURED window, entirely in JavaScript, no BLE involved - handy as a cross-check when a board's own on-device result isn't showing up.
• Added a Serial Monitor command reference (further down this page, in the Protocol notes section) - previously this page only documented the BLE wire protocol, never each board's own USB-Serial menu keys.

What was new in v13:

• Fixed a Nano33 compile error some sketch edits could trigger ("used before declared") - a couple of global variables got referenced (in the v11/v12 model-push fixes) above the point in the file where they were declared. Arduino's build step only auto-forward-declares functions, never variables, so a global has to physically sit above every place that uses it. Moved the affected globals up next to the model-receive state they're tied to, so this class of ordering bug is much less likely to resurface as the file keeps growing.
• Attempted a fix for "Fetch ESP32 Model to Browser" not completing (superseded by the real fix in v14 above) - added a pacing delay between indicate() calls, which helped but didn't fully solve it.
• Nano33 graphs can now be shown/hidden. A "Show graphs" checkbox next to the Nano33's heartbeat label toggles both its charts off and on.

What was new in v12: model pushes to the Nano33 switched from write-without-response to write-with-response (the browser now waits for each chunk's acknowledgement before sending the next, instead of flooding faster than the board could keep up), plus other commands to a board are held off while a push to it is in flight.

Protocol notes (must match the firmware exactly):
Service UUID: 7e400001-b2c3-5d4e-af60-9b3c7d8eaf20
Control char (write): 7e400002-... - browser sends "CAPTURE", "QUICK", "TRAIN", "INFER", "QUICKINFER", "PUSHTONANO", or "PUSHMODEL" as plain ASCII text (Nano33 only understands CAPTURE/QUICK/INFER/QUICKINFER - it never trains)
Heartbeat char (notify, ~2Hz): 7e400003-... - CSV text "ax,ay,az,mic,temp,hum,prox"
Binary char (notify, chunked) [CHANGED v14.2 - was indicate on the ESP32 side; the Nano33's binary char is still indicate, since its transfers are small (up to ~1.6KB) and haven't shown this issue]: 7e400004-... - 4-byte little-endian uint32 length header, then raw float32LE payload in ~180-byte chunks. 407 floats = full 1-second window, 17 floats = QUICK reading, 5872 floats = a full MODEL PACKAGE (only sent by the ESP32 in response to "PUSHMODEL").
Model char (write, chunked) [NEW v02]: 7e400005-... - same 4-byte-header + chunk pattern, browser -> board, carries a model package (numClasses + weights + calibration) into either board's live weights.
Result char (notify) [NEW v02]: 7e400006-... - CSV text "predIdx,conf,p0,p1,...,pN-1" sent after any INFER/QUICKINFER, by whichever board just ran the forward pass.
myFusionWeights.h [NEW v09]: not BLE - a plain-text C++ header download (Save myWeights.h button) with the same numbers as the .bin, for baking a model into either sketch at compile time (USE_BAKED_WEIGHTS).
Phone motion training [NEW v09]: not BLE either - uses the browser's own DeviceMotionEvent API on the phone/tablet the page is open on, feeding straight into the same labeled dataset as the two boards.
On-device result text format [CHANGED v14]: both boards' Serial Monitor AND this page's "On-device result" line now use the exact same format, produced by one shared function on each board (myPrintResultLine in both .ino files): "On-device result: 1normal (0unknown=3%, 1normal=90%, 2issue=7%,)" - the trailing comma before the closing paren is intentional (falls out of the shared per-class loop), kept so the two are byte-for-byte identical.

Serial Monitor commands (type these into each board's own USB Serial Monitor at 115200 baud - these are NOT sent over BLE, they only work with a USB cable plugged into that specific board):

Nano33 (nano33-v14.ino) - no training on this board by design:
  i = Infer once/continuous (real 1s window, 'x' to stop)
  q = Quick continuous infer (fast replicated reading, 'x' to stop)
  s = status (BLE connection + model-loaded state)
  d = toggle debug output [NEW v14] (verbose per-timestep sampling lines)
  z = reset (restart the board)
  ? = show this menu

ESP32S3 (esp32-v14.ino):
  1-3 = Collect a sample for that class (then c to capture, x to exit)
  t = Train (on-device, from SD-collected samples)
  i = Infer once/continuous (real 1s window, 'x' to stop)
  q = Quick continuous infer (fast replicated reading, 'x' to stop)
  r = (re)connect to the Nano33
  k = recalibrate
  s = status
  p = push the current model to the Nano33 for on-device inference
  e = export the current model as pasteable hex text [NEW v14.2] - BLE-free fallback, see "Paste Model From Serial" on this page
  d = toggle debug output (verbose BLE/transfer diagnostics)
  b = cycle BLE mode (Dual / Peripheral only / Central only / Off)
  z = restart now, to apply a newly-selected BLE mode
  ? = show this menu

The Control char commands above ("CAPTURE", "QUICK", etc) are the separate BLE-over-the-air command set this webpage actually sends - use the "Custom command" box on either board's panel to send any of these, or the single letters 'r'/'k'/'s' (the ESP32 also accepts those same letters over BLE, not just USB Serial).