XIAO ML Kit Training over WebSerial. v145

Train a 2 layer Vision CNN AI model in the browser · Export weights to PC and the to ESP32 SD card · WebSerial ESP32 camera capture · Flash firmware from browser

Needs a Chrome or Edge Browser for the webSerial connection. Probably best not to use a cell phone.

A reminder to sponsor a TinyML Hardware kit at the openCollective mysysbook at opencollective.com/mlsysbook to support universities in the Global South

1 · Flash Firmware

Flash compiled ESP32 .bin files directly from the browser using esptool-js v0.5.7.
Requires Chrome/Edge · HTTPS or localhost · USB CDC On Boot: Enabled

If you want to look at the code and load the sketch onto your Arduino IDE it is here in text form firmware.ino

⚠ MUST REFRESH WEBPAGE FIRST before flashing (clears previous WebSerial state), then unplug and replug the ESP32 after flash completes.

▸ Option A — Sketch Only (flash.ino.bin)

Flashes only the application partition at 0x10000. Fast. Preserves bootloader and NVS.
Use when the bootloader is already on the device (standard re-flash).

Browse (optional)

Default auto-loaded: flash.ino.bin · Browse to override

▸ Option B — Full Merged Flash (flash.ino.merged.bin)

Flashes a single merged binary: bootloader + partition table + app at 0x0.
Use for fresh chips or after partition changes.

ℹ Locating .bin files in Arduino IDE 2.x

Enable: File → Preferences → Show verbose output during: compilation
The build path is printed at the end of the compilation log.

Option A — Sketch only: YourSketch.ino.bin → 0x10000
Option B — Merged: YourSketch.ino.merged.bin → 0x0
(Arduino IDE 2.x also exports via Sketch → Export Compiled Binary)

· ESP32 WebSerial & SD Browser

Connect your XIAO ESP32S3 Sense via USB · requires Chrome/Edge · USB CDC On Boot: Enabled

Not Connected

Send command:

Serial Monitor

Serial output will appear here...

Auto-scroll Note: Drag the gray area.

⊿

Decisions Before Training

The defaults have been fully tested

Use Grayscale Mode

Set BEFORE starting — 3x smaller model, needs more data

Input Resolution:

Training resolution: 64x64 — conv2 out: 29x29

Number of Classes:

Change class count here, or use Load config.json in Section 3 to set classes automatically. ⚠ Changing classes resets all image buffers.

Reminder to:

RAM window per class (max images in training buffer):

Maximum images held in browser RAM per class. Lower only if you hit browser memory limits — disk folder holds all images regardless.

SD Card Browser

Browse, view, and delete files on the ESP32 SD card. Requires firmware with SD_LIST / SD_DELETE support.

/ (root)

Connect ESP32 then press Refresh to browse SD card

3 · File & SD Transfer

PC Project Folder None — images held in RAM only
Pick a project root folder — mirrors SD card structure: root/images/ClassName/ and root/header/ created automatically. Every Capture saves to images/ClassName/img_NNNN.jpg.

Load Config & Model

TFJS Model = browser-native format (large, multi-file). Use for resuming browser training only.

		Load config.json to restore class names and settings from a previous session.
		Loads the compact ESP32 weight format back into the browser model. PC button reads `project_folder/header/myWeights.bin`. SD button reads `/header/myWeights.bin` from the ESP32 SD card via WebSerial.
		PC button reads `project_folder/images/ClassName/` for each class. SD button reads `/images/ClassName/` from the ESP32 SD card via WebSerial — ESP32 must be connected. Per-class load buttons are also available in each class row in Section 4.

4 · Camera & Data Collection

Camera Source:

The gray area is draggable.

⊿

📷 Capture = grab frame · auto-saves to images/ClassName/ if folder set · Load from PC = pick individual files into RAM buffer · 📂 Load from SD = pull images from ESP32 SD card · Clear RAM = frees tensors only (disk and SD untouched)

Activity Log

Logs will appear here...

5 · Training Progress

Batches Trained: 0
Min Samples:

Use All Data (systematic epochs)

Target Epochs: INCREASE EPOCHS TO TRAIN LONGER!

Batch Size:
Learning Rate:
Dropout Rate:

Avg Loss: --
Status: Waiting...

Ready...

(ESP32 on-device inference · slightly reduces ESP32 FPS)
(webcam / ESP32 stream · runs in browser, no ESP32 FPS impact)

Confusion Matrix

Auto-save weights every 10 batches (saves .bin + .h to project folder when set. Also saves the config.json to be safe)
Leave unchecked to protect a well-trained model from being overwritten during continued training. Manual Export myWeights.bin and Export myWeights.h buttons always work on demand.

Export to PC

myWeights.bin = binary weights for SD card deployment. myWeights.h = C header for compiling weights into firmware (#define USE_BAKED_WEIGHTS). config.json = class names + training settings.

Send to SD CardBest to manually copy all the files from the PC folder to the SD card
. This is VERY SLOW!

Sends both files to SD card /header/ folder via WebSerial (firmware v78+ required, ESP32 must be connected). Use the big green button after training to deploy your model in one click.

6 · ESP32 Export Settings & config.json Editor

Set export options below, then use Section 3 buttons to send files to PC or SD card.

Firmware Version:

useAugmentation (flip + brightness on ESP32)
useGrayscale (must match model colour mode)
imagesToPsram
Validation: of min class → images/class

validationImages = last N images/class held out. % recalculates the box live as images load. Edit the box directly to override. Written to config.json only at train-start. imagesToPsram = preload all training images to PSRAM before training.

Bulk Image Transfer PC ↔ SD

For config.json and myWeights.bin use the dedicated buttons — they are faster. This panel is for bulk-transferring image datasets. Requires firmware v78+ · Stop camera streaming before sending · Each chunk is checksum-verified.

Project folder: None selected

Status: Idle
Current file: —
Progress:

0 / 0 bytes Files: 0 / 0

config.json — Live Editor

Edit freely before saving. Syncs automatically from the fields on the left.

PC button saves to your computer (project folder or Downloads).
SD button sends directly to SD card /header/config.json via WebSerial (firmware v78+ required).

Sends both critical files to the SD card in one click. Do this after training to deploy your model.