XIAO ML Kit Training over WebSerial.

Train a 2 layer Vision CNN AI model in the browser · Export weights to PC and the to ESP32 SD card · WebSerial ESP32 camera capture · Flash firmware from browser

Base github for this page at github.com/webmcu-ai/webmcu-vision-web.

While proof of concept esp32s3 on-device training github is at https://github.com/webmcu-ai/on-device-vision-ai



1 · Flash Firmware

Flash compiled ESP32 .bin files directly from the browser using esptool-js v0.5.7.
Requires Chrome/Edge · HTTPS or localhost · USB CDC On Boot: Enabled

If you want to look at the code and load the sketch onto your Arduino IDE it is here in text form firmware.ino

⚠ MUST REFRESH WEBPAGE FIRST before flashing (clears previous WebSerial state), then unplug and replug the ESP32 after flash completes.

▸ Option A — Sketch Only (flash.ino.bin)

Flashes only the application partition at 0x10000. Fast. Preserves bootloader and NVS.
Use when the bootloader is already on the device (standard re-flash).

Browse (optional)
Default auto-loaded: flash.ino.bin · Browse to override

▸ Option B — Full Merged Flash (flash.ino.merged.bin)

Flashes a single merged binary: bootloader + partition table + app at 0x0.
Use for fresh chips or after partition changes.

ℹ Locating .bin files in Arduino IDE 2.x
Enable: File → Preferences → Show verbose output during: compilation
The build path is printed at the end of the compilation log.

Option A — Sketch only: YourSketch.ino.bin → 0x10000
Option B — Merged: YourSketch.ino.merged.bin → 0x0
(Arduino IDE 2.x also exports via Sketch → Export Compiled Binary)


· ESP32 WebSerial & SD Browser

Connect your XIAO ESP32S3 Sense via USB · requires Chrome/Edge · USB CDC On Boot: Enabled

Not Connected






Serial Monitor
Serial output will appear here...


⊿
 SD Card Browser

Browse, view, and delete files on the ESP32 SD card. Requires firmware with SD_LIST / SD_DELETE support (esp_all_menu_image_train_infer68).

/ (root)
Connect ESP32 then press Refresh to browse SD card

Sync config from SD card

Read /header/config.json from SD and update all fields including class names.



4 · Camera & Data Collection

Set BEFORE starting — 3x smaller model, needs more data



Training resolution: 64x64 — conv2 out: 29x29







⊿
📁 PC Project Folder None — images held in RAM only
Pick a project root folder — mirrors SD card structure exactly: root/images/ClassName/ and root/header/ are created automatically. Every Capture saves to images/ClassName/img_NNNN.jpg.

Maximum images held in browser RAM per class. Set high enough to hold all your images — disk folder holds all images regardless. Lower only if you hit browser memory limits.



Class 0:
0 samples



Class 1:
0 samples



Class 2:
0 samples




📷 Capture = grab frame · auto-saves to images/ClassName/ if folder set · Load from PC = pick individual files into RAM window · 📂 Load from Folder = loads from existing project folder's images/ sub-folders into RAM · 📂 Load from SD = pull from ESP32 SD card · Clear RAM = frees tensors only (disk and SD untouched) · To copy images to ESP32 SD: eject SD card, copy images/ folder manually from your PC project folder, re-insert.



Ready...




(ESP32 on-device inference · slightly reduces ESP32 FPS)
(webcam / ESP32 stream · runs in browser, no ESP32 FPS impact)

5 · Training Progress

Batches Trained: 0
Min Samples:







Avg Loss: --
Status: Waiting...




Leave unchecked to protect a well-trained model from being overwritten during continued training. Manual Export myWeights.bin and Export myWeights.h buttons always work on demand.

Confusion Matrix


Activity Log

Logs will appear here...

6 · ESP32 Export (SD Format)

Exports weights + config for the XIAO ESP32S3 Sense. Copy files to SD card /header/ folder.






imagesToPsram = preload all training images to PSRAM before training. validationImages = last N images per class held out for validation.


Binary: JSON header + float32 weights. Saves to project_folder/header/myWeights.bin if a project folder is set, otherwise to PC Downloads. Auto-saved every 10 batches when the auto-save toggle is checked — copy to SD card /header/myWeights.bin manually.

C header file — copy to sketch folder alongside your .ino, then uncomment #define USE_BAKED_WEIGHTS to compile weights directly into firmware (v66+). Saves to project_folder/header/myWeights.h if a project folder is set, otherwise to PC Downloads. Auto-saved every 10 batches when the auto-save toggle is checked.

 config.json — Direct Download

Live textarea — edit freely before saving. Syncs from fields above.


Downloads button saves to your computer — copy to SD card /header/config.json manually.


GENERATE .h Header File — Legacy / Optional

For older firmware that reads weights from a compiled-in C header instead of the SD card.




7 · View / Edit Source Code


By Jeremy Ellis (hpssjellis) · LinkedIn · Support opencollective/mlsysbook · XIAO ML Kit $22 USD · Use at your own risk · MIT License