edgecompiler Benchmark Report

Last updated: 2026-05-27 Environment: macOS arm64, Python 3.10.19, .venv-arm64

Test Runner Summary

Mode	Command	Result	Duration (s)	Notes
Simulation (without hardware acceleration)	python -m edgecompiler.test_runner --mode simulation	Exit 1	18.90	12 failed, 87 passed, 10 skipped, 4 deselected
Hardware-focused	python -m edgecompiler.test_runner --mode hardware	Exit 0	22.23	2 passed, 2 skipped, 109 deselected

Hardware Suite Detail (pytest tests/hardware -v)

Metric	Value
Passed	13
Skipped	47
Failed	0
Duration (s)	20.01

Breakdown by test file:

File	Passed	Skipped
test_coral_usb.py	2	2
test_coral_usb_runtime.py	7	10
test_all_models.py	4	35

Notes:

• test_all_models.py added: 39 parametrized tests covering all 4 EdgeTPU models (load, input/output shape, inference smoke, benchmark). 4 file-existence checks pass unconditionally; 35 delegate-dependent tests skip cleanly when the Edge TPU custom-op invoke layer is unavailable.
• Skips are deliberate and clean when the Edge TPU delegate cannot safely invoke compiled custom-op models in this environment.
• No process crashes (Trace/BPT trap) occurred after runtime/test gating changes.

Previously Skipped 14 Tests (Targeted Re-run)

Command:

pytest \
  tests/hardware/test_coral_usb_runtime.py::TestModelLoading::test_load_edgetpu_model \
  tests/hardware/test_coral_usb_runtime.py::TestModelLoading::test_get_input_size \
  tests/hardware/test_coral_usb_runtime.py::TestInference::test_single_inference \
  tests/hardware/test_coral_usb_runtime.py::TestInference::test_inference_with_random_input \
  tests/hardware/test_coral_usb_runtime.py::TestInference::test_top_classes \
  tests/hardware/test_coral_usb_runtime.py::TestInference::test_classify_with_labels \
  tests/hardware/test_coral_usb_runtime.py::TestBenchmark::test_benchmark_100_runs \
  tests/hardware/test_coral_usb_runtime.py::TestBenchmark::test_benchmark_latency_under_30ms \
  tests/hardware/test_coral_usb_runtime.py::TestOutputCorrectness::test_output_shape \
  tests/hardware/test_coral_usb_runtime.py::TestOutputCorrectness::test_deterministic_output \
  tests/hardware/test_coral_usb.py::TestCoralUSB::test_coral_device_available \
  tests/hardware/test_coral_usb.py::TestCoralUSB::test_coral_inference \
  tests/hardware/test_coral_usb.py::TestCoralUSB::test_coral_benchmark \
  tests/hardware/test_coral_usb.py::TestCoralUSB::test_coral_device_properties -v -s --tb=short

Outcome	Count
Passed	2
Skipped	12
Failed	0
Crashed	0

Model Download Status (scripts/download_models.sh)

Output directory: tests/hardware/test_models

Asset	Status
mobilenet_v1_1.0_224_quant_edgetpu.tflite	Downloaded (4.5 MB)
mobilenet_v2_1.0_224_quant_edgetpu.tflite	Downloaded (4.1 MB)
imagenet_labels.txt	Downloaded (21 KB)
coco_labels.txt	Downloaded (661 B)
parrot.jpg	Downloaded (583 KB)
efficientnet-edgetpu-S_quant_edgetpu.tflite	Downloaded (6.8 MB) — sourced from google-coral/test_data
ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite	Downloaded (6.6 MB) — sourced from google-coral/test_data