closed_loop_audio_test_suite/ARTIFACT_DETECTION_README.md

Artifact Detection Test

Overview

This test plays a 1kHz sine wave for a configurable duration (default 60 seconds) and records both channels simultaneously:

• Channel 1: Loopback path (direct audio interface connection)
• Channel 2: DUT/Radio path (through beacon and radio transmission)

The test detects buzzing, clicks, dropouts, and other audio artifacts using multiple configurable algorithms.

Quick Start

python test_artifact_detection.py \
  --serial-number SN001234 \
  --software-version abc123 \
  --comment "Testing new firmware"

Detection Algorithms

The test uses four configurable detection algorithms (spectral_anomaly is disabled by default due to false positives):

1. Spectral Anomaly Detection (DISABLED BY DEFAULT)

• Status: ⚠️ Currently generates too many false positives - disabled by default
• What it detects: Unexpected frequencies that aren't harmonics of the fundamental tone
• Use case: Buzzing, interference, crosstalk
• Configuration: threshold_db - how far below fundamental to search (-60 dB default)

2. Amplitude Spike Detection (WORKING)

• What it detects: Sudden changes in signal amplitude (RMS)
• Use case: Clicks, pops, dropouts
• Configuration: threshold_factor - number of standard deviations (3.0 default)

3. Zero-Crossing Anomaly Detection (WORKING)

• What it detects: Irregular zero-crossing patterns
• Use case: Distortion, clipping, non-linear artifacts
• Configuration: threshold_factor - number of standard deviations (2.0 default)

4. Energy Variation Detection (WORKING)

• What it detects: Rapid energy changes between time windows
• Use case: Dropouts, level fluctuations, intermittent issues
• Configuration: threshold_db - energy change threshold (6.0 dB default)

Configuration

Edit config.yaml to customize the test:

artifact_detection:
  test_frequency: 1000  # Hz
  duration: 60.0        # seconds
  amplitude: 0.5        # 0.0 to 1.0
  detectors:
    spectral_anomaly:
      enabled: true
      threshold_db: -40
    amplitude_spikes:
      enabled: true
      threshold_factor: 3.0
    zero_crossing:
      enabled: true
      threshold_factor: 2.0
    energy_variation:
      enabled: true
      threshold_db: 6.0

Command Line Options

• --serial-number: Serial number (required)
• --software-version: Git commit hash or version (required)
• --comment: Optional comments about the test
• --config: Path to config file (default: config.yaml)
• --duration: Override duration in seconds
• --frequency: Override test frequency in Hz

Example: Quick 10-second Test

python test_artifact_detection.py \
  --serial-number SN001234 \
  --software-version abc123 \
  --duration 10

Example: Custom Frequency

python test_artifact_detection.py \
  --serial-number SN001234 \
  --software-version abc123 \
  --frequency 440

Tuning Detection Algorithms

More Sensitive Detection

To catch more subtle artifacts, make thresholds stricter:

detectors:
  spectral_anomaly:
    threshold_db: -50      # Lower = more sensitive
  amplitude_spikes:
    threshold_factor: 2.0  # Lower = more sensitive
  zero_crossing:
    threshold_factor: 1.5  # Lower = more sensitive
  energy_variation:
    threshold_db: 3.0      # Lower = more sensitive

Less Sensitive Detection

To reduce false positives in noisy environments:

detectors:
  spectral_anomaly:
    threshold_db: -30      # Higher = less sensitive
  amplitude_spikes:
    threshold_factor: 4.0  # Higher = less sensitive
  zero_crossing:
    threshold_factor: 3.0  # Higher = less sensitive
  energy_variation:
    threshold_db: 10.0     # Higher = less sensitive

Disable Specific Detectors

detectors:
  spectral_anomaly:
    enabled: false  # Turn off this detector

Output

The test generates:

1. YAML results file: test_results/{timestamp}_artifact_detection_results.yaml
2. JSON results file: test_results/{timestamp}_artifact_detection_results.json
3. Summary plots (if enabled): test_results/{timestamp}_artifact_detection/
   • Time domain waveforms with artifact markers
   • Frequency spectrum analysis
4. Individual anomaly plots: test_results/{timestamp}_artifact_detection/individual_anomalies/
   • Each anomaly plotted individually with ~20 periods of context
   • Detailed view showing exactly what the anomaly looks like
   • Named by channel, type, and timestamp for easy identification

Results Structure

metadata:
  test_id: "20260317_140530"
  timestamp: "2026-03-17T14:05:30.123456"
  test_type: "artifact_detection"
  pcb_version: "v2.1"
  pcb_revision: "A"
  software_version: "abc123"

artifact_detection_result:
  test_frequency_hz: 1000
  duration_sec: 60.0
  channel_1_loopback:
    total_artifacts: 5
    artifact_rate_per_minute: 5.0
    artifacts_by_type:
      spectral_anomaly: 2
      amplitude_spike: 3
  channel_2_dut:
    total_artifacts: 23
    artifact_rate_per_minute: 23.0
    artifacts_by_type:
      spectral_anomaly: 8
      amplitude_spike: 10
      energy_variation: 5
  detector_config: {...}

Interpreting Results

• Zero artifacts in both channels: Excellent signal quality
• Same artifacts in both channels: Likely environmental interference or audio interface issue
• More artifacts in Channel 2 (radio path): Radio transmission degradation detected
• High spectral_anomaly count: Interference or crosstalk
• High amplitude_spike count: Clicks, pops, or dropouts
• High energy_variation count: Level instability or dropouts

Comparison with Loopback Baseline

The loopback path (Channel 1) serves as a baseline reference. Any additional artifacts in the radio path (Channel 2) indicate degradation introduced by the radio transmission system.

Expected behavior:

• Loopback should have minimal artifacts (ideally zero)
• Radio path may have some artifacts due to transmission
• Large difference indicates issues in radio hardware/firmware