Design and Knowledge Distillation of LPDNet for Efficient Edge Driver Distraction Recognition

Visual driver monitoring covers safe driving, phone use, texting, drinking, radio operation, and other cabin activi-ties. The State Farm competition popularized a ten-class still-image formulation [1], while subsequent work has examined posture classification, fine-grained activity recognition, anomaly detection, multimodal perception, and compact driver-monitoring models [2,5–12]. Images from one driver are strongly related; consequently, driver-aware evaluation is important when the intended claim concerns unseen people [3,4]. An edge implementation adds a second requirement. A classifier that performs well on a workstation can be un-suitable for a small CPU that shares memory and compute with other functions. ResNet, MobileNet, ShuffleNet, Ef-ficientNet, and ConvNeXt represent different accuracy–cost designs [13–18], yet parameter or operation counts do not uniquely determine runtime. Operator layout, memory traffic, software, and hardware can change measured la-tency [16,26,27,30–32]. Recognition, theoretical complex-ity, and target-device behavior therefore need to be con-sidered together. This distinction matters operationally. Pure model la-tency describes the exported network, whereas image and replay pipelines also include decoding and preprocessing. A model can therefore occupy one position in a complexi-ty table and another in a device-throughput comparison. Our central engineering question is how the tested LPDNet operating points compare when recognition, model size, and CPU throughput are considered jointly. We address this application setting with LPDNet. The family uses ShuffleNetV2 stages, replaces eligible depth-wise operations with training-time directional branches that fuse into single deployment convolutions, and forms a classifier from final global and intermediate bilateral fea-tures. A fold-matched ConvNeXt-Tiny teacher [18] sup-plies soft targets during knowledge distillation (KD) [19], but prior work shows that KD depends on the teacher–student pair and training protocol [20–22]. We therefore report matched non-KD controls rather than treating distil-lation as an automatic gain. The contributions are threefold. First, we develop a task-oriented LPDNet family with directional reparameter-ization and bilateral feature modelling. Second, we con-duct matched KD evaluation: under the tested matched protocol, LPDNet-X1.5-KD achieved a higher observed mean than its non-distilled control, whereas Shuf-fleNetV2-X1.5-KD achieved a lower observed mean than its matched control, showing that the observed benefit should not be transferred automatically to another light-weight student. Third, we connect driver-disjoint recogni-tion and deployment-graph complexity to protocol-separated Raspberry Pi 5 measurements, identifying dis-tinct operating points within the tested LPDNet family rather than a universally preferred configuration.

Vision-based driver monitoring spans posture classification, fine-grained or multimodal activity recognition, anomaly detection, and compact or attention-based models [2,5–12]. Chai et al. compared split and metric choices on State Farm and AI City data and emphasized leave-N-driver-out evaluation for unseen drivers [3]; the present zero-overlap result instead comes from local split manifests. Driver-disjoint folds reduce subject leakage but do not establish robustness to a new dataset, vehicle, camera, or population. Recent task-specific compact methods include an EMA-teacher semi-supervised transformer [10], category-specific attention distillation for shallow MobileNetV2 students with CPU/GPU speed reporting [11], and a structurally reparameterized convolution–Vision-LSTM hybrid [12]. The present study brings these concerns together through driver-disjoint five-fold evaluation, matched KD/non-KD controls for two students, deployment-fused LPDNet operations, and protocol-separated Raspberry Pi 5 measurements, without making a priority claim. Generic compact design [14–17,27], KD [19–22], structural reparameterization [23–26], and scenario-specific device measurement [16,26,27,30–32] provide the broader lineages.

Figure 1 reconstructs the executed LPDNet-X1.5 graph. A 3×256×256 image passes through the ShuffleNetV2-X1.5 stem and stages with 4, 8, and 4 blocks. Their outputs are 176×32×32, 352×16×16, and 704×8×8; a final pointwise convolution and global average pooling produce a 1024-dimensional global feature vector. Each eligible stage depthwise operation is replaced during training by 3×3, 1×5, and 5×1 depthwise Conv–BatchNorm branches. An identity–BatchNorm branch is added when stride is one. The branch outputs are summed. Before export, BatchNorm parameters are absorbed and the kernels are padded and combined into one biased 5×5 depthwise convolution. The executed X1.5 model contains 5, 9, and 5 such replacements in stages 2–4, respectively. This follows the deployment-oriented principle of asymmetric and diverse-branch reparameterization [23–26], while retaining LPDNet’s code-defined branch geometry. The directional branches expose anisotropic receptive fields during optimization. Fusion is algebraic, not an ensemble: no parallel path remains at inference, and KD and non-KD X1.5 checkpoints share the same deployed architecture.

The Stage-3 feature map is split equally along the width dimension, and each half is spatially averaged to obtain 352-dimensional left and right feature vectors. Their average, followed by LayerNorm, forms a symmetric descriptor, while their difference, also followed by LayerNorm, forms a directional descriptor. The normalized 1024-dimensional global feature is concatenated with the two 352-dimensional descriptors to form a 1728-dimensional representation, followed by dropout of 0.25 and a ten-class linear layer. The design preserves global context and left–right contrast, but the non-monotonic structure exploration does not establish an independent causal gain for either descriptor. Several classes distinguish left- and right-hand interactions, motivating the head. Because mirroring requires class remapping and some exploration rows also change recipe settings, we do not claim that bilateral statistics always improve recognition.

The fold-specific ConvNeXt-Tiny teacher is frozen during student training and provides logits for both the original and horizontally mirrored views. Mirroring remaps the class indices in the order 0, 3, 4, 1, 2, 5, 6, 7, 8, and 9. For LPDNet-X1.5-KD, the objective combines original-view classification, mirrored-view classification, mirror consistency, and knowledge distillation. Label smoothing is 0.08, the mirror temperature is 2, and the corresponding loss weights are 0.50 for the mirror term, 0.25 for the consistency term, and 0.40 for knowledge distillation. The KD temperature is 3. The KD term uses temperature-scaled teacher-to-student KL divergence weighted by normalized teacher confidence. Teacher confidence is defined as the maximum softmax probability for each sample and is normalized by the mean teacher confidence of the corresponding view within the current minibatch. The confidence exponent is one, and the KD losses from the original and mirrored views are averaged. The matched ShuffleNet pair uses the same formulation but omits the consistency term. Only the student receives gradients. Within each student architecture, the matched non-KD control removes the teacher and KD term while retaining the same driver split, seed, optimizer, learning-rate schedule, augmentations, mirror objective, checkpoint rule, early stopping, micro-batch size 32, and gradient accumulation of two. Early stopping may yield different total epoch counts, so the comparison is protocol-matched rather than a bitwise replay.

The teacher is discarded, all 19 directional modules are fused, and the student is exported as static-batch FP32 ONNX, opset 18. Export-time validation compares the training and fused graphs before the deployment model is executed through ONNX Runtime’s CPUExecutionPro-vider interface [28].

The audited State Farm metadata contains 22,424 unique images, 26 drivers, and ten classes [1]. Five outer folds allocate 5, 5, 6, 5, and 5 test drivers. Training, validation, and test driver lists are disjoint in every fold, with zero recorded driver, path, or image-hash overlap. All models reproduce the same outer protocol, and corresponding matched-control manifests are byte-identical. Each fold uses three validation drivers and 17 or 18 training drivers. Images remain clustered within drivers, so the bootstrap resamples drivers rather than individual images. Runs use seed 42, ImageNet initialization, AdamW, at most 70 epochs, a base learning rate of 0.0003, weight decay of 0.03, warm-up followed by cosine decay, and validation single-view Macro-F1 for checkpoint selection. The common augmentation path includes color perturbation, autocontrast, affine transformation, blur, and random erasing. Early stopping uses the same patience and improvement threshold within each matched pair. Test-single Macro-F1 is the primary metric. Main results are reported as the five-fold mean and sample standard deviation, while pooled out-of-fold (OOF) Macro-F1 is treated as a separate estimand. For each matched comparison, a paired bootstrap resamples all 26 drivers as clusters with identical multiplicities for both models. We use 10,000 valid replicates, seed 42, fixed classes c0–c9, and percentile-based 95% confidence intervals. The authoritative deployment run uses a Raspberry Pi 5 with 4 GB memory, aarch64 architecture, and four 2.4-GHz Cortex-A76 cores [29], together with ONNX Runtime 1.28.0 CPUExecutionProvider, FP32 precision, batch size one, and an input shape of 1 × 3 × 256 × 256. Pure inference uses one or four threads after 50 warm-up runs and five repeats of 500 timed iterations. Fixed-image end-to-end timing includes decoding, preprocessing, inference, and argmax. The fixed-video benchmark replays exactly 300 decoded frames offline; it is a deterministic image stream rather than temporal recognition or live-camera operation. Explicit scenarios and software/hardware disclosure follow the general reproducibility motivation of edge and inference benchmarks [30–32]. Preprocessing preserves aspect ratio, center-pads the image to 256 × 256, and applies ImageNet normalization. The deployment table reports four-thread pure inference, fixed-image end-to-end throughput, and fixed-video offline throughput as distinct measurement categories.

Table 1 summarizes eight representative models spanning the teacher, standard baselines, lightweight baselines, and the two discussed LPDNet configurations. ConvNeXt-Tiny has the highest listed mean Macro-F1 (0.9009), with EfficientNet-B0 and ResNet18 at 0.8938 and 0.8931. The proposed fami-ly yields two measured profiles. Bilateral-only achieved 0.8817 ± 0.0308 with 1.271 million parameters and 0.188 GMACs. LPDNet-X1.5-KD achieved 0.8837 ± 0.0426 with 2.555 million parameters and 0.399 GMACs. Its observed mean Macro-F1 was 0.00201, or 0.20 percentage points, higher than Bilateral-only, while Bilateral-only used 50.2% fewer parameters and 52.9% fewer GMACs. Within the tested LPDNet family, Bilateral-only represents the compact, high-throughput operating point, whereas LPDNet-X1.5-KD repre-sents the wider distilled configuration used for matched teacher–student analysis and achieves the higher observed mean of the two. These labels describe measured operating points and are not formal deployment recommendations or evidence of Pareto optimality. In the broad architecture comparison, ShuffleNetV2-X1.5 is the nearest direct lightweight baseline to LPDNet-X1.5-KD. Their mean Macro-F1 values are nearly the same (0.8832 and 0.8837, respectively), while Their mean Macro-F1 values are nearly the same (0.8832 and 0.8837, respectively), while ShuffleNetV2-X1.5 has higher measured four-thread throughput than LPDNet-X1.5-KD (77.9 versus 60.3 infer-ences/s). The paper therefore does not establish global superi-ority of the distilled LPDNet configuration; its contribution is the proposed family, the matched distillation characterization, and device-level evaluation. The wider structure exploration is non-monotonic. Direc-tional-only, directional-plus-bilateral, plain-5×5-plus-bilateral, and LPDNet-Full do not form an additive accuracy ladder, and some rows also differ in mirror or dropout recipe. This weakens component-isolation claims but not the source-grounded conclusion that the family exposes different meas-ured operating points.

Under the tested matched protocol, LPDNet-X1.5-KD achieved 0.8837 ± 0.0426, compared with 0.8725 ± 0.0547 for its non-distilled control (Table 2). The five-fold mean dif-ference was +0.01125; pooled OOF Macro-F1 was 0.89039 versus 0.87775, a difference of +0.01263. The paired driver-clustered 95% interval for this pooled-OOF difference was [−0.33, 2.89] percentage points. The observed average teach-er gap was 1.72 percentage points for LPDNet-X1.5-KD, compared with 2.84 percentage points for its non-distilled control. Under the tested matched protocol, ShuffleNetV2-X1.5-KD achieved a lower observed mean, 0.8660 ± 0.0399, than its non-distilled control, 0.8741 ± 0.0281; the pooled differ-ence was −0.00653 and its paired driver-clustered interval was [−2.18, 1.08] percentage points. Both primary intervals cross zero, so they do not support a nonzero matched differ-ence or establish equivalence; they also omit new-seed and new-dataset variability. This opposite observed direction indi-cates that the higher LPDNet mean under KD should not be assumed for another lightweight student. It neither identifies architecture as the sole cause nor establishes a universal ar-chitecture-dependent KD mechanism. The cross-student contrast is descriptive rather than a con-trolled architecture-only comparison because the KD and non-KD settings are matched within each student architecture, whereas the executed recipes are not identical across architec-tures, including the mirror-consistency weight.

Table 3 and Fig. 2 separate the profile decision from the teacher comparison. Figure 2 highlights the two discussed LPDNet operating points and the closest direct lightweight baseline, while the teacher and remaining measured models provide contextual deployment references. Bilateral-only reaches 144.9 four-thread inferences/s at 6.90 ms, whereas LPDNet-X1.5-KD reaches 60.3 inferences/s at 16.58 ms. The compact profile is therefore 2.40 times as fast in this category for a 0.20-percentage-point lower mean Macro-F1. Fixed-image throughput is 72.0 versus 42.8 frames/s, and fixed-video offline throughput is 73.2 versus 44.7 frames/s. These categories include different non-model work. Offline replay is a fixed image stream, not temporal recognition, camera ac-quisition, or a complete vehicle system. Matched non-KD checkpoints were not separately de-ployed and are therefore absent from Table 3. Relative to the teacher, LPDNet-X1.5-KD is 1.72 percent-age points lower in mean Macro-F1 but has 90.8% fewer pa-rameters, 93.1% lower GMACs, and 14.24-fold four-thread throughput. Similar proxy complexity still does not guarantee similar latency: original LPDNet-X1.5 and ShuffleNetV2-X1.5 differ by only 2.66% in parameters and 3.76% in GMACs, yet their recorded four-thread latencies are 16.37 and 12.84 ms, a 27.5% difference. The evidence does not isolate its runtime cause; it demonstrates why the target de-vice must be measured. The single authoritative run supports only the recorded conditions; it does not quantify between-device variance or other software/hardware stacks.

Within the tested LPDNet family, Bilateral-only is the compact, high-throughput operating point, while LPDNet-X1.5-KD is the wider distilled configuration with the higher observed mean Macro-F1 and the smaller observed average teacher gap. Under the tested matched protocol, LPDNet-X1.5-KD achieved a higher observed mean than its non-distilled control, whereas the ShuffleNet control showed the opposite observed mean direction. Raspberry Pi 5 measure-ments show that these operating points differ in measured device behavior as well as theoretical complexity. The result is a bounded family-level characterization, not a claim that one configuration is globally preferable.

Limitations The study uses one dataset, 26 drivers, one five-fold split design, one primary seed, one KD recipe, and two matched student architectures. Both primary KD intervals cross zero, so effects are not established beyond the observed drivers and fitted checkpoints. The non-monotonic, partly recipe-confounded structure exploration does not isolate component causality. Deployment uses one physical Raspberry Pi benchmark run without independent device replication. Ex-ternal-dataset generalization, power or energy, live-camera acquisition, temporal recognition, and production-system behavior were not evaluated. Accordingly, the results are an evidence-led design and deployment characterization, not a state-of-the-art, universal KD, or cross-dataset generalization claim. As a final reproducibility limitation, the retained ar-chive omits the original competition images, teacher check-point hashes, source commit identifiers, and exported ONNX binaries.