Budget-Controlled Hyperparameter Optimization for AI-Based Steel Surface Defect Detection

AI-based visual inspection supports industrial quality control by identifying surface defects that affect material integrity and downstream processing. For steel surfaces, subtle texture changes, irregular boundaries, and large differences in defect shape make both classification and localization difficult. A useful inspection model must therefore be assessed with more than a single permissive detection score. Detector performance depends on loss weights, augmentation, and optimization settings. Comparing hyperparameter optimization (HPO) methods is difficult when their search budgets or final-training protocols differ. A favorable result from one retraining seed can also obscure whether a selected configuration remains competitive across repeated training. Equal evaluation budgets and a held-out test split make these comparisons easier to interpret. This study compares random search, Genetic Algorithm (GA)-only, Covariance Matrix Adaptation Evolution Strategy (CMA-ES)-only, GA-to-CMA-ES, and GA-to-CMA-ES-to-hyperlocal tuning, termed Full. All use 45 proxy evaluations and common final training. The original search maximizes validation mean average precision at Intersection over Union (IoU) 0.5, denoted mAP@0.5. Final assessment also reports mAP@0.5:0.95, averaged over IoU thresholds from 0.5 to 0.95 in steps of 0.05, hereafter strict mAP. Our contributions are an equal-budget comparison, an analysis of proxy-to-metric alignment and stage contributions, and completed controls for objective choice and final-stage allocation. A five-seed extension tests selected original configurations. The additional controls directly optimize strict mAP and compare hyperlocal tuning with continued CMA-ES after the same first 40 evaluations. This separates the observed benefit of Full from an unsupported claim that every stage always improves performance.

Steel surface inspection initially relied on handcrafted texture descriptors. Completed local binary patterns and scattering convolution networks address noise and appearance variation in early steel-defect studies [1], [2]. Deep learning has since become a central approach to industrial surface inspection [3]. The YOLO family combines detection accuracy with efficient inference; the Ultralytics framework provides a common implementation for controlled training and evaluation [4], [5]. GA uses population-based exploration through selection, crossover, and mutation [6]. CMA-ES adapts a continuous search distribution from evaluated candidates [7], [8]. Combining these mechanisms may locate and refine useful configurations, but a longer sequence alone does not establish an advantage. Our comparison fixes the evaluation budget and distinguishes selected-configuration retraining variability from variability across independent HPO searches.

Each evaluation trains YOLOv8l from the same pretrained checkpoint for 15 epochs. The original outer objective is the maximum validation mAP@0.5 within that run, with the earliest epoch resolving a tie. All 225 original candidates completed 15 epochs. After search, the best configuration from each method is frozen and retrained for 100 epochs. The short training duration is the main computational approximation; using mAP@0.5 itself does not make training cheaper. Table 1 specifies the budget allocation. The original two-stage and Full methods use different GA and CMA-ES budgets, so their final-score difference does not isolate hyperlocal tuning. We therefore report within-trajectory incumbents and add a shared-prefix control for the final five evaluations.

The 15 requested arguments comprise learning-rate, warmup, loss, and augmentation settings. Log-scaled ranges are lr0: 10⁻⁵–0.05, lrf: 10⁻⁴–0.5, and weight_decay: 10⁻⁶–0.005. Linear ranges are momentum: 0.80–0.99, warmup_epochs: 0–5, warmup_momentum: 0.70–0.95, box: 0.02–0.10, cls: 0.20–1.00, and dfl: 0.50–2.50. Augmentation ranges are hsv_h: 0–0.03, hsv_s and hsv_v: 0.20–0.90 each, degrees: 0–15, translate: 0–0.20, and shear: 0–5. Candidates are represented in normalized coordinates; logarithmic variables are transformed before scaling. GA provides exploration, CMA-ES refines a region around the best GA candidate, and hyperlocal tuning samples a smaller neighborhood. These are intended roles, whose realized contributions must be measured from the incumbent history. The implementation uses optimizer=auto. In the new runs, it selects AdamW with initial learning rate 0.001 and beta1=0.9, overriding requested lr0 and momentum; warmup_momentum is inactive on this optimizer path. Consequently, 15 requested arguments must not be interpreted as 15 demonstrated effective dimensions. All compared paths retain this common implementation.

A separate experiment defines A as Full maximizing peak validation mAP@0.5 and B as Full maximizing peak validation strict mAP. Both allocate 27 GA, 13 CMA-ES, and 5 hyperlocal evaluations. C shares B’s first 40 physical evaluations and CMA-ES state, then continues CMA-ES for five evaluations. Thus, each path has 45 logical evaluations, while only 95 physical proxy runs are needed. C tests the final allocation conditional on this prefix; it is not CMA-ES-only from initialization. For these new searches, GA starts with eight uniformly sampled candidates and uses tournament selection of size three. Arithmetic crossover occurs with probability 0.7 using coordinate weights sampled from −0.5 to 1.5. Mutation is applied to a child with probability 0.5; each coordinate is then perturbed with probability 0.25 by Gaussian noise of standard deviation 0.15. Candidates are clipped to the normalized bounds. GA evaluates 8 initial candidates followed by batches of 8, 8, and 3, retaining the best observed point for the next stage. CMA-ES starts at that point with normalized step size 0.18 and population size five. Its 13 evaluations comprise batches of 5, 5, and 3, including an update from the final partial batch. Hyperlocal tuning perturbs the incumbent independently and uniformly by ±1% of the search range for lr0, momentum, box, and cls, and ±2% for other coordinates. Logarithmic arguments are perturbed in log space. Bounds are enforced and strict improvements update the center. NumPy and CMA-ES random streams use seed 2026; proxy training uses seed 1000 plus the trial index. New proxies disable patience and all complete 15 epochs. The original patience was 10, but every original proxy also completed 15 epochs. The new searches have explicit CMA randomness and are distinct realizations, not exact replays of the original search. Changing the outer objective also changes which peak epoch is selected.

NEU-DET contains six classes: crazing, inclusion, patches, pitted_surface, rolled-in_scale, and scratches. The fixed split contains 1,260 training, 270 validation, and 270 test images. No byte-identical images occur across splits. Validation guides search and checkpoint selection; the held-out test split is used for final assessment and is reused by the additional experiments. All final models use image size 640, batch size 4, and 100 epochs. Common final augmentation uses scale=0.8 and perspective=0.0002, whereas proxies use scale=0.5 and perspective=0. Original all-method retraining uses seeds 0–2; the fixed Baseline, CMA-ES-only, and Full configurations are extended to seeds 0–4. Each new A/B/C winner is frozen before three-seed retraining. All nine new final runs reach 100 epochs. Original validation summaries use epoch 100; independent test results evaluate best.pt. In the verified Ultralytics 8.3.229 implementation, strict-validation fitness selects best.pt. This checkpoint rule is separate from the outer HPO objective. We report mean ± sample standard deviation (SD), precision (P), recall (R), and the first epoch reaching 90% of each run’s own best validation mAP@0.5. Retraining seeds measure fixed-configuration variability, not independent HPO repetitions. Tables 2-3 and Fig. 1 summarize the original comparison.

Tables 2 and 3 and Fig. 1 retain the original three-seed comparison. Full obtains the highest validation means: 0.7512 for mAP@0.5 and 0.4023 for strict mAP. On the test split, CMA-ES-only has the highest mAP@0.5, 0.7439, whereas Full has the highest strict mean, 0.4046. The tradeoff is therefore metric-dependent: Full is a promising choice when threshold-averaged localization performance is prioritized, while CMA-ES-only is a strong simpler alternative. Full’s original paired strict-test advantage over CMA-ES-only is 0.249 percentage points, positive in two of three pairs, with an exploratory 95% paired t interval of [−1.596, 2.094]. No original test comparison against Baseline survives Holm adjustment within its five-comparison metric family. These small-sample results do not establish general optimizer superiority. Validation-only significance should not be interpreted as equivalent test evidence.

Table 4 extends only the three fixed original configurations. Full improves strict test mAP over Baseline by 2.015 percentage points, positive in all five pairs, with a 95% paired t interval of [1.118, 2.911]. Its advantage over CMA-ES-only is 0.155 points, positive in three of five pairs, with an interval of [−0.520, 0.830]. Exact two-sided sign-flip p-values are 0.0625 and 0.625, respectively. The t intervals are exploratory and their apparent precision should be read alongside these exact small-sample checks.

The per-class profile in Table 5 shows that patches and pitted_surface are easier for the baseline than crazing and rolled-in_scale. These values come from a separate baseline API validation evaluation; their macro-average, 0.3825, is not the epoch-100 strict validation value of 0.3780 in Table 2. They describe baseline class difficulty and do not demonstrate class-specific improvement by Full. Fig. 2 shows the original validation learning curves.

Fig. 2 and Table 2 summarize the original final-training trajectories. Baseline reaches 90% of its own best validation mAP@0.5 at 52.67 epochs on average, compared with 47.33 for CMA-ES-only and 49.33 for Full. The corresponding SDs are 6.81, 3.79, and 5.03 epochs. This relative threshold describes curve shape: its target differs between runs. Earlier crossing does not demonstrate fewer training epochs or measured compute savings because all final runs use 100 epochs.

For each original candidate, we pair its maximum validation mAP@0.5 with strict mAP from the same earliest peak epoch. Table 6 reports Pearson and Spearman correlations and top-five overlap within each 45-evaluation search. The 225 observations include 147 distinct requested settings; searches are adaptive and share some candidates, so correlation coefficients are descriptive rather than independent-sample significance tests. Full shows strong candidate-level alignment, with Spearman ρ=0.942 and Pearson r=0.926. However, its mAP@0.5-selected candidate has same-epoch strict mAP 0.32669, below the pool maximum of 0.33373 by 0.704 percentage points; the top-five sets overlap in only three candidates. Thus, strong overall correlation does not guarantee identical winner selection. This within-proxy relationship must also be distinguished from prediction of long-training performance. Across the five selected optimizer configurations, the Spearman correlation between the proxy objective and mean final strict performance is 0.50 on validation and 0.20 on test. With only five selected configurations, this is weak evidence of predictive fidelity. Original strict-metric gains are downstream outcomes of the mAP@0.5 outer objective and the common final checkpoint rule, rather than direct outer optimization of strict mAP.

Table 7 gives the best objective observed at each stage boundary. GA gains are measured against the first proxy candidate, not against the 100-epoch baseline. In the original Full trajectory, the first candidate scores 0.59394 and GA reaches 0.65780, a gain of 6.386 percentage points. CMA-ES then adds 1.556 points, while the five hyperlocal evaluations add zero. Thus, the original Full gain cannot be attributed to hyperlocal tuning. Original Full and A use mAP@0.5; B and C use strict mAP. The original GA-stage winner matches the configuration retrained for GA-only. The mean test strict score is 0.37868 after GA and 0.40459 after both later stages. The latter two entries refer to the same selected configuration and three models; they are not independent retraining experiments. For new A, CMA-ES adds 1.139 points and hyperlocal tuning adds 0.604 points. For B and C, the first strict-objective candidate scores 0.24348; GA adds 6.374 points and CMA-ES adds 1.813 points through evaluation 40. B’s hyperlocal tail then adds 0.291 points, compared with 0.007 for C’s five continued CMA-ES evaluations. The final strict-proxy advantage of the hyperlocal allocation is 0.284 points under the shared prefix. Only terminal winners are retrained in the new experiment.

Table 8 reports the independent test performance of the three new terminal winners. Directly optimizing the strict proxy in B increases mean strict test mAP relative to A, while reducing mAP@0.5. B also exceeds C in strict test performance after the shared-prefix tail control. These comparisons address objective choice and final allocation separately within the new experiment. The mean paired strict-mAP differences are +0.720 points for B−A and +0.555 points for B−C. Both are positive in all three matched seed pairs, but their exploratory 95% paired t intervals are [−0.587, 2.028] and [−0.822, 1.931] points, respectively. Both exact two-sided sign-flip p-values are 0.25. The corresponding mAP@0.5 differences are −1.404 and −0.695 points. The observations support a localization tradeoff and a conditional benefit from hyperlocal refinement, with substantial uncertainty. The new searches also show strong same-epoch proxy alignment. Pairing metrics at each candidate’s mAP@0.5 peak gives Spearman correlations of 0.939, 0.912, and 0.903 for A, B, and C, respectively; Pearson correlations are 0.932, 0.904, and 0.910. Each path has 45 logical observations, with B/C sharing 40. These coefficients again describe adaptive candidate pools, not independent replicated searches. A’s selected candidate has peak strict mAP 0.32057, compared with 0.32826 for B’s selected candidate. However, A contains an unselected candidate with peak strict mAP 0.33091. Objective alignment changes selection and the adaptive trajectory but does not guarantee the largest strict score across all observed candidate pools. Because each objective also selects its own peak epoch, this comparison tests objective-and-epoch selection together. The new A and B test strict means are below the original Full mean. They must not be pooled with the original Full runs or described as absolute improvements over that earlier winner. Rather, the new experiment supplies a matched comparison showing that directly targeting strict mAP and allocating the final budget to hyperlocal tuning can improve localization within this particular search realization.

Table 9 reports measured elapsed time for the original optimizer loops and final-training processes. Search takes 20.02–20.34 hours per method; search plus three final-training runs takes 28.89–29.43 hours. Original Full requires 20.23 hours for search and 29.12 hours through final training. Baseline’s three training runs take 8.98 hours. The six added runs that extend three configurations from three to five seeds take 17.37 hours, plus 1.51 minutes for their test evaluations. The new controlled experiment runs on one NVIDIA RTX 3050 with nominal 6 GB memory, using PyTorch 2.5.1+cu121, CUDA 12.1, and Ultralytics 8.3.229. Mixed precision is enabled. Table 10 gives cost per completed logical path, including 45 proxies, three final-training runs, and three tests. The paths have nearly equal cost, so their observed differences are not explained by a larger completed evaluation budget. Final time covers three 100-epoch runs. Total time also includes approximately 0.013 hours of testing per path. B and C share 17.405 hours of prefix evaluation; this is included in each logical path, so their path totals must not be summed as separate physical workloads. These measurements cover completed training and evaluation processes, including CPU work and validation; they are not GPU-kernel hours. Original timings exclude separate testing, and historical hardware provenance is insufficient for a direct hardware-speed comparison with the new runs.

Fig. 3 shows the original same-image comparison. In crazing image 136, Full labels the defect as crazing, whereas Baseline predicts patches and some alternatives display both classes. In pitted-surface image 234, Baseline displays no box, while all HPO methods detect the surface defect. In scratches image 233, all methods detect elongated regions and GA-to-CMA-ES additionally predicts an inclusion. These examples illustrate differences in predictions that aggregate scores alone do not show. The displayed cases and representative seeds were selected with knowledge of test outcomes, and ground-truth boxes are not drawn. The figure is therefore illustrative rather than an unbiased estimate of false-positive rates or localization error. It does not establish that Full consistently reduces manual inspection cost or remains robust to changes in production lighting and cameras.

Full provides the highest original strict-test mean and a five-seed improvement over Baseline, but its margin over CMA-ES-only remains uncertain. The original hyperlocal stage contributes no incumbent gain. The new shared-prefix experiment supplies a positive hyperlocal increment and an observed strict-test gain, while retaining the accompanying loss in mAP@0.5. This combination supports metric-aware configuration selection rather than a universal ranking of optimizers. The study uses one NEU-DET split and one detector. Training seeds do not replicate the adaptive HPO process; each new path has only one search realization. Additional datasets, detectors, independent HPO repetitions, and multi-objective optimization remain unevaluated. Test reuse and selected qualitative examples further limit generalization. The reported compute is training/evaluation process time, not deployment latency or industrial cost.

An equal 45-evaluation budget and common final training make the HPO comparison interpretable. Full achieves the highest original test strict mean; the five-seed fixed-configuration extension gives a 2.015-point gain over Baseline. CMA-ES-only remains competitive and has the highest original test mAP@0.5. The new objective and shared-prefix controls show conditional localization gains from directly optimizing strict mAP and using hyperlocal refinement. Strong proxy correlation does not ensure identical selection, and the small-seed uncertainty prevents a general superiority claim. The framework supports transparent budget and metric tradeoffs for steel-defect detector selection.

This work was supported by the Ministry of Trade, Industry and Resources (MOTIR), Republic of Korea, under Grant No. RS-2024-00431769, and by the Ministry of the Interior and Safety (MOIS, Korea) under Development of Technology in Cooperation with the Ministry of Disaster and Safety Program (No. 20026195).