Cluster-Guided Minority Enhancement for Imbalanced Classification

M achine-learning applications frequently operate on datasets in which one class is much rarer than the other [1]-[4]. This pattern is common in disease screening, fraud detection [5], customer default prediction, and insurance claims. A learner trained on such data can obtain high overall accuracy while still missing a large proportion of minority cases, because minimizing total error favors the dominant class [6]. For these problems, recall and F1 score are therefore more informative than accuracy alone. Imbalance can be addressed at the data level, the algorithm level, or through hybrid methods [7]. Data-level approaches are attractive because they can be combined with many classifiers without modifying the underlying learning algorithm. Random oversampling is simple but duplicates observations and may increase overfitting [8]. Random undersampling avoids duplication but discards majority information [9]. SMOTE instead synthesizes minority samples by interpolating between neighboring minority instances [10], and has become one of the most widely used oversampling methods. The limitation is that SMOTE treats every selected minority point as a valid seed. If an isolated outlier or a borderline point is selected, the interpolation path may cross an unrepresentative region or even approach the majority class. The resulting synthetic samples can distort the local minority distribution and reduce classification reliability. Existing variants such as Borderline-SMOTE [11], ADASYN [12], MWMOTE [13], and filtering-based oversampling [14], [15] partly address this problem by changing where or how samples are generated. However, noisy or sparsely populated minority regions remain difficult to handle consistently. This study reformulates the oversampling stage as a two-level clustering problem. K-means is first used to identify small, isolated minority clusters that are treated as outlier groups. Those groups are excluded from sample generation. K-means is then applied again to the retained minority observations, and SMOTE is performed only within each resulting subcluster. The approach, referred to here as outlier-aware K-means SMOTE (OA-KM-SMOTE), is designed to preserve representative minority structure while preventing a typical observation from dominating interpolation. The main contribution is not a new classifier but a classifier-independent resampling pipeline. The method is tested with four standard learners on three datasets that differ substantially in dimensionality and imbalance level. The reported comparisons show consistent gains over conventional SMOTE in both recall and F1 score.

Resampling methods seek to change the training distribution before model fitting. Oversampling adds minority observations, undersampling removes majority observations, and hybrid strategies combine both operations. SMOTE generates a synthetic point between a minority seed x_i and one of its k nearest minority neighbors x_nn: xsyn = xi + λ(xnn − xi), 0 ≤ λ ≤ 1. By creating new points instead of copying existing ones, SMOTE typically reduces the direct overfitting risk of random oversampling. Nevertheless, its neighbor relation is purely geometric. It does not distinguish a representative sample from an outlier, and it can connect points belonging to different local minority modes. Several SMOTE variants refine the sampling region. Borderline-SMOTE emphasizes minority samples near the decision boundary. ADASYN assigns more synthetic data to difficult minority observations, whereas MWMOTE uses weighted selection to focus on informative minority points. SMOTE-IPF combines oversampling with filtering, and other outlier-filtering methods remove suspicious observations before generation. These methods demonstrate that sample quality matters as much as the total number of synthetic instances. K-means offers a simple mechanism for discovering compact groups in high-dimensional data. Given K clusters C_i with centroids μ_i, the standard objective minimizes within-cluster squared distance: J = ∑i ∑x∈Cᵢ ‖x − μi‖2. Small clusters located far from the main minority distribution can therefore be interpreted as candidate outlier groups [16]. Compared with pointwise detection, cluster-level filtering is easy to implement, scalable, and directly compatible with subsequent cluster-wise oversampling. The proposed method uses this property to separate atypical minority structure from regions that should support synthetic generation.

A.Design Rationale The proposed pipeline is based on two assumptions. First, minority outliers are more likely to form very small and relatively isolated clusters than ordinary minority observations. Second, synthetic samples are more plausible when both endpoints belong to the same locally coherent minority region. The method therefore separates outlier handling from oversampling and avoids interpolation across minority subpopulations. B.Processing Steps Step 1. Split the data. The dataset is divided into training and test subsets before any balancing operation. Only the training subset is modified, while the test subset preserves the original class distribution. Step 2. Preprocess the training subset. Missing values are handled, categorical variables are encoded, and majority and minority observations are separated. Feature scaling is applied for distance-based processing and model training. Step 3. Detect minority outlier clusters. K-means is fitted only to minority observations. The number of clusters is selected through grid search. In the reported implementation, a cluster is flagged as an outlier group when its membership is at least three times smaller than that of the next larger cluster. Samples in these sparse clusters are removed from the oversampling pool. Algorithm 1 OA-KM-SMOTE Input: training set D, candidate cluster counts K, target ratio r 1. Separate majority Dmaj and minority Dmin. 2. Select K by grid search and cluster Dmin. 3. Remove clusters satisfying the sparse-cluster rule. 4. Recluster the retained minority observations. 5. Apply SMOTE independently inside each subcluster. 6. Merge synthetic minority samples with Dmaj. Output: balanced training set Dbal. Step 4. Cluster the retained minority class. K-means is applied again after outlier removal. The retained observations are divided into internally similar subclusters, which restricts the neighborhood from which synthetic samples can be generated. Step 5. Apply SMOTE within clusters. Conventional SMOTE is executed separately inside each retained minority subcluster until the required class ratio is reached. This prevents synthetic segments from connecting minority instances that belong to distant or structurally different regions. Step 6. Merge and train. The generated minority data are combined with the original retained minority and majority samples. The balanced training set is then used to fit the selected classifier. The unchanged test set is used for final evaluation. The complete workflow of OA-KM-SMOTE is illustrated in Fig. 1. C.Method Characteristics OA-KM-SMOTE differs from ordinary K-means-assisted oversampling because clustering is used twice for different purposes. The first clustering stage is a filtering stage: it identifies sparse minority groups that should not guide interpolation. The second clustering stage is a generation stage: it confines nearest-neighbor search to locally coherent regions. Separating these roles makes the synthetic-sample mechanism easier to interpret and reduces the chance that a single atypical point influences a large number of generated observations. The additional computational cost is limited because both clustering operations are applied only to the minority portion of the training set. The procedure does not alter the classifier, the loss function, or the test data, so it can be attached to an existing machine-learning pipeline as a preprocessing component. Its principal tuning decisions are the number of K-means clusters, the cluster-size criterion used for outlier identification, and the target class ratio after oversampling. From a geometric perspective, the method narrows the support of the synthetic distribution. Conventional SMOTE may draw line segments across the entire minority manifold, whereas OA-KM-SMOTE restricts interpolation to retained local components. This restriction is especially relevant when minority observations contain disconnected modes or when a sparse point lies near a majority region. The method does not guarantee that every retained sample is noise-free, but it reduces the number of synthetic descendants that can be generated from structurally atypical seeds.

A.Datasets and Preprocessing Experiments use three binary-class datasets: stroke prediction, credit-card default, and car-insurance claim prediction. The datasets contain 11, 8, and 43 input features, respectively, and cover different imbalance regimes. TABLE I summarizes the class counts and the minority outliers identified by the clustering stage. Table I Dataset characteristics reported in the original experiments. Dataset Neg./Pos. Feature Outliers Stroke 42,617 / 783 11 15 Credit default 18,677 / 5,323 8 60 Insurance claim 54,844 / 3,748 43 80 Each dataset is divided into 75% training data and 25% test data. The test partition is not resampled. Categorical attributes are converted to numeric form, missing observations are removed where necessary, and the final training features are standardized. For stroke and credit-card default, the resampled negative-to-positive ratio is approximately 13:12. For insurance claims, the target ratio is 49:40. The stroke dataset represents a medical screening problem in which the positive class denotes patients who experienced a stroke. It contains demographic and health-related variables such as age, gender, hypertension, heart disease, smoking status, and work type. Its raw negative-to-positive ratio is approximately 272:5, making missed positive cases a central concern. The credit-card dataset predicts whether a customer will default on credit services. Its attributes describe age, salary, marital status, credit limit, credit category, education level, and related customer information. Although its class ratio is less extreme than those of the other datasets, the minority class is still sufficiently underrepresented to bias standard training. Preprocessing follows the same sequence for both oversampling strategies. Binary yes/no attributes are mapped to numeric values, nominal categories are encoded, and records with unresolved missing values are removed. Standardization is applied after the balanced training set is formed so that distance-based learning and K-means are not dominated by variables with larger numeric ranges. The test set receives the corresponding transformation but is never used to estimate synthetic samples. The insurance dataset predicts whether a policyholder will file a claim within the following six months. It contains 43 attributes related to policy tenure, vehicle age and condition, population density, and customer characteristics. The combination of high dimensionality, a negative-to-positive ratio near 73:5, and 80 detected positive outliers makes it the most demanding test case in the study. B.Classifiers and Metrics OA-KM-SMOTE and conventional SMOTE are paired with XGBoost, random forest, k-nearest neighbors (KNN), and decision tree classifiers. XGBoost provides regularized gradient boosting [17], random forest aggregates bootstrapped trees, KNN offers a distance-based learner that is directly sensitive to local sample geometry, and decision tree provides a non-ensemble baseline. Hyperparameters and the K-means cluster count are selected through grid search. For a binary confusion matrix, recall is TP/(TP+FN), while F1 is 2·Precision·Recall/(Precision+Recall). A higher recall indicates that fewer positive observations are missed, and F1 summarizes the balance between positive predictive accuracy and sensitivity. The reported comparison therefore emphasizes whether the resampling method improves minority recognition without relying on the potentially misleading overall accuracy of an imbalanced test set. Performance is reported using recall and F1 score. Recall measures the proportion of positive cases correctly identified, whereas F1 is the harmonic mean of precision and recall. These measures are appropriate because false negatives are especially important in the three prediction tasks and can be obscured by accuracy on imbalanced data [18]. C.Comparison Protocol The comparison keeps the train-test partition, data cleaning, feature representation, target balancing ratio, and classifier family unchanged. The only systematic difference is the oversampling stage: conventional SMOTE is applied directly to the complete minority class, whereas OA-KM-SMOTE first filters outlier clusters and then generates samples within retained minority subclusters. This design isolates the effect of the proposed resampling strategy. All outlier detection and oversampling operations are performed after the data split and only on the training portion. The original test distribution is preserved, which prevents synthetic observations from leaking into evaluation and avoids an overly optimistic performance estimate [19]. The best model selected under the reported grid-search setting is then evaluated on the test set using recall and F1 score. Because the target balance ratio differs across datasets, the experiment is intended to compare the two resampling mechanisms under matched dataset-specific conditions rather than to identify a universal optimal ratio. The reported differences should therefore be interpreted as evidence about where synthetic points are generated, not simply about how many points are added.

A.Stroke Prediction The stroke dataset is the most imbalanced in raw class counts, with 783 positive cases among 43,400 observations. Fifteen positive samples are identified as outliers. TABLE II shows that OA-KM-SMOTE improves both recall and F1 for all four classifiers. The largest F1 increase occurs with KNN, rising from 0.8829 to 0.9485, a gain of 6.56 percentage points. XGBoost, random forest, and decision tree also improve by 1.35, 2.26, and 2.31 points, respectively. The corresponding F1-score comparison is presented in Fig. 2. The consistent improvement suggests that filtering sparse minority regions reduces harmful interpolation without sacrificing the main minority structure. The stronger effect for KNN is also plausible within the reported framework: because KNN relies directly on local neighborhoods, it benefits more visibly when the synthetic samples follow coherent minority geometry. B.Credit-Card Default Credit-card default is the least imbalanced of the three datasets. Sixty outliers are detected among 5,323 positive cases. As reported in TABLE III, OA-KM-SMOKE improves recall and F1 score for all four classifiers. The corresponding F1-score comparison is shown in Fig. 3. The improvements are smaller than for stroke, but every classifier still benefits. KNN again records the largest F1 gain, from 0.7768 to 0.8009. XGBoost and random forest change only modestly, indicating that ensemble learners already provide relatively stable decision boundaries on this dataset, while local neighborhood quality remains more influential for KNN. C.Car-Insurance Claims The insurance dataset combines strong class imbalance with 43 features and contains 80 detected minority outliers. Conventional SMOTE produces low F1 scores for all four classifiers in the reported experiment. TABLE IV reports the recall and F1 results on the car-insurance claim dataset. After outlier filtering and cluster-wise sampling, recall rises to 0.8134–0.9332 and F1 rises to 0.7811–0.9601. The substantial improvements are also illustrated in Fig. 4. The largest absolute F1 gain is observed for XGBoost (83.64 percentage points), followed by random forest (76.87), KNN (72.42), and decision tree (41.58). These unusually large gains show that direct SMOTE was particularly vulnerable to the structure of this dataset under the reported setup. Removing sparse minority clusters before interpolation appears to prevent unrepresentative synthetic points from dominating the balanced training set. The result also supports the use of cluster-level rather than purely pointwise outlier handling when the feature space is high-dimensional [20]. D.Overall Interpretation Across all twelve classifier–dataset combinations, OA-KM-SMOTE produces higher recall and F1 than conventional SMOTE. Fig. 5 summarizes the mean F1-score improvement across the four classifiers for each dataset. The magnitude of improvement depends on data geometry. When the baseline is already strong, as in credit-card default, filtering produces modest but consistent gains. When outliers and high dimensionality interact with severe imbalance, as in the insurance data, the choice of synthetic sampling region becomes decisive. The method is computationally simple because it adds only two K-means stages around a standard SMOTE implementation. It is also classifier-independent and can be inserted into an existing preprocessing pipeline. Its main practical requirement is reliable selection of the cluster count and the outlier-cluster rule. Because a fixed size ratio may not describe every dataset, the threshold should be validated rather than treated as a universal constant. The study focuses on binary classification and cluster-level removal. Very small but legitimate minority subgroups could be incorrectly discarded, while non-spherical clusters may not be represented well by K-means. These limitations motivate alternative robust initializations, K-medoids, density-based clustering, or modified distance measures in future work.

This paper presents a condensed formulation of an outlier-aware K-means SMOTE method for imbalanced binary classification. The method first detects sparse minority clusters, removes them from the synthetic-sampling pool, and then performs SMOTE within the remaining minority subclusters. By limiting interpolation to representative local regions, it reduces the influence of outliers and avoids connecting structurally dissimilar minority samples. Experiments on stroke, credit-card default, and car-insurance claim datasets show higher recall and F1 scores than conventional SMOTE for XGBoost, random forest, KNN, and decision tree classifiers. The findings support minority-cluster filtering as a practical enhancement to data-level imbalance handling, particularly for high-dimensional datasets with severe imbalance.