Experiments & Benchmarks

Evaluate InsideForest as supervised clustering and region discovery. Random-forest classification accuracy is intentionally excluded from the clusterer decision.

Primary metrics

• Coverage and unmatched rate, always retaining cluster -1 in clustering metrics.
• AMI, NMI, ARI, homogeneity, completeness, and weighted cluster purity.
• Region lift, entropy, support, target-class margin, and class-level coverage.
• Number of raw leaves, final regions, compression, runtime, and memory.
• Assignment and geometric stability across seeds or resamples.

Current class-region benchmark

python experiments/validate_class_region_clusters.py --profile quick

The benchmark compares InsideForestRegionClusterer and InsideForestClassRegionClusterer under the current no-fallback contract. In a local two-fold Iris smoke run, the class-aware clusterer reached coverage 1.000 and AMI about 0.824; the general clusterer reached coverage 0.440–0.480 and AMI 0.537–0.548. These values are smoke evidence, not a universal ranking.

Deciding whether branch aggregation adds value

Keep branch_aggregation="none" as the baseline. Adopt an aggregation only if repeated evaluation shows median compression of at least 20%, non-inferiority in coverage, AMI, and purity, no stability degradation, and acceptable compute cost. Aggregation must only combine regions with the same target class and must recompute their distribution, support, and score.

Reproducible protocol

1. Use stratified repeated splits and fixed seeds.
2. Fit only on train and evaluate assignments on untouched holdout rows.
3. Test string labels, remapped IDs, numeric permutations, and column permutations.
4. Report per-fold results and robust summaries, not only a single average.
5. Archive the exact parameters, package versions, raw regions, and final region table.

Regression regions

python experiments/validate_regression_regions.py --profile quick validates InsideForestContinuousRegionClusterer through η², coverage, unmatched rate, target dispersion reduction, compression, stability, runtime, and memory. Forest R²/RMSE remains a separate generator diagnostic.