GEOBIA Accuracy Assessment Methods

Two-Stage Accuracy Framework

Unlike pixel-based classification, GEOBIA introduces an intermediate step—segmentation—that directly influences final classification accuracy. Errors propagate through the workflow: poor segmentation leads to incorrect features, which leads to misclassification.

A comprehensive accuracy assessment should therefore evaluate both stages:

1. Segmentation quality: How well do segments correspond to real-world objects?
2. Classification accuracy: How correctly are segments assigned to thematic classes?

Segmentation Quality Assessment

Evaluating segmentation quality is challenging because it requires reference data—delineations of "true" object boundaries. Assessment methods fall into two categories: supervised (reference-based) and unsupervised (goodness measures).

Supervised Segmentation Metrics

When reference polygons are available, several metrics quantify segmentation quality:

• Over-segmentation: Reference objects split into multiple segments
• Under-segmentation: Multiple reference objects merged into one segment
• Area Fit Index (AFI): Measures area overlap between segments and reference
• Quality Rate (QR): Combines over- and under-segmentation into single metric

Unsupervised Goodness Measures

When reference data is unavailable, unsupervised measures assess segmentation based on internal properties:

• Intra-segment homogeneity: Low spectral variance within segments
• Inter-segment heterogeneity: High contrast between adjacent segments
• Global score: Combined measure of internal homogeneity and boundary strength

These measures can guide parameter selection but do not guarantee segments match real-world objects.

Classification Accuracy Assessment

Classification accuracy assessment in GEOBIA largely follows established remote sensing practices, with some object-specific considerations.

Confusion Matrix

The confusion (error) matrix remains the foundation of accuracy assessment. For GEOBIA, the matrix can be constructed using:

• Object counts: Each object is one sample (simple but ignores object size)
• Area-weighted: Objects contribute proportionally to their area
• Point sampling: Random points classified by the containing object

Derived Accuracy Metrics

• Overall accuracy: Proportion of correctly classified samples
• Producer's accuracy: Probability that reference data is correctly classified (completeness)
• User's accuracy: Probability that classified data matches reference (correctness)
• Kappa coefficient: Agreement corrected for chance (though its use is debated)
• F1 score: Harmonic mean of precision and recall, useful for imbalanced classes

Sampling Design

Robust accuracy assessment requires careful sampling design. Key considerations include:

Sample Size

Minimum sample sizes depend on the number of classes and desired confidence level. A common guideline is at least 50 samples per class, with more for rare or important classes.

Sampling Strategy

• Simple random: Equal probability for all locations
• Stratified random: Ensures representation of all classes
• Systematic: Regular grid, easy to implement
• Two-stage cluster: Efficient for large areas

Object-Specific Considerations

In GEOBIA, sampling units can be points, objects, or polygons. Point-based sampling may intersect object boundaries, requiring decisions about how to handle mixed reference areas. Object-based sampling must address size variability—large objects should not dominate accuracy estimates unless area-weighting is appropriate.

Reference Data Collection

Reference data quality directly impacts accuracy assessment reliability. Common sources include:

• Field surveys: Ground truth visits (most reliable, most expensive)
• Higher-resolution imagery: Visual interpretation of detailed imagery
• Existing maps/databases: Use with caution—may have their own errors
• Crowdsourced data: Emerging source, requires quality control

Reference data should be collected independently from training data to avoid overly optimistic accuracy estimates.

Cross-Validation for Machine Learning

When using machine learning classifiers, cross-validation provides robust accuracy estimates:

• K-fold: Data split into k subsets, each used once for validation
• Stratified k-fold: Maintains class proportions in each fold
• Spatial cross-validation: Accounts for spatial autocorrelation by separating folds geographically

Spatial cross-validation is particularly important for remote sensing applications where nearby samples are not independent. Standard k-fold cross-validation may overestimate accuracy when spatial autocorrelation is present.

Reporting Accuracy Results

Transparent reporting enables proper interpretation and comparison of results:

• Report full confusion matrix, not just summary statistics
• Include confidence intervals for accuracy estimates
• Describe sampling design and sample sizes per class
• Document reference data source and collection methods
• Note any known limitations or biases

Common Pitfalls

• Using training data for validation: Always use independent samples
• Ignoring spatial autocorrelation: Nearby samples are not independent
• Insufficient sample size: Results may not be statistically significant
• Ignoring segmentation quality: Classification accuracy may be limited by segmentation errors
• Over-relying on overall accuracy: Per-class metrics reveal important patterns

For guidance on avoiding errors throughout the GEOBIA workflow, see our article on common GEOBIA mistakes.