Cloud Masking

Cloud masking is the process of identifying and flagging cloud-contaminated pixels in optical satellite imagery so they can be excluded from analysis. Clouds are the single largest obstacle in optical Earth observation — at any given time, roughly 67% of the Earth's surface is obscured by clouds. Without reliable cloud masks, spectral indices, classification algorithms, and time series analyses will produce erroneous results because cloud pixels have reflectance signatures that can be confused with snow, bright sand, or built-up surfaces. Cloud shadow detection is an equally critical companion task. Effective cloud masking is a prerequisite for virtually every optical remote sensing workflow.

Cloud masking algorithms generally fall into three categories. Rule-based (physical threshold) methods apply spectral tests to individual pixels based on known physical properties of clouds — high reflectance in visible bands, low brightness temperature in thermal infrared, and specific spectral ratios. Fmask (Function of Mask), originally developed for Landsat, is the most widely used rule-based algorithm. Sen2Cor, ESA's processor for Sentinel-2, includes a scene classification module using spectral thresholds.

Machine learning methods treat cloud detection as a classification problem. S2cloudless, developed by Sinergise for Sentinel Hub, uses gradient boosting. More recent approaches use deep learning — CNNs and U-Net architectures — that consider spatial context and can detect cloud boundaries more precisely, including thin cirrus.

Multi-temporal methods like MAJA compare each new acquisition against previous cloud-free observations. Pixels that suddenly brighten are flagged as cloud candidates. This approach is effective for persistent thin clouds but requires a stack of recent acquisitions.

Cloud shadow detection typically uses the identified cloud mask combined with sun and satellite geometry to project where shadows should fall, then confirms by checking for darkened pixels at predicted locations.

No cloud masking algorithm achieves perfect accuracy. Thin cirrus clouds and cloud edges are the most challenging — they partially transmit surface reflectance, making them spectrally ambiguous. Bright surfaces like snow, ice, salt flats, and white sand are frequently confused with clouds (false positives), while dark or warm clouds can be missed (false negatives). Cloud shadow detection remains less accurate because projected shadow positions depend on estimated cloud heights. Rule-based methods struggle to generalize without threshold tuning, while machine learning methods require representative training datasets. Multi-temporal methods need frequent revisit data. Over-aggressive cloud masking removes usable data, while under-masking allows contaminated pixels to corrupt analyses — finding the right balance is an ongoing challenge.

Cloud masking has been a concern since the earliest days of satellite remote sensing. Simple brightness thresholds were used with early Landsat missions. Zhe Zhu and Curtis Woodcock introduced Fmask in 2012, which became the standard for Landsat and was later adapted for Sentinel-2. ESA released Sen2Cor as the operational processor for Sentinel-2 at its launch in 2015. The MAJA algorithm, developed by CNES and DLR, brought multi-temporal cloud detection into operational use. S2cloudless was released by Sinergise around 2018, becoming widely adopted through Sentinel Hub and Google Earth Engine. Since 2020, deep learning approaches have demonstrated superior performance for cloud edge delineation and thin cloud detection.