MixChannel: Satellite Images Can Now Be Processed with Minimal Datasets, Thanks to a New Technique from Skoltech

Computer vision algorithms are responsible for converting series of satellite images into a coherent and interpretable output. However, it takes time, and considerable computing power to achieve - and both can be toned down thanks to a new training method.

Researchers from Skolkovo Institute of Science and Technology (Skoltech) have found a method to assist computer vision algorithms in processing satellite images of the Earth with greater accuracy with limited datasets for training. This makes different remote sensing tasks easier for machines and, consequently, the people who use them and the data they provide.

Details of the new training method for computer vision algorithms are detailed in the report "MixChannel: Advanced Augmentation for Multispectral Satellite Images," appearing in the latest Remote Sensing journal.

ALSO READ: Genetic Markers That Control Sunflower Oil Traits Discovered in New Skoltech Study

Improving Computer Vision Algorithms

Machine learning and computer vision strategies have been a part of developing satellite images of Earth, which are then used for various applications. Tasks that are repetitive and usually prone to human error are easily accomplished by well-trained systems. However, before a machine learning system can successfully and accurately carry out its tasks, it needs to be trained first - a process that requires specialized skill, time, and a huge dataset.

Meanwhile, satellite images basically different from the standard smartphone photos, which can easily take multiple shots consecutively. These orbital cameras can only take a few shots of a certain location for each orbit, not to mention the resolution from such a distance is limited and that natural obstructions such as clouds affect the images. To work around this, a technology called image augmentation has been used by researchers to work around and have machine learning systems "fill in" to create quality images.

"While they are very powerful, neural networks demand a lot of training data to achieve top results. Unfortunately, in practical tasks, we usually don't have enough data," explains Sergei Nesteruk, co-author of the new study and a Ph.D. student from Skoltech, in a press release from the institute. He explains that to overcome this challenge, scientists usually apply different techniques to artificially increase the size of their datasets. One of the most common techniques is image augmentation, where images are altered to introduce variability to the existing dataset.

Developing MixChannel

It led Skoltech professor Ivan Oseledets and colleagues to develop a new image augmentation technique called MixChannel for multispectral satellite images. The method is built on substituting bands from original images using the same bands from images of the same location taken from another date.

"It is easy to use image augmentation for generic RGB images. But multispectral data is very complicated, and there was no efficient way to augment it," explains Svetlana Illarionova, also a co-author of the paper and Ph.D. student from Skoltech. "MixChannel is the novel augmentation technique designed to work specifically with multispectral data."

To test the MixChannel performance, researchers used satellite images from Sentinel-2, those of conifer and deciduous boreal forests in the Arkhangelsk region in northern European Russia to train a convolutional neural network to identify these forests. Although the region is notoriously cloudy, the new Skoltech method managed to outperform state-of-the-art solutions. Furthermore, researchers note that it can be combined with other image augmentation methods to improve machine learning systems.