New capability for developing environmental intelligence applications from satellite data
2 September 2020
The NERC Earth Observation Data and Acquisition and Analysis Service (NEODAAS) is pleased to announce the successful install of its state-of-the-art MAssive GPU cluster for Earth Observation (MAGEO).
MAGEO is an incredibly powerful computer intended to apply artificial intelligence algorithms to Earth observation (EO) data, to help develop ‘environmental intelligence’ applications that leverage patterns in the natural world. This includes research on wildfires, oil spills, macroplastic detection and habitat mapping.
The NEODAAS team intend to use MAGEO in three key areas of development: large scale computer vision neural networks applied to satellite imagery to provide automated data analysis and outputs that can be used for key decision making; development of “micro-Artificial Intelligence (AI)”, that can be used to improve the accuracy of small functions contained in larger applications; data fusion approaches, that can translate many sources into a single continuous dataset, and lastly, accelerate processing of satellite imagery, allowing the creation of higher resolution products at faster speeds using the high performance GPU computation capabilities MAGEO provides.
The system, commissioned by NEODAAS and funded by the Natural Environment Research Council’s (NERC) Transformational Capital Award, was built by OCF Limited and is hosted at Plymouth Marine Laboratory. With more than two hundred thousand Graphics Processing Unit cores, 100 times more than in a modern gaming PC, MAGEO is able to run a huge number of simultaneous simulations.
Above: The newly built MAGEO data centre
Although only just installed, MAGEO has already proved highly beneficial. At the beginning of the UK COVID lockdown, it was used in COVID-19 medical research rapidly making its way into the top 1% of computers contributing to folding@home.
MAGEO has also dramatically accelerated research efforts in the estimation of chlorophyll concentrations in the upper ocean. Due to the highly parallel environment and significant individual GPU computation power, research that would have taken 16 months on a single GPU machine took 10 days on MAGEO.
AI is a tremendous step forward within the Earth observation field. A staggering amount of data is produced by satellite every day and by using AI, researchers can tease out changes, trends and anomalies far easier and quicker than by standard analysis. The computing power provided by MAGEO alongside the existing Earth Observation expertise in NEODAAS, which is overseen by The National Centre for Earth Observation (NCEO), will enable researchers in the UK to fully utilise AI.
Deep Learning is a subfield of machine learning and while both fall under the broad category of artificial intelligence, deep learning is behind the most human artificial intelligence. Deep learning structures algorithms in layers to create an "artificial neural network” that can learn more flexibly and make intelligent decisions. Deep learning applications already make our lives easier from improving photos on our phones and identifying their contents automatically, to voice recognition in assistants like Siri or Alexa.
Not only is the hardware impressive but the software set up on the system has been developed to accommodate the requirements of different users. Deep Learning techniques use multiple libraries, with specific versions of each often needed to reproduce research. It would be difficult to setup MAGEO to suit everyone so utilising singularity containers, specific environments can be created to fit the needs of each researcher. The use of containers also ensures research can be reproduced as all libraries used are fixed within the container.
Combining Earth Observation data with Deep Learning has the potential to address many environmental challenges. However, the computing requirements can have a negative impact on the environment due to the resources needed to build systems and energy to power and cool them. PML commissioned Infiniti to build a state-of-the-art high performance micro datacentre (MDC) to house MAGEO and it contains high efficiency, in-row air conditioning units. Also the five NVIDIA DGX-1 units that make up MAGEO are MAXQ units, these are only available to research organisations, and use 50% of the power of the standard model while still delivering 80% of the performance.
Above: The NVIDIA DGX-1 hardware
In addition, building MAGEO as a shared resource that can be utilised by many researchers and supporting them to optimise their code, will mean NEODAAS can minimise any impact MAGEO has on the environment, relative to the benefits it will provide.
Dr Daniel Clewley, NEODAAS Manager and Research Software Engineer at Plymouth Marine Laboratory, comments: “Deep Learning has the potential for us to extract information from EO data in a way we have not been able to before but it requires a lot of computing power. MAGEO addresses this need for computing power combined with the world-class EO expertise of NEODAAS”.
MAGEO is available for researchers eligible for NERC funding by either including as part of a grant proposal or through direct access (free at the point of use). For more details contact NEODAAS.
Follow MAGEO’s progress on www.neodaas.ac.uk or follow on Twitter @NEODAAS.
Above: This animation shows satellite data from the SUOMI VIIRS sensor acquired over the UK from 2018 to the present day and processed by NEODAAS. Each frame represents a composite using images from a 7-day period. Colours are enhanced to highlight phytoplankton blooms forming and disappearing. While humans are very good at spotting features like blooms in satellite data, the volume of data being acquired means it is not practical to analyse all data but by using the power of MAGEO, far more data can be analysed.
Above: A European Space Agency (ESA) Climate Change Initiative Ocean Colour dataset showing the monthly average of chlorophyll concentration, one of the datasets being produced at Plymouth Marine Laboratory, which will be available for researchers to utilise with MAGEO.