Sponsored Feature: Humans have wanted to know what the weather is going to do since first we understood how the forces of nature determined our expectations of survival. And as far back in our history as 650 BCE, the Babylonians studied the appearance of cloud patterns as the basis of short-term forecasts.
Methods of weather prediction have moved on ever since, to the point where many of the world’s highest-performing computers are now deployed to inform forecasts and help us make sense of climatical phenomena. Typically, that involves using Numerical Weather Prediction (NWP) models – objective calculations of changes to the mapped weather based on physics-based mathematical equations.
These techniques have limitations though and are not always the most expedient way to meet the growing and diversifying demand for advance knowledge around impending weather events. According to a research team led by system solutions experts at Lenovo and the University of Connecticut (UConn) now devising a new AI-based approach to plotting weather futures, the Babylonians’ interest in cloud formations provides an instructive vector for fresh thought about an age-old challenge.
“The practice of weather forecasting is, to a significant extent, one of weather interpretation,” says Zaphiris Christidis, weather segment leader at Lenovo’s Infrastructure Solutions Group (ISG).
“Human forecasters interpret constantly updated information supplied by the weather bureaus and agencies. They use their forecasting experience and intelligence to reach an informed judgement on what the weather will likely do. With UConn, we are developing a platform that uses AI and neural networks to deliver forecasts but using machine intelligence instead of human acumen.”
The Business Of Prediction
Despite its long and well-established history, the weather prediction sector continues to thrive and expand. It’s not known if the weather watchers of old charged a fee for their prognostications, but today, the weather forecasting services market size is projected to grow to $5,483 million by the end of 2027 (ResearchAndMarkets). It could also expand to be worth between $4.6 billion (Precedence Research) and $4.19 billion (Allied Market Research) by 2030 – that’s an estimable CAGR of around 10 percent over the forecast periods.
Certainly, demand for weather forecasting services has grown markedly in recent years, driven by burgeoning verticals like transport and logistics, agriculture, insurance, and environmental management, and maritime. Additional interest from emergent sectors like disaster management and hazard mitigation have added to this demand.
Technological innovation is also playing a crucial part in driving this exponential growth, with the availability of high performance computing resources alongside institutional supercomputers, high-end radars, small satellites, data analytics – and, of course, AI.
Broadly, the weather forecasting services market segments into short-range (up to three days), medium-range (up to 10 days), long-range (up to 30 days) and extended-range (two to three month) forecast types. But it is ultra-short-range predictions – or ‘nowcasts’ – of weather outcomes in the period of three-to-six hours, that are the focus of the Lenovo ISG and UConn collaboration.
Lenovo has proven reach into the global weather systems sector and is already working with a lengthy list of partners in operational weather centers and weather research centers located around the world. That includes partnerships with the National Center for Meteorology in Saudi Arabia and the Korea Meteorological Administration, for example.
The Lenovo/UConn ‘AI Nowcasting Project’ leverages a neural network and draws on spatio-temporal modelling techniques from the field of anomaly detection developed to perform unsupervised video anomaly detection for surveillance video use-cases.
As a generic application, ‘nowcasting’ is weather forecasting on a very short-term mesoscale meteorology period of between two and six hours (definitions vary). Such forecasts rely on methods of extrapolation over time of known weather parameters, rather than mathematics-based models.
“There are, of course, several different ways in which AI and Deep Learning can be deployed to aid weather forecasters, including to augment NWP,” Christidis says. “However, most of the work in these fields is still ongoing. To understand what we think are the advantages of our approach, you first have to understand the characteristics of the traditional prediction methodologies.”
Raining Buckets Of Data
With or without AI, NWP models are proficient at foretelling typical weather systems, but they have their limitations and blindsides, adds Christidis. Macroscale meteorological events, for instance, can occur almost without warning. Weather patterns for the medium-term outlook can also change drastically in the time between initial data collection and a forecast being issued and acted on.
One of the limitations imposed on ‘time-to-prediction’ is the time required for data processing. Manipulating the vast datasets and crunching the complex calculations necessary to most NWP is time-consuming, even for the most powerful supercomputers used by the world’s top meteorological bureaus.
“For supercomputers to make weather predictions, they need to obtain weather data,” Christidis says. “The data used in weather forecasting is supplied by a wide variety of sources – satellites, weather stations, balloons, airplanes, ships at sea, and more. Forecasters can also access the Global Telecommunication System that collects and disseminates data four times a day in six-hour intervals. In short, there are massive data sets available for NWP models.”
The size of the datasets supplied by these diverse sources can be anything between 1 GB and 1 TB, explains Christidis. And before the data can be used, it must be put through a quality control stage. Once completed, the data is then imported into mathematical models which use them to base forecasts on.
“These models are equations that describe the state, motion, and time evolution of various atmospheric parameters such as wind and temperature,” Christidis says, “so a pretty complex and time-consuming undertaking all in all.”
Turning these equations into accurate forecast information for a given geographical area requires an additional factor – masses of compute power.
“To derive the information from these models takes quite a time… you have this weather data that arrives at the weather center that then takes maybe up to two hours to be quality-checked and processed,” explains Christidis. “Then they distribute the data to other smaller weather services for localized weather forecasts… so that adds another hour or so to the delivery timeframe.”
Shortening Time To Prediction
The AI Nowcasting Project’s rationale is predicated in part by shortening time-to-prediction by using AI to come up with basic information based on observed phenomena rather than massively scaled data crunching. What fine detail might be lacking is made-up for in its speed of delivery.
“Different organizations have very different weather forecast requirements – for example, where maintenance engineers are scheduled to work in exposed and hazardous places, rocket launches, or high-profile sporting events,” Christidis says. “Many use-cases exist where it is most important to know quickly what the weather is going to do in the imminent future – the next three-to-four hours, say – rather than in details over a longer timescale,”
In view of this, the AI Nowcasting Project uses observed reflectivity radar data images – past and present – as its primary source data. Reflectivity radar data images present a high-definition picture of the weather from the energy reflected back to radar receivers. Reflectivity images are the radar images that most often appear in TV weather updates.
The AI Nowcasting Project first analyzes a set of reflectivity radar data images that pertain to actual past weather events that occurred over a specified geographical area – these have come from UConn’s extensive database of radar images of weather events. By doing this, the system’s AI component is ‘trained’ in how those weather events evolved and unfolded over set start-to-finish points.
The AI then applies this knowledge to real-time satellite images from current weather patterns for the same geographical area, using a Convolutional Transformer methodology to derive an extrapolative view of how it ‘expects’ the weather will develop.
“Very simply put, our prototype interprets ongoing weather movements in the context of past specific weather events it has been trained on and posits a ‘view’ on how the current situation will unfold,” explains Christidis. “It is interpreting current weather patterns on the basis of past weather patterns and outputting an informed view – or forecast – of how things will turn out.”
Lenovo and UConn’s collaboration on the AI Nowcasting Project began in June 2022 and is due to conclude its first development phase in August 2023.
“With UConn, we continue to refine the algorithms, but we have had good results that are extremely promising,” Christidis reports. “Even at this stage we believe that forecasts based on AI analysis of cloud formations is probably no more or no less accurate than equivalent predictions based on NWP.”
Sponsored by Lenovo.
Be the first to comment