AIESD230090_proof.pdf

Dynamical Tests of a Deep Learning Weather Prediction Model

GREGORY J. HAKIM

AND SANJIT MASANAM

Department of Atmospheric Sciences, University of Washington, Seattle, Washington

Department of Physics, University of California at Santa Barbara, Santa Barbara, California

(Manuscript received 23 October 2023, in ﬁnal form 21 April 2024, accepted 15 May 2024)

ABSTRACT: Global deep learning weather prediction models have recently been shown to produce forecasts that rival

those from physics-based models run at operational centers. It is unclear whether these models have encoded atmospheric

dynamics or simply pattern matching that produces the smallest forecast error. Answering this question is crucial to estab-

lishing the utility of these models as tools for basic science. Here, we subject one such model, Pangu-Weather, to a set of

four classical dynamical experiments that do not resemble the model training data. Localized perturbations to the model

output and the initial conditions are added to steady time-averaged conditions, to assess the propagation speed and struc-

tural evolution of signals away from the local source. Perturbing the model physics by adding a steady tropical heat source

results in a classical Matsuno–Gill response near the heating and planetary waves that radiate into the extratropics. A local-

ized disturbance on the winter-averaged North Paciﬁc jet stream produces realistic extratropical cyclones and fronts, in-

cluding the spontaneous emergence of polar lows. Perturbing the 500-hPa height ﬁeld alone yields adjustment from a state

of rest to one of wind–pressure balance over ;6 h. Localized subtropical low pressure systems produce Atlantic hurricanes,

provided the initial amplitude exceeds about 4 hPa, and setting the initial humidity to zero eliminates hurricane develop-

ment. We conclude that the model encodes realistic physics in all experiments and suggest that it can be used as a tool for

rapidly testing a wide range of hypotheses.

KEYWORDS: Dynamics; Idealized models; Machine learning

1. Introduction

In the past few years, deep learning (DL) weather predic-

tion models demonstrate forecast skill comparable to those

from government operational centers (Bi et al. 2023; Kurth

et al. 2023; Lam et al. 2023; Ben Bouall

egue et al. 2024).

These models are trained on ERA5 analyses and have fore-

cast skill on initial conditions not contained in their training

data. In contrast to DL approaches that explicitly enforce

physical constraints (e.g., Beucler et al. 2021), it is unclear

whether these models have encoded atmospheric physics,

such as the dynamics of air motion and propagation of distur-

bances, or simply patterns that minimize the squared error of

the next pattern in a sequence. If these models can be shown

to produce physically realistic solutions, they offer an enor-

mous opportunity for testing hypotheses much faster than is

currently possible. More importantly, these models may offer

a new path to discovery for multiscale problems, where solu-

tions from physics models reﬂect uncertain parameterizations

of poorly resolved processes such as moist convection, small-

scale mixing, and surface ﬂuxes.

Physical tests that examine the evolution of spatially local-

ized disturbances are particularly effective in analyzing model

physics, since the propagation of signals away from these dis-

turbances is constrained by dynamics. For example, in the

small-amplitude limit, the group velocity in linear wave theory

sets the speed of energy dispersion away from a local distur-

bance. Here, we apply the localized-disturbance approach

to the Pangu-Weather model of Bi et al. (2023) using four

canonical experiments, one involving perturbations to the

model output and the other three involving perturbed initial

conditions. The perturbations are applied to climatological

time-mean steady states, which are smoother than any indi-

vidual state that the model was trained on. These experiments

are subjectively chosen, and while solutions are not compared

directly to identical experiments in a physics-based model,

they provide an important plausibility study to motivate such

additional experiments. Our hypothesis at the start of this re-

search was that localized features will immediately produce a

global response, because no constraint was imposed to pre-

vent this during model training.

In addition to running ord ers of magni tude faster than physics-

based models, these experiments with the Pangu-Weather model

are comparatively easy to conﬁgure. Performing any one of

the experiments described here with a modern physics-based

weather model is a signiﬁcant undertaking, primarily due to

complexities associated with model initialization (e.g., Daley

1993; Kalnay 2003). For example, one case we consider here

involves idealized extratropical cyclo ne development over

the Paciﬁc Ocean, for which there have been various ap-

proaches using idealiz ed ba sic-stat e jet s treams and initial

perturbations. Rotunno et al. (1994) provide an analytical

approximation of an idealized jet stream originally shown

in Simm ons and Hoskins (1975), which is also used in later

studies (e.g., Menchaca and Durran 20 17). Rotunno et al.

(1994) numerically condition the initial perturbation on th e

linearly most unstable normal mode of the jet stream, whereas

Supplemental information related to this paper is available

at the Journals Online website: https://doi.org/10.1175/AIES-D-

23-0090.s1.

Corresponding author: Gregory J. Hakim, gha[email protected]du

DOI: 10.1175/AIES-D-23-0090.1 e230090

Ó 2024 American Meteorological Society. This published article is licensed under the terms of the default AMS reuse license. For information regarding

reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

HAKIM AND MASANAM 1JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

Menchaca and Durran (2017) perturb the jet with a localized

disturbance deﬁned by potential vorticity inversion, followed

by a time-ﬁltering procedure to remove gravity waves. Recog-

nizing the need for a standardized baroclinic wave test case,

Polvani et al. (2004) and Jablonowski and Williamson (2006)

developed prot ocols for such experiments in general circula-

tion models. In particular, Jablonowski and Williamson

(2006) provide analytical expressions for the jet stream and

initial disturbance and compare solutions for four different

models and varying grid resolutions to assess solution con-

vergence. To apply this test to deep learning mo dels, uni-

form surface boundary conditions are needed for these

models, which the current-generation models do not have

since they are trained on reanalysis grids. However, similarly

conﬁgured deep learning models could be directly compared,

and one motivation of this study is to provide a starting point

for such “unit test s” of the model.

We proceed in section 2 with a description of the experi-

ments and the data used to conduct them. Results for the four

experiments described above are presented in section 3. Con-

clusions are drawn in section 4.

2. Method and experiment design

The Pangu-Weather model uses a vision transformer architec-

ture trained on ERA5 rean alysis data from 1979 to 2017 (Bi et al.

2023); the trained model weights are publicly available. Model

variables consist of global gridded ﬁelds of geopotential height,

speciﬁc humidity of water vapor, temperature, and vector wind

components on 13 isobaric levels (1000, 925, 850, 700, 600, 500,

400, 300, 250, 200, 150, 100, and 50 hPa), and surface ﬁelds (mean

sea level pressure, 2 m air temperature, and 10 m vector wind

components). Data reside on the native 0.25

latitude–longitude

grid of ERA5. There are four models, which are trained sepa-

rately for different forecast lead times: 1, 3, 6, and 24 h. Bi et al.

(2023) indicate that solutions are most accurate when using the se-

quence of mode ls with the smallest number of steps to reach a de-

sired lead time (e.g., a 32 h forecast uses the 24 h model, followed

bythe6hmodelandthentwostepsofthe1hmodel).

Our experiments involve adding perturbations to a steady

climatological mean atmosphere. We perform the simulations

by solving

x(t 1 1) 5 N[x(t)] 2 d

x 1 f: (1)

Here, x represents the model state vector, N represents the

Pangu-Weather model, and t represents time indexed according

to the version of the model (i.e., t 1 1 means a 1-day forecast

when using the 24-h version of the model and a 3-h forecast for

the 3-h version). The term f is a modiﬁcation to the model out-

put, taken here to be zero for all experiments except steady

tropic al heating, when it is ﬁxed at a speciﬁed value. The term

x repre sents the one-step solution of the model that renders

the climatological mean atmospheric steady state:

x 5 N(x) 2 x: (2)

We may then take

x independent of time. The full state vector,

which we send to the Pangu-Weather model, is deﬁned by

x 5

x 1 x



, where x



are anomalies from the climatological

mean state. For x



5 f 5 0, (1) with (2) gives x(t 1 1) 5

x) 2 N(x) 1 x 5 x;i.e.,x is time independent.

Since we are interested in spatially localized perturbations,

we deﬁne f and the initial perturbations x



(t 5 0) using a func-

tion that decays to zero from a local maximum at a speciﬁed

distance. For this purpose, we use the function deﬁned by

Eq. (4.10) in Gaspari and Cohn (1999) and deﬁne the distance

at which the disturbance reaches zero by L:

G(r; L)

1 10# d # L/2

2 5r 1 4 2

L/2 # d # L

0 L # d

⎧

⎪

⎨

⎪

⎩

(3)

Here, r 5 2d/L and d is the distance on the sphere from a cen-

tral reference point.

For the steady heating experiment, we set f to be a constant

vector with zeros everywhere except for the temperature ﬁeld

within a horizontal region at all levels between 1000 and

200 hPa, where it is set to 0.1 K (day)

. The region is deﬁned

in longitude by (3) with L 5 10 000 km and centered at 120

and in latitude

by cos(6

) within 15

of the equator. The ini-

tial condition is given by

x, which is set to the 0000 UTC

1979–2019 December–February (DJF) ERA5 time average.

For the extratropical cyclone experiment, we deﬁne the

anomaly ﬁeld x



(t 5 0) by simple linear regression of all ﬁelds

in the state vector against a standardized time series of DJF

500-hPa geopotential height at the point 40

N, 150

E. The re-

gressed ﬁeld is then multiplied by (3) with L 5 2000 km to en-

sure a spatially localized disturbance and added to the DJF

time-mean ﬁeld. We use the same perturbation initial condi-

tion for the geostrophic and hydrostatic adjustment experi-

ments, except we set to zero all variables at all levels except

the 500-hPa geopotential height.

For the hurricane experiments, we take the same approach

as for the extratropical cyclone experiment, except in this case

we use the July–September (JAS) mean state. The distur-

bance is deﬁned by simple linear regression of all ﬁelds in the

state vector against a standardized time series of JAS mean

sea level pressure at the point 15

N, 40

W. The regressed ﬁeld

is then multiplied by (3) with L 5 1000 km and added to the

JAS time-mean ﬁeld. For the results in this case, we perform

simulations by scaling the perturbation ﬁeld by a multiplicative

constant to vary the strength of the initial low pressure system.

3. Results

a. Steady tropical heating

Tropical heating anomalies associated with long-lived phe-

nomena such as El Niño and the Madden–Julian oscillation

produce global changes in circulation. As a result, there is a

long history of research to understand the mechanism relating

tropical heating to circulation changes. The most well-known

ARTIFICIAL INTELLIGENCE FOR THE EARTH SYSTEMS VOLUME 32

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

experiments examine the response of a climatological atmo-

spheric state to a steady heating pattern (e.g., Hoskins and

Karoly 1981; Sardeshmukh and Hoskins 1988; Jin and Hoskins

1995), which reveal planetary waves radiating into the extra-

tropics along great circle paths. Details of the response depend

on the mean state, through the potential vorticity gradients that

affect planetary wave propagation, and the structure of the heat-

ing ﬁeld (Ting and Sardeshmukh 1993). A representative exam-

ple showing the effect of the lo cation of the heating on the wave

response, and sensitivity to details in the mean state, is found in

Fig. 7 of Ting and Sardeshmukh (1993).

The Pangu-Weather response to weak DJF tropical heating

(0.1 K day

) shows a small 500-hPa height increase over the

heating region after 5 days, with a negative anomaly to the

north (Fig. 1a). The extratropical wave train extends down-

stream and increases in amplitude during days 5–20, with

maximum anomalies over 100 m at day 20 (Figs. 1b,c). A

wave train appears in both hemispheres, with larger ampli-

tude in the Northern (wintertime) Hemisphere, which has

the stronger waveguide for stationary waves. This response is

qualitativel y similar to c lass ical results (e.g., Hoskins and

Karoly 19 81 ; Sardeshmukh and Hoskins 1988), with differ-

ences in details dependent on the location, shape, and tem-

poral structure of the heating, se asonalit y, and other factor s.

A closer examination of the response in the lower tropo-

sphere near the heating reveals a pattern similar to the classi-

cal Matsuno–Gill (Matsuno 1966; Gill 1980) response to

steady tropical heating (Fig. 2). Along the equator, wind

anomalies are convergent toward the western end of the heat-

ing region. This signature is associated with a Kelvin-wave re-

sponse to the heating. Off the equator, the western end of the

heating is ﬂanked by cyclonic gyres in both hemispheres,

which are associated with a mixed-Rossby–gravity wave re-

sponse. Unlike idealized experiments, typically using shallow-

water equations, these solutions are inﬂuenced by surface

boundary conditions, so that there are ﬂow distortions over

the Maritime Continent in particular and myriad multiscale

moist processes involving clouds and convection.

This experiment suggests that the Pangu-Weather model

responds qualitatively, if not quantitatively, consistent with

idealized experiments for tropical heating. Anomalies emerge

smoothly and locally from the heat source and increase in am-

plitude with time as a nearly stationary wave response. Ideal-

izing the problem further to the zonal-mean DJF basic state

produces a similar response, with a wave train extending

across the North Paciﬁc to North America (Fig. S1 in the on-

line supplemental material), but with differences in phase and

amplitude related to the basic state on which the waves propa-

gate. The Southern Hemisphere response is also notably

weaker for the zonal-mean state, which may be related to a

weaker extratropical waveguide (smaller potential vorticity

gradients) in summer.

b. Extratropical cyclone development

The next experiment considers the time evolution of a lo-

calized 500-hPa trough at the western end of the North Paciﬁc

storm track (Fig. 3a), which is the canonical initial condition

preceding surface cyclogenesis (e.g., Gyakum and Danielson

2000; Hakim 2003; Yoshida and Asuma 2004). Observations

and idealized modeling results show the development of a

localized extratropical cyclone, with subsequent cyclones ap-

pearing downstream as the disturbance evolves into a spreading

wave packet (e.g., Hakim 2003; Jablonowski and Williamson

2006).

Results for Pangu-Weather show that after 2 days the trough

has pr ogr esse d to the cen tral Paciﬁc and begun to disperse, with

the appearance of anticyclonic circulation s both upstream and

downstream (Fig. 3b). A surface cyclone develops to the east

of the upper trough, with a smaller-scal e secondary cyclone

appearing upstream (Fig. 4b). By day 4, the upper trough has

ampliﬁed and spread into a wave packet, with the leading edge

along western North America (Fig. 3d), and a surface cyclone

nearly coincident with the upper trou gh (Fig. 4d). Vertical align-

ment of extratropical cyclones is the hallmark of a developing

cyclone that has reached the occluded phase of the life cycle.

In contrast, the upstream surface cyclone remains downstream

of the 500-hPa trough and continues to deepen past day 4. A

second upstream cyclone appears at day 4 west of the date line.

These cyclones are accompanied by temperature anomalies hav-

ing the largest horizontal gradients near the surface cold front

(Fig. S2).

All aspects of this idealized baroclinic development are

consistent with observations and modeling (e.g., Jablonowski

and Williamson 2006) of localized extratropical cyclone devel-

opment. In particular, disturbances at the upstream end of the

storm track produce a baroclinic wave packet (Simmons and

Hoskins 1979) that disperses and moves downstream at the

group velocity (faster than the phase of individual troughs).

As we ﬁnd here, these solutions also show both upstream sur-

face development and downstream upper-level development

(Simmon s and Hoskins 1979; Chang 1993; Hakim 2003).

Moreover, the upstream surface development we observe

here has a relatively smaller spatial scale, resembling a “polar

low,” which is frequently observed in winter over the North

Paciﬁc (e.g., Mullen 1983; Rasmussen 2003). Curiously, these

polar lows appear ﬁrst at the surface and have a warm core,

suggestive of the importance of surface ﬂuxes due to cold air

moving over relatively warmer water (Emanuel and Rotunno

1989).

Idealizing the problem further to the zonal-mean DJF at-

mosphere produces a similar response, with a wave packet

that spreads downstream toward Europe by day 10 (Fig. S3).

Furthermore, repeating the experiment, but for summer

conditions (JAS time m ean), shows much weaker cy clone

development and an absenc e of polar lows (not shown). W e

conclude that Pangu-Weather appears to have implicitly en-

coded the seasonally varying physical p rocesses of oceanic

extratropical cyclone development in the neural network

weights that govern the dynamical evolution of its prognostic

variables.

c. Geostrophic and hydrostatic adjustment

Here, we test an initial perturbation similar to the extra-

tropical cyclone case, except that it is localized completely to

HAKIM AND MASANAM 3JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

the 500-hPa ﬁeld; it does not extend in the vertical, and every

other ﬁeld has zero anomaly. We note that this initial condi-

tion lies outside all aspects of the ERA5 training data for the

Pangu-Weather model, since ERA5 is produced with a hydro-

static model. This type of initial condition is unbalanced since

there are no wind or temperature anomalies, whereas outside

the deep tropics one commonly ﬁnds the wind ﬂowing along

the height contours (as evident in Fig. 3a). The classic text-

book example is the Rossby adjustment problem, consisting

of an initially stationary shallow layer of water with a jump in

the free surface, which evolves to a state of geostrophic bal-

ance (e.g., Holton and Hakim 2013, section 5.6). This is a

FIG. 1. Response in DJF 500-hPa geopotential height to steady tropical heating of 0.1 K day

within the region outlined by the dashed red line. The DJF-averaged geopotential height is

shown by gray lines every 60 m, and anomalies are shown by red (positive) and blue (negative)

lines; the zero contour is suppressed. Solutions are shown for (a) 5 days (contours every 0.3 m);

(b) 10 days (contours every 2 m); and (c) 20 days (contours every 20 m).

ARTIFICIAL INTELLIGENCE FOR THE EARTH SYSTEMS V

OLUME 34

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

particularly hard test and one that likely cannot be performed

without additional modiﬁcation using a physics-based model,

since unbalanced initial conditions produce rapid oscillations

that are difﬁcult to resolve. An example showing the initial re-

sponse of a localized anomaly analogous to the one consid-

ered here (but of opposite sign) is shown in Lelong and

Sundermeyer (2005, their Fig. 5). The initial response is diver-

gent ﬂow, which under the action of Coriolis turning rotates

over an inertial period to a state of wind–pressure balance.

Since we are interested in the initial response at short time

scales, we use the 1-, 3-, and 6-h versions of the Pangu-

Weather model here. Results show that, after 1 h, the wind ac-

celerates from rest in the initial conditions to about 5 m s

and is convergent on the area of low geopotential height

(Fig. 5a). The center of convergence is to the west of the

lowest height, which increases to 289 m from 2100 m in the

initial condition. At 3 h, the wind accelerates to a maximum

of about 10 m s

and remains convergent on the area of low

height, for which the minimum has increased to 274 m (Fig. 5b).

The wind direction has turned clockwise at all locations com-

pared to the 1-h solution, as one expects from the Coriolis

turning of the accelerating wind in the direction of the pressure

gradient force. At 6 h, the wind direction has continued to ro-

tate clockwise such that it is nearly parallel to the geopotential

height contours everywhere, reﬂecting a closer balance be-

tween the wind and geopotential height ﬁelds (Fig. 5c). The

height minimum has increased to 258 m, reﬂecting a conver-

sion of available potential energy to kinetic energy.

A quarter turn of a Foucault pendulum at 40

N takes ;9h,

so the adjustment in the wind ﬁeld indicated by the Pangu-

Weather solution is consistent with physical expectations. Once

again, we conclude that the solution for this idealized initial-

value problem is qualitatively, if not quantitatively, consistent

with the expected dynamics.

Repeating the experiment, except for an initial disturbance

on the equator, produces a notably different response (Fig. S4).

The velocity ﬁeld is again convergent on the area of low geopo-

tential height, except in this case convergence is directed on the

center of the low. The difference may be due to the basic-state

jet stream in the previous case, with fast westerly winds and a

strong meridional potential vorticity gradient that promotes

westward Rossby-wave propagation. Another notable aspect of

the equatorial case is the slower Coriolis turning of the wind

and the fact that the model has learned about the asymmetry in

this turning about the equator. An analysis of the time differ-

ence in the anomalous zonal wind on the equator reveals signals

that propagate in both directions at around 20 m s

(Fig. S5),

typical of tropical gravity waves. A weaker signal is also evident

at the speed of sound (dashed black line s). Fina lly, we no te that

this ﬁgure shows the incompatibility between the different

versions of the Pangu-Weather model, with abrupt differ-

ences in the time tendency at intervals of 3, 6, and 24 h. These

single-step “shocks” do not appear to adversely affect the so-

lution at subsequent times but will affect temporal diagnostic

calculations that span several time steps of the model.

d. Atlantic hurricane development

The last example concerns the evolution of a localized dis-

turbance in the subtropics for the JAS averaged conditions.

Seeds of Atlantic hurricanes take the form of weak low pres-

sure systems, which may develop into mature storms given

the right environmental conditions. Finite-amplitude distur-

bances are thought to be needed to reduce the time to devel-

opment while the storm is in a favorable environment (e.g.,

McBride and Zehr 1981; Rotunno and Emanuel 1987; Nolan

et al. 2007). Here, we perform experiments for a localized

area of low pressure at a reference location (15

N, 40

W) and

vary the initial amplitude. The three-dimensional perturba-

tion is constr ucted si milarly to the initial cond ition for the

extratropical cyclone case, by regressing all variables and

locations on to the mean sea level pressure at the reference

location.

Results show that the low pressure systems take a familiar

track toward the northwest around the climatological subtrop-

ical area of high pressure (Fig. 6). Stronger initial conditions

take a progressively northward track, which is consistent with

the known physical basis due to the increasing amplitude of

azimuthal wavenumber-1 asymmetries (“

gyres”). Although

Pangu-Weather may at best poorly resolve these features, the

FIG. 2. The 850-hPa anomaly wind vectors for the steady heating experiment after 20 days. The

red dashed line outlines the region of steady heating.

HAKIM AND MASANAM

5JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

FIG. 3. Solution at 500 hPa for a localized disturbance on the DJF atmosphere. The

full geopotential height is shown by gray lines every 60 m, and anomalies from the DJF

average are shown by red (positive) and blue (negative) lines every 20 m; the zero

contour is suppressed. Green arrows show the anomalous vector wind. Solutions are

shownat(a)0(thespeciﬁed initial condition); (b) 2; (c) 3; and (d) 4 days.

ARTIFICIAL INTELLIGENCE FOR THE EARTH SYSTEMS V

OLUME 36

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

FIG. 4. Surface cyclones associated with the solution in Fig. 3. Anomalies in mean sea level

pressure are shown every 2 hPa, with red (blue) lines for positive (negative) values; the zero

contour is suppressed. Water vapor speciﬁc humidity anomalies (g kg

) at 850 hPa are shaded.

Solutions are shown at (a) 0 (the speciﬁed initial condition); (b) 2; (c) 3; and (d) 4 days.

HAKIM AND MASANAM

7JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

FIG. 5. Solution at 500 hPa for the geostrophic adjustment problem consisting of a localized

geopotential height disturbance on the DJF-averaged atmosphere. The full geopotential height

is shown by gray lines every 60 m, and negative anomalies are shown by blue lines every 20 m;

the zero contour is suppressed. Green arrows show the anomalous vector wind. Solutions are

shown at (a) t 5 0 (the speciﬁed initial condition); (b) 1; (c) 3; and (d) 6 h.

ARTIFICIAL INTELLIGENCE FOR THE EARTH SYSTEMS V

OLUME 38

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

neural network has identiﬁed this physical relationship be-

tween the strength of tropical cyclones and a northward track.

For initial disturbances with anomalous mean sea level

pressure less than about ;4 hPa, the storms do not intensify,

whereas initial disturbances stronger than this rapidly intensify

(Fig. 7). An additional experiment for the 103 disturbance was

performed by setting the water vapor speciﬁc humidity to zero,

and unlike the original case that rapidly develops, the dry sys-

tem rapidly decays. Pangu-Weather does not explicitly model

condensational heating, but the model has the conditional asso-

ciation between water vapor content and the development of

tropical cyclones.

4. Conclusions

We have tested the Pangu-Weather deep learning weather

prediction model on a set of four canonical experiments

aimed at probing its dy namical response to local pertur-

bations. These pe rturbations are helpful for determining

whether disturbances evolve and propagate in a physically

meaningful manner. Our hypothesis at the outset of this

work was that these localized features would immediately

produce a global response because t here is no constraint

to prevent this during model training. The fact that each

experiment produced signal propagation and structural evolu-

tion qualitatively in accord with previous research in meteoro-

logy suggests that the model has encoded realistic physics.

While we do not make a direct comparison to solutions from a

physics-based model, the results here provide a proof of concept

motivating such experiments. We note that, due to differences

in numerics, boundary conditions, and parameterizations for

unresolved scales and processes, solutions from physics-based

models for these experiments will differ in details, and it

would be interesting to see whether the Pangu-Weather solu-

tions fall within the uncertainty of the physics-based models.

Perhaps equally useful is an application of the experiments

presented here for comparing deep learning models. Since

many models are trained on the same reanalysis data, issues

related to surface boundary conditions are less important,

facilitating model comparisons f or what may be regarded a s

unit tests.

Results from the canonical experiments show qualitative, if

not quantitative, agreement with studies of similar phenomena

in observations and numerical simulations. This agreement

ranges from hourly time scales for the geostrophic adjustment

process to approximately steady features beyond 10 days as-

sociated with stationary tropical heating. Highlights from

these experiments, summarized in Table 1, include a Matsuno–

Gill response and extratropical planetary wave response to

steady tropical heating; baroclinic wave packet emergence and

polar low development in the cold air mass associated with a

North Paciﬁc extratropical cyclone; divergent ﬂow yielding to

rotational ﬂow for an unbalanced initial condition; and the

importance of initial-vortex amplitude and water vapor in the

development and track of Atlantic hurricanes.

We conclude that the Pangu-Weather model encodes real-

istic physics for the experiments considered here, motivating

future basic research using this tool. Several attributes make

this model particularly powerful for atmospheric dynamics

and scientiﬁc hypothesis testing. First, the simulations are

computationally inexpensive compared to traditional global

weather models. This enables large ensembles, including iter-

ations over varying parameters, initial conditions, and pertur-

bations to model output. Second, experiments are extremely

easy to conﬁgure, and the model is very forgiving in aspects

that physics models are not. For example, initial imbalances

in physics-based models can produce spurious oscillations at

the model time step that are difﬁcult to remove or ﬁlter with-

out affecting the resolved scales of interest. Therefore, we

speculate that models like Pangu-Weather might be particu-

larly useful for the rapid evaluation of hypotheses, allowing

FIG. 6. Tracks of mean sea level pressure minima for experiments

seeding Atlantic hurricanes on the July–September-averaged

atmosphere. All experiments are initialized with a surface-based

low pressure system at 15

N, 40

W, and initial amplitude by a

scaling factor on the climatological JAS standard deviation at that

location (indicated in the legend).

FIG. 7. Intensity of the low pressure systems tracked in Fig. 6 in

terms of anomalous mean sea level pressure (hPa) as a function of

time (days).

HAKIM AND MASANAM

9JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

tests over a wide range of ideas to quickly narrow the scope of

investigation for experiments using expensive physics-based

models. Among many possibilities, one particularly interest-

ing path of research employs deep learning models to exam-

ine multiscale phenomena involving convective clouds, such

as the Madden–Julian oscillation, where physics-based models

and theory have not yet approximated the essential physical

processes.

Acknowledgments. We thank Steve Penny for conversations

related to deep learning models in the geosciences and Mike

Pritchard for comments on an earlier draft of the manuscript.

Comments from three anonymous reviewers and editor David

John Gagne were helpful in improving the clarity of the

manuscript. GJH acknowledges support for this research

from NSF Award 2202526 and Heising-Simons Foundation

Award 2023-4715.

Data availability statement. Code and data to reproduce

results in this paper can be found at this GitHub repository:

https://github.com/modons/DL-weather-dynamics.

REFERENCES

Ben Bouall

egue, Z., and Coauthors, 2024: The rise of data-driven

weather forecasting: A ﬁrst statistical assessment of machine

learning-based weather forecasts in an operational-like con-

text. Bull. Amer. Meteor. Soc., 105, E864–E883, https://doi.

org/10.1175/BAMS-D-23-0162.1.

Beucler,T.,M.Pritchard,S.Rasp,J.Ott,P.Baldi,andP.Gentine,

2021: Enforcing anal ytic constraints in ne ural networks

emulating physical systems . Phys.Rev.Lett., 126, 098302,

https://doi.org/10.1103/PhysRe vLett.126.0983 02.

Bi, K., L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, 2023:

Accurate medium-range global weather forecasting with 3D

neural networks. Nature, 619,533–538, https://doi.org/10.1038/

s41586-023-06185-3.

Chang, E. K. M., 1993: Downstream development of baroclinic

waves as inferred from regression analysis. J. Atmos. Sci., 50,

2038–2053, https://doi.org/10.1175/1520-0469(1993)050,2038:

DDOBWA.2.0.CO;2.

Daley, R., 1993: Atmospheric Data Analysis. Cambridge University

Press, 472 pp.

Emanuel, K. A., and R. Rotunno, 1989: Polar l ows as Arctic

hurricanes. Tellus, 41A (1), 1–17, https://doi.org/10.34 02/tellusa.

v41i1.11817.

Gaspari, G., and S. E. Cohn, 1999: Construction of correlation

functions in two and three dimensions. Quart. J. Roy. Meteor.

Soc., 125, 723–757, https://doi.org/10.1002/qj.49712555417.

Gill, A. E., 1980: Some simple solutions for heat-induced tropical

circulation. Quart. J. Roy. Meteor. Soc., 106, 447–462, https://

doi.org/10.1002/qj.49710644905.

Gyakum, J. R., and R. E. Danielson, 2000: Analysis of meteoro-

logical precursors to ordinary and explosive cyclogenesis in

the western North Paciﬁc. Mon. Wea. Rev.,

128, 851–863,

https://doi.org/10.1175/1520-0493(2000)128,0851:AOMPTO.2.

0.CO;2.

Hakim, G. J., 2003: Developing wave packets in the North Paciﬁc

storm track. Mon. Wea. Rev., 131, 2824–2837, https://doi.org/

10.1175/1520-0493(2003)131,2824:DWPITN.2.0.CO;2.

Holton, J. R., and G. J. Hakim, 2013: An Introduction to Dynamic

Meteorology. Academic Press, 552 pp.

Hoskins, B. J., and D. J. Karoly, 1981: The steady linear response

of a spherical atmosphere to thermal and orographic forcing.

J. Atmos. Sci., 38, 1179–1196, https://doi.org/10.1175/1520-

0469(1981)038,1179:TSLROA.2.0.CO;2.

Jablonowski, C., and D. L. Williamson, 2006: A baroclinic instability

test case for atmospheric model dynamical cores. Quart. J. Roy.

Meteor. Soc., 132, 2943–2975, https://doi.org/10.1256/qj.06.12.

Jin, F., and B. J. Hoskins, 1995: The direct response to tropical

heating in a baroclinic atmosphere. J. Atmos. Sci., 52,307–319,

https://doi.org/10.1175/1520-0469(1995)052,0307:TDRTTH.2.

0.CO;2.

Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and

Predictability. Cambridge University Press, 341 pp.

Kurth, T., and Coauthors, 2023: FourCastNet: Accelerating global

high-resolution weather forecasting using adaptive Fourier

neural operators. Proc. of the Platform for Advanced Scientiﬁc

Computing Conf., Davos, Switzerland, Association for

Computing Machinery, 1–11, https://doi.org/10.1145/3592979.

3593412.

Lam, R., and Coauthors, 2023: Learning skillful medium-range

global weather forecasting. Science, 382,1416–1421, https://

doi.org/10.1126/science.adi2336.

Lelong, M.-P., and M. A. Sundermeyer, 2005: Geostrophic adjust-

ment of an isolated diapycnal mixing event and its implications

for small-scale lateral dispersion. J. Phys. Oceanogr., 35, 2352–

2367, https://doi.org/10.1175/JPO2835.1.

Matsuno, T., 1966: Quasi-geostrophic motions in the equatorial area.

J. Meteor. Soc. Japan, 44,25–43, https://doi.org/10.2151/jmsj1965.

44.1_25.

McBride, J. L., and R. Zehr, 1981: Observational analysis of tropical

cyclone formation. Part II: Comparison of non-developing

versus developing systems. J. Atmos. Sci., 38, 1132–1151,

https://doi.org/10.11 75/1520-0469(1 981)038,1132:OAOT CF.2.

0.CO;2.

Menchaca,M.Q.,andD.R.Durran,2017:Mountainwaves,

downslope winds, and low-level blocking forced by a midlati-

tude cyclone encountering an isolated ridge. J. Atmos. Sci.,

74,617–639, https://doi.org/10.1175/JAS-D-16-0092.1.

Mullen, S. L., 1983: Explosive cyclogenesis associated with cy-

clones in polar air streams. Mon. Wea. Rev., 111, 1537–1543,

TABLE 1. Summary of subjective evaluation of the physics experiments. Qualitative summary evaluation labels are physically

“consistent,”“inconsistent,” or “mixed.” The remaining columns summarize notable strengths and weaknesses.

Experiment Evaluation Strength Weakness

Tropical heating Consistent Teleconnections; Matsuno–Gill response Weak remote response

Baroclinic development Consistent Cyclogenesis; downstream development None

Geostrophic adjustment Mixed Divergent wind; Coriolis turning Shocks from different models

Tropical cyclone Consistent Development sensitive to moisture Poleward turning too strong?

ARTIFICIAL INTELLIGENCE FOR THE EARTH SYSTEMS V

OLUME 310

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC

https://doi.org/10.1175/1520-0493(1983)111,1537:ECAWCI.2.0.

CO;2.

Nolan, D. S., E. D. Rappin, and K. A. Emanuel, 2007: Tropical

cyclogenesis sensitivity to environmental parameters in radia-

tive–co nvective equilibrium. Quart. J. Roy. Meteor. Soc., 133,

2085–2107, https://doi.org/10.1002/qj.170.

Polvani, L. M., R. K. Scott, and S. J. Thomas, 2004: Numerically

converged solutions of the global primitive equations for test-

ing the dynamical core of atmospheric GCMs. Mon. Wea.

Rev., 132,2539–2552, https://doi.org/10.1175/MWR2788.1.

Rasmussen, E. A., 2003: Polar lows. A Half Century of Progress

in Meteorology: A Tribute to Richard Reed, Springer, 61–78.

Rotunno, R., and K. A. Emanuel, 1987: An air–sea interaction

theory for tropical cyclones. Part II: Evolutionary study using

a nonhydrostatic axisymmetric numerical model. J. Atmos. Sci.,

44, 542–5 61, https://doi.org/10.1 175/1520-0469(1987 )044,0542:

AAITFT.2.0. CO;2.

}}, W. C. Skamarock, and C. Snyder, 1994: An analysis of

frontogenesis in numerical simulations of baroclinic waves.

J. Atmos. Sci., 51, 3373–3398, https://doi.org/10.1175/1520-0469

(1994)051,3373:AAOFIN.2.0 .CO;2.

Sardeshmukh, P. D., and B. J. Hoskins, 1988: The generation of

global rotational ﬂow by steady idealized tropical divergence.

J. Atmos. Sci., 45, 1228–1251, https://doi.org/10.1175/1520-

0469(1988)045,1228:TGOGRF.2.0.CO;2.

Simmons, A. J., and B. J. Hoskins, 1975: A comparison of spectral

and ﬁnite-difference simulations of a growing baroclinic

wave. Quart. J. Roy. Meteor. Soc., 101,551–

565, https://doi.

org/10.1002/qj.49710142912.

}},and}}, 1979: The downstream and upstream development

of unstable baroclinic waves. J. Atmos. Sci., 36, 1239–1254,

https://doi.org/10.1175/1520-0469(1979)036,1239:TDAUDO.2.

0.CO;2.

Ting, M., and P. D. Sardeshmukh, 1993: Factors determining the

extratropical response to equatorial diabatic heating anomalies.

J. Atmos. Sci., 50, 907–918, https:// doi.org/10.1175/1520-04 69

(1993)050,0907:FDTERT .2.0.CO;2.

Yoshida, A., and Y. Asuma, 2004: Structures and environment of

explosively developing extratropical cyclones in the north-

western Paciﬁc region. Mon. Wea. Rev., 132, 1121–1142,

https://doi.org/10.11 75/1520-0493(2 004)132,1121:SAEOE D.2.

0.CO;2.

HAKIM AND MASANAM

11JULY 2024

Unauthenticated | Downloaded 10/07/24 04:18 PM UTC