From taylor13 at llnl.gov  Thu Nov  2 11:46:59 2006
From: taylor13 at llnl.gov (Karl Taylor)
Date: Thu Nov  2 11:59:59 2006
Subject: [CCSM-AMWG] CCSM3+ development simulations
In-Reply-To: <453E501B.7030005@ucar.edu>
References: <453E501B.7030005@ucar.edu>
Message-ID: <454A3D23.7030700@llnl.gov>

Dear AMWG:

There may be some confusion concerning the interpretation of one of the 
sets of "Taylor Diagrams" mentioned in Richard Neile's email copied 
below.  The set of diagrams that are supposed to display information 
concerning how well the models simulate the annual cycle phase and 
magnitude (labeled "ANN: TIME") are in one respect wrong.  The centered 
RMS error that can normally be inferred from the distance between each 
plotted point and the "REF" point (i.e., the observed point) is not in 
fact a measure of the RMS error averaged over all spatial points, as 
incorrectly implied in these plots.  This is because the procedure used 
in computing the variance and correlation statistics breaks the 
relationship that is the basis of the diagram, namely:

E**2 = sig1**2 + sig2**2 - 2*sig1*sig2*R

where E is the centered RMS error, sig1, and sig2 are the standard 
deviation of the observed and simulated fields, respectively, and R is 
the correlation.

The web page description of the Taylor diagrams pointed to in the email 
below indicates that in computing statistics for the "ANN: TIME" case, 
the standard deviation and correlation are first computed at each grid 
cell.  Then the spatial means of these statistics are computed. If these 
mean statistics are used in the above equation, E is *not* the RMS 
error, averaged over all grid cells.  If the true RMS errors at each 
grid cell are spatially averaged, the result will not satisfy the above 
equation.  In short the above equation is not linear, so that averaging 
the statistics spatially destroys the relationship.

It is therefore inappropriate to use a Taylor diagram to display the 
"ANN: TIME" statistics.  I would recommend that if you want to focus on 
the annual cycle, simply subtract the time mean field, and then compute 
the space-time statistics for the resulting field.  This is the way we 
have always done this at PCMDI. [Note that the time mean field and the 
annual cycle field (with time-mean removed) are mathematically 
orthogonal so the sum of component variances and the sum of the 
mean-square errors add to give the total statistics found in the "ANM: 
SPACE-TIME" plots.]

By the way, in June of 2003 Peter Gleckler and I produced a whole series 
of Taylor diagrams evaluating competing versions for CAM2.X.  We made 
these plots accessible via the web to the AMWG, and allowed the user to 
select any particular component of interest, including the full 
space-time statistics, statistics based on the annual mean fields, and 
statistics based on each of the seasonal means.  In addition to the full 
spatial fields, the user could choose to view statistics based only on 
zonal mean fields or only on deviations from the zonal means. Each 
figure emphasized a different aspect of the simulations. I think the 
number of figures made available at that time might have been 
overwhelming (although the web-based interface to the figures made it 
easy to select), so I'm not advocating expanding the breadth of analysis 
-- just pointing out that it could be done quite easily.

I'm glad that routinely preparing figures like these has now apparently 
become practical at NCAR.  Richard Neale deserves thanks for that.

Best regards,
Karl


Richard Neale wrote:
> AMWG and working group co-chairs,
> 
> In preparation for an interim CCSM development simulation using the new 
> pop2 ocean we have produced a number
> of CAM (atmosphere-only) experiments examining possible incremental 
> improvements to the
> Zhang-McFarlane deep convection scheme. Therefore, we are requesting the 
> community's feedback as to the
> configuration which shows the greatest promise for inclusion in this 
> coupled integration.
> 
> The convection changes represent the initial internal efforts of the 
> NCAR Climate Modeling Section.
> We, of course, recognize the ongoing efforts of external collaborators 
> and so any changes
> included in the up-coming CCSM experiment by no means represent any 
> final version of CCSM4.
> We intend to have a similar evaluation whenever other groups submit 
> model formulations to be
> considered for CCSM.
> 
> We hope the community will examine and provide feedback on 3 
> experiments, with the following changes to the convection scheme.
> 1. The addition of convective momentum transports. (cam3_3_fv_cmt)
> 2. Changing the deep convective plume calculation from a non-entraining 
> to an entraining calculation. (cam3_3_fv_dilute)
> 3. Combining changes (1) and (2). (cam3_3_fv_cmt_dilute, 
> cam3_3_fv_cmt2_dilute)
> 
> A range of diagnostics have been produced for each experiment including 
> climate mean and variability diagnostics as well as a first
> effort at providing metrics in the form of an extended Taylor diagram.  
> They can be found on the following website:
> 
> http://swiki.ucar.edu/cam-dev/25
> 
> This webpage is an editable wiki based webpage and is currently 
> restricted to UCAR and guest users.
> If you do not have a UCAR general username and password then please 
> contact me (rneale@ucar.edu)
> and I will send the guest password and username.
> 
> The initial conclusion of the Climate Modelling Section is that ** 
> option 3 (cmt2_dilute) ** represents the most promising configuration.
> However, community members may find aspects which are unacceptable in 
> the simulation and we
> encourage feedback regarding any comments, suggestions and reservations 
> you may have.
> 
> Many thanks
> Richard Neale
> (for AMWG and the Climate Modeling Section)
>