TOUGH: Targeting Optimal Use of GPS Humidity Measurements in Meteorology

Extract of Project Workplan

Introduction

The main objectives for the present project proposal are to improve the use of GPS data for numerical weather prediction and climate monitoring. This will be done by innovation of new techniques and methodologies enabling proper correction of error sources identified in previous work, as well as by initiating use of the more detailed information available in the form of the individual delays between each receiver and the GPS satellites visible to it. In order to make the required progress to meet the objectives, research efforts and technical developments over a wide range of problem areas need to be carried out. This research and development require active participation from the geodetic and the meteorological communities. To get an initial overview of the required efforts, we here mention a few scientific and technical key problems that will be solved:

·        The pre-processing of the raw GPS measurements will be handled by a number of Processing Centres. In order to meet the future operational timeliness requirements from the numerical weather prediction community, algorithms for near-real-time pre-processing will be introduced. Furthermore, this pre-processing will be carefully co-ordinated and monitored in order to guarantee the meteorological community a homogeneous data set, with stable and known (documented) error characteristics.

·        Early trials to assimilate ground-based GPS data have indicated that these data may be affected by systematic observation errors (error biases) as well as spatially and temporally correlated observation errors. Significant efforts will be devoted in the present project to (a) increase our understanding of the origin of these observation errors; (b) eliminate these errors to the extent possible and (c) model the characteristics of the observation errors. Realistic statistical models of the observation errors are needed for an optimal assimilation of the data.

 

·        It is foreseen that the most significant impact of ground-based GPS measurement will be possible only through application of 4-dimensional assimilation techniques. First of all, GPS data have a high temporal resolution. More important may be that GPS data provides information mainly on the atmospheric moisture. In order to derive atmospheric pressure, temperature and wind fields that are consistent with the moisture field as seen by the GPS data, the forecast model must be utilised in the assimilation process. This is exactly what is done in 4-dimensional data assimilation. Two forms of 4-dimensional data assimilation, namely 4 dimensional variational data assimilation (4D-Var) and nudging, will be applied in the present project in order to maximise the impact of GPS data.

 

·        The ground-based GPS data provide information only about the vertical integrated atmospheric moisture content. In order improve the vertical distribution of the observed water vapour during the assimilation process, the GPS data assimilation will be supplemented in the present project with assimilation of moisture measurements from surface stations.

 

·        Ground-based GPS information has so far been utilised in the form of Zenith Total Delay (ZTD) data. Each ZTD data value is obtained through a mapping from a number of slant delay measurements. It is expected that the meteorological data assimilation would benefit from a direct assimilation of slant delays. The explicit mapping to zenith delays, which may introduce unnecessary errors, is avoided and information on horizontal gradients may also become possible to extract.

 

·        To assess the impact of the ground-based GPS data, and the best way in which to process and use such data, three types of studies will be carried out. First, cases of significant weather events will be selected. Data assimilation experiments will be conducted to assess the impact of the data on these cases. Special attention will be given to short range precipitation forecasts, as we expect that most additional information from the data should be in humidity.  Secondly, data assimilation experiments using different data assimilation systems will be conducted for long periods (e.g. a month) and for all seasons, in order to draw general conclusions on the operational use of the data. Forecasters and other end-users will evaluate the quality of the resulting weather forecasts, produced with and without the GPS data.

 

The scientific and technical work of the proposal has been divided into 7 basic work-packages WP 3000 – WP 9000, the content of which is briefly described below. These basic work-packages have been further sub-divided into sub-work-packages, described with details later in this section. Three additional work-packages involve Project Management (WP 1000), User requirements (WP 2000) and Exploitation and dissemination (WP 10000).

 

Modelling of observation error characteristics for data assimilation (WP 3000)

Data assimilation for NWP (Numerical Weather Prediction) optimally estimates the atmospheric state using observation information. The observed values always contain observation errors. In case these errors are un-correlated between different observations, more plentiful observations lead to a more accurate state estimate. Observation error correlation generally implies reduced information content of the observations. Use of more observations does not in this case improve, but degrade the quality of the state estimate, unless the error correlation is properly accounted for.

In static data assimilation schemes, such as 3D-Var (3-dimensional variational assimilation), observations are used from one instant close to the analysis time. Serially correlated observations errors from one station, i.e. temporal observation error correlations, do not play any role in this case. Horizontal error correlations, i.e. observation error correlations between stations at one instant, need to be accounted for by error modelling in order to obtain an optimal state estimate. In temporally extended data assimilation schemes, such as 4D-VAR, observations are used at appropriate time over a data-window. In this case also temporal correlations of observation errors need to be accounted for.

Mean observation errors, i.e. error biases, need a specific treatment of bias reduction. Generally it is very difficult, however, to distinguish the slowly varying horizontal observation error correlation from the mean observation errors, or from the systematic errors of the NWP model. Comparison of ground-based GPS measurements with forecast model data and with radiosonde data have revealed that the GPS measurements may be affected by error biases. Early data assimilation experiments have indicated that it is necessary to apply bias reduction algorithms in order to avoid detrimental effects of these error biases on, for example, precipitation forecasts. Ideally, these error bias problems should be avoided by applying remedy actions as close as possible to the source of the information, e.g. at the GPS station or by improving the pre-processing algorithms. It is foreseen, however, that the need for bias reduction schemes will remain. Statistical comparison between GPS observations and model data will be applied to design the bias reduction algorithms.

The design of the ground-based GPS measurements and pre-processing systems implies theoretically the measurements to be affected by spatially correlated errors. Simulation studies by Jarlemark et al. (2001) and studies of empirical spatial correlations by Stoew et al. (2001) support this theory. These studies suggest that the length scale of the GPS observation error correlation may be significantly larger than the length scale of the forecast error. This separation of length scales can possibly be utilised for a determination of the spatial (horizontal) correlation of GPS observations errors from innovation vectors, i.e. the differences between GPS observations and the model data. Other observations of the atmospheric moisture could in principle serve as references for the estimation of the GPS observation errors, but the limited spatial resolution and relatively poor quality of radiosonde moisture measurements do not make this approach meaningful. It will furthermore be investigated whether the observation error and forecast error contributions to the spatial correlation of the GPS data innovation vectors can be separated through a modelling of the forecast error correlation by simulation techniques, based on ensemble assimilation experiments. 

With the introduction of 4-dimensional variational data assimilation (4D-Var), several observations from the assimilation window, for example a 6 hour period, and from the same station may be utilised. Experiences from the 4D-Var assimilation of surface observations have shown that the sensitivity of the assimilation to systematic observation errors may become critical and that models for the temporal correlation of observation error need to be specified (Järvinen et. al., 1999). Models for the temporal correlation will alternatively be developed from innovation vectors, i.e. differences between GPS observations and model data, or from differences between GPS observations and high quality radiosonde observations.

The efficiency of the developed bias reduction schemes and the developed models for spatial and temporal observation error correlation will be tested through data assimilation and forecast experiments.

Development and testing of 4-dimensional data assimilation techniques (WP 4000)

It is foreseen that ground based GPS observations due to their high time resolution, and due to the use of the forecast model in the assimilation will have the highest impact when assimilated using 4D-Var assimilation systems. 3D-Var assimilation systems are currently in operational use by DMI, MetO, and SMHI, while 4D-Var assimilation systems are under development.

The 4D-Var assimilation schemes will be developed to handle GPS observations in an optimal manner. Since GPS observations mainly are related to the moisture variables of the forecast model, it is important to include condensation and precipitation processes in the 4D-Var schemes. This requires mathematical formulations, called parameterization, of these processes and their computer codes in nonlinear, tangent linear and adjoint forms. In most of the state of the art NWP models, these schemes are highly nonlinear and non-differentiable. Therefore, they often need to be simplified or regularized in mathematical formulations before the development of the tangent linear and adjoint schemes needed by 4D-Var.. The application of 4D-Var to GPS data will be tested and validated through case studies and through data impact studies, covering at least ten days.

DMI and MetO expect to apply complete 4-dimensional variational data assimilation (4D-Var) schemes for their operational NWP forecast models, while LAQ will apply a simplified 4-dimensional assimilation based on nudging to a mesoscale forecast model (MM5) and compare this with 3D-Var assimilation.

Optimisation of GPS and surface humidity assimilation (WP 5000)

The ground-based GPS measurements of Zenith Total Delay (ZTD) in principle only provide information on the vertically integrated water vapour in the atmosphere above the GPS stations. In case no other water vapour information is available, 3-dimensional variational data assimilation (3D-Var), for example, will use statistical knowledge only to distribute the observed information in the vertical. It was shown by Kuo et al. (1996) in an observing system simulation study that more information on the vertical distribution of water vapour could be retrieved by adding humidity observations from surface stations. This possibility to improve the utilisation of ground-based GPS measurements will be investigated by running a 3D-Var data assimilation and forecast experiment over one month with and without 2 meter relative humidity observations.

This shall be done by INM and SMHI. The variational data assimilation system to be applied by these two project partners already includes preliminary observation operators based non-linear, tangent-linear and adjoint versions of the post-processing for 2 meter relative humidity. These observation operators will be upgraded to be consistent with the latest version of the forecast model and complemented with models for observation error statistics.

Development of methods for assimilation of slant GPS delays (WP6000)

 Instead of deriving zenith quantities, GPS signal delay and integrated water vapour can also be measured along slant paths from ground-based receivers to GPS satellites. By using not only the zenith delay of a receiver but also the slant delays the number of observations will increase by roughly a factor ten. By applying variational algorithms a three-dimensional water vapour field can be retrieved from slant observations, at least from a dense network of receivers. Furthermore, the horizontal resolution of the retrieved water vapour field will also profit from this larger amount of observations.

The derivation of zenith and slant GPS delays from GPS observations involves several assumptions about the atmospheric structure. In particular, assumptions about atmospheric homogeneity and receiver multipath when observing satellites are at low elevation angles (close to the horizon) influence the results. The multipath must be carefully modelled as a function of receiver environment while the atmospheric model used for the mapping must be carefully chosen in cases of atmospheric inhomogeneities. Even when estimating only slant delays, mapping functions are still needed in order to separate receiver clock errors from atmospheric delays. Traditionally, mapping functions are empirical functions derived from multi-year averages of radiosonde data. A new approach is to derive the mapping function directly from NWP model output. This could result in a significant improvement of IWV measurements for low elevations. Pre-processing of raw slant delays before assimilation will be investigated, using additional input from NWP analysis. This will help to discriminate site dependent effects (multipath, antenna phase center variations) and receiver clock errors from atmospheric delays. It can also be used to derive intermediate quantities such as ZTD, horizontal gradients, scale height and or timing information, which could be used as an alternative to assimilating slant delays. Currently used software will be modified, if necessary, and additional modules to estimate slant delays and model multipath will be developed. Furthermore, mapping procedures based on forecast model input will be developed and tested by one weather service, KNMI,  and  a geodetic institute, TUD.

 

In order to obtain realistic results the error biases and correlations of the GPS slant measurements must be modelled. Observations for a network of ground-based receivers will be simulated from a 3-D water vapour field and used for assimilation trials. The goal of these simulations is to test our software and to estimate the capability of a network of GPS receivers to reconstruct refractivity field inhomogeneities at different scales. In addition we need to determine an optimal discretisation and interpolation scheme of the refractivity field to be used for the processing of observational data. The retrieved fields will be validated against water vapour radiometer measurements during the CLIWANET campaign.

 

The natural first step towards using slant-delay measurements in NWP assimilation is to properly evaluate them against the model counterparts. For this task an appropriate observation operator[1] is needed. The zenith delay observation operator is simple to develop, as the observation geometry is relatively straightforward and similar to the NWP model geometry. The slant-delay observation operator, in contrast, requires a model profile along a slanted path with unknown intersections with the model levels. Once the problem of interpolating the model variables on a slanted path is solved, the associated delay calculation problem can be fairly easily solved.

 

A demonstration version of a GPS slant delay observation operator will be developed by FMI and KNMI in co-operation, and this observation operator will be adapted to the HIRLAM three-dimensional variational data assimilation system. The operational NWP model of KNMI will be used for impact studies with a resolution of at least 10 km x 10 km. The performance of the assimilation of these slant delays will be investigated by conducting observation system simulation experiments (OSSE). Impact studies will be performed with the analysed water vapour fields, obtained from the GPS data of a dense GPS network (Observation System Experiment, OSE). DMI will perform assimilation tests using the software developed by KNMI and FMI.

 

Impact studies and extreme case studies (WP 7000)

DMI will monitor the operational forecasts and information about the actual weather in order to identify periods and areas in which the forecasts are particularly poor, or in which “special” weather occurred in areas with good coverage of GPS stations partaking in the project. For the selected cases, each participating institute will carry out extensive, full-scale data assimilation experiment. Month long assimilation experiments will  be carried out for each of the four seaasons. Standard statistical methods will be used for objective verification. Analyses and forecasts with and without the ground-based GPS data will be verified against observations and analyses. Special attention will be given to short range forecasts of moisture, clouds and precipitation. Forecasters will participate with subjective verification of the forecasts.

 

One of the objectives of the EUCOS program of EUMETNET is to increase the cost-efficiency of the European observing system while staying at the same overall cost. It is proposed to replace some radiosonde stations by AMDAR aircraft soundings. Comparing with radiosondes, one of the drawbacks of the current AMDAR is the lack of humidity information. The ground-based GPS ZTD data could provide useful complementary humidity information that allows this cost-redistribution with less negative effect on numerical weather predictions. A well documented EUCOS observation period will be selected, see e.g. Amstrup (2000), and the impact of replacing radiosonde data with combined AMDAR/GPS data will be studied.

 

GPS ZTD data provision and monitoring (WP 8000)

Currently GPS data is available from regional geodetic networks under pre-existing agreements with regional processing centres. In past research methodology has been developed to process the data to retrieve atmospheric properties. This methodology will be used in demonstration mode in this project, to allow the users to gain experience using the EO products in their NWP application. The GPS data will be retrieved from the sites and quality checked. The refractive delays in the GPS signals will be calculated and then geometrically mapped to the zenith delay (ZTD. For a period of at least one year this will be done in near real time (NRT), as necessary for operational NWP. These products will be used by NWP groups, which are developing ZTD assimilation algorithms. The data will also be further processed to remove the hydrostatic component of the delay based on surface pressure measured at the site. This non-hydrostatic, or "wet" delay will then be transformed to integrated water vapour. These products will be used by NWP users, which are developing nudging assimilation systems.

 

Each regional data processing centre will be responsible for retrieving the GPS data, processing the data, and transferring the data to the project ftp site in NRT. In processing the data, the centres will include stations from a common reference network in their solutions to provide a means for cross-checking the quality of the data and to ensure that the reference frames used are consistent. Similar products that are available from organisations outside the consortium that cover other regions will also be made available to the meteorological users.

 

The first 3 months are to be used to improve the raw data flow as necessary, to verify the robustness of the processing system and to make any adjustments to the processing concerning the station distribution, following the recommendations of the work-package leader and a processing committee. During these 3 months and the following 21 months, the products will be provided continuously to the users as a demonstration prototype system. 6 months into the project quality control standards will be implemented.

 

Radiosonde observations can be used as an important independent data set for validating GPS ZTD data both on a daily basis and on long term statistics. The quality of the radiosondes is high, but the temporal and spatial resolutions sometimes lead to problems. NWP analyses and forecasts, on the other hand, can be used as another source of data with a uniform resolution in 4 dimensions. The database will contain radiosonde data, NWP data and precipitation data that is collected for validating the short term precipitation forecasts.

 

GPS ZTD system research (WP 9000)

In previous work developing the methodology and its validation, it was established that the GPS ZTD and IWV products are of a quality comparable or superior to existing data sources available to the NWP user community. In particular, the products were shown to be in overall good agreement with radiosondes (less than 10mm of delay). However, the products occasionally had epochs of unexplained poor data quality. In addition, long spatial and temporal signals in the residuals from radiosonde and NWP comparisons have been detected. This work-package will investigate the source of these errors and contribute new techniques to the methodology implemented in the demonstration processing.

 

Most of the GPS software packages provide the standard deviation of the estimated Zenith Total Delay (ZTD) parameter as an estimate of the quality of the solution. The standard deviation is a formal measure of quality computed from the inverse of the normal matrix. As a measure of quality it is seriously flawed because it does not take into account the actual quality of the observations, it is unaware of important errors such as multipath, and it assumes the orbits (and sometimes satellites clocks) are perfect. The standard deviation is always too optimistic and cannot be used to model the errors during the assimilation into NWP. A new quality indicator for the ZTD will be developed and tested. The new indicator will be computed from the estimated least squares residuals by using variance component estimation techniques, taking into account the degree of freedom over the domain of the ZTD parameter.

 

The strength of ground-based GPS is certainly not its absolute accuracy. Because of its sensitivity to signal multipath effects, varying the elevation angle cut-off limits - or using different schemes for down-weighting low elevation angle observations - will typically have a significant impact on the estimated ZTD value. A constant bias over decades is in principle not a problem but if there are variations at the time scales of years it will influence both NWP models and long term climate monitoring. We will use long time series (> 5 years) of independent radiosonde and microwave radiometer data to study these effects and believe that a correct assessment can be made at the 5-10 mm level in ZTD. Very-Long-Baseline Interferometry (VLBI) is another method, which will be used. Several European VLBI sites, e.g., Wettzeell, Matera, and Onsala, are co-located with important GPS sites in the IGS network, where data are publicly available. The VLBI estimates of ZTD are obtained from the same type of estimation technique as in GPS but due to the large directional antennas used the multipath effect is in practise eliminated. VLBI observations are, however, not continuous, but 24-hour observing sessions bi-weekly or monthly for more than five years provide a sufficient data base.

 

GPS tropospheric zenith delay is correlated with the site co-ordinates, especially with the vertical one. For meteorological applications there is no need to estimate them when processing GPS data, but, in order to derive the ‘best’ possible ZTD estimates, there is a need to know site co-ordinates with a certain level of accuracy. Generally they are obtained averaging daily station estimates over a longer period of time. So even for pure meteorological applications there is the need of station co-ordinates monitoring. Of course, they are related to the terrestrial reference frame (TRF) in which they have been computed. The changing of TRF could introduce biases into the GPS ZTD and IWV products. Furthermore constrains to the reference frame are also induced by fixing the GPS orbits (IGS orbits are given in a TRF) during the data reduction, what is commonly done when regional network are considered. Therefore it is an interesting question to understand how to deal with the biases related to the reference frame, even for climate investigations. Furthermore, the geodetic reference frame is always being improved. There are occasionally slight changes which can lead to offsets in the long term trend of GPS ZTD. The influence different reference frames have on GPS ZTD estimates will be evaluated and a methodology for dealing with updates to the reference frame will be established. It will be verified that differences between processing centres estimates for the reference IGS stations are not due to orbit errors,  co-ordinate errors or reference frame errors. Guidelines for verifying the quality of GPS ZTD and IWV data will be established by examining repeatability of co-ordinates and these guidelines will be implemented in the GPS ZTD and IWV processing.

 

Results from the EC MAGIC project showed that the difference between GPS ZTD and radiosondes increased in magnitude in high humidity regimes, producing a seasonal signal in these differences.  These signals limit the ability to separate a climatic signal from the noise in the of GPS ZTD products. Biases correlated with seasonal signals due to systematic differences in actual and modelled vertical structure will be investigated as well as noise sources in the radiosonde and GPS ZTD data that could have a seasonal variation.

 

The International GPS Service (IGS) has developed a method for combining ZTD solutions from different processing centres by removing a bias between processing centres and averaging the results. The same method is applied for the 12 analysis centres of the EUREF Permanent GPS Network (EPN). Typical for IGS and EUREF is, that almost all stations are processed by at least three processing centres. In our distributed network, only a subset of stations will be common among processing centres, but these can be used to verify that there are no offsets. The batch type of processing used by IGS and EUREF will be converted into a Kalman filter approach that can be used in near real-time applications. The differential biases between the analysis centres will be modelled for the stations in common. Special techniques for the detection, identification and adaptation of outliers and biases will be used. Algorithms will be developed and tested and possible refinements will be investigated. For example, the NRT combination could be further combined with bias reduction algorithms (using output from NWP analysis) to model absolute biases. TUD  will also develop automated methodology for a regional combination of solutions following the EUREF model, in order to provide the best integrated product from the regional products. They will aid in the implementation of this methodology at the processing centres.


Project planning and time table

WP#

Wpname

Start

End

Year 1

Year 2

Year 3

1000

Management

00

36

 

 

 

 

 

 

1100

Overall Management

00

36

 

 

 

 

 

 

1200

Scientific Coordination

00

36

 

 

 

 

 

 

1300

Data supply co-ordination

00

36

 

 

 

 

 

 

1400

Meeting preparation and participation

00

36

 

 

 

 

 

 

2000

User Requirements

00

03

 

 

 

 

 

 

3000

Error Modelling for variational assimilation

00

24

 

 

 

 

 

 

3100

Bias reduction schemes

00

24

 

 

 

 

 

 

3200

Modelling of spatial error correlation

00

24

 

 

 

 

 

 

3300

Modelling of temporal error correlation

00

24

 

 

 

 

 

 

4000

Variational assimilation development and tests

00

36

 

 

 

 

 

 

4100

Develop and optimise 4Dvar assimilation

00

33

 

 

 

 

 

 

4200

Mesoscale data assimilation development and tests

00

36

 

 

 

 

 

 

5000

Optimisation of GPS/surface humidity assimilation

00

36

 

 

 

 

 

 

5100

Refining methods of surface humidity assimilation

00

24

 

 

 

 

 

 

5200

Testing combined GPS / surface humidity assimilation

24

36

 

 

 

 

 

 

6000

Development of methods for use of slants delays

00

36

 

 

 

 

 

 

6100

Slant delay retrievals

00

30

 

 

 

 

 

 

6200

Slant delay validation and observation error studies

06

24

 

 

 

 

 

 

6300

Observation operator development

00

18

 

 

 

 

 

 

6400

Assimilation tests

18

36

 

 

 

 

 

 

7000

Assimilation impact statistics / extreme case studies

00

36

 

 

 

 

 

 

7100

Co-ordination of case studies and compiling results

00

36

 

 

 

 

 

 

7200

Case studies and extensive impact studies

06

30

 

 

 

 

 

 

7300

EUCOS scenario impact studies

00

24

 

 

 

 

 

 

8000

GPS ZTD data provision and monitoring

00

36

 

 

 

 

 

 

8100

Product quality monitoring and reporting

00

36

 

 

 

 

 

 

8200

NWP User GPS ZTD/IWV data server maintenance

00

36

 

 

 

 

 

 

8300

Regional GPS ZTD data production

00

36

 

 

 

 

 

 

8400

Furnishing continuous radiosonde and NWP output

00

36

 

 

 

 

 

 

8500

Validation database development and maintenance

00

36

 

 

 

 

 

 

8600

User validation and feedback

03

33

 

 

 

 

 

 

9000

GPS ZTD System Research

00

30

 

 

 

 

 

 

9100

Robust quality indicators

00

09

 

 

 

 

 

 

9200

Long term bias elimination

00

30

 

 

 

 

 

 

9300

Co-ordinate system biases

00

24

 

 

 

 

 

 

9400

Biases correlated with seasonal signals

00

24

 

 

 

 

 

 

9500

Optimal combination of regional solutions

00

09

 

 

 

 

 

 

10000

Exploitation and dissemination

00

36

 

 

 

 

 

 

Project planning and timetable.