LROSE Standard data formats
LROSE-SUPPORTED DATA FORMATS IN RADIAL COORDINATES
The following table shows the various formats, in native polar coordinates, that are supported by the Radx library in LROSE-core:
| Radx file format support | |||
|---|---|---|---|
| Format | Read access | Write access | |
| CfRadial-1 - NCAR/EOL/UNIDATA | Yes | Yes | |
| CfRadial-2 - NCAR/EOL/UNIDATA | Yes | Yes | |
| ODIM HDF5 | Yes | Yes | |
| DORADE - NCAR/EOL | Yes | Yes | |
| UF - Universal Format | Yes | Yes | |
| Foray-1 netCDF - NCAR/EOL | Yes | Yes | |
| DOE ARM netcdf - precedes CfRadial | Yes | No | |
| MDV radial - NCAR/RAL | Yes | Yes | |
| NEXRAD msg31 level 2 archive | Yes | Yes | |
| NEXRAD msg1 level 2 archive | Yes | No | |
| SIGMET - raw format (Vaisala) | Yes | No | |
| RAPIC - BOM Australia | Yes | No | |
| HRD - HRD NOAA | Yes | No | |
| LEOSPHERE LIDAR, ASCII format | Yes | No | |
| EEC - now supports CfRadial | Yes | N/A | |
| GAMIC HDF5 | Yes | No | |
| WMO BUFR | Yes | No | |
| CFARR (NCAS, UK) | Yes | No | |
| D3R (CSU) | Yes | No | |
| Gematronik XML/Binary | Yes | No | |
| NOXP - OU | Yes | No | |
| TDWR | Yes | No | |
| TWOLF (OU Lidar) | Yes | No | |
Standardized formats
Standardized data formats are fundamental to the development ot tools that will address community needs.
Keeping the formats to a limited number of standards simplifies the work of the developers and frees them up to handle more interesting needs such as algorithms and displays.
It is important that, to the extent possible, the data formats be 'self-describing'. The primary candidates are:
Sometimes a binary format is essential - for example, for radar time series data the size is so large that only a binary format makes sense. For this circumstances, the binary format will be properly documented.
CfRadial
The CfRadial format was developed to faithfully record all aspects of radar moments data in radial coordinates, i.e. the native coordinate system for the instrument.
CfRadial is designed to follow the Climate and Forecasting conventions for NetCDF data. This helps to facilitate the use of the data by numerical models, through the use of agreed-upon standard names for field variables. For details see: CF-compliant NetCDF.
CfRadial is hosted on GitHub at http://ncar.github.io/CfRadial/
See http://opensky.library.ucar.edu/collections/OSGC-000-000-021-355 for documentation for CfRadial version 1.3 in PDF form.
See cf-conventions.1.7.pdf for a copy of the latest CF conventions document.
ODIM HDF5 radial data format
The BALTRAD group, and EuMetNet, use the ODIM standard formats for representing both radial and Cartesian data.
The ODIM specification is available as a PDF.
Cf-compliant NetCDF for Cartesian data
For gridded data that is in Cartesian coordinates, the recommended format is CF-compliant NetCDF.
IWRF time series format
The Integrated Weather Radar Format (IWRF) was developed to provide an efficient binary, platform-independend data exchange format for time series data.
The IWRF documentation is available in PDF.
Legacy formats
A number of legacy formats are supported by LROSE, because so much historical data is stored in these formats.
DORADE Format
DORADE was developed in the 1990's, to handle the more complex nature of Doppler radars, especially airborne platforms. DORADE is a binary format composed of distinct header and data blocks, making it extensible.
Many NCAR field projects from 1990 to 2010 stored the raw radial radar data, as well as derived products, in DORADE format.
The NCAR/EOL soloii display is a primary tool for displaying, editing and manipulating DORADE files.
From 2010, DORADE files will gradually be replaced with the new CfRadial format.
The DORADE documentation was updated in 2010.
UNIVERSAL Format (UF)
Commonly known as UF, the Universal Format is formally known as the Common Doppler Radar Exchange Format. UF was documented by S.L Barnes in 1980 in the Bulletin of the American Meteorological Society (BAMS), Vol 61, 1401 - 1404.
UF was primarily designed for access via FORTRAN. A FORTRAN binary block holds the data for each ray (or beam). UF is composed exclusively from ASCII characters and 16-bit integer words. Floating point values are computed by applying scale factors and offsets to the integer values.
UF has some significant limitations. Field names are limited to 2 characters. For example, reflectivity is often referred to as ZH, or DZ, and radial velocity as VR, because the 2-character limitation prevents use of longer names such as DBZ or VEL. Also, the headers are limited in length, so that the metadata associated with more sophisticated radars cannot be accurately represented.
Nevertheless, UF is still popular with many researchers, and much analysis code has been written for UF data.
The UF documentation is based on the 1980 BAMS article.
FORAY NetCDF Format
The FORAY project at EOL developed a radar data exchange format based on the Unidata netCDF scientific data format.
FORAY netCDF is based on NetCdf version 3, i.e. it does not support data compression.
The FORAY netCDF sweep file format was the primary format used for the TIMREX field experiment in Taiwan. TIMREX also stored the radial data in DORADE format.
An example ncdump from a FORAY netCDF file is the best way to understand the format.
The FORAY project has now ended. TIMREX will be the only NCAR field project which will use FORAY netCDF files. All subsequent projects will use CfRadial and DORADE.