Documentation for 2A-21, version 6 Fri Mar 26 14:53:43 EST 2004 ------------------------------------------------------------------- Algorithm 2A-21: Surface Cross Section Name of Contact: Bob Meneghini NASA/GSFC Ph: 301-614-5652 email: Robert.Meneghini-1@nasa.gov ------------------------------------------------------------------- 1. Objectives and functions of the algorithm The primary objective is to compute the path integrated attenuation (PIA) using the surface reference technique (SRT). The surface reference technique rests on the assumption that the difference between the measurements of the normalized surface cross section within and outside the rain provides a measure of the PIA. An estimate of the non-rain normalized radar surface cross section (sigma-zero or NRCS) is used as a reference value for computing the PIA. The algorithm selects the "best" of four types of reference estimates that may be computed. The estimates used are * Along-track Spatial Average. An average of the Ns most recent non-rain sigma-zero measurements in same angle bin and with the surface type (currently Ns=8). In the text below, this estimate is referred to as the 'spatial average'. * Temporal Average. A latitude-longitude-angle bin data base (computed for 1 x 1-degree latitude-longitude cells) computed over the previous month. * Global Average. A global mean sigma-zero classified by surface type and angle bin, computed for a "typical" month. * Cross-Track/Spatial-Average Hybrid. An estimate computed from a cross-track fit of the current spatial average data. This is computed only if the entire current scan is over ocean. In addition to the sample mean of the NRCS as a function of angle bin (and in some cases location or surface type), the reference data sets also include the standard deviation of each of the sample means. This quantity is used to estimate the stability of the reference rain-free mean NRCS. In the spatial surface reference data set, the mean and standard deviation of the NRCS are calculated over a running window of Ns fields of view before rain is encountered (currently, Ns=8). These operations are performed separately for each of the 49+2 incidence angles of TRMM, corresponding to the cross-track scan from -17 degrees to +17 degrees with respect to nadir. The 2 additional angle bins (making the total 51 rather than 49) are used to take care of non-zero pitch/roll angles that can shift the incidence angle outside the normal range. For the temporal surface reference data set, the running mean and standard deviation are computed over a 1 x 1-degree (latitude, longitude) grid. Within each 1 x 1-degree cell, the data are further categorized into incidence angle categories (26). The number of observations in each category, Nt, are also recorded. Note that in the temporal reference data set no distinction is made between the port and starboard incidence angles so that instead of 49 incidence angles, there are only 25+1, where the additional bin is used to store data from angles outside the normal range. As of version 6, a new method of estimating surface reference values is used for ocean scans. This "hybrid" method takes the spatial surface reference data set for the entire scan and computes a cross-track fit. The assumption is that the cross-track angular dependence of the surface cross section can be approximated by a quadratic. As implemented, this method applies only to scans that are entirely over ocean. Whenever this condition is met, and hybrid estimate can be computed, then it will be chosen in preference to the spatial and temporal estimate. See section 8 for more detail about the hybrid method. When rain is encountered, the mean and standard deviations of the reference sigma-zero values are retrieved from the spatial and temporal surface reference data sets. To determine which reference measurement is to be used, the algorithm checks whether Nt>=Ntmin and Ns>=Nsmin, where Ntmin and Nsmin are the minimum number of samples that are needed to be considered a valid reference estimate for the temporal and spatial reference data sets, respectively. (Currently, Ntmin=50 and Nsmin=8.) If neither condition is satisfied, no estimate of the PIA is made and the flags are set accordingly (see below). If only one condition is met, then the surface reference data which corresponds to this is used. If both conditions are satisfied, the surface reference data is taken from that set which has a smaller sample standard deviation. If a valid surface reference data set exists (i.e., either Nt>=Ntmin or Ns>=Nsmin or both) then the 2-way path attenuation (PIA) is estimated from the equation: PIA = - sigma-zero(in rain) where sigma-zero(in rain) is the value of the normalized radar surface cross section over the rain volume of interest and is the mean value obtained from either the temporal or spatial reference data sets, the choice of which depends on the considerations discussed above. The hybrid estimate is computed after the entire scan has been processed as above. If the hybrid estimate is computable (entire scan is over ocean, and spatial estimates exist for a sufficient number of angle bins, Nh>=Nhmin [currently, Nhmin=5]), then the hybrid estimate is substituted for the previously selected estimate. To obtain information as to the reliability of this PIA estimate we consider the difference between the PIA, as derived in the above equation, and the standard deviation as calculated from the no-rain sigma-zero values and stored in the reference data set. Labeling this as std_dev(reference value), then the reliability factor of the PIA estimate is defined as: reliabFactor = PIA/std_dev(reference value) When this quantity is large, the reliability is considered high and conversely. This is the basic idea. Specific definitions of the reliability flag and reliability factors are given in the definitions of the output variables. Description of the HDF output variables for 2a-21 can be found in Volume 4 - levels 2 and 3 file specifications available at: Two comments should be made. i. The PIA is often defined as the one-way path attenuation rather than the 2-way attenuation used here. Note also that the PIA(2-way) is related to the specific attenuation or attenuation coefficient k (dB/km) by the equation: PIA(2-way) = 2 * integral[0, rs] k(s) ds where the path integral is taken along the direction of the main beam and where the integration limits range from the radar to the surface. Since the attenuation from the radar to the storm top is negligible, the integral can also be thought of as going from the storm top to the surface. ii. A case can be made for defining the reliability factor other than that given above. For example, we can define reliability factors by: Rel = [PIA - std_dev(reference value)]/PIA Rel' = PIA - std_dev(reference value) so that Rel=1 (or Rel' = PIA) would correspond to a perfect estimate whereas increasingly smaller values of Rel (including negative values) would correspond to lower reliabilities. Since the numerator and denominator of the expressions for Rel and Rel' can be computed from the output data, these quantities can be examined. 2. Command line arguments: 1B21.inputfile.HDF the 1B-21 HDF file 2A21.outputfile.HDF the 2A-21 HDF output file 2A21.int_tr.dat the read-only temporal intermediate file 2A21.int_tw.dat the write-only temporal intermediate file 2A21.int_s.dat the spatial intermediate file 2A21.diag the verification (diagnostic) file Note that the verification file is created in the program and should not exist prior to execution. 3. Definitions of Output Variables sigmaZero(49) [real*4]: Normalized backscattering radar cross section of the surface (dB) (NRCS) for the 49 angles bins in the radar scan (unitless). rainFlag(49) [integer*2]: Rain/no-rain flag (rain=1; no-rain=0) The rain possible category from 1B-21 is included in the no-rain category; only the rain-certain category is considered rain. incAngle(49) [real*4]: Incidence angle wrt nadir (in degrees); pitch/roll correction is included. pathAtten(49) [real*4]: Estimated 2-way path-attenuation in (dB) where pathAtten = 2*int[0,rs] k(s) ds where k(s) is the atten. coeff. in dB/km and integral runs from storm top to the surface. The path attenuation is often designated as the PIA, the path-integrated attenuation. reliabFlag(49) [integer*2]: Reliability Flag for the PIA estimate, pathAtten, defined below. reliabFactor(49) [real*4]: Reliability Factor for the PIA estimate, pathAtten, defined below. Definition of reliabFlag reliabFlag = 10000*iv + 1000*iw + 100*ix + 10*iy + iz where iv is a rain/no-rain indicator iw is an indicator of the reliability of the PIA estimate ix indicates the type of surface reference used iy provides information about surface detection iz gives the background type iv = 1 (no rain along path) = 2 (rain along path) iw = 1 (PIA estimate is reliable) - see definitions below = 2 ( is marginally reliable) = 3 ( is unreliable) = 4 ( provides a lower bound to the path-attenuation) = 9 (no-rain case) ix = 1 (spatial surface reference is used to estimate PIA) = 2 (temporal " " " PIA) = 3 (neither exists - i.e. insufficient # of data points) = 4 (unknown background type) = 5 (no-rain case & low snr - do not update temporal or spatial SRs) = 6 (global surface reference) = 7 (cross-track-spatial hybrid surface reference) = 9 (no-rain case) iy = 1 (surface tracker locked - central angle bin) = 2 ( unlocked - central angle bin) = 3 (peak surface return at normally-sampled gate - outside central swath) = 4 ( not at normally-sampled gate - outside central swath) iz = 0 (ocean) = 1 (land) = 2 (coast) = 3 (unknown or of a category other than those above or 'mixed' type) Note: for missing data set reliabFlag = -9999 Definition of reliabFactor reliabFactor = pathAtten/std_dev(reference value) where PIA is the 2-way path-integrated attenuation (dB), and std_dev(reference value) is the standard deviation as calculated from the no-rain sigma-zero values. Both quantities are in dB. It is important to note that in previous versions (versions 1-4) the reliabFactor was defined as the difference: pathAtten - std_dev(reference value) rather than the ratio pathAtten/std_dev(reference value) The parameter iw (in reliabFlag) is determined from reliabFactor and the SNR of the surface return (in dB). As currently defined: iw = 1 (reliable) if ((reliabFactor.ge.3).and.(SNR(dB).gt.3)) = 2 (marginally reliable) if: ((reliabFactor.ge.1) and (reliabFactor.lt.3) and (SNR(dB).gt.3)) = 3 (unreliable) if either (reliabFactor.lt.1) or ((SNR(dB).le.3).and.(reliabFactor.lt.3)) = 4 (lower bound) if ((reliabFactor.ge.3) and (SNR(dB).le.3)) [The iw = 4 case is defined because, while the attenuation estimate will be negatively biased because of a low signal-to-noise ratio, it may lead to the best rain estimate possible under the circumstances.] [SNR is the signal-to-noise ratio; expressed in dB this is given by the difference between the noise-corrected radar return power (dBm) and the radar noise power (dBm)] Note: for missing data, set reliabFactor = -9999.9 Note: pathAtten is output as long as the estimate is greater than 0, regardless of the value of iw. Note: pathAtten can assume both positive and negative values. Negative values have a negative reliabFactor, they are marked unreliable (iw=3). Before version 6, negative pathAtten were set to zero. 4. Description of the Processing Procedure: At each angle bin, calculate the normalized radar surface cross section, sigma-zero, and check whether rain is present. Also, find the (1 x 1-degree x angle-bin) element into which the measurement falls. If rain is present, retrieve the mean and standard deviations from the temporal and spatial reference data sets (formed from previously measured data under no-rain conditions). If both temporal and spatial reference data sets satisfy certain conditions, check which sample mean has the lower variance. Using the sample mean associated with the smaller variance, compute an estimate of the path-integrated attenuation and an associated reliability factor. If neither the spatial nor temporal reference data satisfy these conditions, use the global reference. In version 6, check the following conditions: the entire scan is over ocean and the spatial reference exists in at least Nhmin (Nhmin=5) angles bins (out of a total of 49) within the scan; if these two conditions hold, then the hybrid reference data is used (section 8). If rain is absent, update the temporal statistics (mean and mean-square) of sigma-zero at the relevant (1 x 1-degree x angle-bin) element. Also, update the spatial statistics of sigma-zero. 5. Interfaces to other algorithms: All input data for this algorithm is from 1B-21; the outputs are used by 2A-25, 3A-25 and 3A-26. 6. Comments and Issues: a. A gaussian beam approximation is used to represent the TRMM antenna pattern. b. The radar return power used in computing sigma-zero is that for which a 2.5 dB correction has been made. The factor accounts for the logarithmic averaging loss. Like the rain, the surface is treated as a Rayleigh target. c. Sigma-zero is being computed from that (single) gate where the return power is a (local) maximum. d. The algorithm assumes that rain is present only if minEchoFlag = 2 (rain certain); minEchoFlag = 1 (rain possible) and minEchoFlag = 0 (rain absent) are treated as no-rain cases. Note that the minEchoFlag variable is read from 1B-21. e. Before version 6, images of path attenuation from 2a-21 sometimes showed a striated or streaky pattern where the attenuation estimates at one or more angles are larger than the estimates at adjacent angles. This occurred more often at near-nadir angles where high values of the surface cross section are observed under no-rain conditions. Where it is used, the cross-track-spatial hybrid method appears to eliminate most of these streaky patterns. f. The diagnostic file includes attenuation and reliability results from several alternative methods: "standard" [version 5], cross-track, and hybrid, with 3 alternative reliabFactor formulae. Each entry is written on one line with the following fields: method - A text string identifying the method scan number angle bin number PIA reliabFactor reliabFlag The method ID values are: stdPIA Version-5 standard PIA (spatial or temporal) xTrack Cross-track method (ocean only) xtHyb1 Hybrid method, reliabFactor=A/chisqr xtHyb2 Hybrid method, reliabFactor=A/rms(sd) xtHyb3 Hybrid method, reliabFactor=A/sd In the reliabFactor formulae above, A is the Hybrid PIA, chisqr is the reduced Chi-square for the cross-track fit, sd is the standard deviation for the individual spatial average for the current anglebin, and rms(sd) is the root-mean-square of the sd's over all angle bins for the current scan. The value written in the HDF file is xtHyb2. The "xTrack" method is another experimental cross-track method that uses non-rain angle bins in the current scan. It is computed only if the scan includes a mixture of rain and non-rain angle bins, all over ocean. This method was experimentally introduced in version 5. Because each line is labeled with the method ID, the diagnostic file can easily be dissected into separate files for each method. Examples of this are shown in the following Unix commands: egrep 'stdPIA' 2A21.990913.10321.6.diag | cut -d: -f2 > stdpia egrep 'xTrack' 2A21.990913.10321.6.diag | cut -d: -f2 > xtrack egrep 'xtHyb1' 2A21.990913.10321.6.diag | cut -d: -f2 > xthy1 egrep 'xtHyb2' 2A21.990913.10321.6.diag | cut -d: -f2 > xthy2 egrep 'xtHyb3' 2A21.990913.10321.6.diag | cut -d: -f2 > xthy3 g. It is important to note that in previous versions (versions 1-4) the reliabFactor was defined as the difference: pathAtten - std_dev(reference value) In version 5 and 6, reliabFactor is defined as the ratio: pathAtten/std_dev(reference value) h. Prior to version 6, when the PIA < 0, pathAtten was set to 0.0. reliabFactor, which in version 5 is proportional to PIA, was computed before pathAtten was set to 0, so it could be negative; pathAtten was never less than zero. In version 6, pathAtten is no longer set to zero. Negative values are possible, although they will always be marked unreliable in the reliabFlag. 7. Description of Temporal Intermediate File Two temporal intermediate files are used to store (write-only file) and read (read-only file) the no-rain statistics of the normalized surface cross sections as a function of incidence angle (26 categories) and location (1 x 1-degree latitude-longitude grid). For the first month of data (December, 1997), the read-only and write-only temporal intermediate files are initialized to zero. During the processing of data from this month, the no-rain statistics are continuously updated and stored in the write-only file. At the end of the month, the write-only file for December is used as the read-only file for January and the write-only file is re-initialized to zero. This means that for the data processed in January, the statistics compiled in December will be used for the temporal reference data set. At the end of the processing of the January data, the write-only file, used to store the statistics for January, is used as the read-only file for the month of February. In general, the write-only file from the previous month is used as the read-only file (i.e. the reference data set) for the present month and where the write-only file is re-initialized at the beginning of each month. At the beginning of the post-boost data processing, the temporal files are set to zero. In particular, for the first segment of post-boost data (Aug. 2001), the read-only intermediate files are zeroed out. For the next month, the read-only files are converted to the write-only file and used as the temporal reference data for Sept. 2001. 8. The Cross-track Hybrid Surface Reference Method 2A-21 v5 computes the Path Integrated Attenuation (PIA) for each angle bin using spatial and temporal averages, selecting the method that gives the largest reliability factor. Version 6 also implements two alternative "cross-track" methods for ocean scans. The results of these methods are written in the diagnostic file along with the v5 result. When appropriate, the "cross-track hybrid" PIA is chosen as the 2A-21 product. Both methods perform a cross-track quadratic fit of the reference surface cross sections. The xtrack method fits a quadratic using data from all rain-free angle bins in the current scan; the hybrid method fits a quadratic using data from the current spatial-average at each angle bin. If the rain region is extensive, the xtrack method may have few or no data points to fit; if enough data are available to fit, the points will always be near the rain observations because they are taken from the same scan. The hybrid method always (or almost always) has a full scan worth of data to fit. The 49 spatially-averaged data points have the same potential problems as the standard spatial average, namely, that in some cases the spatial average might be computed in a region remote from the current IFOV, so that the along-track spatial averages are computed over ocean areas that are far from the rain cell in questions. (By examining the reference data for an orbit of data processed in reverse order we find that the spatial averaged reference data can differ significantly particularly in coastal regions.) The spatial average for each angle bin is computed over the last 8 rain-free observations over the same background type (i.e., ocean, land, or coast). The standard deviation from the spatial average is used as a weight in the fitting procedure. The fitted function is a quadratic where the fitting routine is based on the LFIT routine from Numerical Recipes (Press, et. al., 1989). The spatial average at the i-th angle bin is the average of the last 8 no-rain surface reflectivity values, sigma0_NR, y(i) = spatial_mean( sigma0_NR(theta(i)) ) and the standard deviation is stddev(i) = sqrt(var( sigma0_NR(theta(i)) )) The fit function is the quadratic yfit(i) = a + b*theta(i) + c*theta(i)^2 We determine the parameters a, b, and c by minimizing chi^2 = sum{i=1..49}( (y(i) - yfit(i))/stddev(i) )^2 The (2-way) hybrid PIA is the difference between the Surface Reference value, yfit(i), and the attenuated surface cross section, sigma0(i): A(i) = yfit(i) - sigma0(i) At this point, it is unclear what reliability factor is appropriate for the cross-track hybrid PIA estimate. Currently, the program computes three candidate values and writes them to the diagnostic file. The reliability factor is a ratio of the PIA estimate to an estimate of the uncertainty. The difference between the candidate factors is the choice of uncertainty estimate. For the standard spatial average product, the uncertainty estimate is the standard deviation of the spatial average. (For the temporal average it is the standard deviation of the temporal average. The question of whether these choices are the most appropriate, and whether or not they represent the best criteria for selecting between spatial and temporal estimates is unresolved.) The first hybrid reliability factor is based on the reduced chi-square for the fit, chisqr(N-3) = chi^2/(N-3) N is the number of angle bins, usually N=49. The reliability factor is reliabFactor_1(i) = A(i)/chisqr(N-3) Two alternate reliability factors are based on the sum of the standard deviations across the scan, and on the rms of the standard deviations across the scan: reliabFactor_2(i) = A(i)/[ 1/49 sum{j=1..49}( sigma(j) ) ] and reliabFactor_3(i) = A(i)/[ rms(sigma) ] where rms(sigma) = sqrt( 1/49 sum{j=1..49}( sigma(j)^2 ) ) In version 6, reliabFactor_3 is the value included in the HDF product. 9. Revised Angle Bin Definitions 2A-21 defines two angle bins. The first, angle1, is computed from the absolute value of the incidence angle, and is used to categorize observations for the temporal Surface Reference Method. It ranges from 1 to 26 (Fortran array convention). The other angle bin, angle2, depends on the signed incidence angle, and is used for the spatial SRM. It ranges from 1 to 51. The angle bins angle1 and angle2 are defined such that (angle1 - 1 - 0.5)*dtheta <= abs(theta) < (angle1 - 1 + 0.5)*dtheta (angle2 - 26 - 0.5)*dtheta <= theta < (angle2 - 26 + 0.5)*dtheta where theta is the incidence angle and dtheta is the angle bin size. These relations can be expressed more simply by angle1 = int((abs(theta)/dtheta) + 1 + 0.5) angle2 = int((theta/dtheta) + 26 + 0.5) respectively. There was a problem with the implementation of this algorithm before version 6: dtheta was defined as the cross-track beam width, which is read from the 1B21 Ray Header. The beam _positions_ (as opposed to the beam width) are uniformly spaced with steps about 0.75 degrees. One effect of the old angle bin definition is that the angle bins were not uniformly populated. In general, each scan should have one beam position in each "angle2 bin" except for the extra edge bins (1 and 51). In fact, some bins were under-populated and some were over-populated because the angle bins did not correspond with the actual beam spacing. In version 6 of the 2A-21 algorithm, dtheta has been set to a constant value of 0.75 degrees. 10. References Caylor I.J., G.M. Heymsfield, R. Meneghini, and L.S. Miller, 1997: Correction of sampling errors in ocean surface cross-sectional estimates from nadir-looking weather radar. J. Atmos. Oceanic Technol., 14, 203-210. Iguchi, T. and R. Meneghini, 1994: Intercomparisons of single-frequency methods for retrieving a vertical rain profile from airborne or spaceborne radar data. J. Atmos. Oceanic Technol., 11, 1507-1516. Kozu, T., 1995: A generalized surface echo radar equation for down-looking pencil beam radar. IEICE Trans. Commun., E78-B, 1245-1248. Marzoug, M. and P. Amayenc, 1994: A class of single- and dual-frequency algorithms for rain rate profiling from a spaceborne radar. Part I: Principle and tests from numerical simulations. J. Atmos. Oceanic Technol., 11, 1480-1506. Meneghini, R., T. Iguchi, T. Kozu, L. Liao, K. Okamoto, J.A. Jones, and J. Kwiatkowski, 2000: Use of the surface reference technique for path attenuation estimates from the TRMM Radar. J. Appl. Meteor., 39, 2053-2070. Meneghini, R. and K. Nakamura, 1990: Range profiling of the rain rate by an airborne weather radar. Remote Sens. Environ., 31, 193-209.