Topic: accuracy assessment and technique's senstivity without validation data

I am working on “measuring coseismic displacements using COSI-Corr". I have produced correlation results (~ 80 results) using different technique's parameters given as:

•    Using:   1- ASTER DEM    2- SRTM DEM
•    Resampling method (Bilinear, Bicubic, Sinc)
•    Correlator (frequential or statistical)
•    Window sizes (8, 16, 32, 64)
•    Step Size (8, 12, 16)
•    Threshold (0.90, 0.95)

I do not have validation dataset except some reference values from the literature. My objectives are:
1.    To measure accuracy of my results (correlations, displacements).
2.    To check sensitivity of the technique (COSI-Corr) based on these results.
3.    To investigate residual errors due to topography, attitude or CCD alignment.

I have seen following paper which describe accuracy assessment for glacier movement. It describe some tests to evaluate the results without validation dataset.

Scherler, D., Leprince, S., and Strecker, M.R., 2008, Glacier-surface velocities in alpine terrain from optical satellite imagery--Accuracy improvement and quality assessment: Remote Sensing of Environment, v. 112, p. 3806-3819.

Can you please give me any idea or directions how I can design (devise) some tests in order to perform the above mention tasks for measuring coseismic displacements. OR how I can check the accuracy of my results or technique’s sensitivity without having validation data.

Thanks for your consideration.


Re: accuracy assessment and technique's senstivity without validation data

Hi Yaseen,

To validate your correlation results, you need to be able to identify and isolate the source of each errors. In your correlation results, you will have a superimposition of errors from;
- geometric modeling errors due to the sensor and platform,
- geometric errors due to topography errors,
- noise and correlation bias due to aliasing from the optical system and resampling method,
- decorrelation due to temporal changes,
- noise and bias in the correlation due to changes in lighting condition, shading differences, etc...

You need to understand all these sources of errors to have a better understanding of the uncertainty and potential bias on the displacement field measured.

You can try to evaluate most of these effects using images with no ground displacement.

You also need to understand what is affected when you change the options. For instance, the bilinear and bicubic resampling methods are in COSI-Corr only for fast visual results. They should not be used for quantitative results as the implementation does not account for local image warping. Hence they'll introduce resampling artifacts depending on the topography and incidence angle of the acquisition.

The difference between ASTER DEM and SRTM depends on the error of each DEM and angle of incidence difference between acquisitions.

The threshold simply performs a filtering, cutting-off high image frequencies to reduce noise. The lower the threshold, the more the frequencies are being cut.

The step size should be related to the window size. The step size should sample well the deformation field you're trying to recover. If the step size is too small, the displacement field will be oversampled and no additional information will be gained. If its too large, you'll miss details.

The larger the window size, the more you average the displacement field, you're trading spatial smoothness for spatial resolution. Also, if the window size is too small, its FFT won't make sense because of the time-frequency uncertainty principle.

You should also have clear objectives as to which characteristics of the displacement field you want to evaluate (consistency of the fault trace, consistency of the relative fault offset, consistency of the long wavelength deformation?). I think you should generate synthetic images to test all the errors simply due to the algorithms. Then use the real images to characterize the effects of decorrelation and geometrical uncertainties.

In practice, you can also create models of fault dislocation and compare them with the correlation results.

To summarize, I don't think you can rigorously evaluate what's going on simply by looking at the processing results from a pair of images, and trying the combination of all parameters. Most combination actually won't  make any sense, you have to first understand the theoretical effects of each parameter, then make hypotheses, then test your hypotheses using your practical examples.

I think you also have to be clear on your objective: are you trying to test the methodolgy implemented in COSI-Corr, or are you rather trying to test the COSI-Corr software itself? These are two different things.

Let me know,