Back to INAG Homepage | Back to UAG-104 Contents page |

Use of ground-based and satellite data for an improved procedure for testing the accuracy of ionospheric maps

P A Bradley and M I Dick

Rutherford Appleton Laboratory, Chilton, Didcot, Oxon OX11 0QX, UK

Abstract

Present methods of testing the accuracy of empirical maps of ionospheric characteristics, made from past measurements of ground-based vertical-incidence sounders, employ the same data from the same locations. pA new method is proposed leading to a figure-of-merit of relative accuracy by which different mapping methods can be compared, making use of various other measurement and theoretical data sets. Proposals are made for using appropriate weighting factors to allow comparisons involving a mix of different types of measurement data.

Introduction

Planners of HF radio propagation circuits and assessors of the effects of the ionosphere on Earth-space links need long-term models of the height distributions of electron density for input to ray-tracing routines. These models have anchor points determined from predicted values of the standard vertical-incidence ionospheric characteristics given by empirical expressions fitted to past measurement data sets. The morphology of the regular E and, to a lesser extent, the F1-layer characteristics is governed by simple formulae in terms of solar-zenith angle, epoch of the solar cycle and latitude but the F2 layer exhibits more detailed structure and the most commonly used global maps of foF2 and M(3000)F2 are those generated by the Jones-Gallet numerical mapping procedure [1]. In this technique median measured data for each month of reference low and high sunspot numbers are separately mapped in terms of harmonic orthogonal Fourier functions and associated sets of numerical coefficients to give time-of-day, latitude and longitude variations. The numerical coefficients for the separate ionospheric characteristics are determined with different orders of harmonics. The maximum harmonic orders and sets of coefficients adopted (typically 988 coefficients for each month in the case of foF2) should be those for which the rms residuals between the mapped and measured values attain a lower limit.

The Radiocommunication Sector (former CCIR) of the International Telecommunication Union has adopted a set of such maps [2] based on data from some 160 measurement stations collected over the years 1954-58 and 1964. Reference values are given for twelve-monthly smoothed sunspot numbers R12 = 0 and 100, with an assumed linear variation in terms of R12 up to R12 = 150 and complete saturation for higher R12. An example of one map of the set showing F2(ZERO)MUF = foF2 + 0.5 MHz for R12 = 100 is given in Figure 1. Although these maps have proved useful they suffer from a number of limitations, principal amongst which are (i) the measured data are non-uniform in geographical position and no specialised method has been produced to eliminate this potential bias, (ii) ocean regions are filled by assuming we understand the physics of the F region and by generating data given in terms of synthesised 'screen-point' values chosen to ensure that the harmonic functions remain stable in these regions and (iii) it is now possible in principle to augment old measurements with more recently collected data.

The 1984 Florence URSI General Assembly established Working Group G5, tasked to produce a new set of foF2 maps, took account of more recent vertical-sounding and satellite measurement data and applied latest thermospheric wind theory to aid in the geographical smoothing [3]. Following Rush et al [4], sets of coefficients were generated from up to 50 years of vertical-sounding data collected at the Australian Ionospheric Prediction Service, including data from the most recent years. The same maximum harmonic orders as the CCIR maps were retained to preserve software compatibility and new maps were published [5]. The URSI Working Group compared their maps and the CCIR maps with some 250,000 monthly median hourly vertical-sounding measurements. Mean differences for the separate months in both cases were generally less than 0.2 MHz and standard deviations in the range 1.1-1.5 MHz. Differences for the URSI maps were marginally less than those for the CCIR maps, but despite the large data samples many of the test data were the same as those involved in the maps. In particular, to reproduce the greater ocean structure there was degrading of the fit at the measurement locations. This means that if the same criteria had been applied to the URSI maps in choosing the maximum orders of harmonics, these orders would undoubtedly have been greater than in the CCIR case, resulting in a reduction in residuals. In retrospect, noting that one of the present authors was a member of the URSI group, the decision to retain the same orders, whilst numerically convenient, turns out to have been a mistake. An additional consideration is that the method of comparison did not allow testing at the screen points.

Despite these efforts, the CCIR community would not adopt these as a replacement for the maps already in use and the position remains unchanged. Although there are no international efforts under way to generate further global maps, some studies are in train to produce improved regional maps, in particular in limited geographical areas [6]. This points to a need to review the adopted map-testing procedure. This present paper makes proposals through the use of appropriate weighting factors to allow comparisons involving a mix of different types of measurement data.

In the comparison of different types of empirical map, the question arises how to balance accuracy and complexity of approach with a reluctance to change from existing procedures. The solution has to depend on the application. Where extreme day-to-day variability exists, how accurate does the predicted median need to be? For telecommunications planning, for example, the requirements are likely to be less stringent than in following long-term ionospheric changes, perhaps related to global warming or secular drifts in the Earth's magnetic field. Ultimately the adopted approach involves subjective factors. Here we address only accuracy considerations. The aim is to formulate a figure-of-merit for the maps giving their overall relative accuracy's, rather than to quote a separate accuracy figure for the different places and times. The criteria to apply in assembling a measurement data base for map testing is also considered.

Data availability

Earlier tests used only vertical-sounding data, resulting in two particular problems. The same measurement data should not be used for map generation as for map testing and ideally half the data should be applied in each role. However, in the real situation where vertical-sounding data locations are sparse, there is a tendency to employ as many as possible for map generation. This leaves an inadequate number of locations for testing, or means that greater weight is given to those locations selected for testing than those used in making the maps. In fact, the testing is undertaken with test data for the same locations as those used in map generation. Hence the accuracy's in other world areas where the maps are determined by interpolation/extrapolation procedures or theory, and may well be least accurate, are not tested. This then points to the value of examining the availability for testing using data sets from other types of measurements as shown in Table 1.

We need to consider the epochs for which data are needed. Many groups have examined the sunspot-cycle variations of monthly median foF2 for different stations and some workers [7, 8] have noted differences on the upwards and downgoing legs of a given cycle, leading to a so-called 'hysteresis effect'. Nonetheless, Bradley [9], from an examination of Slough foF2 and M(3000)F2 data since 1932, the longest sequence of available measurements, has concluded that changes between cycles and differences on the rising and falling half cycles are irregular and small in comparison with the scatter. Figure 2 is an example from these earlier results in which noon monthly median foF2 for June is shown plotted as a function of smoothed sunspot number R12 and displayed separately for the different solar cycles. Also given is the same curvilinear best-fit line over the whole period drawn through the data points for each cycle. Agreement is good in all cycles supporting the contention that any measurement data for any past epoch may be used for map testing.

The data used should be temporally distributed as uniformly as possible to incorporate all times-of-day, seasons and solar epochs. Testing also should cover the full geographical regions that are mapped. Vertical-sounding data cover a wide range of different years and months. This suggests the need to weight the test data sample to provide a uniform assessment, using as many data as feasible in map testing without compromising map generation. Weighting is particularly important if limited additional measurements are made in new areas specifically for map testing.

Whereas vertical-incidence ionosondes are irregularly distributed geographically, polar-orbiting satellites offer the possibility of uniform area testing. A difficulty lies in the temporal coverage of the satellite data and in the significance of the information obtained. A most valuable data set is that provided by the Japanese ISS-b topside-sounder satellite from which constant UT maps have been constructed seasonally by combining the data for the separate orbits (Figure 3). Tests of Rush et al [5] for 65,000 data points over six separate seasons in 1978/79 resulted in rms differences of 19.2-24.5% for the CCIR maps in comparison with 16.7-21.7% for the URSI maps. There was a tendency for the CCIR residuals to be smaller over land and for the URSI residuals to be appreciably less over the Pacific Ocean. However, it should be noted that the ISS-b measured seasonal data sets each consist of around 65,000 data points, or only some 2,700 values for each UT hour. Yet, for example, with comparisons made on a 5° latitude-longitude grid this would provide just 2-3 samples per geographical cell (Figure 4), which is a gross under sampling. So there are sampling problems in using topside sounder or other satellite data that need to be compensated with appropriate weighting factors.

Bilitza [10] has reviewed the availability of satellite sounder and direct probe data. Figure 5 taken from his survey shows the lifetimes and heights of relevant satellites. Topside-sounder data were collected from the Alouette and ISIS series of satellites but synoptic data sets of foF2 were never published and the raw measurements are understood no longer to be available. Other topside soundings were collected aboard Intercosmos 19 and Cosmos 1809 satellites. At present these data are stored in original form, some with hard copy. A sample of topside height profiles has been generated [Pulinets et al, 11]. Interpretation of direct satellite plasma-probe data creates additional problems as reviewed by King [12]. Besides knowing the laws of extrapolation to the F2-layer peak height there are additional uncertainties in estimating that height itself. Nonetheless Kutiev and Stankov [13] have developed extrapolation techniques and applied these subject to varying approximations.

Total electron content (TEC) data exist with significant geographical coverage around each ground-based measuring site. With TEC given as the product of equivalent slab thickness and maximum plasma density and with slab thickness showing much less spatial structure than foF2, the use of an appropriate slab thickness model would enable measured TEC to be converted to corresponding foF2. To the best of the authors' knowledge this has not yet been done, perhaps because hitherto there has been no motivation. However, this possibility should be considered.

Normally incoherent-scatter radar measurements do not give absolute peak electron densities but rather relative values that are to be calibrated in terms of nearby foF2 vertical-sounding measurements. On the other hand they do yield the heights of peak densities. Empirical formulae are available (Dudeney and Kressman [14]) to relate peak height to corresponding M(3000)F2 with acceptable accuracy. Although the numbers of incoherent scatter sounders distributed throughout the world are relatively low and although these do not usually make synoptic measurements, the quantities of data that exist are considered sufficient to be of value in global map testing.

Anderson [15] has pioneered the use of semi-empirical ionospheric models in which relatively simple empirical expressions are formulated to approximate to the results of full calculations based on equations of continuity and momentum. By normalising to ground-based measurements, it is possible to use the theory to give absolute values in neighbouring regions. This technique generates considerably more ionisation structure over the oceans than is given by conventional polynomial smoothing procedures. The question arises whether it is valid to test maps using data based on the same theory as went into their generation. This is an important point, since it is over the oceans that existing maps are least accurate.

Methods exist for inverting oblique-incidence ionograms to equivalent mid path electron-density profiles [16,17], thereby yielding associated foF2 and M(3000)F2 values. However, not all ionograms are suitable for inversion because there is insufficient visible trace or because of spread effects. The assumption is usually made that there is a concentric ionosphere so that for path lengths beyond about 3000 km and for multiple-hop propagation, procedures cannot meaningfully be applied. Inversion methods tend to be complicated so that automated implementations are complex. This means that few reliable data are available using these techniques.

We recommend the establishment of a test data set composed of as may of the above types of data as are feasible.

Weighting factors

Typically the testing data base consists of different data sets of unequal size, have different accuracy's related to the techniques and monthly sampling and cover different geographical areas. In some cases these areas will overlap, whereas other regions are relatively sparsely served (see Table 1). Any means of reconciling these separate factors is subjective. The proposals presented here apply to the combination of any number of such different test data sets where these varying considerations are taken into account by means of appropriate weighting factors. Suggestions are offered for values of the weighting factors.

TABLE 1 - Relative quantities of available test data, their inherent accuracy's, monthly sampling and numbers of locations for which they apply

Data Source

Quantity

Inherent Accuracy

Monthly Sampling

Numbers of Locations

Routine vertical ionosondes

Much

Good

Good

Many

Temporary ionosondes

Little

Good

Poor

Few

Satellite probe

Medium

Medium

Medium

Many

Topside sounder

Medium

Good

Medium

Many

TEC

Medium

Medium

Medium

Medium

Incoherent scatter

Little

Good

Medium

Few

Theory

Infinite

Poor

Perfect

Infinite

Oblique ionograms

Little

Medium

Poor

Few

Weighting factors need to take account of inherent data usefulness from a location standpoint; the daily sampling and accuracy. In formulating numerical values for the weighting factors the dominant influences are reviewed.

location

If test data relate to nearby locations at the same time eg two adjacent ionosondes or measurements along a satellite track, then the weighting should be diminished in inverse proportion to the quantities of such data. On the other hand, as separation increases the data sets become independent and should have full weighting. According to Edwards et al

,[18] the correlation distance for foF2 fluctuations about the monthly median is typically some 500 km in a N-S direction and 1000 km E-W. Other ionospheric characteristics vary over comparable or enlarged distances. So it is proposed to define an array with each element corresponding to a geographical area of size 5° in latitude by 10° in longitude where the number of such elements depends on the extent of the region mapped. For a given epoch, all test data, within an element, are weighted by the number of other values within that element for that epoch. Satellite data will be quantised to one value at the centre of the area for each time. Thus W_{location}is the weighting factor for location andW

_{location}= 1/S (1)for S sets of data

daily sampling

Sampling accuracy effects on estimates of the monthly median value derived from a limited number of daily samples can be assessed [19] assuming a normal law of variation of the daily values. The standard error E for a sample of D days drawn from a population of standard deviation s is determined as:

(2)

This equation states (Figure 6) that E tends to zero as the sample size rises to 30 days. E remains relatively low for sample sizes exceeding 5-10 days, but increases for smaller samples. It is appropriate to adopt a weighting factor which is the inverse of the standard error with a limiting maximum of 10: i.e.

(3)

accuracy

Somewhat arbitrarily an accuracy weighting factor, W

_{accuracy}, is proposed and values are given in Table 2.TABLE 2 -Proposed values of W

_{accuracy}

Data Source

W

_{accuracy}Reason

Fixed vertical ionosondes

1

arbitrary

Temporary ionosondes

1

same as above

Topside sounder

1

data availability?

Satellite probe

0.7

some interpretation problems

TEC

0.7

slab height model needed

Incoherent scatter radar

1

limited data sets

Theory

0.5

depend on complexity of approval

Oblique ionograms

0.7

some interpretation problems

Now suppose for a given time period and element there are ni test data values of type i each derived from D days of sampling. Then the overall weight factor associated with these can be taken as:

W

_{ci}= W_{location}xW_{sampling}xW_{accuracy }(4)with the separate terms given as above.

Proposed procedure

For a uniform set of N test data values, each of equal weight, the figure-of-merit F is taken as the standard deviation between the test data and mapped values -

where T = test data value

M = mapped value

The map with lowest F is taken as the best. If the test data, for example, are daily values and the mapped figures are regarded as monthly medians, then equation (5) provides an indication of the accuracy of the maps on a given day. In the present application it is assumed that the test data are estimates of the monthly median values so F is a smoothed figure having relative rather than absolute significance.

Equation. (5) can be extended to the case where the separate data have different weights. Suppose there are C classes of data composed of Nc values with weighting Wc. Then we have that -

Equation (6) applies where the separate classes correspond to different data sources and test locations and the respective values to the various times-of-day, season and solar epoch encompassed within the test.

Conclusions

The problems of limited sampling and associated significance of comparison results, as for example in the case of use of topside-sounder data cited above, highlighted the need to adopt a procedure, of the type described, for future mapping accuracy assessments. Future work will apply the above procedure to test various alternative mapping methods [eg 20-23] being developed under EEC project COST 238 (PRIME) [6]. Consideration might also be given to a reassessment of the existing URSI and CCIR maps by this approach.

Acknowledgments

The work reported in this paper has been performed as part of the UK National Radio Propagation Program and has been funded by the Radiocommunications Agency of DTI.

References

1. Jones, W.B., and Gallet, R.M., 1962 'Methods for applying numerical maps of ionospheric characteristics',

J Res NBS-D,66D, 649-662.2. CCIR Report 340, 'CCIR atlas of ionospheric characteristics',

ITU, Geneva, 1990.3. Rush, C.M., PoKempner, M., Anderson, D.N., Stewart, F.G., and Perry, J., 1983 'Improving ionospheric maps using theoretically derived values of foF2',

Radio Sci,18, 95-107.4. Rush, C.M., PoKempner, M., Anderson, D.N. ,Perry, J., Stewart, F.G., and Reasoner, R., 1984 'Maps of foF2 derived from observations and theoretical data',

Radio Sci, 19, 1083-1097.5. Rush, C.M., Fox, M., Bilitza, D., Davies, K., McNamara, L., Stewart, F., and PoKempner, M., 1989 'Ionospheric mapping: an update of foF2 coefficients',

Telecomm. J, 56, 179-182.6. Bradley, P.A., 1993 'COST-PRIME (Prediction and retrospective ionospheric modelling over Europe)',

INAG59, 12-16, WDC-A Boulder, Colorado 80203, USA.7. Mikhailov, A.V., 1993 'On the dependence of monthly median foF2 on solar activity indices',

Advances in Space Research, 13(3), 71-74.8. Adler, N.O., Ezquer, R.G. and Manzano, J.R., 1993 'On the relationship between ionospheric characteristics and solar indices',

Advances in Space Research. 13(3), 75-78.9. Bradley, P.A., 1993 'A study of the differences in foF2 and M(3000)F2 between solar cycles', Proceedings of the PRIME COST 238 Workshop,

Wissenschaftlicher Bericht2, 67-77, Karl-Franzens Universitat, Graz, Austria.10. Bilitza, D., 1989 'The worldwide ionospheric data base', National Space Science Data Center,

WDC-A Report89-03, Goddard Space Flight Center, Washington DC.11. Pulinets, S.A., Karpachev, A.T. and Shoya, L.D., 1993 'The role of topside sounding in modelling the topside of the ionospheric F-region: the present situation and future possibilities', Proceedings of the PRIME COST 238 Workshop,

Wissenschaftlicher Bericht2, 29-37, Karl-Franzens Universitat, Graz, Austria.12. King, J.W., 1973 'The determination of foF2 and hmF2 from satellite-borne probe data',

Telecomm J, 40, 364-368.13. Kutiev, I., and Stankov, S., 1992, 'Satellite data for use in the ionospheric mapping', Proceedings of the PRIME/URSI joint Workshop on 'Data validation of ionospheric models and maps (VIM),'

Memoria, COST 238TD(93)001, 100-104.14. Dudeney, J.R., and Kressman, R.I., 1986, 'Empirical models of the electron concentration of the ionosphere and their value for radio communications purposes',

Radio Sci, 21, 319-330.15. Anderson, D.N., 1993 'Global ionospheric modelling', Modern Radio Science,

Oxford Science Publication, Oxford, UK, 159-188.16. Gething, P.J.D., 1969 'The calculation of electron density profiles from oblique ionograms',

J Atmosph Terr Phys, 31, 347-354.17. Krasheninnikov, I.V. and Liannoy, B.E., 1990 'Estimation of the true ionospheric height profile',

J Atmosph Terr Phys, 52(2), 113-117.18. Edwards, W.R., Rush, C.M., and Miller, D.M., 1975 'Studies on the development of an automated objective ionospheric mapping technique',

Air Force Surveys in GeophysicsNo 302, AFCRL-TR-75-0124, Air Force Cambridge Research Laboratories, Bedford, Mass 01730, USA.19. CCIR Report 253, 'Measurements of sky-wave signal intensities at frequencies above 1.6 MHz',

ITU, Geneva, 1990.20. Zolesi, B., Cander, Lj.R., and De Franceschi, G., 1991 'Mapping of some ionospheric characteristics over a restricted area using SIRM', Seventh International Conference on Antennas and Propagation,

IEE Conference Publication333, 512-515.21. De Santis, A., De Franceschi, G., Zolesi, B., Pau, S., and Cander, Lj.R., 1992 'Regional mapping of the critical frequency of the F2 layer by spherical cap harmonic expansion',

Ann. Geophysicae, 9, 401-406.22. Singer, W., and Taubenheim, J., 1990 'Application of the expansion into empirical orthogonal functions to ionospheric characteristics',

Advances in Space Research, 10(11), 59-64.23. Teryokhin, Yu.L., and Mikhailov, A.V., 1992 'A new approach to solving the problem of a spatial approximation of the field of observed foF2 values',

Geomag and Aeronomy, 32(2), 205-210.