In Proc. of IEEE Globecom 2006, San Francisco, CA, USA, November 2006.

Comparative Evaluation of Prediction Heuristics for Wireless Channels Ana Aguiar, Adam Wolisz TKN, TU Berlin [email protected], [email protected] Abstract— Impairments in wireless data communication due to time and location dependent errors can be overcome by using channel-adaptive techniques, like channel-aware scheduling or adaptive modulation. These techniques require information about channel behaviour obtained from channel predictors. Unfortunately enough the accuracy of predictors usually suggested in the literature has not been confirmed and comparative studies do not exist (with one exception). In addition, existing predictors are frequently difficult to compute and sensitive to numerous parameters, e.g. noise and channel variability. In this paper, we investigate and compare the performance accuracy of three simple heuristics for channel prediction. Our investigation includes a comparison with a verified reference predictor using Rayleigh channels and measured WLAN traces. Our study indicates the usefulness of the investigated heuristics for fixed systems and pedestrian speed mobility.

I. I NTRODUCTION Data communication over wireless channels is strongly influenced by the pattern of channel quality changes. Thus the idea of adapting transmission parameters to the channel behaviour (e.g. adaptive modulation and coding [19], channelaware scheduling [3], [16] or multi-user diversity [14], [18]) in order to improve the transmission quality is very popular. Any such adaptation strategy depends, however, not only on the knowledge of the actual (and past) channel parameters, but has also on a prediction of the future channel behaviour. In most studies a perfect knowledge of the future behaviour of the channel is assumed, much less frequently some inaccuracy is assumed. Surprisingly enough, very little attention has been devoted to a systematic investigation of the accuracy of channel predictors. In fact, up to our best knowledge, [17] contains the only publicly available comparative study! Unfortunately the channel predictors presented therein—based on auto-regressive models of the fading process or on adaptive flters—are complex and their accuracy sensitive to the time-variance of the channel, making them unattractive. Nevertheless we select the best of those predictors as basis for our study, it will be referred to as Modified Covariance (MC). In this scenario, heuristics are an attractive low-complexity alternative, but an evaluation of their accuracy and a comparison to existing predictors is missing. This paper is devoted to a fundamental investigation of the accuracy of three frequently used prediction heuristics, which can intuitively be defined as follows: • the channel parameters will not change, i. e. the predicted

values are the same as the last measured oneswhich we call one step (OS) prediction; • the channel parameters change according to the average of N most recent channel sampleswhich we call moving average (MA) prediction; the channel pyrameters change linearly according to the trend of N most recent channel samples which we call linear prediction (LP). For a more realistic evaluation, we conducted a measurement campaign in several indoor and outdoor environments typical for WLAN and used the traces to assess the accuracy of the prediction, additionally to theoretical Rayleigh channels. Although the reference algorithm is more accurate is theoretical scenarios, it is not usable at all in realistic scenarios due to noisy samples and error propagation. For the realistic WLAN scenarios, one of the proposed heuristics is a much simpler and accurate way to predict the channel behaviour. II. W IRELESS C HANNEL P REDICTION A. Related Work Reference [9] introduces the Long Range Predictor (LRP): a prediction algorithm based on an auto-regressive model (AR) for the fading process, that uses the maximum entropy method (MEM) for the estimations of the AR model coefficients. The channel is undersampled to achieve a longer prediction horizon with the same AR model order and complexity; to get prediction values at the data sampling rate, the undersampled predicted values are interpolated. The performance of the algorithm (measured as mean squared error) is very good for a stationary Jakes channel, but degrades for non-stationary channel parameters. In [5] the authors remark that prediction accuracy is reduced by noisy samples and that non-stationarity produces mismatch in the AR coefficients. Reference [4] proposes two predictors, one based on an AR model using least squares estimation and another one based on Kalman filters. As above, the authors identify the need for noise filtering, proposing Wiener filters, and the use of sub-sampling to extend the achievable prediction range. The evaluation of these prediction algorithms is made in different measurement scenarios and the metric used is not comparable to the previous ones. Besides, the algorithms proposed could not be reproduced from the information available in the papers. Reference [13] studies a wireless channel predictor based on neural networks that perform better than the Modified Covariance algorithm, the best in [17], but neural networks are far more complex than an AR model.

The following heuristic has often been proposed [6], [7], [15]: assume that the channel will not change from the last channel sample, i. e. take the last reported channel indicator as a prediction for the future channel indicator. Although [7] shows an evaluation of the accuracy of this heuristic, it is based on a good/bad Gilbert-Elliot channel model, which is realistic only to a certain extent, and inappropriate for adaptation of transmission parameters like modulation or coding. In the single comparative study known to us, Semmelrodt in [17] proposes several methods for estimating the coefficients of a channel prediction algorithm based on an auto-regressive (AR) models and adaptive filters, and evaluates the prediction results using theoretical Rayleigh channels. The metric used is the normalised MSE of the prediction error for a certain fraction of wavelength, a metric that can be compared to that used in [9]. According to the results shown in both references prediction of the signal envelope obtained with the best algorithms in [17] are more accurate than LRP [9] for one wavelength. But we must take into account that in the first case only very low mobility was considered (1 m/s) whereas in the second case speeds of 26 m/s were used. Except for the heuristic, the channel predictors presented are complex and, except for the neural networks, require different calibration for specific environments. Further, they are sensitive to the time-variant nature of the wireless channel as well as to noise. Although the latter can be reduced by filtering, nothing can make the channel invariant. B. Reference Predictor—Modified Covariance (MC) As a reference for our study we chose the prediction algorithm that performed best in the comparative study [17]— Modified Covariance (MC). The MC models the received signal s(i) as an AR process of order p: s(i) =

p X

ak · s(i − k) + e(i) i = 0 . . . N − 1

(1)

k=1

where e(i) is a complex white Gaussian noise process and ak the coefficients of the AR polynomial. A linear predictor is used to extrapolate the behaviour of the process beyond the available channel samples [17], [8]: sˆ(i) =

p X

ak · s(i − k)

(2)

k=1

To obtain a predicted value it is necessary to estimate the coefficients of the model ak . Thus, the time-variant wireless channel is modeled as an AR process with time-variant coefficients ak (i), that are calculated anew each time that channel prediction is required. The MC prediction algorithm uses the least squares method to solve the system of linear equations for calculation of the model coefficients ai . We use a model order p=15 as proposed in [17]. This linear predictor allows only a single signal sample to be predicted at time i. To predict several samples in the future, several linear predictors are cascaded, each using the previously extrapolated signal samples as input. Since the input to predictors down the

chain are extrapolated samples (themselves inaccurate), the prediction error propagates for increasing horizons. C. Heuristics for Wireless Channel Prediction Take s(i) as the received signal amplitude at discrete time ˆ instant i and s(i) h the amplitude of the received signal at time i + h, predicted at time i. We compare the performance of the following three heuristics for prediction of the wireless channel: One Step (OS) ˆ s(i) h = s(i)

(3)

Moving Average (MA) 1 ˆ s(i) h = N

N −1

X

s(i − k),

(4)

k=0

i. e. the predicted value is the average of the last N channel samples. Linear Prediction (LP) PN −1 PN −1 PN −1 s(i − k) ∗ k − s(i − k) k k=0 k=0 k=0 ˆ , s(i)h = (i−h)∗ PN −1 PN −1
2 k=0

k2 −

k=0

k

(5)

i. e. the predicted value is given by a linear regression over the last N channel samples. III. P ERFORMANCE E VALUATION We use a simulation study for the performance evaluation. We predict, at each discrete time instant i, the values of the amplitude of the received signal for prediction horizons h between 1 and 15 ms. Then, we calculate the prediction error—the difference between the predicted and the actual ˆ − s(i + h)— for each horizon signal amplitudes e(i)h = s(i) h h. A. Metric The metric used to compare the performance of the channel predictors is the the mean squared value of the normalised prediction error (NMSE) in dB, as in [9], [17].  2 K−1 X 1 e(i)h q  , (6) NMSE(h) = 10 · log PK−1 K i=0 1 2 |s(i)| i=0 K where K is the number of channel samples. Since the metric is expressed in dB, the more negative the values of the MSE, the higher the accuracy of the predictor. Also, an MSE of 0 dB expresses errors with power similar to the power of the signal. B. Rayleigh Channels The theoretical behaviour of the wireless channel fading can be described as the sum of the several incident waves and its amplitude characterised by the Rayleigh distribution. We will evaluate the performance of the predictors on Rayleigh fading channels synthesised according to the Jakes method [11] for a center frequency of 2.4 GHz and Doppler spreads corresponding to speeds of 6 km/h, 15 km/h, 30 km/h, 50 km/h and 100 km/h—here we show only results for 6 km/h, 50 km/h

TABLE I M EASUREMENT SCENARIOS . N

IS THE NUMBER OF MEASUREMENT RUNS MADE IN EACH SCENARIO . K IS THE TOTAL NUMBER OF SAMPLE VALUES IN EACH SCENARIO AFTER RE - SAMPLING AT

Scenario Archi Carpark Maths

N 7 7 4

K 888249 1084395 618950

Mensa

7

1088694

Environment Busy roundabout Parking lot surrounded by buildings on 3 sides Foyer of Maths building during intervals between lectures Student canteen of the TU Berlin at busy hour

Road Walk

7 3

892994 408778

Busy street Grass surrounded by trees and bushes

1

ACF

0.5

0

-0.5 6km/h 50km/h 100km/h

-1 0

2

4

6

8 lag [ms]

10

12

14

Fig. 1 AUTOCORRELATION FUNCTION OF THE R AYLEIGH FADING FOR LAGS UP TO 15 MS AND VARIOUS SPEEDS .

and 100 km/h. Each channel trace has 500000 values sampled at 1 kHz. The autocorrelation functions of the Rayleigh fading for lags up to 15 ms are shown in Figure 1 and will be helpful for understanding the performance of the predictors. C. WLAN Channel Measurements In practice, however, the channel indicator available to adaptive mechanisms is the output of a measurement system. Usually, it expresses the average signal over the duration of a received packet in a proprietary scale of discrete values, which can be converted to dBm. Nothing guarantees that these channels indicators behave similarly to Rayleigh fading. Although WLAN cards are not an accurate way to measure the channel, what they deliver is often the only indication of channel quality available to channel adaptive techniques, especially those employed at the link layer like scheduling. Thus, it is necessary to evaluate the accuracy of channel predictors using more realistic scenarios. We conducted channel measurements using WLAN cards in environments where WLAN coverage is plausible. The criteria for the choice of the measurement scenarios was that they should have the characteristics, in terms of mobility and surroundings, of environments where WLAN coverage could be available. Table I shows a brief description of the environments where we carried out the measurements. The measurements were done using two laptops where one was used as a base station (Base) and the other one as the wireless terminal (Mobile). The laptops were running the

1 KH Z .

Mobility Traffic between Base and Mobile No mobility People moving between and around Base and Mobile People moving between and around Base and Mobile Traffic between Base and Mobile Pedestrian speed

Linux operating system (kernel 2.4.17), and were equipped with Lucent WLAN cards with the the PRISM2 [10] chipset 1 . To gather as many channel samples as possible we used a packet generator at the Base that sent UDP packets carrying 1 Byte of data every 1.3 ms 2 , we changed the driver of the WLAN cards [12] so that no acknowledgments were sent 3 and packets with a wrong CRC-check were not discarded 4 . The WLAN card measured the received signal power averaged over the duration of the packet, and the values were recorded together with the time of the measurement for every received packet both at the Base and at the Mobile. Later, we resampled the measured data at 1 KHz using linear interpolation. Further, to reduce the noise in the signal measurements, we used a 30 point moving average filter (a compromise between the bandwidth and the noise reduction capability)5 . The environments Archi and Road show deep fades and fast variations; Walk shows deep fades and shadowing; Maths, Carpark and Mensa show very little variations, and the received signal looks often like white noise with some deep fades. Details on the measurement campaign and the results can be found in [1], [2] and the traces are publicly available at http://www.tkn.tuberlin.de/∼aaguiar/wlan measurements.html. IV. R ESULTS For lack of space, we do not show all results figures here, but they are available at http://www.tkn.tuberlin.de/∼aaguiar/prediction.html. 1 For practical reasons relating to the availability of open-source drivers we only used one type of cards. 2 1.3 ms was the minimum possible sending interval for our measurement setup, due to the delays in the Linux protocol stack; although we tried shorter send intervals, packets never arrived at shorter intervals at the card. 3 Waiting for link layer acknowledgements would increase the time interval between two channel samples, and we are not interested in wether packets suffer bit errors or not. 4 In this way, we could also have signal samples for packet that suffered from bit errors, increasing the number of channel samples. 5 The FFT of the received signal traces before and after noise filtering can be seen in [2]. They show that the frequencies below 10 Hz are not changed; only high frequency noise is filtered. For a carrier frequency of 2.4G Hz the 10 Hz correspond to speeds of 4.5 km/h and we find this adequate for the low mobility environments involved.

0

20 10 0 MSE[dB]

MSE[dB]

-5

-10

-10 -20 N=2 N=5 N=10 N=15 N=20 N=30

-30

-15

N=5 N=10 N=20 N=30

-20 0

2

4

6 8 10 Horizont [ms]

12

-40 -50 14

0

2

4

(a) 6 km/h

6 8 10 Horizont [ms]

12

14

(a) 6 km/h

0

20 10 0 MSE[dB]

MSE[dB]

-5

-10

-10 -20 N=2 N=5 N=10 N=15 N=20 N=30

-30

-15

N=5 N=10 N=20 N=30

-20 0

2

4

6 8 10 Horizont [ms]

12

-40 -50 14

(b) 50 km/h

Fig. 2 MSE OF THE PREDICTION ERROR FOR THE MA

0

2

4

6 8 10 Horizont [ms]

12

14

(b) 100 km/h

PREDICTOR FOR

R AYLEIGH FADING .

A. For Rayleigh channels Figure 2 shows the NMSE for the MA heuristic and varying values of the number of samples used for speeds of 6 and 50 km/h, the latter exemplifying the behaviour observed also for 100 km/h. The heuristic shows somewhat low error power only for pedestrian speed and short horizons, and the best performance is achieved for N=5. For long horizons or high speeds the MSE grows to values close to 0, indicating errors with the same power as the signal, which is intuitively as good as guessing. Figure 3 shows the performance for the LP heuristic with varying number of channel samples used in the regression. Here we also show only the results for 6 and 100 km/h, where the latter is representative of the results for 50 km/h. Like the MA, the LP heuristic works only for pedestrian speed and short horizons, in which case the highest accuracy is obtained when only 2 samples are used. Figure 4 shows a comparison of the three heuristics and the reference predictor for Rayleigh fading and all speeds studied, where for the MA and LP heuristics the best value of N was chosen. Here we can see again that the accuracy for high mobility is similarly bad for all speeds. Neither the heuristics nor the reference predictor are good for high mobility, all showing values of the NMSE very close to 0. High channel variability for high speeds leads to errors of high magnitude of the MC. The reason why the heuristics do not work for high speeds is that the autocorrelation is below 90% even for

Fig. 3 MSE OF THE PREDICTION ERROR FOR THE LP PREDICTOR FOR R AYLEIGH FADING .

a lag of 1 ms, as can be seen in Figure 1, expressing the fast variations of the signal within short time intervals. Under such condition, the heuristics cannot perform well since they assume a channel behaviour with little or no changes. In the same figure, we can see that the autocorrelation for pedestrian speed (6 km/h) is greater than 90% up to 7.67 ms and greater than 99% up to 2.39 ms, which means that the channel varies very little within those horizons. This is why OS performs better than MA: the first assumes the channel does not change. For horizons up to 3 ms, the LP with N=2 performs better than the reference scheduler MC (see Fig. 4-a), due to the low variability of the channel within this horizon (recall that the autocorrelation is greater than 98%), however, for h > 3 ms, the MSE grows quickly to values close to 0. All in all, for prediction horizons greater than 3 ms, the MC is the only predictor that is accurate enough up to 12 ms. B. For WLAN measurement traces Although the measurement environments differed, the accuracy of the predictors changes only minimally from one environment to the other. This is because, although for bigger timescales things look different, within timescales of 15 ms all environments look similar due to the low mobility, with only very little changes in the received signal. The results per scenario are available at http://www.tkn.tu-berlin.de/∼aaguiar/prediction /prediction.html. We studied the influence of the number of channel samples used

10

10

0

5

NMSE[dB]

MSE[dB]

-10 -20 -30

-50 0

2

4

6 8 10 Horizont [ms]

12

-5 MC P=5 MC P=10 MC P=15 MC P=20 MC P=30 MC P=40

-10

OS MA N=5 LP N=2 MC

-40

0

-15 14

-20

(a) 6 km/h

0

10

VARYING

-10 MSE[dB]

4

6 8 10 Horizon [ms]

12

14

Fig. 5 C OMPARISON OF THE NMSE OF THE REFERENCE PREDICTOR MC FOR

0

AR MODEL ORDER AVERAGED OVER ALL MEASURED WLAN CHANNEL TRACES .

-20 -30 OS MA N=5 LP N=20 MC

-40 -50 0

2

4

6

8

10

12

14

Horizont [ms]

(b) 50 km/h 10 0 -10 MSE[dB]

2

-20 -30 OS MA N=5 LP N=20 MC

-40 -50 0

2

4

6 8 10 Horizont [ms]

12

14

(c) 100 km/h Fig. 4

C OMPARISON OF THE MSE OF THE PREDICTORS FOR R AYLEIGH FADING .

NMSE does not increase with prediction horizon. We also varied the order of the AR model for the reference predictor MC, and found that no significant differences were seen for p ≥15 and small h—Figure 5. The results also show that P=20 leads to the lowest NMSE, although for h>5 ms the NMSE becomes huge due to error propagation. Figure 6 shows the NMSE of the proposed heuristics and the reference predictor when the NMSE is calculated over all samples in all measurement scenarios. The MC predictor is no longer usable with the measured channel traces, showing the highest NMSE for all horizons, and raising to values greater than 0 dB for h > 5 ms, in the best configuration. This is due to problems tracking the coefficients of the AR model because of noisy samples, which lead to an inaccurate prediction for small h. Since prediction for farther horizons is obtained by cascading linear predictors, the error propagates leading to huge errors for big h. For h=1 ms, the OS or LP perform best due to the channel stability within this time interval. This is only true for this shortest prediction horizon. Otherwise, the OS heuristic, which performed good for Rayleigh fading, produces inaccurate prediction due to the noisy samples. Also the LP shows bad performance because, due to the noise, the predicted signal for big values of N is close to the average. The highest accuracy is achieved by the MA heuristic, in most cases with N=30. Since the channel does not change much in the interval in consideration, and taking that the samples are noisy, the moving average with many samples is intuitively the best way to predict the channel. However, this also implies that for h>1 ms, the best achievable prediction is the average received signal.

V. C ONCLUSIONS in the prediction for the MA and LP predictors, but we do not show the plots for lack of space. Since the measured channels change more sharply than Rayleigh channels due to noise, the LP with few samples leads to high gradients, that lead to the big prediction errors for h>2 ms. When the regression uses many samples, the output of the LP varies only little around the average signal value. All in all, the MA and LP produce lower NMSE the more samples N they use, but they only predict the average channel value, and not the variation. The low NMSE is due to the variations being of small amplitude. Since the predicted value is close to the average signal value, the

We presented an evaluation of the accuracy of three heuristics for prediction of wireless channel behaviour and compared them to a reference prediction algorithm—the MC predictor that has the best performance in the only comparative study that we know of. The accuracy is measured in terms of the mean squared value of the prediction error, and we evaluated it through simulation for both theoretical Rayleigh fading and measured channel traces in WLAN environments. The results for Rayleigh fading show that the reference predictor only works for low mobility. Further, although it performs best for Rayleigh fading at 6 km/h, it makes big errors for the measured WLAN channels due to error propagation. Of all

10

NMSE[dB]

5 0 -5 -10 OS MA N=30 LP N=30 MC P=20

-15 -20 0

2

4

6 8 10 Horizon [ms]

12

14

Fig. 6 C OMPARISON OF THE NMSE OF THE PREDICTORS FOR THE MEASURED WLAN CHANNEL TRACES (NMSE OVER ALL SCENARIOS ).

heuristics, the linear prediction performs best with Rayleigh channels; however, for the measurement traces, the moving average has the highest accuracy. Although we did not measure all possible WLAN environments, our measurements cover a wide range of WLAN scenarios. In all of them, the more complex reference predictor MC showed very poor accuracy, whereas the moving average over 30 ms heuristic produces less errors. According to these results, in WLAN scenarios, especially for link layer channel-adaptive mechanisms usually implemented in the driver like scheduling, a heuristic is a more accurate and simple channel predictor than the MC: for time horizons up to 1 ms the OS should be used, otherwise a moving average over 30 ms. .

R EFERENCES [1] A. Aguiar and J. Klaue. Bi-directional wlan channel measurements in different mobility scenarios. In Proc. Vehicular Technology Conference (VTC Spring), Milan, Italy, May 2004. [2] A. Aguiar and J. Klaue. Bi-directional wlan channel measurements in wlan environments with mobility. Technical Report TKN-04-002, Telecommunication Networks Group, Technische Universit¨at Berlin, Apr. 2004. [3] P. Bhagwat, P. Bhattacharya, A. Krishna, and S. Tripathi. Using channel state dependent packet scheduling to improve tcp throughput over wireless lans. Wireless Networks Journal, vol. (3)1, 1997. [4] T. Ekman, M. Sternad, and A. Ahl¡E9¿n. Unbiased power prediction on broadband channels. In Proceedings of IEEE Vehicular Technology Conference 2002 — VTC 2002 Fall, Sep 2002. [5] T. Eyceoz, S. Hu, A. Duel-Hallen, and H. Hallen. Adaptive prediction, tracking and power adjustment for frequency non-selective fast fading channels. In Communication Theory Mini-Conference, pages 1–5, 1999. [6] C. Fragouli, V. Sivaraman, and M. Srivastava. Controlled multimedia wireless link sharing via enhanced class-based queueing with channelstate-dependent packet scheduling. In Proc. of IEEE INFOCOM. IEEE, 1998. [7] J. Gomez, A. Campbell, and H. Morikawa. The havana framework: Supporting application and channel dependent qos in wireless networks. In Proceedings of the 7th International Conference on Network Protocols (ICNP’99), Nov 1999. [8] S. Haykin. Adaptive Filter Theory, 3rd Ed. Prentice Hall, 1996. [9] S. Hu, A. Duel-Hallen, and H. Hallen. Long-range prediction of fading signals. IEEE Signal Processing Magazine, 17(3):61–75, May 2000. [10] Intersil Corporation, 4243-3 Medical Drive, San Antonio, TX. PRISM Driver Programmer’s Manual (Version 2.10), Aug 2001. [11] W. C. Jakes. Microwave Mobile Communications. IEEE Press, Wiley Interscience, 1994.

[12] J. M. ([email protected]). Host AP driver for Intersil Prism2/2.5/3. http://hostap.epitest.fi/. [13] J. Klaue and A. Aguiar. Robust real-time channel prediction based on inaccurate instantaneous measurements: an approach. In Proc. of 6th IEEE International Workshop on Signal Processing Advances for Wireless Communications (SPAWC), New York, NY, USA, June 2005. [14] R. Knopp and P. Humblet. Information capacity and power control in single cell multiuser communications. In Proc. of IEEE Int. Conf. on Communications, pages 331–335, Jun 1995. [15] S. Lu, T. Nandagopal, and V. Bharghavan. Design and analysis of an algorithm for fair service in error prone channels. ACM/Baltzer Wireless Networks Journal, Vol. 6(4), pp. 323-343, Aug 2000. [16] T. S. E. Ng, I. Stoica, and H. Zhang. Packet fair queueing algorithms for wireless networks with location-dependent errors. In Proceedings of the 17th Annual Joint Conference of the IEEE Computer and Communications Societies — INFOCOM ’98, volume 3, pages 1103– 1111, Mar 1998. [17] S. Semmelrodt and R. Kattenbach. Performance analysis and comparison of different fading forecast schemes for flat radio channels. COST Technical Report, Jan 2003. [18] D. Tse. Multiuser diversity in wireless networks: Smart scheduling, dumb antennas and epidemic communication. Presentation at IMA Wireless Workshop, Aug 2001. [19] S. Vishwanath and A. Goldsmith. Adaptive turbo-coded for flat-fading channels. IEEE Trans. on Communications, 51(6):964–972, Jun 2003.