Abstract
Communicationbased train control (CBTC) network is an automated control network for railways that ensures the safe operation of rail vehicles using data communications. CBTC networks have stringent requirements for communication availability and latency. Wireless local area network (WLAN) is a popular choice in CBTC networks due to the available commercialofftheshelf WLAN equipments. However, handoffs in WLANs may result in communication interruption and long latency in WLANsbased CBTC networks. In this article, we propose a handoff management scheme for CBTC networks using stream control transmission protocol (SCTP) and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks. We formulate the handoff decision problem as a stochastic SemiMarkov Decision Process (SMDP) with the objectives of minimizing the handoff latency and maximizing the SCTP throughput. Simulation results are presented to show that the proposed scheme can significantly improve the handoff performance in CBTC networks.
Keywords:
Communicationbased train control (CBTC); Handoff; Stream control transmission protocol (SCTP); IEEE 802.11p wireless local area networks (WLANs)Introduction
Communicationbased train control (CBTC) network is an automated control network for railways that ensures the safe operation of rail vehicles using data communications [1]. CBTC is based on two important technologies that marked profoundly the development of our society in the last century: railways and communication technologies. It is a modern successor of traditional railway signaling systems that provide a limited control using track circuits, interlockings and signals. In most CBTC networks, data between trains and trackside equipments are transferred bidirectionally by wireless communication networks, such as global system for mobile communicationsrailway (GSMR) and wireless local area network (WLAN). For urban mass transit systems, WLAN is a better choice due to the available commercialofftheshelf equipments [2]. WLANbased CBTC has been deployed in many real systems, such as New York City Canarsie Line, Beijing Metro Line 10 from Siemens [3], and Las Vegas Monorail from Alcatel [4]. We will focus on WLANbased CBTC networks in this article.
Communicationbased train control networks have stringent requirements for wireless communication availability and latency [5]. Whereas in commercial wireless networks, less service availability and long latency mean less revenues or/and poor quality of services (QoSs) [6]; in CBTC networks, less service availability could cause train derailment, collision or even catastrophic loss of life or assets. Therefore, it is important to ensure the wireless communications are available when they are needed, and the latency is minimized in CBTC networks. Furthermore, in recent years, there have been significant developments of high speed train systems around the world (e.g., China railway highspeed (CRH) systems with the maximum speed of 352 km/h [7]), which introduce new nontrivial challenges to the CBTC designs in the high speed environment.
Most existing WLANbased CBTC networks are using traditional IEEE 802.11 technologies [8], such as 802.11a/b/g. However, IEEE 802.11a/b/g WLANs were not originally designed for high speed environments. Particularly, when a train moves away from the coverage of a WLAN access point (AP) and enters the coverage of another AP along the railway, a handoff procedure occurs, and this handoff process may result in communication interruption and long latency. The handoff procedure can be divided into four steps, namely probing (also referred to as scanning), channel switching, authentication and association. This whole procedure may take up to several hundreds milliseconds [9].
There are several schemes proposed in the literature to decrease WLAN handoff latency. Fitzmaurice [9] and Mishra et al. [10] have shown that over 90% of the time in the handoff process is spent in the scanning stage. Therefore, most of previous works in optimizing WLAN handoffs focus on making the scanning process more efficient. A SyncScan technique is proposed in [11], in which appropriate time synchronization is required between APs and clients. A topology inferencing technique in both clients and APs is proposed in [12] to improve the scanning process. A cooperative handoff framework is proposed in [13] to utilize mechanism for information sharing to reduce the delays during the scanning/probe phases. In [14], a fast handoff scheme that skips all mentioned stages is proposed, where handoff is controlled and prepared by the access network and is triggered by sending a hop request message to the mobile station (STA). There are some schemes using multiradio in mobile clients trying to reduce the WLAN handoff latency. Adya et al. [15] proposed a protocol to allow multiradio mobile nodes in a mesh network to potentially establish two separate wireless links between a pair of nodes. This work primarily focuses on improving efficiency of wireless mesh networks, which is different from the CBTC networks considered in this article.
It is necessary to look at the handoff management at multiple layers of the protocol stack, not just at the data link layer as considered in the past [16]. Indeed, the handoff management problem can be solved at transport layer [1720]. For example, stream control transmission protocol (SCTP) [21,22], a new IETFstandardized transport layer protocol in addition to transmission control protocol (TCP) and User Datagram Protocol (UDP), can be used to solve the handoff management problem. The multihoming, multistream and partial reliable [23] data transmission features of SCTP are especially attractive for applications that have stringent performance and high reliability requirements. Compared to other handoff management approaches, transport layer schemes have many advantages, such as improved throughput and latency performance. Moreover, no third party other than the endpoints participates in handoff process, and no modification or addition of network components is required, which makes transport layer approaches attractive in WLANbased CBTC networks, where commercialofftheshelf equipments are widely used.
Although some works have been done for the handoff management in CBTC networks, most of them are focused on handoff protocols and network architectures, and handoff decision policy issues (i.e., when to execute handoff) are largely ignored in CBTC networks. However, due to the high mobility environment, as well as the high availability and low handoff latency requirements, handoff decision policy issues are very important in designing CBTC networks, which will significantly affect the overall system performance.
In this article, we study the handoff decision policy issues in CBTC networks based on SCTP and IEEE Std 802.11p2010 WLANs [24], which is an emerging technology for vehicular communication networks. To the best of our knowledge, the design of handoff management in CBTC networks based on SCTP and IEEE 802.11p WLANs has not been done in previous works. The distinct features of the proposed scheme are as follows.
· We propose a handoff management scheme based on SCTP and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks.
· We formulate the handoff decision problem as a stochastic SemiMarkov decision process (SMDP) [25], which has been successfully used to solve finance [26] and admission control [27] problems, among others. This article focuses on the application of SMDP to the handoff decision problem in CBTC networks.
· Minimizing the handoff latency is one of the objectives in the proposed scheme. In addition, since multimedia information, such as train schedule, weather forecast, live news, sports and finance, is more and more popular in railway communication networks [28], we also consider maximizing the SCTP throughput in our scheme.
· Extensive simulation results are presented. It is illustrated that the proposed scheme can significantly decrease the handoff latency and improve SCTP throughput in CBTC networks.
The rest of this article is organized as follows. The 802.11p and SCTP based CBTC network with the corresponding handoff procedure is presented in Section “The proposed CBTC network based on SCTP and IEEE 802.11p”. The SMDP based handoff decision model, optimality equation, and value iteration algorithm are described in Section “Optimal Handoff the CBTC network using SCTP and IEEE 802.11p WLANs”. Some implementation issues are given in Section “Implementation issues”. Simulation results are presented and discussed in Section “Simulation results and discussions”. Finally, we conclude our study in Section “Conclusions and future work”.
The proposed CBTC network based on SCTP and IEEE 802.11p
In this section, we first present an overview of CBTC. Then, we introduce IEEE 802.11p WLANs and SCTP. The proposed CBTC network based on SCTP and IEEE 802.11p WLANs is also presented in this section.
Overview of CBTC
Figure 1 describes a CBTC network. In this network, continuous bidirectional wireless communications between each station adapter (SA) on the train and the wayside AP are adopted instead of the traditional fixedblock track circuit. The railway line is usually divided into areas or regions. Each area is under the control of a zone controller (ZC) and has its own radio transmission system. Each train transmits its identity, location, direction and speed to the ZC. The radio link between each train and the ZC should be continuous so that the ZC knows the locations of all the trains in its area at all the time. The ZC transmits to each train the location of the train in front of it and gives it a braking curve to enable it to stop before it reaches that train. Theoretically, as long as each train is traveling at the same speed and they all have the same braking capability, they can travel together as close as a few meters in between them.
Figure 1. A communicationbased train control (CBTC) network.
When a train moves away from the coverage of an AP and enters the coverage of another AP along the railway, the handoff procedure may result in communication interruption and long latency. In CBTC networks, it is important to maintain communication link availability in order to guarantee train operation safety and efficiency. To this end, we present a handoff management scheme based on SCTP at transport layer and IEEE 802.11p at data link and physical layers to provide high link availability in CBTC networks. Brief introductions of IEEE 802.11p and SCTP are given in the next two subsections.
IEEE 802.11p
IEEE Std 802.11p2010, also known as Wireless access in vehicular environment (WAVE), is an amendment to the IEEE Std 802.112007 standard that adds applications to fast changing vehicular networks [24].
It deals essentially with the data link and physical layers of the OSI model. The medium access control (MAC) protocol in IEEE 802.11p uses the enhanced distributed channel access (EDCA) mechanism originally provided by IEEE Std 802.11e2005 [29], while the physical layer is a variation of the OFDM based IEEE Std 802.11a1999 standard, with a 10 MHz wide channel instead of the 20 MHz one usually used by IEEE 802.11a devices.
The focus of IEEE 802.11p lies on fast adaptation to rapid changes occurring in a highly mobile vehicular network, sacrificing identification and authentication procedures that are usually part of the IEEE 802.11 WLAN standards. For more efficient data exchange between high speed vehicles or between a vehicle and a Road Side Unit (RSU), IEEE 802.11p specifies a minimized set of parameters for the execution phase of the handoff process [30].
Stream control transmission protocol
Stream control transmission protocol is a transport layer protocol, which inherits many of the core features of TCP such as congestion control and reliability [21]. It also includes enhancements over TCP. The multihoming feature enables an SCTP session to be established over multiple interfaces identified by multiple IP addresses. SCTP normally sends packets to a destination IP address designated as the primary address, but can redirect packets to an alternate secondary IP address if the primary IP address becomes unreachable. Accordingly, the path between two SCTP hosts using their primary addresses is the primary path, and a path between two SCTP hosts using one or more secondary addresses is a secondary path. While only one primary path exists between two SCTP hosts, more than one secondary paths can be available. The set of available connecting paths forms an SCTP association. An SCTP association between two hosts, say, A and B, is defined as:
Any of the IP addresses at either host can be used as the corresponding source or destination address in an IP packet sent by one host to the other. Before data can be exchanged, the two SCTP hosts must exchange the set of available IP addresses in the association establishment stage. The mobile extension of SCTP (mSCTP) enables the endpoints to dynamically add, delete, or change the IP addresses during an active SCTP association [17]. The multihoming mechanism was originally designed for faultresilient communications between two SCTP endpoints over wired networks. This powerful feature can be used to design a handoff management scheme in CBTC networks.
The proposed CBTC network based on SCTP and IEEE 802.11p WLANs
Figure 2 describes the network architecture and protocol stack of the proposed CBTC network based on SCTP and IEEE 802.11p. There are two radios in the SA on a train. Each radio is related to a different MAC interface and IP interface. The ZC has only one IP interface. Normally only one pair of IP addresses (the IP address in the ZC and the IP address in the SA) is active, which is called the primarily path. When a handoff is triggered, the SA on the train will try to exchange necessary information with the ZC to establish another path with another radio, which is called the secondary path. In this article, we do not use standard SCTP, since standard SCTP can result in a long interruption time during a handover [17]. Instead, inspired by mSCTP [17] and MMPSCTP [19], we let the communication nodes copy all the buffered data from the primary path to the second path so that the data in sendingbuffers of the two paths are completely the same. The SA also needs to cut off the primary path to finish the handoff process at an appropriate time. The whole handoff procedure is shown in Figure 3.
Figure 2. Network architecture and protocol stack of the proposed trainground communication network based on SCTP and IEEE 802.11p.
Figure 3. The proposed handoff procedure.
A critical issue in the above network is the handoff decision policy, i.e., when to perform handoff. In high speed environments, wireless channels are changing dynamically in CBTC networks. The communication QoS is not simply determined by the geolocation of the train and AP. If the handoff decision policy is not designed carefully, communication interruption, long latency and low throughput may occur, which will significantly affect the performance of a CBTC network. Therefore, an efficient handoff decision policy is needed to decide at what time the second path should be established and when to cut off the primary path, which will be studied in the following sections.
Optimal Handoff decision policy in the CBTC network using SCTP and IEEE 802.11p WLANs
In this section, we first present an overview of SMDP modeling. Then, the states, actions, reward functions, state transition probability, constraints, optimality equations, and value iteration algorithm in the CBTC system are presented.
SMDP modeling
In this article, the handoff decision problem in the CBTC network presented above is formulated as an SMDP [25]. Markov decision process (MDP) provides a mathematical framework for modeling decisionmaking in situations where outcomes are partly random and partly under the control of a decision maker. MDP has been successfully used in heterogeneous wireless networks [31]. Besides the basic features, an SMDP generalizes a MDP by allowing decision maker to choose actions whenever the system state changes and allowing the time spent in a particular state to follow an arbitrary probability distribution. In this article, we use SMDP to solve the handoff decision problem in CBTC networks using SCTP and IEEE 802.11p. The optimal handoff decision policy can be obtained from the value iteration algorithm in this formulation. In our proposed CBTC network, the SA on the train makes handoff decisions at specific time instances according to the current state s(t), and the system moves into a new state based on the current state s(t) as well as the chosen decision a(t). Given s(t) and a(t), the next state is conditionally independent of all previous states and actions. This Markov property of state transition process makes it possible to model the handoff problem as an SMDP.
An SMDP model consists of the following five elements: (1) decision epochs, (2) states, (3) actions, (4) rewards, and (5) transition probabilities, which will be described in the following.
The SA on a train has to make a decision whenever a certain time period has elapsed. The instant times are called decision epochs.
Let S and A be the state space and action space, respectively. Given the current state s(t)∈S and the chosen action a(t)∈A, the state transition probability function for the next state s(t + 1) is denoted as P[s(t + 1)s(t),a(t)]. This function is Markovian because the state transition probability depends on the current state and action but not on the previous states.
A decision rule prescribes a procedure for action selection in each state at a specified decision epoch. Markov decision rules are functions δ(t) : S → A, which specify the action choice a(t) when the system occupies state s(t) at decision epoch t. A policy Π = (δ(1),δ(2),…,δ(t)) is a sequence of decision rules to be used at all decision epochs.
Let υ^{Π}(s(0)) denote the expected total reward from the first decision epoch until the handoff decision period elapses, given that the policy Π is used with an initial state s(0). We have
where r(s(t),a(t)) is the reward function, denotes the expectation with respect to policy Π and initial state s(0), and E_{N}denotes the expectation with respect to random number N. The sequence T = {1,2,…,N} represents the times of successive decision epochs. Let the time between two successive epochs be τ. The product of random variables N and τdenotes the time that the train stays between two successive APs. We refer that time as the handoff decision period. The random variable N, which depends on the AP deployment space, train speed and the time between successive decision epochs, is assumed to be geometrically distributed with mean 1/(1 − λ).
Given N geometrically distributed with mean 1/(1 − λ), according to [25], (1) can be rewritten as
where λ can also be interpreted as the discount factor of the model, and 0 ≤ λ ≤ 1, denotes the expectation with respect to policy Π and initial state s(0).
Our optimization problem is to maximize the expected total discounted reward. We define a policy Π^{∗} to be optimal if . A stationary policy has the form . For convenience, we simply denote Π by δ. Our objective is to determine an optimal stationary policy δ^{∗}, which maximizes the expected total discounted reward given by (2).
In order to obtain the optimal solution, it is necessary to identify the states, actions, reward functions, state transition probability and constrains in our SMDP model.
Action and state
In our SMDP model, at each decision epoch, the SA on the train has to decide whether the connection should use the current chosen AP or connect to the next AP (we assume the SA on the train will not be in the coverage of three successive APs). We assign every AP along the railway with a distinct number. Let M be the AP that covers the SA, then the other one is M + 1. According to our handoff scheme, we define the action space as A = {M,M + 1,M∗(M + 1)}. An action a(t) ∈ A is defined as follows.
(1) If a(t) = M, the SA communicates with AP M;
(2) If a(t) = M + 1, the SA communicates with AP M + 1;
(3) If a(t) = M ∗ (M + 1), the SA communicates with both AP M and AP M + 1.
Each state s(t) = [ γ_{1},γ_{2},η,ξ ] ∈ S has the following information:
(1) The measured signaltonoise ratio (SNR) from two APs, γ_{1} and γ_{2};
(2) The current SCTP congestion window, η;
(3) The path(s) currently used by the SA, ξ;
Reward function
In our formulation, we define the reward function as
where f(s(t)) reflects the QoS provided by the chosen path(s) at epoch t, and g(s(t),a(t)) captures the cost under state s(t) and action a(t).
Given the current state s(t) and the chosen action a(t), f(s(t)) is defined as
where f_{b}(s(t)) is the SCTP throughput, f_{d}(s(t)) is the SCTP packet delay, α and β are two independent dimension weight factors to make the SCTP throughput and packet delay comparable, ϕ and (1 − ϕ) are importance weight factors to indicate the importance of SCTP throughput and packet delay. In (4), we combine SCTP throughput and delay into a single function. This is a common approach used in the optimization literature, which is called aggregate objective function (AOF), to solve an optimization problem with multiple objectives [32,33]. In reality, different CBTC networks have different throughput and packet delay requirements. By adjusting the parameters in (4), the proposed scheme is generic enough to accommodate different requirements in real CBTC networks.
Stream control transmission protocol throughput and packet delay will be derived later in Section “SCTP throughput and packet delay”
The cost function g(s(t),a(t)) under the current state s(t) and the chosen action a(t) is defined as
where ξ is the currently used path, K is the multipath penalty when the SA is working in the multipath mode, which is mostly caused by the interference of two wireless links when they are working simultaneously.
When the action is a single path action and no handoff happens, the penalty is 0. The penalty will change to be K when the current action is a multipath policy. As shown in Figure 3, there are information exchanges between the SA and AP when handoff is triggered and finished. We assume the reward for these actions to be zero since most of the bandwidth is occupied by the communication overhead. The cost for these actions is f(s(t)).
SCTP throughput and packet delay
In this section, we derive SCTP throughput and packet delay, which are used in (4).
Orthogonal frequency division multiplexing (OFDM) is adopted as the physical technology in 802.11p. There are several transmission speeds in 802.11p physical layer. A communication node will choose a transmission speed according to adaptive modulation and coding. The SNR can be partitioned into a number of consecutive nonoverlapping intervals with boundary points obtained by rearranging the rate expression [34]
where C is the achievable data rate, W is the OFDM subchannel bandwidth in 802.11p, BER is the bit error rate.
To get BER from a specific SNR under fixed data rates, (6) can be rewritten as
We choose this capacity formula in our formulation because it has been commonly used in the literature, and has reasonable accuracy [35,36]. Nevertheless, our proposed SMDP model is not dependent on a particular capacity formula. If there is a new physical layer technology available for WLANs in the future, a different capacity formula for this new physical layer technology can be used in our formulation as well.
When the SA measures the SNRs from two APs, the corresponding BER can be obtained from (7), and the corresponding frame error rate (FER) is derived as
where L_{fr} is the frame length in bits which can be considered approximately equal to the SCTP chunk size.
As we introduced in section “The proposed CBTC network based on SCTP and IEEE 802.11p”, the MAC protocol in IEEE 802.11p uses the EDCA mechanism originally developed in IEEE 802.11e. In EDCA, a window based backoff mechanism is used such that a node willing to transmit will sense the medium first, and if the medium is not free it will choose a backoff time uniformly at random from the interval [0,CW + 1], where CW is Contention Window and the initial value equals to . The CW will be doubled if the subsequent transmission attempt fails until it reaches .
With packet losses, when a packet is transmitted n times at MAC layer, the corresponding packet delay T_{delay11p}(n) can be ****calculated as follows [37].
where T_{aifs} is the Arbitration Inter Frame Space (AIFS), T_{data} is the time needed to transmit a data frame, T_{sifs} is the Short Inter Frame Space (SIFS), T_{ack} is the time needed to transmit the ACK frame, T_{backoff(i)} is the backoff time of the retransmission at i times, and T_{transfer} is the propagation time of the data. We need to point out that, in (9), an acknowledgment is actually sent only during the last T_{ack}, while no acknowledgement is sent in the previous T_{ack}. The client will wait for T_{ack} even there is no acknowledgment actually sent when a transmission fails.
Therefore, the average 802.11p packet transmission delay T_{11p} with maximum retransmission time R is given by
Given the constant delay T_{wired} in the wired network, finally we get the SCTP packet delay
The wireless delay T_{11p}, which is described in (10), is determined by the SNR in the state parameter s(t), as well as the current action a(t).
Then the instant SCTP throughput can be obtained as
where η is the current congestion window in state parameter in s(t), and RTT is the roundtrip time which is approximately 2∗f_{d}(s(t)).
State transition probability
Given the current state s(t) = [ γ_{1},γ_{2},η,ξ ] and the chosen action a(t), the probability function of the next state is given by
Here, for simplicity of formulation and presentation, we assume that the wireless channels γ_{1},γ_{2}, SCTP congestion window η and currently used path ξare independent. This assumption is reasonable in practice, because the two wireless channels from two APs are independent, and the currently used path is solely determined by the last action. Moreover, the channels change much faster than the SCTP congestion window size, which makes it reasonable to assume that the SCTP congestion window size and the channels are independent. Then, we have
where and are the channel state transition probabilities for the two wireless links, respectively, P[η^{′}  η] is the SCTP congestion window state transition probability, and P[ξ^{′}  ξ,a(t)] is the currently used path transition probability. These state transition probabilities will be derived in the following.
Channel state transition probability
In CBTC networks, due to the restrictions of railways and trains, the ground antenna cannot be installed too high. The low antenna height means that the Fresnel Zone typically limits the propagation of radio to fairly short ranges. Green and Obaidat [38] propose an equation for RF propagation close to the ground for 5.8 GHz frequency [38],
where P_{loss} is the path loss, h_{t} and h_{r} are the heights of the transmitting and receiving antennas, respectively, and d is distance between the train SA and the AP. Combined with largescale path loss and smallscale fading (Rayleigh distribution is used to describe the fading envelope), we get the received SNR γ as
where P_{t} is the transmitted power, ϑ is a Gaussian random variable with a variance of ς and a mean of 0, χ is a Rayleigh random variable with a mean of 1, G_{t} and G_{r}are the antenna gains for the transmitter and receiver, respectively, and P_{noise} is the noise power.
In this article, we use finitestate Markov channel (FSMC) models in CBTC networks. FSMC models have been widely accepted in the literature as an effective approach to characterize the correlation structure of the fading process, including satellite, indoor, Rayleigh fading, Ricean fading, and Nakagami fading channels [3945]. Considering FSMC models may enable substantial performance improvement over the schemes with memoryless channel models [46].
In FSMC models, the range of the received SNR can be partitioned into discrete levels. Each level corresponds to a state in the Markov chain.
Assume there are L states in the model. Let i and κdenote the instantaneous channel state and SNR, respectively. When the channel is in state i, the corresponding SNR is κ_{i}. Then we have κ_{i} < κ < κ_{i + 1}, 0 ≤ i ≤ L − 1. The probability of transition from state i to state j in the Markov model is channel transition probability.
In real networks, the values of the above transition probability can be obtained from the history observation of the CBTC network.
SCTP congestion window transition probability
In order to derive the congestion window transition probability, we refer to the SCTP behavior model in [18]. The SCTP behavior is modeled in terms of “rounds”, where a round starts when the sender begins the transmission of a window of chunks and ends when the sender receives the last acknowledgment for chunks in this window. SCTP doubles its congestion window size in the slowstart stage when the current congestion window, denoted by η, is less than a threshold, denoted by _{ηth}. SCTP increases the congestion window linearly in the congestion avoidance stage if none of the chunks in the previous window is lost during the previous RTT. If one or more chunks in the previous window are lost, the congestion window is set to half of the current window. When the maximum congestion window, denoted by , is reached, the congestion window will not increase if no chunks get lost in the last round. The congestion window will not change until all the chunks in the window are sent out.
Given the current congestion window η, when SCTP is in slowstart, the congestion window in the next epoch can be η, η/2, or 2η. For the congestion avoidance stage, the congestion window can be changed to η, η/2, or η + 1, which depends on the decision epoch and packet losses. We derive the congestion window state transition probability as
where τ is the time between two successive decision epochs, q is the probability that a chunk is successfully received, which can be calculated as, q = 1 − FER^{n}, where FER is the FER obtained in (8), and n is the number of WLAN retransmissions. Particularly, when the current congestion window size is one, it will not change even if the packet is lost. We then derive the probability that the congestion window size transitioning from one to one as follows.
Currently used path transition probability
Because the next used path is completely determined by the chosen action, the currently used path transition probability is simply derived as
Constraint
As described in Section “The proposed CBTC network based on SCTP and IEEE 802.11p”, SA first needs to establish a new path before cutting the old path when a handoff happens. Therefore a constraint related to action decision is needed. Given the currently used path ξ, when ξ is in a single path, the chosen action should not be the other single path action.
Optimality equations and value iteration algorithm
The optimality equation for an SMDP is given by [25]
where υ(s) denotes the maximum expected total reward, given the initial state s, and s^{′}represents next state. That is
The solutions of the optimality equation correspond to the maximum expected total reward υ(s) and the SMDP optimal policy δ^{∗}. Note that the SMDP optimal policy δ^{∗} indicates the decision as to which action to choose.
There are various algorithms available to solve the optimization equation [25]. We use a value iteration algorithm in this article to determine a stationary optimal policy and the corresponding expected total reward. The algorithms is described as follows.
· Set υ^{0}(s) = 0 for each state s. Specify ε > 0, and set k = 0;
· For each state s, compute υ^{k + 1}(s) by
· If υ^{k + 1}(s) − υ^{k}(s) < ε(1 − λ)/2λ, go to step 4). Otherwise, increase k by 1 and return to step 2).
· For each s ∈ S, compute the stationary optimal policy
The value iteration algorithm is proved to be an efficient and stable iteration algorithm [25]. The algorithm operates by calculating successive approximation to the value function υ(s). The computation complexity of the algorithm is O(AS ^{2}), where A is the action set, and S is the state space [25].
Implementation issues
In this section, we briefly explain how to implement our proposed handoff decision algorithm.
In order to determine the optimal handoff decision policy δ^{∗}, we need to measure and estimate the parameters in the SMDP model. In constructing these parameters, we assume that most properties in the network can be made known, which should be realistic particularly for CBTC networks where initial planning and network management is a crucial priori requirement. The multipath penalty K in (5) can be determined by the wireless interface performance. We can measure this parameter by comparing the multiradio performance with the single radio performance. The weight factor ϕ in (4) can be set according to the CBTC networks’ requirements. The handoff decision time space (i.e., discount factor λin (2)) can be estimated based on the AP locations and train moving speeds. The channel state transition probability can be estimated by field measurements. Given the values of all the parameters, the value iteration algorithm described above can be used to derive the optimal handoff decision policy δ^{∗}. The calculation of the optimal policy is performed offline and should be updated whenever the system parameters are changed.
Once the optimal policy is obtained, it can be stored in a table format. Each entry of the table specifies the optimal action (handoff decision), given the current state (i.e., channel state, currently used path and SCTP window size). For the online process, at each decision epoch, each SA on the train lookups the table to find out the optimal action corresponding to its current state, and then executes the handoff decision. Online looking up tables can be designed with little computational complexity in practice.
Simulation results and discussions
In this section, simulation examples are used to illustrate the performance of the proposed handoff scheme based on SCTP and IEEE 802.11p. We use NS2.29 simulator in our simulations. The University of Delaware’s SCTP model, which has been merged into NS2, is extended so that the multihoming feature can work over wireless links. We consider a simulation scenario with a train moving between successive APs. The average distance between two successive APs is 600 m. The train speed is 80 km/h. The delay for the wired part of SCTP is 100 ms. There are four traffic classes defined in 802.11p. We only consider traffic class four in our model since there is only one kind of traffic in the CBTC system. The traffic class considered in the article has the lowest priority with the following parameters: T_{aifs} = 9us, and . We assume the IEEE 802.11p operates in service channel 174 and 175, and the combined channel provides a date rate of 18 Mbps. The parameters used in the simulations are shown in Table 1.
Table 1. Simulation parameters
We need to point out that our handoff scheme is not limited to train speed. The optimal handoff policy can be calculated only if we can measure the channel transition probability under specific train speed from field tests.
We compare the performance of the proposed scheme with that of the existing handoff scheme based on UDP and traditional IEEE 802.11a and two other heuristic handoff decision policies. The results show that the proposed scheme can significantly decrease the handoff latency, as well as improve SCTP throughput and the expected total reward in CBTC networks.
We first present the FSMC model derived from real field tests that were conducted around Yonghegong metro station in Beijing, China. Since the sample interval is much smaller in the field tests compared to the time between successive decision epochs, we average the measured SNR between successive decision epochs. The averaged received SNR is shown in Figure 4a. We divide the received SNR into four levels, , [15,20], [20,25], and , which are marked as 1, 2, 3, and 4, respectively, in Figure 4b.
Figure 4. The received SNR from two APs.
The channel state transition probability matrices for the two paths are
respectively.
Handoff delay improvement
Figure 5 shows the sequence numbers of the packets at the corresponding time on the X axis. From this figure, we can see that, in the traditional UDP and 802.11a based handoff scheme, the SA on the train has no communication with any AP for a period of time during the handoff process, and the packets are lost, which is unacceptable for realtime and highsafety applications. This is caused by the handoff delay in the traditional handoff scheme, in which the old path is broken before the new path is setup with the new AP.
Figure 5. Packet sequence number in different schemes.
By contrast, we can observe from Figure 5 that the proposed handoff scheme supports handoff between adjacent APs. During the handoff process, the SA associates to the new AP with the other radio, obtains a new IP address for the new path, communicates with the new address before the old path is terminated. No packet is lost during this handoff procedure. The small delay in the handoff process is caused by the information exchange before the establishment of the new path.
In addition, we illustrate the behaviour of normal SCTP in Figure 5. Instead of multicasting data over two paths during the handoff process, the lost packet is retransmitted on the second path for the normal SCTP behaviour. As shown in Figure 5, compared to our proposed handoff scheme, extra transmission latency occurs in the normal SCTP behaviour during the handoff process. This is because it takes time for the SCTP sender to get to know the packet is lost, and then to retransmit it.
Expected total reward and SCTP throughput improvement
In this section, we compare the performance of our proposed handoff decision policy with two other heuristic handoff decision policies. For the first heuristic policy, the path to be selected in each decision epoch is the one that always has the better SNR. For the second heuristic policy, we also model the handoff decision as an SMDP, but SCTP congestion window variation is not considered in this model.
Figure 6 shows the handoff procedures under the three policies with different multipath penalties K defined in (5). In this figure, the Y axis represents the current path: Y = 1 means that the SA is communicating with AP M; Y = 2 means that the SA is communicating with AP M + 1; Y = 3 means that the SA is communicating with both AP M and AP M + 1. With the increase of the multipath penalty, the proposed SMDP policy considering SCTP congestion window changes. When the multipath penalty is very small, the SA tends to stay in the multipath state to get the best expected total reward. The multipath state duration decreases rapidly with the increase of the multipath penalty. When the multipath penalty K = 0.05, the multipath state duration is very short during the handoff process. For the SMDP policy without considering SCTP congestion window, the policy is not so sensitive to the multipath penalty, because it does not have enough information to make an appropriate decision to get the best expected total reward. For the policy without using SMDP, as it does not consider the long term total expected reward, the pingpong handoff shows up in the handoff procedure, and the policy does not change when the handoff parameters change.
Figure 6. Handoff procedures under three policies. (When the current path is 1, the SA is communicating with AP M; When the current path is 2, the SA is communicating with AP M + 1; When the currently used path is 3, the SA is communicating with both AP M and AP M + 1.).
Figure 7 shows the expected total reward under three policies. We can observe that the SMDP policy considering SCTP congestion window always gives the best expected total reward compared to the other two policies. Figure 8 shows the expected total reward versus the weight factor ϕ, defined in (4), for the three policies. The SMDP policy considering SCTP congestion window gives the highest expected total reward for all different values of ϕ. As we can see, the reward improvement is more obvious with the increase of weight factor. This is because when ϕincreases, the SCTP throughput reward in (12) becomes more important than the SCTP packet delay reward in (11), the two heuristic handoff decision policies do not care about the SCTP congestion window, which is a very important factor for SCTP throughput.
Figure 7. The expected total reward under three policies with different multipath penalties.
Figure 8. The expected total reward under three policies with different weight factors.
The average SCTP throughput of different policies is shown in Figure 9. The SCTP throughput increases with the weight factor in our proposed policy. This is because the optimal policy is designed to maximize SCTP throughput. When the weight factor increases in (4), more emphasis is put on the throughput in the reward function. Similarly, the two heuristic policies have worse throughput performance compared to our proposed decision policy. Moreover, the throughput of the two heuristic policies does not change when the weight factor increases. This is because these two policies do not care about SCTP congestion window variation. We also observe that, when the weight factor decreases to zero, our proposed decision policy gives the same throughput performance as one of the heuristic policies.
Figure 9. The throughput under three policies with different weight factors.
Structure of the optimal policy
The SMDP optimal handoff decision policy δ^{∗} is numerically computed by implementing the value iteration algorithm. In our scheme, the state space S has 4 dimensions. To present the optimal policy clearly, one dimension needs to be fixed to a specific value. Therefore, in the following example, we fix the currently used path to be M, which means the SA is currently communicating with AP M. The structure of the optimal policy is shown in Figure 10. The cubes represent the handoff policy: When the cube’s height is 1, the SA does not trigger a handoff; When the cube’s height is 3, the SA makes a handoff decision and begins to work in the multipath mode.
Figure 10. Structure of the optimal policy. (The current path is M, which means the SA is currently communicating with AP M. The cubes represent the handoff policy: When the cube’s height is 1, the SA does not trigger a handoff; When the cube’s height is 3, the SA makes a handoff decision and begins to work in the multipath mode.).
We can observe from Figure 10 that when the congestion window changes from 1 to 32, the SA tends not to execute a handoff. This is because when the congestion window increases, even if the SNR is low, the throughput is acceptable. Compared to the multipath penalty caused by a handoff, making a handoff decision would not improve the expected total reward.
Conclusions and future work
Communicationbased train control networks using WLANs have stringent requirements for wireless communication availability and latency. In this article, we studied the handoff management issues in CBTC networks. We presented a CBTC network based on SCTP and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks. The handoff decision problem was modeled as a SMDP with the objectives of minimizing the handoff latency and maximizing the SCTP throughput. In simulation results, we showed that the handoff delay is very close to zero in our proposed handoff management scheme, and the proposed SMDP based handoff decision algorithm can significantly improve SCTP throughput. We also observed that both SCTP congestion window and the measured SNR from APs are important in making the handoff decisions.
We are currently implementing the proposed scheme in a real testbed to further evaluate the performance. We have a project “Channel modeling in metro tunnels and its applications” supported by the National Science Foundation of China (Project NO. 61132003). The testbed is in Beijing YiZhuang Line, which is a part of Beijing Urban Rail system. Moreover, we plan to extend the proposed model to multipleinput multipleoutput (MIMO) systems, where the tradeoff between multiplex and diversity will be considered in our model. Moreover, in our SMDP model, only transport layer, data link layer and physical layer were considered as crosslayer information. Extending our model to application layer is also our future work.
Competing interests
We declare that this paper has no competing interests.
Acknowledgements
We thank the reviewers for their detailed reviews and constructive comments, which have helped to improve the quality of this article.
This work was in part supported by Beijing Jiaotong University Technology Funding Project (NO.KWJB11082536, and NO.2011JBZ014), National Science Foundation of China (NO.61132003), Key Projects in State Key Lab. of Rail Traffic Control and Safety (No. RCS2011ZT007, NO. RCS2010ZT007, NO. RCS2010ZZ003), State Key Laboratory of Rail Traffic Control and Safety of Beijing Jiaotong University within the frame of the project (No. I11K00060), Beijing Science and Technology Star Project (2010B015), and by the Natural Sciences and Engineering Research Council (NSERC) of Canada and industrial and government partners, through NSERCDIVA Strategic Research Network.
References

RD Pascoe, TN Eichorn, What is communicationbased train control. IEEE Veh. Tech. Mag 4(4), 16–21 (2009)

M Aquado, E Jacob, P Saiz, JJ Unzilla, MV Hiquero, J Matias, Railway signaling systems and new trends in wireless data communication. Proc. IEEE VTC’2005Fall, ((Dallas, TX, Sept 2005), pp), . 1333–1336

R Lardennois, Wireless communication for signaling in mass transit. Siemens Transportation Systems (2003)

E Kuun, Open standards for CBTC and CCTV radio based communication. Technical Forums of Alcatel (2004)

F Whitwam, Integration of wireless network technology with signaling in the rail transit industry. Alcatel Telecommun. Rev 1(1), 43–48 (2003)

WF Wei, TL Sheu, A channel preemption model for vertical handoff in a WLANembedded cellular network. Wirel. Netw 16(4), 929–941 (2010). Publisher Full Text

H Dong, B Ning, B Cai, Z Hou, Automatic train control system development and simulation for highspeed railways. IEEE Circ. Syst. Mag 10(2), 6–18 (2010)

M Fitzmaurice, Use of wireless local area networks in rail and urban transit environments. Transport. Res. Record J. Transport. Res. Board 1916(1), 42–46 (2005). Publisher Full Text

A Mishra, M Shin, W Arbaugh, An empirical analysis of the IEEE 802.11 MAC layer handoff process. SIGCOMM Comput. Commun. Rev 33, 93–102 (2003). Publisher Full Text

S Shin, AG Forte, AS Rawat, H Schulzrinne, Reducing MAC layer handoff latency in IEEE 802.11 wireless LANs. Proceedings of the, Second International Workshop on Mobility Management and Wireless Access Protocols ((Philadelphia, USA, 2004), pp), . 19–26

I Ramani, I Ramani, Syncscan: Practical fast handoff for 802.11 infrastructure networks. Proc. IEEE Infocom’05, ((Seoul, Korea, Mar 2005), pp), . 675–684

M Shin, A Mishra, WA Arbaugh, Improving the latency of 802.11 handoffs using neighbor graphs. Proc.MobiSys’04, ((Boston, USA, June 2004), pp), . 70–83

G Athanasiou, T Korakis, L Tassiulas, An 802.11k compliant framework for cooperative handoff in wireless networks. EURASIP J. Wirel. Commun. Netw 2009, 14 (Article ID 350643) (2009) Publisher Full Text

B Jooris, A Schoutteet, F Vermeulen, I Moerman, Access network controlled fast handoff for streaming multimedia in WLAN. Proc. 16th IST Mobile and Wireless Communications Summit, ((Budapest,Hungary, July 2007), pp), . 15

A Adya, P Bahl, J Padhye, A Wolman, L Zhou, A multiradio unification protocol for IEEE 802.11 wireless networks. Proc. BROADNETS’04, ((San Jose, USA, Oct 2004), pp), . 344–354 PubMed Abstract

FR Yu, VWS Wong, JH Song, VCM Leung, HCB Chan, Next generation mobility management: an introduction. Wiley Wirel. Commun. Mobile Comput 11(4), 446–458 (2011). Publisher Full Text

L Ma, F Yu, V Leung, T Randhawa, A new method to support UMTS/WLAN vertical handover using SCTP. IEEE Wirel. Commun 11(4), 44–51 (2004). Publisher Full Text

L Ma, FR Yu, VCM Leung, Performance improvements of mobile SCTP in integrated heterogeneous wireless networks. IEEE Trans. Wirel. Commun 6(10), 3567–3578 (2007)

D Pareit, N Gheysens, T Van Leeuwen, I Moerman, W Van Brussel, W Torfs, P De Cleyn, C Blondia, QoSenabled Internetontrain network architecture: interworking by MMPSCTP versus MIP. Proc. ITST’07, (Jun 2007)

D Pareit, I Moerman, P Demeester, W Torfs, P De Cleyn, C Blondia, SCTP as mobility protocol for enhancing Internet on the train. Proc. ITST’08, (Oct 2008)

R Stewart, Stream control transport protocol. IETF RFC4960, (2007)

R Stewart, Q Xie, M Tuexen, S Maruyama, M Kozuka, Stream control transport protocol (SCTP) dynamic address reconfiguration. IETF RFC5061 (2007)

R Stewart, M Ramalho, MTQ Xie, P Conrad, Stream control transport protocol. IETF RFC3758, (2004)

IEEE IEEE standard for information technology–telecommunications and information exchange between systems–local and metropolitan area networks–specific requirements part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) specifications amendment 6: Wireless access in vehicular environments. IEEE Std 802.11p2010, ((2010), pp), . 1–51

M Puterman, in Markov Decision Processes: Discrete Stochastic Dynamic Programming (Wiley, New York, 1994)

EA Feinberg, A Schwartz, (eds.), (eds.), in Handbook of Markov Decision Processes (ch), . Markov Decision Processes in Finance and Dynamic Options (Kluwer, Boston, 2002)

F Yu, V Krishnamurthy, V Leung, Crosslayer optimal connection admission control for variable bit rate multimedia traffic in packet wireless CDMA networks. IEEE Trans. Signal Process 54(2), 542–555 (2006)

K Zografos, K Androutsopoulos, V Spitadakis, Design and assessment of an online passenger information system for integrated multimodal trip planning. IEEE Trans. Intell. Transport. Syst 10(2), 311–323 (2009)

IEEE , Wireless medium access control (MAC) and physical layer (PHY) specification: Medium access control (MAC) enhancement for quality of service (QoS), July 2003

A Bohm, M Jonsson, (Handover in IEEE 802), . 11pbased delaysensitive vehicletoinfrastructure communication(Research Report, Halmstad University, Sweden, 2007)

E StevensNavarro, Y Lin, VWS Wong, An MDPbased vertical handoff decision algorithm for heterogeneous wireless networks. IEEE Trans. Veh. Tech 57(2), 1243–1254 (2008)

A Messac, E Melachrinoudis, CP Sukam, Aggregate objective functions and pareto frontiers: Required relationships and practical implications. Optim. Eng 1(2), 171–188 (2000). Publisher Full Text

MH Gadallah, On multiobjective optimization problem: modeling issues and numerical verification. Proc. 34th Int’l, Conf. Computers and Industrial Eng. ((San Francisco, USA, Nov 2004), pp), . 635–640

X Qiu, K Chawla, On the performance of adaptive modulation in cellular systems. IEEE Trans. Commun 47(6), 884–895 (1999). Publisher Full Text

S Ali, V Leung, Dynamic frequency allocation in fractional frequency reused ofdma networks. IEEE Trans. Wirel. Commun 8(8), 4286–4295 (2009)

G Song, Y Li, Crosslayer optimization for ofdm wireless networkspart i: theoretical framework. IEEE Trans. Wirel. Commun 4(2), 614–624 (2005)

Z Li, Z Yan, Trainground communication in CBTC based on 802.11b: Design and performance research. Proc. WRI Int’l Conf. Commun. and Mobile Computing ((Kunming, China, Jan 2009), pp), . 368–372

DB Green, MS Obaidat, An accurate line of sight propagation performance model for adhoc 802.11 wireless LAN (WLAN) devices. Proc. IEEE ICC’02 ((New York, USA, Apr 2002), pp), . 3424–3428

F Babich, G Lombardi, E Valentinuzzi, Variable order Markov modeling for LEO mobile satellite channels. Electron. Lett 35(8), 621–623 (1999). Publisher Full Text

P Si, FR Yu, H Ji, V Leung, Distributed sender scheduling for multimedia transmission in wireless mobile peertopeer networks. IEEE Trans. Wirel. Commun 8, 4594–4603 (2009)

F Babich, G Lombardi, A measurement based Markov model for the indoor propagation channel. Proc. IEEE VTC’97, vol. 1 ((Phoenix, AZ, May 1997), pp), . 77–81

HS Wang, PC Chang, On verifying the firstorder Markovian assumption for a rayleigh fading channel model. IEEE Trans. Veh. Tech 45(2), 353–357 (1996). Publisher Full Text

FR Yu, B Sun, V Krishnamurthy, S Ali, Application layer qos optimization for multimedia transmission over cognitive radio networks. Wirel. Netw 17, 371–383 (2011). Publisher Full Text

C Pimentel, TH Falk, L Lisbôa, Finitestate Markov modeling of correlated Ricianfading channels. IEEE Trans. Veh. Tech 53(5), 1491–1501 (2004). Publisher Full Text

YL Guan, LF Turner, Generalized FSMC model for radio channels with correlated fading. IEE Proc. Commun 146(2), 133–137 (1999). Publisher Full Text

Y Wei, FR Yu, M Song, Distributed optimal relay selection in wireless cooperative networks with finitestate markov channels. IEEE Trans. Veh. Tech 59, 2149–2158 (2010)