Mobility load balancing in LTE/5G


Given below is a high-level description of an example of how reinforcement learning can be used with NetSim 5G

  • Load transfer from an overloaded cell to an under-loaded neighbouring cell
  • The default association in NetSim is Max RSS i.e., the UE associates itself to the BS from which it sees max signal strength
  • Now, let us denote c_ij as the instantaneous rate of a UE and is theoretically a log function of SINR 

  • And let R_ij be the long-term rate and y_ij is the fraction of resources allocated, to UE_i by BS_j 


  • Note that the max RSS association does not balance the load between BSs. The load balancing problem can be solved from optimization theory for a fixed topology
  • Now let's say the SINR changes with time due to user mobility
  • Then RL can be used to decide a "load-aware" UE-BS association i.e., the association is not based on max RSS
  • We explore, Markov decision process/Q-learning based (model-free) RL
    • At state s_t  RL agent selects action a_t by following policy π and receives reward r(s_t, a_t). 
    • The MDP has value function V^π (s), and action value function Q^π (s, a) where α (0≤α≤1) is the discount factor 
    • We assume that the update interval (epoch) ≫ LTE/5G frame length of 10ms
  • State: UE SINRs (γ_1,…, γ_N  ), based on the current association at time t 
  • Action: 
    • Association x_ij (indicator variable showing association of UEi_ to BS_j)
    • Resource allocation y_ij (equals 1/sum_j x_ij, i.e., the reciprocal of the number of UEs associated with a BS. This is exact for Round robin, and on average for PFS)




Variations

  • Consider GBR and Non GBR users
  • Split between GBR PRB usage and Non GBR PRB usage
  • Time-varying network traffic
  • Use deep neural networks to approximate the Q and Value functions
  • Additional constraint: Minimum throughput per user (i.e., minimum SINR , γ, to all users)
  • Objective: Latency minimization


Other examples

  • Association based on logical cell boundaries using Cell individual offset (CIO)
  • Power control in multi-tier HetNets


Attachment


A toy example (4 slide PDF) showing RL for 5G is attached