A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfer

R. Ghosal, Gaurav and Ghosal, Dipak and Sim, Alex and V. Thakur, Aditya and Wu, Kesheng
2020 IFIP Networking Conference, 2020

We consider data sources connected to a software defined network (SDN) with heterogeneous link access rates. Deadline-driven data transfer requests are made to a centralized network controller that schedules pacing rates of sources and meeting the request deadline has a pre-assigned value. The goal of the scheduler is to maximize the aggregate value. We design a scheduler (RL-Agent) based on Deep Deterministic Policy Gradient (DDPG). We compare our approach with three heuristics: (i) P_FAIR, which shares the bottleneck capacity in proportion to the access rates, (ii) V_D_Ratio, which prioritizes flows with high value-to-demand ratio, and (iii) V_B_EDF, which prioritizes flows with high value-to-deadline ratio. For equally valued requests and homogeneous access rates, P_FAIR is the same as an idealized TCP algorithm, while V_B_EDF and V_D_Ratio reduce to the Earliest Deadline First (EDF) and the Shortest Job First (SJF) algorithms, respectively. In this scenario, we show that RL-Agent performs significantly better than P_FAIR and V_D_Ratio and matches and in over-loaded scenarios out-performs V_B_EDF. When access rates are heterogeneous, we show that the RL-Agent performs as well as V_B_EDF even though the RL-Agent has no knowledge of the heterogeneity to start with. For the value maximization problems, we show that the RL-Agent out-performs the heuristics for both homogeneous and heterogeneous access networks. For the general case of heterogeneity with different values, the RL-Agent performs the best despite having no prior knowledge of the heterogeneity and the values, whereas the heuristics have full knowledge of the heterogeneity and V_D_Ratio and V_B_EDF have partial knowledge of the values through the ratios of value to demand and value to deadline, respectively.

PDF    

@inproceedings{Networking2020,
  author = {R.\ Ghosal, Gaurav and Ghosal, Dipak and Sim, Alex and V.\ Thakur, Aditya and Wu, Kesheng},
  title = {A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfer},
  booktitle = {2020 {IFIP} Networking Conference},
  year = {2020}
}