Unmanned aerial vehicles (UAVs) can be used as low-altitude flight base stations to satisfy the coverage requirements of wireless users in various scenarios. In practical applications, since the transmitted power and energy resources of the UAVs are limited and the propagation environments are complicated and time-variant, it is challenging to control a group of UAVs to ensure coverage performance while preserving the connectivity and safety of the UAV networks. To this end, a two-step environment-learningbased method is proposed for the intelligent deployment of the UAVs. First, a machine learning algorithm is used to establish an accurate prediction model of the link qualities from the UAVs to the users under a specific scenario for the next step. Then, a modified deep deterministic policy gradient (DDPG) algorithm is employed to control the movements of the UAVs according to the predicted link qualities and to maximize the proportion of covered users. The prioritized experience replay mechanism is introduced to the standard DDPG algorithm to accelerate the deployment procedure. The coverage performance is analyzed in both the interference-free situation and the situation with co-channel interference. Simulation results have shown that the proposed method has a higher convergence speed than the standard DDPG method. Additionally, the proposed deployment method can achieve higher coverage performance and better adaptability to the dynamic environment than three commonly used methods, the random method, the K-means-based method, and the statistical-channel-model-based method.

Introduction

In recent years, unmanned aerial vehicles (UAVs) have attracted great attention due to small size, low price, and high flexibility. The UAVs with variable positions can establish line-of-sight (LoS) communication links to the users. Therefore, the UAVs are suitable to be used as low-altitude flight base stations (BSs) to reduce the signal attenuation and improve the coverage performance [1], [2]. For example, in the case of a terrestrial BS failure, the UAV-BSs can be rapidly deployed to satisfy temporary coverage demands for wireless services [3]–[۵]. The cellular networks can also be assisted by the UAV-BSs in the temporary hotspot area [6]–[۹]. The deployment problem of the UAVs is more complicated than that of the terrestrial BSs. First, in practical application scenarios, due to the limited transmitted power and energy resources, multiple UAVs are often required to be deployed together to ensure the large coverage. Second, since the propagation environment is complex and changeable, the UAVs are expected to have certain adaptability to the environment to rapidly satisfy the coverage requirements of the users. In addition, multiple UAVs should be set with a certain distance limitation to maintain both the connectivity to ensure the robustness of the network and the security to prevent collisions caused by unexpected situations. Therefore, how to effectively deploy the positions of the UAVs to improve the coverage performance is a challenging problem.