I- Introduction

II- System Model

III- MDP Analysis of Transimission

IV- Deep-Q-Learning-Based Optimal Action Acquisition Scheme

V- Algorithm Comparisons

VI- Simulation Experiment and Analysis

VII- Conclusion

References

Abstract

Cognitive networks (CNs) are one of the key enablers for the Internet of Things (IoT), where CNs will play an important role in the future internet in several application scenarios, such as healthcare, agriculture, environment monitoring, and smart metering. However, the current low packet transmission efficiency of IoT faces a problem of the crowded spectrum for the rapidly increasing popularities of various wireless applications. Hence, the IoT that uses the advantages of cognitive technology, namely the cognitive radio-based Internet of Things (CIoT), is a promising solution for IoT applications. A major challenge in CIoT is the packet transmission efficiency using CNs. Therefore, a new Q-learning-based transmission scheduling mechanism using deep learning for the CIoT is proposed to solve the problem of how to achieve the appropriate strategy to transmit packets of different buffers through multiple channels to maximize the system throughput. A Markov decision process based model is formulated to describe the state transformation of the system. A relay is used to transmit packets to the sink for the other nodes. To maximize the system utility in different system states, the reinforcement learning method, i.e., the Q learning algorithm, is introduced to help the relay to find the optimal strategy. In addition, the stacked auto-encoders deep learning model is used to establish the mapping between the state and the action to accelerate the solution of the problem. Finally, the experimental results demonstrate that the new action selection method can converge after a certain number of iterations. Compared with other algorithms, the proposed method can better transmit packets with less power consumption and packet loss.

Introduction

In the future, wireless sensor networks are expected to be integrated into the Internet of Things [1-2], where reconfigurable, flexible, and intelligent sensors dynamically join the Internet and use it to collaborate and accomplish their tasks for a wide range of applications in various domains [3-8], such as big data applications, Internet of Things, E-commerce, medical device [9-10], virtual reality & augmented reality, and environment monitoring. The network environment also tends to become increasingly complicated, and the communication resources become increasingly scarce. It is a great challenge to the wireless sensor networks and Internet of Things. Coincidentally, the cognitive network technology can compensate for these deficiencies [11-15]. Cognitive nodes are intelligent wireless devices that can sense the environment, observe the network changes, use the knowledge learnt from the previous interaction with the network, and make intelligent decisions to seize the opportunities to transmit. The process of continuously sensing the environment information, exchanging control information, learning information, deciding and executing a strategy in the network can provide the ability of intelligence and adaptability to the wireless sensor networks and the future Internet of Things. Therefore, the cognitive radio technology is a key communication approach for resource-constrained wireless sensor networks and future wireless network [16-20]. When cognitive users, i.e., sensors in wireless sensor networks, access the spectrum, to effectively use the network resources and satisfy the throughput demand for multimedia applications, effective mechanisms are required to coordinate the actions of the cognitive users (transmission power control, spectrum access, transmission scheduling, et al.) [21-24]. With the rapid increase in number of wireless devices in the Internet of Things, more data will be stored in the network nodes. Thus, the method to rapidly forward data with the limited storage space and bandwidth is a great challenge for the current wireless network of Internet of Things.