Reinforcement learning is a remarkable aspect of the artificial intelligence field with many applications. Reinforcement learning facilitates learning new tasks based on action and reward principles. Motion planning addresses the navigation problem for robots. Current motion planning approaches lack support for automated, timely responses to the environment. The problem becomes worse in a complex environment cluttered with obstacles. Reinforcement learning can increase the capacity of robotic systems due to the reward system's capability and feedback to the environment. This could help deal with a complex environment. Existing algorithms for path planning are slow, computationally expensive, and less responsive to the environment, which causes late convergence to a solution. Furthermore, they are less efficient for task learning due to post-processing requirements. Reinforcement learning can address these issues using its action feedback and reward policies. This research presents a novel Q-learning-based reinforcement algorithm with deep learning integration. The proposed approach is evaluated in a narrow and cluttered passage environment. Further, improvements in the convergence of reinforcement learning-based motion planning and collision avoidance are addressed. The proposed approach's agent converged in 210th episodes in a cluttered environment and 400th episodes in a narrow passage environment. A state-of-the-art comparison shows that the proposed approach outperformed existing approaches based on the number of turns and convergence of the path by the planner.
Artificial Intelligence of Things (AIoT) edge computing has emerged as a critical enabler for consumer technology, easing the load on distribution grid networks through enhanced data transmission and processing capabilities. The Internet of Things (IoT), a modern network paradigm, connects devices using sensors and communication mediums to enable seamless information exchange. However, the limited computational capacity of edge nodes poses challenges in optimizing AIoT edge resources compared to cloud computing. To address this, a cloud-edge three-layer framework for task offloading and edge resource allocation is proposed in the context of consumer technology and distribution grids. This model incorporates random tasks, limited resources, unequal processing power, and high latency requirements. It features two key phases: a resource auction using a multi-round iterative process and compute offloading managed by a Deep Reinforcement Learning (DRL) approach. Enhanced algorithms such as Double DQN and Dueling DQN are integrated into the job offloading process to improve performance. Simulations validate the convergence of the task offloading algorithm and demonstrate that the proposed methods significantly enhance computational efficiency and resource utilization for edge nodes. These advancements offer promising solutions for the dynamic needs of consumer technology and future communication networks.
Transactions on Emerging Telecommunications Technologies · مجلد 36 (10)
ABSTRACT Utilizing blockchain in vehicular ad‐hoc networks (VANETs) can proficiently resolve concerns pertaining to data security and privacy. The limited throughput of blockchain obstructs its extensive implementation in VANETs. Current studies on enhancing blockchain throughput frequently encounter the issue of action space proliferation, leading to inadequate scalability. This research presents a strategy for optimizing blockchain performance on VANETs using deep reinforcement learning (DRL). The suggested method enhances blockchain throughput by selecting block producers and consensus methods, and by modifying block size and intervals, while maintaining decentralization, low latency, and security in VANET‐based blockchain systems. Furthermore, to augment network security, an anomaly detection mechanism is incorporated, utilizing machine learning techniques to identify and mitigate potential attacks aimed at VANETs. The proposed system enhances throughput and fortifies resilience against malicious operations by identifying anomalous patterns in network behavior. The method uses the BDQ framework to meticulously partition the action space, tackling the action space explosion issue that occurs with conventional DRL techniques in blockchain throughput optimization. Simulation results indicate that the suggested solution significantly improves the throughput and security of the VANET‐based blockchain system.
vehicular ad hoc networksblockchainanomaly detectiondeep reinforcement learning
The integration of beyond diagonal reconfigurable intelligent surfaces (BD-RIS) with non-orthogonal multiple access (NOMA) in the Internet of Vehicles (IoV) networks presents the transformative potential for 6G vehicular communications, promising unprecedented gains in energy and spectral efficiency. However, existing research fails to address the joint optimization of BD-RIS configuration, NOMA power allocation, and dynamic IoV association–a critical oversight given their inherent interdependence in practical deployment scenarios. Current literature predominantly examines these components in isolation, with conventional RIS architectures and orthogonal multiple access schemes, leading to suboptimal performance in high-mobility vehicular environments. This work bridges this gap by proposing a novel hybrid optimization framework for NOMA-aided BD-RIS assisted IoV networks. The proposed framework simultaneously optimizes the IoV association with the base station (BS), NOMA power allocation, and BD-RIS phase shift design to maximize the energy efficiency of the system while ensuring the minimum signal-to-interference plus noise ratios (SINRs) of IoVs. The proposed framework is formulated as non-linear optimization due to joint decision variables and rate expressions of IoVs, resulting in an NP-hard problem where achieving a joint optimal solution is computationally complex. To handle this complexity, the original joint formulation is decomposed into three subproblems: IoV association with BS, BS power allocation, and BD-RIS phase shift design. An efficient iterative solution is then developed through the synergistic combination of deep reinforcement learning, first-order Taylor expansion, and manifold optimization methods. For comprehensive evaluation, we introduce a NOMA-aided conventional RIS assisted IoV framework as a benchmark. Extensive numerical results based on Monte Carlo simulations demonstrate that the proposed NOMA-aided BD-RIS assisted IoV network achieves significant improvements of 32% in energy efficiency and 28% in spectral efficiency compared to conventional RIS-assisted architectures, validating its potential for next-generation vehicular networks.
beyond-diagonal reconfigurable intelligent surfacenon-orthogonal multiple accessInternet of Vehiclesenergy efficiency optimization
OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · 1-6
The deep Q-network (DQN) technique, as a classic value-based deep reinforcement learning process, has been widely applied in fields such as multi-robotic agent motion planning. However, the DQN method faces a series of tasks, such as overestimating Q-values, complex Q-value computations, neural networks lacking historical memory proficiencies, and inefficient exploration when using ε-greedy strategies. This paper proposes a DQN-based multi-robot agent deep reinforcement learning motion planning approach to solve these problems. To accomplish their objectives without colliding with one another, agents can acquire stable and efficient motion planning techniques with the aid of this method. Initially, a Dueling-based optimization tool is suggested, which is based on the DQN approach. It simplifies and improves the calculation of Q-values by calculating state values and advantage function values, and by choosing the appropriate action based on the parameters of the presently updating Q-value network. Additionally, we provide a GRU-based memory mechanism that incorporates GRU modules to facilitate the network's temporal data acquisition and handling of agents' historical information. And lastly, we offer a noise-based exploration technique that works. By introducing parameterized noise, the exploration technique in DQN is modified, improving the examination efficiency of agents, and achieving a balance between exploration and exploitation in multi-robotic agent systems. Tests were conducted in six different simulation scenarios on the PyBullet simulation platform. Experimental results show that the proposed method enables efficient collaboration among multi-robotic agent teams, agreeing them to go where they're going without running into each other, and the methods' training is steady.
In Industry 4.0, industrial robot trajectory planning requires real-time collision avoidance. This is solved by our innovative Collision Criticality (CC) LSTM-DRL approach. This work suggests navigating many landscapes with diverse barriers using deep reinforcement learning (DRL) and long-short-term memory for data analytics in digital transformation. The robot’s current state is used by Crit LSTM DRL to estimate collision timing and obstacle CC. Obstacle courses use CC ranks. Robotics creates a fixed-dimensional vector that accurately represents reality using LSTM models. After that, the robot’s current state is added to this vector to complete its perception and cognition. Deep Reinforcement learning (DRL) optimizes state values using input data. At each time step, Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) identify the next state values for all possible actions. System operation depends on the highest value’s action. We learned three environmental models in this research. A scenario with five, 10, or one to ten barriers was classified. These models were tested for adaptability to different obstacle densities. In obstacle-rich situations, researchers tested these three models’ strengths and weaknesses. Testing the algorithm in various settings proves its efficacy. The program also considers the combined influence of impediments on the robot by portraying them as a joint state. CC-LSTM-DRL performs well in Industry 4.0’s dynamic and challenging conditions, according to experiments. Mobile robots can confidently navigate changing environments due to their versatility and flexibility, making them ideal industrial tools. Industry 4.0, Collision Criticality, Industrial Robot, Autonomous Systems, Mobile Robots, Collision Risk
deep reinforcement learningreal-time analyticsindustrial robotsdigital transformation
The Internet of Things (IoT) is an emerging technology that allow smart devices to communicate through various heterogeneous channels (wired or wireless). However, for conventional networks, it has become a challenging task to efficiently control and manage the data flows of a huge number of devices. Software-defined networking (SDN) is a new way of thinking about networking. Because it is programmable, flexible, agile, and gives you a big picture of the network, it has tried to solve some IoT problems, like scalability, heterogeneity, and complexity. In large-scale SDN-IoT networks, there is a requirement for routing protocols that are both efficient and secure in order to ensure a superior level of quality of service (QoS) and quality of experience (QoE). To address the above stated challenges, a novel deep reinforcement learning (DRL) known as DQQS model is proposed. The aim is to achieve QoS and QoE while also ensuring the security of the SDN-IoT network. The proposed DQQS model dynamically extracts patterns from the past network history by interacting with the underlying network and generating optimized routing policies. This article employs three network metrics—throughput, latency, and the probability of avoiding malicious nodes—to measure the performance of DQQS. Simulations reveal that the proposed framework outperforms four state-of-the-art routing algorithms: OSPF, L-L Routing, Sailfish Routing, and RL-Routing in terms of both throughput and latency. For instance, in an attacked environment, the proposed DQQS model achieved the highest throughput value of 14.5 Mbps, surpassing OSPF at 8 Mbps, L-L at 8.2 Mbps, Sailfish at 9 Mbps, and RL at 9.5 Mbps. Similarly, this model exhibited superior performance in latency, recording the lowest latency value of 52 ms, compared to OSPF 88 ms, L-L 85 ms, Sailfish 72 ms, and RL 75 ms routing algorithms. The experimental results demonstrate that this new DQQS model is a pioneering deep reinforcement learning-based technique that optimally addresses secure routing in the SDN-IoT environment, ensuring enhanced quality of service and experience, and outperforming state-of-the art DL methodologies in both security and network performance metrics.
software-defined networkingInternet of Thingssecure routingdeep reinforcement learning
IEEE International Conference on Innovations in High Speed Communication and Signal Processing (IHCSP) · 1-8
In Industry 4.0, industrial robot trajectory planning requires real-time collision avoidance. This is solved by our innovative Collision Criticality (CC) LSTM-DRL approach. This work suggests navigating many landscapes with diverse barriers using deep reinforcement learning (DRL) and long-short-term memory for data analytics in high Speed Communication. The robot's current state is used by Crit LSTM DRL to estimate collision timing and obstacle CC. Obstacle courses use CC ranks. Robotics creates a fixed-dimensional vector that accurately represents reality using LSTM models. After that, the robot's current state is added to this vector to complete its perception and cognition. Deep Reinforcement Learning (DRL) optimalizes state values using input data. At each time step, Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) identify the next state values for all possible actions. System operation depends on the highest value's action. We learned three environmental models in this research. A scenario with five, 10, or one to ten barriers was classified. These models were tested for adaptability to different obstacle densities. In obstacle-rich situations, researchers tested these three models' strengths and weaknesses. Testing the algorithm in various settings proves its efficacy. The program also considers the combined influence of impediments on the robot by portraying them as a joint state. CC-LSTM-DRL performs well in Industry 4.0's dynamic and challenging conditions, according to experiments. Mobile robots can confidently navigate changing environments due to their versatility and flexibility, making them ideal industrial tools.
collision avoidanceLSTMdeep reinforcement learningobstacle density
International Conference on Artificial Intelligence and Teacher Education (ICAITE) · 117-124
Scheduling courses is a routine yet important task in Smart educational activities. Traditional manual scheduling is time-consuming and labor-intensive, prone to errors, and unable to meet the demands of large-scale scheduling. Classic genetic algorithms for scheduling have issues such as rapid convergence and decreased scheduling efficiency as constraint factors increase. To address these problems in existing scheduling genetic algorithms, we propose a self-learning genetic algorithm for scheduling based on deep reinforcement learning (GAGDRL). The GAGDRL algorithm utilizes the Q-learning algorithm to achieve adaptive adjustment of crossover and mutation parameters, enhancing the search capability of the genetic algorithm. Establishing a parameter dynamic adjustment model for the Markov decision process (MDP) analyzes the state set of the population fitness function to achieve a comprehensive evaluation of the overall performance of the population. Additionally, the deep Q-network (DQN) algorithm is introduced into the scheduling problem to address issues related to the large number of population states and the volume of Q-table data in scheduling. Experimental results show that, compared to classical and improved genetic algorithms for scheduling, the GAGDRL algorithm improves accuracy and optimization capability. The proposed algorithm can also be applied to problems such as exam scheduling, cinema seating arrangements, and airline route planning.
International Journal of Communication Systems · مجلد 38 (1)
Summary As the popularity of the Internet of Things (IoT) increases, so do the energy requirements of IoT terminal equipment. To address the energy shortage problem of equipment and ensure continuous and stable operation in light of renewable energy and an uncertain environment, a rational and efficient energy allocation strategy is required. This paper proposes a deep reinforcement learning energy allocation strategy that uses the DQN algorithm to directly interact with the unknown environment. The best energy allocation method is independent of environmental knowledge, and a pretraining algorithm is proposed to maximise the initialization state of the strategy. Experiments of comparison and simulation are conducted under various channel data circumstances. Results indicate that the proposed energy allocation strategy outperforms the current strategy in multiple channel conditions and has a high capacity for adaptation to changing conditions.
energy-efficient resource allocationdeep reinforcement learningdeep Q-networkInternet of Things