Description:
INV-20004
Background
Recent work has demonstrated that cutting-edge advances in deep reinforcement learning (DRL) can be leveraged to empower wireless devices with the essential ability to "sense" current spectrum and network conditions and "react" in real-time by either exploiting known optimal actions or exploring new actions. Research going on whether real-time DRL can be applied in the resource-challenged embedded IoT domain, as well as designing IoT-tailored DRL systems and architectures, remains mostly uncharted territory.
Technology Overview
This technology uses Deep Wire-less Embedded Reinforcement Learning (DeepWiERL), a general-purpose, hybrid software/hardware DRL framework specifically tailored for embedded IoT wireless devices. DeepWiERL provides abstractions, circuits, software structures, and drivers to support the training and real-time execution of state-of-the-art DRL algorithms on the device’s hardware. Moreover, DeepWiERL includes a novel supervised DRL model selection and bootstrap (SDMSB) technique that leverages transfer learning and high-level synthesis (HLS) circuit design to orchestrate a neural network architecture that satisfies hardware and application throughput constraints and speeds up the DRL algorithm convergence.
Experimental evaluation on a fully-custom software-defined radio testbed
(i) proves for the first time the feasibility of real-time DRL-based algorithms on a real-world wireless platform with multiple channel conditions.
(ii) shows that DeepWiERL supports16x data rate and consumes 14x less energy than a software-based implementation, and
(iii) indicates that S-DMSB may improve the DRL convergence time by 6x and increase the obtained reward by 45%.
Benefits
- The feasibility of real-time DRL-based algorithms on a real-world wireless platform with multiple channel conditions
- DeepWiERL supports 16x data rate and consumes 14x less energy than a software-based implementation
- Can improve the DRL convergence time by 6x
Applications
- Wireless Self-Optimization
- Military IoT Applications
- Civil IoT Applications
Opportunity
- License
- Research collaboration
- Partnering