An ever-increasing number of mobile applications are leveraging deep learning models to provide novel and useful features, such as real-time language translation and object recognition. However, current mobile inference paradigm requires application developers to statically trade-off between inference accuracy and inference speed during development time. As a result, mobile user experience is negatively impact given dynamic inference scenarios and heterogeneous device capacity.
The MODI project proposes new research in designing and implementing a mobile-aware deep inference platform that combines innovations in both algorithm and system optimizations. The proposed work will address mobile deep inference performance problems by enabling flexible, fine-grained model partition and layer-based inference execution, as well as mobile-specific model designs. In addition, MODI enables a scalable mobile deep inference paradigm with efficient model management both on-device and in the cloud.
The project will empower deep learning to provide useful features for mobile applications with significantly improved performance, opening doors for further optimizing deep learning models to run on much more resource-constrained devices such as embedded devices. The MODI project can be used as a standalone cloud system or integrated with existing general inference serving platforms by incorporating its mobile-specific optimizations, therefore increasing adoption.
The broader impacts of the project will include graduate and undergraduate courses that incorporate research results, outreach to expose undergraduates, high school and K-12 students to research in this interdisciplinary field of computer systems and deep learning. In addition, project related source code and other resources will be released to the research community through our project website.
Coming up soon.
Publicly Released Data Sets and Code
Coming up soon.
This project is sponsored by NSF award CNS #1815619.