Print Email Facebook Twitter Efficient Neural Network Architecture Search Title Efficient Neural Network Architecture Search Author YANG, MINGHAO (TU Delft Mechanical, Maritime and Materials Engineering) Contributor Pan, Wei (mentor) Zhou, Hongpeng (mentor) Gavrila, Dariu (graduation committee) van de Plas, Raf (graduation committee) Degree granting institution Delft University of Technology Programme Mechanical Engineering | Vehicle Engineering Date 2019-07-05 Abstract One-Shot Neural Architecture Search (NAS) is a promising method to significantly reduce search time without any separate training. It can be treated as a Network Compression problem on the architecture parameters from an overparameterized network. However, there are two issues associated with most one-shot NAS methods. First, dependencies between a node and its predecessors and successors are often disregarded which result in improper treatment over zero operations. Second, architecture parameters pruning based on their magnitude is questionable. In this thesis, classic Bayesian learning approach is applied to alleviate these two issues. Unlike other NAS methods, we train the over-parameterized network for only one epoch before update network architecture. Impressively, this enabled us to find the optimal architecture in both proxy and proxyless tasks on CIFAR-10 within only 0.2 GPU days using a single GPU. As a byproduct, our approach can be transferred directly to convolutional neural networks compression by enforcing structural sparsity that is able to achieve extremely sparse networks without accuracy deterioration. Subject NASDeep LearningICMLArtificial Intelligence To reference this document use: http://resolver.tudelft.nl/uuid:9985c543-cb4e-4259-b6f8-b44ba433f1e3 Part of collection Student theses Document type master thesis Rights © 2019 MINGHAO YANG Files PDF Master_Thesis.pdf 2.37 MB Close viewer /islandora/object/uuid:9985c543-cb4e-4259-b6f8-b44ba433f1e3/datastream/OBJ/view