Surrogate network-based sparseness hyper-parameter optimization for deep expression recognition

Weicheng Xie Wenting Chen Linlin Shen Jinming Duan Meng Yang

School of Computer Science & Software Engineering, Shenzhen University, PR China
Shenzhen Institute of Artificial Intelligence and Robotics for Society, PR China
Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University, PR China
School of Computer Science, University of Birmingham, UK, School of Data and Computer Science, Sun Yat-sen University, PR China

Abstract

For facial expression recognition, the sparseness constraints of the features or weights can improve the generalization ability of a deep network. However, the optimization of the hyper-parameters in fusing dif- ferent sparseness strategies demands much computation, when the traditional gradient-based algorithms are used. In this work, an iterative framework with surrogate network is proposed for the optimization of hyper-parameters in fusing different sparseness strategies. In each iteration, a network with signifi- cantly smaller model complexity is fitted to the original large network based on four Euclidean losses, where the hyper-parameters are optimized with heuristic optimizers. Since the surrogate network uses the same deep metrics and embeds the same hyper-parameters as the original network, the optimized hyper-parameters are then used for the training of the original deep network in the next iteration. While the performance of the proposed algorithm is justified with a tiny model, i.e. LeNet on the FER2013 database, our approach achieved competitive performances on six publicly available expression datasets, i.e., FER2013, CK+, Oulu-CASIA, MMI, AFEW and AffectNet.

Visualization

The iterative framework of the proposed hyper-parameter optimization

Four sparseness strategies and their hyper-parameters

The configuration of ‘original+surrogate’ network

Example images of the FER2013, CK+, Oulu-CASIA, MMI, AFEW and AffectNet databases

The evolution curves of the hyper-parameters and the testing accuracies

The evolution curves of the eight hyper-parameters of LeNet optimized with PSO for the FER2013 database

The evolution curves of the validation and testing accuracies

The confusion matrices (%) of the proposed algorithm


@article{xie2021surrogate,
    title={Surrogate network-based sparseness hyper-parameter optimization for deep expression recognition},
    author={Xie, Weicheng and Chen, Wenting and Shen, Linlin and Duan, Jinming and Yang, Meng},
    journal={Pattern Recognition},
    volume={111},
    pages={107701},
    year={2021},
    publisher={Elsevier}
}