Nowadays, person recognition has received significant attention due to broad applications in the security system. However, most person recognition systems are implemented based on unimodal biometrics such as face recognition or voice recognition. Biometric systems that adopted unimodal have limitations, mainly when the data contains outliers and corrupted datasets. Multimodal biometric systems grab researchers’ consideration due to their superiority, such as better security than the unimodal biometric system and outstanding recognition efficiency. Therefore, the multimodal biometric system based on face and fingerprint recognition is developed in this paper. First, the multimodal biometric person recognition system is developed based on Convolutional Neural Network (CNN) and ORB (Oriented FAST and Rotated BRIEF) algorithm. Next, two features are fused by using match score level fusion based on Weighted Sum-Rule. The verification process is matched if the fusion score is greater than the pre-set threshold. The algorithm is extensively evaluated on UCI Machine Learning Repository Database datasets, including one real dataset with state-of-the-art approaches. The proposed method achieves a promising result in the person recognition system.