This study was conducted to assess the quality of the self-assessment speaking rubric adapted by Montgomery from Bill Heller in 2000. The rubric consisted of six aspects with a four-point rating scale and was originally written in English and aimed to be used by the English language learners. As the respondents of this research were the Indonesian students who learn Arabic language as a foreign language, the rubric was therefore modified and translated into Indonesian language. Rasch measurement model approach provides various analyses with empirical evidence about the quality of instrument by looking at the rating scale analysis, summary statistics, item fit, principal component analysis and Wright map. About 43 Arabic language learners from a university in Salatiga, Indonesia, were involved in this study. The finding showed that the four rating options were clearly understood by the respondents. All six items in the rubric were also appropriately measure students’ speaking skills. High value of person (0.84) and item reliability (0.94) indicated good quality of both respondents and instrument. The Cronbach alpha value 0.83 indicated high reliability. To sum up, the self-assessment speaking rubric has a good quality to measure speaking skills and is appropriate to be used by students to self-assess their Arabic speaking ability.