Evaluating Machine Translation Post-editing Training in Undergraduate Translation Programs-An Exploratory Study in Saudi
The modern translation industry is using machine translation post-editing (MTPE) widely, and the
translation industry in the Arab World is following the global lead. However, while MTPE training
is offered in many language pairs around the world, MTPE training in English-Arabic is still not
officially offered in translation training programmes in the Arab World. The aim of this study is to
evaluate the effectiveness of MTPE training in a female undergraduate translation programme in
Saudi Arabia by examining students’ opinions about MTPE and comparing its productivity and
quality with an established practice in the translation classroom, i.e., human translation (HT). To
achieve its aim, this study used a mixed-method design of the ‘Kirkpatrick Model of Learning
Evaluation’. Focus group discussions and retrospective pre-test surveys were used to examine
students’ opinions as well as a pre-post experiment which involved two groups of students (29 in
the control group and 31 in the experimental group) that was used to compare the productivity of
students and the quality of translated texts when using MTPE as compared with HT. Students’
opinions that were revealed through the pre-intervention focus group discussions were generally
mixed with a preference shown in favour of HT, except for translation speed as most of the
students thought that MTPE was the faster method of translation. As for the survey, students’
pre-intervention responses supported those opinions revealed in the focus group discussions.
However, post-intervention responses revealed a statistically significant shift towards more
acceptance of MTPE training and use, indicating that the more students learned about the
features of MT and MTPE skills and practiced them, the more positive their opinions became.
Statistical results from comparing students’ productivity showed a medium effect size which
indicates that MTPE cannot be ignored as a method to increase productivity in translation. The
effectiveness of MTPE in translation quality was evaluated by measuring error count and error
type. Error count analysis indicated that students who used MTPE have increased scores in a
similar manner to those who used HT but not more. The analysis of error type showed that while
MTPE helped students avoid deletion and technical errors, the number of errors relating to
accuracy, comprehension and grammar were more frequent in Arabic MTPE translated texts.
The modern translation industry is using machine translation post-editing (MTPE) widely, and the
translation industry in the Arab World is following the global lead. However, while MTPE…