خلاصة:
Although some piecemeal efforts have been made to investigate the validity and use of the Iranian PhD exam, no systematic project has been specifically carried out in this regard. The current study, hence, tried to attend to this void. As such, to ensure a balanced focus on testinterpretation and test consequence, and to track evidence derivedfrom a mixed–method study on the validity of Iranian PhD entrance exam of TEFL (IPEET), this study drew on a hybrid of two argument-based structures: Kane's (1992) argument model and Bennett's (2010) theory of action. Resting on the network of inferences and assumptions borrowed from the hybridized framework, the study investigated the extent to which the proposed assumptions would be supported by empirical evidence. It also examined the unintended consequences that may possibly be revealed through this validity investigation. Three sources of data informed the present study: (a) Test score data from about 1000 PhD applicants' taking IPEET test administered in 2014, (b) questionnaires completed by university professors and PhD students of TEFL, and finally, (c) telephone and focus-group interviews with university professors and PhD students of TEFL, respectively. The results from the analysis of mixed-method data indicated that all the inferences proposed for this study were rebutted, suggesting that some unintended consequences have happened to the technical as well as the decision quality of this test, hence its invalidity. Findings alsoReceived: 09/10/2015 Accepted: 29/06/2016∗ Corresponding authorprovided valuable insights and suggestions for the betterment of the present content and current policy of IPEET in Iran.
ملخص الجهاز:
Three sources of data informed the present study: (a) Test score data from about 1000 PhD applicants' taking IPEET test administered in 2014, (b) questionnaires completed by university professors and PhD students of TEFL, and finally, (c) telephone and focus-group interviews with university professors and PhD students of TEFL, respectively.
1. 1 Research questions This study primarily aimed at investigating the content and use of IPEET in light of argument-based validity and theory of action, throwing some light on the betterment of the technical and decision quality of this test in Iran.
The interview items were aimed at soliciting university professors' opinion regarding: the relevance of the IPEET test tasks to the PhD courses, PhD students’ success in PhD programs, the quality of decisions made by testing agencies and possible suggestions for the betterment of IPEET, informing the claims on evaluation and extrapolation, intermediate actions, ultimate effects and ultimate actions.
As regards the PhD students and university professors' opinions, a series of Binomial tests of significance were used to report the participants' responses to the specified questionnaire items in the form of observed proportions.
3 Extrapolation inference To find a reasonable answer for the third research question, we sought evidence from questionnaires and interviews responded by experts (university professors) with regard to: a) the relevancy of content of the IPEET test tasks to the content of PhD credit courses exercised in universities and b) PhD students' success in PhD courses taught at PhD programs, results for each will be dealt with in turn.