WiWiSET

WiWiSET - Validation of an Entrance Examination in the Study Domain of Business and Economics - A National and International Comparative Study of Universities and Universities of Applied Sciences

Pant, H. A., Zlatkin-Troitschanskaia, 0., Schipolowski, S. & Förster, M.

The WiWiSET project employs a German edition (so called TEL-D) of the standardized Test of Economic Literacy IV (TEL IV), developed by the U.S. Council for Economic Education (CEE). The test has been translated according to the TRAPD process model (including transla­tion, review, adjudication, pretesting, and documentation) and has been adapted according to the International Test Commission's Guidelines on Test Adaptation for initial diagnostic assessment in higher education in the domain of business and economics. We analyze eco­nomics-related competencies at the beginning of studies in an objective, reliable, and valid way. Fur­thermore, we examine correlations with additional, theoretically significant personal variables, such as prior knowledge and motivation for enrolling in business and economics studies, as well as institu­tional variables, such as types of higher education institution and degree course. By means of various statistic models, such as structural equation modeling and multilevel analysis, we examine the effects that can be explained, according to the preliminary study, by students' individual preconditions, by differences between degree courses, such as business administration and economics, and by differ­ences between higher education institutions, that is, universities compared to universities of applied sciences. Among others, we examine whether the test enables measurement-invariant assessment of economics-related competencies at the beginning of studies.
Building on validation work from the preliminary study, conducted according to the international Standards for Educational and Psychological Testing, the WiWiSET project pursues discriminant, predictive, and incremental validation perspectives. We examine whether subject-re­lated competencies at the beginning of studies assessed by means of the TEL-D can be empirically differentiated from general cognitive abilities (GCA) measured by means of intelligence tests and uni­versity entrance qualifications (discriminant validation). Moreover, we use the TEL-D to predict sub­ject-related academic performance or study success in economics (predictive validation). We expect that the TEL-D should be able to explain significant proportions of variance in subject-related academic performance and should have greater explanatory power than subject-independent predictors such as intelligence test scores and university entrance qualifications (incremental validation). In addition, pre­dictive validation can include prediction of dropouts; we examine the extent to which business and economics students' results on the TEL-D at the beginning of their studies can predict discontinuation of studies within the first academic year, which is the most critical one in this study domain. To be judged a suitable predictor of study success at the beginning of studies and to serve as a useful infor­mation tool for intervention planning in higher education teaching, the TEL-D must increase the relia­bility of predictions of study success when used alongside university entrance qualifications.
The population for the WiWiSET study includes students at the beginning of a Bachelor's degree pro­gram in the domain of business and economics (the largest study domain in Germany) at a university or university of applied sciences in Germany. As an efficient selection process, we will use a duster sample with clustering according to types of higher education institutions (universities or universities of applied sciences). In the sampled clusters, we will assess first-year students in the domain of busi­ness and economics. This method of sampling implies that the data has a nested structure, as students are clustered in higher education institutions. This nesting must be considered in the data analysis in order to distinguish intra-institutional differences among students at one institution from inter-insti­tutional differences between students from different institutions. This aspect of the multilevel ap­proach defines the required sample size in terms of the amount of clusters (level 2 units) and the num­ber of students (level 1 units). To estimate ideally undistorted and generalizable fixed and random effects, we must assess at least 30 higher education institutions. The German Federal Sta­tistical Office has provided a list (04/2015) of all higher education institutions included in the popula­tion, from which we will choose 40 institutions at random. For each geographical area, we have deter­mined two substitute institutions with similar characteristics (such as similar degree courses and de­partment sizes).
To test the research hypotheses of the WiWiSET project, we need two assessment rounds. In the first round of assessment, which is scheduled to take place before the beginning of the winter term 2016 (September-October 2016), first-year students will be assessed at randomly chosen higher education institutions during introductory courses in economic studies. In the second round of assessment to take place at the beginning of the winter term 2017, the same students will be assessed a second time after having completed one year of studies. This way, we aim to assess their study progress during the orientation phase of the Bachelor's degree program so as to examine the predictive validity of the TEL­D in terms of academic courses completed du ring the first year of studies.

Project data and contact