The purpose of the coursework or examination must be maintained

According to JGU’s Corona-Statutes, “deviations from the formats and length of the coursework and examinations according to the examination regulations are permitted in consultation with the examination committee, as long as the coursework and examination goals can still be reached.“ (§ 2 (2) 1) This also applies to take-home examinations as a new examination format, as well as to possible changes to examination formats and tasks.

 

The Corona-Statutes do not regulate the content of a take-home examination

Excepting the directive that the purpose of the coursework or examinations must be maintained, the Corona-Statutes do not stipulate any other conditions for the content of a take-home examination: “A take-home examination is defined as working on one or more tasks assigned by the examiner within a limited time frame, without supervision, and using the methods common to the subject.“ (§ 4 (1) 1)

 

The examination committee decides whether or not a transition to take-home examinations is permissible

The responsible examination committee decides if and in which format the transition from examinations to take-home examinations is permissible.

As a rule, standardized examination questions (multiple-choice) are not recommended for take-home examinations

Take-home examinations take place without supervision and with access to resources (such as course materials, literature, the internet). Therefore, examination tasks should be designed in a way that simple searches for information as well as cooperation between students is either not expedient or is a part of the examination.

While standardized multiple-choice examinations with accordingly chosen didactical questions can make simple searches for information futile, students may still cooperate to find the answers. As a result, carrying out a take-home examination based on multiple-choice questions is not recommended.

Instead, open-ended questions going above and beyond the rote memorization of knowledge (such as explaining connections between specialized terms, transfer, synthesis, or assessments of subject-specific knowledge) are recommended. Didactic hints for designing exam questions accordingly can be found in the following information.

The didactic potential of take-home examinations is mainly in:

• giving students the option to demonstrate higher-tier levels of learning (such as the ability to argue, organize, synthesize, or assess knowledge, analyze situations and find solutions);
• using the examination questions to simulate challenging situations similar to future professional/academic practice (especially generating adequate solutions befitting the situation with limited time, available knowledge, and, if applicable, tapping into new sources of information)
• and therefore making competence-oriented examinations possible.

 

This is derived from the following context:

• Choice of examination format and type of examination questions strongly influence student learning processes (“washback effect”).
• Written classroom examinations, in which supervised students without access to other materials must correctly answer a number of (possibly standardized) knowledge questions favor memorizing knowledge (such as facts, theoretical knowledge, or subject-specific methods).
• The ability to reproduce subject-specific knowledge only requires the most basic academic competence developed in institutes of higher learning. Above and beyond is the capability to apply learned knowledge in the context of subject-specific practice for academic competence. This requires the goal of student learning to be aimed at understanding subject-specific connections as well as the transfer, synthesis, and/or assessment of subject-specific knowledge.
• Due to the characteristic qualities of a take-home examination (unlimited access to material, possible student collaboration), exam questions based on the rote memorization of knowledge are not very suitable (especially in the case of multiple-choice questions). Instead, questions in formats which require higher competence (transfer, synthesis, assessment) are recommended, as these require individualized examination performances and do not benefit from simply looking up information or collaborating with others.
• The same types of tasks which make unethical student behavior pointless justify the didactic potential of take-home examinations: since successful responses to transfer, synthesis or assessment-based questions require profound student knowledge, take-home examinations (with relevantly formulated questions) can result in competence-oriented examinations (and promote corresponding student learning behavior when preparing for the exam).
• You can find more information on the didactic potential of competence-oriented examinations with take-home examinations here.

Designing examination questions which require higher learning competence (transfer, synthesis, assessment)

Designing exam questions is the main challenge for take-home examinations for two reasons. For one, the didactic potential of competence-oriented take-home examinations is only activated with questions which go beyond the rote memorization of knowledge and require students to apply, analyze, synthesize, or assess. In written take-home examinations, this can be achieved by asking open-ended knowledge-based questions and using answering formats which require individual transfer or synthesis competence or reflection on a subject. In contrast, (standardized) knowledge-based questions with set answers (single- or multiple-choice questions) are not suited for this and are therefore not recommended.
On the other hand, due to access to the materials, missing supervision, as well as the asynchronous completion of tasks, take-home examinations allow students to research information and collaborate with each other. Therefore, exam questions should be designed in a way in which copying another’s work, researching information, or collaborating with others is not helpful – or researching information/student cooperation can be made part of the successful task completion. Here, too, open-ended questions and answer formats requiring higher and individual competence for transfer and synthesis are a good option. Additionally, this can be achieved by contextualizing questions, for example, when questions are tied to specific course content or students are required to show what and how they learned in the course (tasks requiring personal reflection).

>> Therefore, for teaching staff, the transition from classroom to take-home examination can be connected with the challenge of designing new examination questions and replacing existing (standardized) questions.

 

Significant increase in grading effort

If the transition to take home-examinations goes hand in hand with the transition from (standardized) knowledge-based questions to open-ended question and answer formats, the grading effort greatly increases. The automated grading of examinations requiring individual student output is not possible.

 

Constructive alignment must be kept in mind

The choice of examination format and the type of examination questions strongly influence student learning processes. How students learn depends on which strategy promises the most success in the examination (so-called “washback effect”, Elton 1987, Alderson/Wall 1993). Teaching staff can use this fact in their teaching: The desired student learning process can be encouraged by harmonizing the learning goals, teaching-learning process, and examination requirements of a course or module.

This so-called “constructive alignment” means,

1. designing the teaching-learning process of a course in a way which encourages precisely those student learning processes necessary for achieving the determined learning goals and
2. designing the examination in a way which requires students to demonstrate the actions named in the learning goals in order to achieve the best possible examination results.

In the context of a possible transition to take-home examinations and the likely resulting new questions, it is important to keep in mind the principle of “constructive alignment”.
One example: The learning goals of a course are mainly the conveying of factual knowledge. At the beginning of the semester, the examination format was announced as a written classroom examination. If the member of teaching staff wants to transition to a take-home examination in which transfer or synthesis competence is required, the consequences could be terrible. It is likely the students will have focused their learning process on the memorization of factual knowledge in order to be successful in the expected (standardized) fact-based exam. A change in examination format to a competence-oriented take-home exam can result in significantly worse exam results.

On the other hand, it is possible to encourage better exam results through “constructive alignment”. Another example: In a different course, the learning goals aim for a deeper understanding of the course material, as well as the ability to transfer or synthesize knowledge, but previously, a (possibly standardized) written classroom exam was scheduled as final exam. Because of this exam format, the students focus on memorizing the course material – and don’t achieve the actual learning goals. By transitioning to a take-home exam with non-standardized transfer and synthesis questions, the required exam performance can be better attuned to the actual learning goals and the course’s teaching-learning process, therefore promoting the desired student learning process.

>> By adapting the level of the examination questions, students are encouraged to work through the course materials during their examination preparation with the aim of a deeper understanding (as long as they have been informed of the examination requirements) in order to be prepared for examination questions requiring the transfer or synthesis of knowledge.

Literature
Alderson, J. Charles; Wall, Diane (1993): Does Washback exist? Applied Linguistics 14 (115-129).
Elton, L. (1987): Teaching in Higher Education: Appraisal and Training, London: Kogan Page.

The following information is not only valid for take-home examination questions. Much of it is generally valid for designing examinations and will be familiar for experienced examiners. In the following, we have collected the 6 most important recommendations to help in constructing and reflecting on exam questions:

 

1. Avoid standardized exam questions requiring rote memorization

Check your existing examination for questions requiring the rote memorization of subject-specific knowledge and questions in standardized form (such as multiple-choice, matching, or fill-in-the-blank). These should be replaced with open-ended questions which require higher learning competence (application, assessment, synthesis/creation of subject-specific knowledge).
Reasoning: (Standardized) questions requiring rote memorization are easily answered through research (internet or course materials) as well as through collaboration among students and are therefore not recommended. Additionally, open-ended answer formats can promote higher learning competence.

 

2. Didactic design of take-home examination questions

Take-home examination questions should be designed in a way that they
a) promote a higher and constructive student learning performance (application, analysis, assessment, synthesis/generation of knowledge)
b) use open-ended answer formats
c) are designed as contextualized question formats (such as realistic case examples, connections to case examples discussed in the course).
You can find detailed information on how to didactically design take-home examination questions here (in German).

 

3.Keep in mind constructive alignment

As a rule, the student performance expected in the examinations should correlate to what was taught and practiced in the course (so-called “constructive alignment”). If your course‘s learning process is based on the construction of subject-specific knowledge and students’ ability to memorize it, and higher levels of student learning performances were not practiced/expected, a transition to a take-home examination requiring higher levels of learning performances with open-ended question formats might lead to worse examination results. In this case, the transition to a take-home examination is not recommended.

 

4. Determine the time frame for taking the take-home examination

The time students are allowed to take for working on a take-home examination at JGU is a maximum of 4 hours. This does not mean that all of the students must take the examination at the same time. You can determine a longer time frame during which students can choose the starting time of the examination individually. For example, the students might be granted a time frame of 24 hours, during which the take-home examination must be completed in three hours.

Please keep in mind that students might also be confronted with challenges (such as care duties, changed circumstances in securing their livelihoods, etc.). Flexibility in taking the examination can accommodate changes in student living and working conditions.

On the other hand, asynchronous examinations offer greater opportunity for student collaboration. However, the more student examination performance is individualized through accordingly asked questions (by requiring higher, constructive examination performances, contextualized questions, etc.), the less helpful student collaboration will be.
In view of the time frame granted (synchronous or asynchronous take-home examinations), technical and organizational conditions are to be kept in mind in addition to these didactic points (see also "How can take-home examinations be implemented online? ")

 

5. Evaluation of take-home examinations

As open-ended questions result in individualized student answers, the automated grading of these take-home examinations is not possible. By transitioning to take home-examinations, the grading effort can therefore greatly increase.

For evaluating take-home examinations, we recommend developing sample answers or criteria grids.
They can increase evaluative objectivity and reduce the grading effort by determining the standards for evaluation ahead of time using sample answers or criteria grids.

This can be especially helpful for examination questions with short answers (such as questions in ILIAS with limited characters for the answers).

For essay questions requiring longer answers (such as the development of an argument), a criteria grid is helpful. You can find detailed information on creating a criteria grid here (in German). Determine ahead of time how you want to proceed when grading aspects such as grammar, spelling, citations, writing style and flow, use of terminology, irrelevant/imprecise information, and let the students know.

 

6. Clearly communicate the examination conditions ahead of time

Inform the participants clearly and explicitly of the examination conditions of the take-home examination ahead of time:

- Which type of examination questions?
Inform the students of the type of questions awaiting them in the examination in a timely manner. As take-home examinations allow access to resources, some students might assume that intensive preparation is not necessary for the examination. Therefore, make sure the students know the examination questions will not be answerable through rote memorization, but that other kinds of learning performances are required, and research will not be a major part of the examination or a great help for performing well. Ideally, give the students a sample problem for every type of question in order to make this clear.

- Work time and, if applicable, time frame to complete the examination
Inform the students of how much time they will have to complete the exam ahead of time. The maximum time is 4 hours.
If you are granting the students a longer time frame during which they may complete the examination, make sure you communicate that clearly, as well.
For example, the students could be informed as follows: „The examination must be completed within the time frame of midnight on July 15, 2020 to midnight on July 16, 2020. Once begun, the time you will have to complete the examination is 4 hours.” For an examination carried out online in ILIAS, include: “Therefore, if you start the exam at 2 pm on July 15, 2020, the examination will automatically be concluded at 6 pm.“
For an examination taking place as a task on moodle, include: “Therefore, if you call up the examination questions at 2 pm on July 15, 2020, you must upload your results by 6 pm on July 15, 2020, at the latest. Otherwise the examination will be counted as failed.”

- Scope and weighting of tasks
Inform students of the maximum number of points to be earned for each question and what scope the answer should have. For essay questions requiring longer answers: How many pages are students expected to write? For questions with shorter answers: Specify the maximum length the answers may be (for open-ended questions in ILIAS, the maximum number of characters can be set).

- Citation rules
Set clear rules for what (such as literature, lecture notes, contributions from others in the course) and how to cite.

- Use of resources
Clearly indicate your expectations regarding the sources of information to be used to answer a question (limited to course materials, use of further literature, internet research). As a rule, students have the option of accessing every kind of resource at any time.

PLEASE NOTE: Since only ILIAS has an established archiving management system at the moment, this platform should be used for online examinations. If other platforms are used, storage periods and archiving must be guaranteed by the teaching staff some other way.

 

Handing out the examination questions and submitting the results can be done via e-mail or by downloading and uploading in a Learning Management System (LMS), alternatively, students can complete the work entirely in an LMS:

 

Option 1) Examination questions are handed out and the results are submitted via e-mail, the students work on the tasks offline.

• Handing out and submission via the student e-mail address.
• Make the declaration of independent work available to your students, so they can submit the filled-out form with the completed work. You can find a template for the form here (Word-Form, in German).
• Please note that a precise submission time (e.g. 3 pm on the dot) is not recommended for technical reasons, as e-mails can take some time to arrive.
• Please note that that you may only send e-mails to a maximum of 1000 addressees in a rolling 24-hour time window and a maximum of 100 e-mails may be sent per minute. Each e-mail may have a maximum of 200 addressees. To be on the safe side, it is recommended that you keep your e-mail client open until all of the e-mails in the outbox have been sent.

 

Option 2) Examination questions are handed out and the results are submitted via download and upload in an LMS, the students work on the tasks offline.

• Every learning management system available at JGU (JGU-LMS, JGU-Reader, Ilias, OpenOlat) has the option to make documents available for download for course participants. Course participants can upload their work, which is visible only for the teaching staff.
• Make the declaration of independent work available to your students, so they can submit the filled-out form with the completed work. You can find a template for the form here (Word-Form, in German).
• In JGU-LMS (moodle basis), Ilias and JGU-Reader, a precise submission time can be determined. After that, student work cannot be uploaded any more. In order to prevent disadvantages resulting from technical problems, we recommend allowing students to submit their work via e-mail as an alternative.
• Due to the high traffic of the LMS in the digital summer semester 2020, problem-free simultaneous download requests coming from a great number of students cannot be guaranteed.

 

Therefore, we recommend:

• Using JGU-LMS and the JGU-Reader for take-home examinations with a set examination period longer than 24 hours. By stretching the amount of time available, the use of these LMS should be possible for a greater number of course participants (>100) as well.

• For shorter examination periods, the end of the submission period should be set to a point during which the ZDV’s employees are still at work, in order for them to be reachable if there are technical problems. An immediate solving of the problem cannot be guaranteed, however.
• Use Iliasfor take-home examinations with set examination periods of up to 4 hours. For several reasons, you need to have your Student Advising Office reserve a virtual examination room (Take-Home 1-4) using RAPS for that time period: Otherwise too many participants might access the server simultaneously. At the same time, support can be guaranteed through the ZDV’s e-learning team and examination periods can be taken into account when doing maintenance.
• You can find an FAQ on take-home examinations in ILIAS on the e-learning team’s pages (in German)
• OpenOlat is hosted by VCRP, therefore we cannot offer any information on the limitations of the system.
• According to § 3 (3) 2 d of the Corona-Statutes, the candidates are to be given the opportunity to familiarize themselves with the electronic system before the examination. Therefore, we recommend giving the students the option to down- and upload within the framework of a corresponding practice examination.

Example on setting the examination period: Activity “Test” in the JGU-LMS.

 

Option 3) The tasks are completed online in a Learning Management System (TH-Online Examination) 

• For a smooth process, a stable internet connection and uninterrupted access to the Learning Management System must be available for the complete examination period.
• The ZDV recommends usage of the LMS Ilias. Over the course of many years, JGU has had good experiences with online examinations of Ilias. Specified technical support can only be offered for this system. You can find the take-home examination hotline for Ilias here.
• Please inform your Student Advising Office regarding date and time of the take-home examination. The Student Advising Offices will enter the dates in RAPS so the ZDV can register the number and time periods of the take-home examinations in the support-planning tool, which is connected to RAPS. This way, periods of heavy traffic can be avoided, and maintenance work can be scheduled to not interfere with examination periods.
• The examiner should be available for students’ questions either by phone or via chat during the examination period. Similarly, students should be available for their examiner in case of corrections to an examination question, for example.
• The declaration of independent work can be represented in Ilias through prerequisites, corresponding documentation is currently still being developed. The upload of a separate declaration of independent work is then no longer necessary.
• According to § 3 (3) 2 d of the Corona-Statutes, the candidates are to be given the opportunity to familiarize themselves with the electronic system before the examination. Therefore, we recommend offering the students a sample examination.
• While it is possible to change a regular e-examination to a take-home examination, this scenario is only be recommended in certain situations for didactical reasons.You can find information on the technical transition here (in German).

You can find an FAQ on take-home examinations in ILIAS on the e-learning team’s pages (in German).