CHAPTER 2 : ELEMENTS OF EFFECTIVE TA-ING

EVALUATING WHAT YOU'VE DONE

Evaluation is an important aspect of teaching and learning, and if done correctly can provide important information to both you and your students.

Evaluation involves both TESTING and GRADING. In addition to evaluating how well your students are doing, you may gain valuable insights about your instruction by EVALUATING YOUR OWN SUCCESS in class.

CONSTRUCTING TESTS

Undoubtedly you will be called upon to construct or help to construct a quiz, midterm, or final during your career as a TA. Depending upon your discipline, the test that you create will generally be either an OBJECTIVE test or an ESSAY test. Whichever type of test you need to construct, the planning phase of your test should be the same.

PLANNING THE TEST. If you followed the suggestion voiced earlier in this manual and specified objectives for your class, you will probably discover that writing a test is a fairly straightforward task.

REVIEW YOUR INSTRUCTIONAL OBJECTIVES. Compare these written statements of what you expected your students to be able to do to the instruction which was presented to them. You need to be aware if an objective that you wrote was not covered in class, or implied a different process than what was taught BEFORE you write the test!

DELIMIT THE CONTENT AND SKILL AREAS TO BE TESTED. Decide on some way of selecting test items (a sampling scheme) to ensure that your quiz/test is representative of what was covered in class and to ensure that it gives an adequate sampling of student capabilities.

OBJECTIVE TESTS ⁽⁹⁾

Although by definition no test can be truly "objective" (existing as an object of fact, independent of the mind), an objective test in this handbook refers to a test made up of multiple choice, matching, fill-in, true/false, or short answer items. Objective tests have the ADVANTAGES of allowing an instructor to assess a large and potentially representative sample of course material, measure most types of learning, and allow for reliable and efficient test scoring. The DISADVANTAGES of objective tests include a tendency to emphasize only "recognition" skills, the ease with which correct answers can be guessed on many item types, and the inability to measure students' organization and synthesis of material. Suggestions for constructing high quality objective tests are offered below:

DO'S:

• Write the test items simply and clearly
• If an item represents a particular opinion, identify the author of the opinion.
• Write clear and unambiguous directions for the test as a whole and for each specific section of the test.
• Assemble items into a test according to some systematic plan, e.g., similar item-types grouped together.
• Devise a system to facilitate scoring the test (e.g., a separate answer sheet and an answer key in the same format).

 

DON'TS:

• Don't lift a statement verbatim from a textbook.
• Don't provide clues in one item for answers to other items.
• Don't intersperse item-types on the test.
• Avoid writing interdependent items such that the answer to one item is necessary to respond to the next item.
• Avoid items dealing with trivia.
• Avoid trick questions.
• Avoid ambiguity in items.

 

TRUE/FALSE ITEMS

These items can assess student recognition of facts and definitions. To write good true/false items, we make these suggestions.

• Be sure a statement is unequivocally true or false.
• Avoid specific determiners such as "always" or "never"-they can be dead giveaways for FALSE items.
• Beware of indefinite terms of degree or amount such as "in most cases" and "great".
• Beware of negative statements and double negatives.
• Beware of including more than one item in the same statement, especially if both are not equally true or false.
• Beware of giving clues to an answer by the length of an item.

 

MULTIPLE-CHOICE ITEMS

This is often the most effective of the objective-type items. Multiple-choice items can measure such things as acquisition of information or vocabulary, application of principles, or evaluation of sample data. Here are some tips to help to ensure high quality in the multiple-choice items.

• The item-stem should clearly formulate a problem.
• As much of the item as possible should be included in the stem.
• Randomize the occurrence of correct responses (i.e., you don't always want "C" to be the right answer).
• Be sure there is only one clearly correct answer (unless you are instructing students to select more than one).
• Make the wording in the response choices (distracters) consistent with the item stem.
• Don't load the stem down with irrelevant material.
• Beware of using answers such as "none of these" or "all of the above".
• Use negatives or double negatives sparingly in the question or stem.
• Beware of using sets of opposite answers unless more than one pair is presented (e.g., go to work, not go to work).
• Beware of providing irrelevant grammatical cues.

 

COMPLETION AND SHORT ANSWER ITEMS

These are variation of the same thing. They are suitable for testing knowledge of facts (e.g., dates, vocabulary). Numerical problems are short answer items. The following suggestions should aid you in writing effective completion or short answer items.

• Don't leave too many blanks in a statement.
• Put blanks at the end rather than at the beginning of an item.
• Beware of indefinite items (many answers could be correct).
• Omit only key words for completion. Don't test for common words (e.g., Fall quarter BEGINS in September).
• In numerical problems indicate the type of units in which the answer is to be expressed if it doesn't give clues to the answer.

 

MATCHING ITEMS

This is a variation of the multiple-choice item. Matching items are efficient for measuring associations of names, dates, etc. In addition, these items can be used to label charts, maps, and the like. Matching items are best when these guidelines are followed.

• Possible answers are homogeneous in nature (i.e., all choices are names, dates, body parts, etc.).
• The number of answer choices exceeds the number of problems.
• Sets of answers are kept short. Make more matching items rather than one items with 400 answers.
• Answers are put in a logical order (chronological, alphabetical (if one exists).
• Directions specify whether answers may be used more than once and specify the basis for matching.

Questions that you write should assess whether or not students CAN DO what you expect them to be able to do after an instructional sequence. For a more detailed description of item-types and techniques for writing items, see A. G. Wesman's article, "Writing the Test Item" (Thorndike, R. L. (Ed.) Educational Measurement, 1971, pp. 81-130.

ESSAY TESTS

The principal advantage of an essay test is that it requires the student to produce and organize an answer, select or create a pattern of ideas, and to demonstrate critical thinking. Its principal DISADVANTAGES are that it typically assesses only a small sample of course objectives. Again, the following are some suggestions which may enhance the quality of the essay tests that you produce.

• Have in mind the processes that you want measured (e.g., analysis, synthesis).
• Start questions with words such as "compare", "contrast", "explain why". Don't use "what", "who", "when", or "list". (These latter types of things are better measured with objective-type items).
• Write items so as to define the parameters of expected answers as clearly as possible.
• Don't have too many answers for the time available.

 

ESSAY EXAMPLES ⁽¹⁰⁾

If an essay test is carefully planned and written, it can be a stimulating exercise in reasoning. In addition, such a test can serve to encourage students to summarize or expand upon a learning sequence.

The following pairs of examples are included to illustrate how questions may be reworded to demand new, more complex understandings and to help students synthesize and evaluate the course content.

	Original:	List the reasons for Weld's conversion.
	Reworded:	Noting both the traditional religious explanation for conversion and Erickson's treatment of the same phenomenon, use these theories to offer your own analysis of Weld's conversation.
	Original:	What were the accomplishments of F.D.R.'s first and second Presidential terms of office?
	Reworded:	How can you reconcile the two seemingly contradictory statements made in our class discussion that "F.D.R. saved capitalism" and tha "F.D.R. was a reformer-liberal"?
	Original:	In the "fluctuation test" of Salvadore Luria and Max Delbruck, what was the fundamental question they were addressing?
	Reworded:	Two contrasting results were possible in the experiments of Salvadore Luria and Max Delbruck. What were they, and how does each elate to the fundamental question they were addressing?

DETERMINING STUDENT GRADES

"...a grade {is}...an inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite amount of material." (Paul Dressel, BASIC COLLEGE QUARTERLY, Michigan State University, Winter 1957, p.6).

As this quote indicates and many undergraduates claim, grading can be unjust, arbitrary, and just plain problematic. In conjunction with the instructor of record for a course, you can do much to alleviate the criticisms and barbs directed at the grading process. The following sections offer some useful suggestions for grading both essay and objective tests.

GRADING OBJECTIVE TESTS

• PREPARE AN ANSWER KEY. If you design the answer key to follow the format of the students' answer sheets, the grading process will be much faster. If the test is based on your class objectives, you can also refer students to lecture notes and readings relevant to a given question on the answer key.
• DETERMINE IF THERE IS MORE THAN ONE CORRECT RESPONSE FOR ANY OF YOUR QUESTIONS. Decide, in advance, if certain incorrect answers merit partial credit.
• DECIDE, IN ADVANCE, THE POINT-VALUE OF EACH QUESTION. Ideally, you should give students this information on the test itself.
• POST THE ANSWER KEY OUTSIDE YOUR OFFICE. Students can then come by to check their performance against the key.
• DISTRIBUTE CORRECTED TESTS IN CLASS OR ASK STUDENTS TO DROP BY YOUR OFFICE TO PICK THEM UP. Don't leave corrected exams out in the hall near your office. Your "A" student may be delighted, but it's important to consider the feelings of those students who do poorly as well. (It is illegal to post test scores along with names, or even with alpha numbers.)

 

GRADING ESSAY TESTS ⁽¹¹⁾

• READ YOUR STUDENTS' EXAMS WITH SCRUPULOUS CARE. Your students have invested a good many hours in preparing for, and taking, examinations. Evaluating their responses deserves your undivided attention.
• PREPARE MODEL ANSWERS IN ADVANCE. Whether you've written the test yourself or you're reading an exam prepared by someone else (e.g., the professor in charge of the course), preparing a key with model answers clarifies the major points that should be covered in student responses on the test.
• DETERMINE WHETHER THERE IS MORE THAN ONE ACCEPTABLE ANSWER OR MORE THAN ONE POSSIBLE SOLUTION TO A PROBLEM.
• PREPARE A LIST OF COMMON IMPROPER ANSWERS, TOGETHER WITH THE PENALTIES FOR EACH. This will help to ensure that your grading is consistent. If you do not hold to uniform grading standards, it is likely that at some point you'll be faced with at least one angry student presenting you with two nearly identical responses to which you have assigned different scores. That student will justifiably want you to account for your grading procedure.
• READ ALL THE ANSWERS TO ONE QUESTION (BY ALL STUDENTS) BEFORE BEGINNING TO GRADE OTHER QUESTIONS.
• GRADE PAPERS AS NEARLY ANONYMOUSLY AS POSSIBLE. Oftentimes it is useful to ask students to record their alpha numbers on an exam rather than read their names. This means matching the numbers to their names after reading the exams, but it helps to remove biases you may have acquired toward students in your class.
• DECIDE, IN ADVANCE, HOW IMPORTANT SUCH FACTORS AS SPELLING AND GRAMMAR WILL BE IN GRADING THE EXAMS. Decisions of this nature should not be arbitrary. What's right or wrong for one student's paper should be right or wrong for all students.
• AFTER YOU'VE GRADED THE TEST, POST OR OTHERWISE MAKE AVAILABLE AN EXAMINATION KEY. This allows your students to see the major points their essays were expected to cover or to see the correct method of solving problems on the exam. In addition, it is often useful to cite common errors that were made on the test and the points removed for such errors.
• ESTABLISH THE CORRESPONDENCE BETWEEN POINT SCORES AND LETTER GRADES BEFORE RETURNING THE EXAMS. Students tend to mark their progress in terms of grades and will want to know how they fared. Remember, you must be consistent in assigning grades based on test points.
• SUBJECTIVITY SHOULD NOT ENTER INTO ASSIGNING GRADES. If, however, you want to introduce some element of subjectivity into your grading (e.g., adding points for student participation in class discussion) make it clear to students that this is your intention. Students should be told at the outset of the course or prior to the exam the maximum and average number of points that you are allotting to this category of your evaluation.
• IF TIME ALLOWS, WRITE COMMENTS ABOUT STUDENT RESPONSES. Let your students know when they've really done an outstanding job as well as providing them with correct information (or a source to find it) when their responses are inaccurate.

 

EVALUATING YOUR TEACHING

As many TAs have already discovered, it is generally desirable to obtain some form of student input regarding their teaching and the course they are instructing while that course is still in progress. Such feedback can be used to make changes while a section is still going on and may be used in conjunction with end-of-term evaluation to plan for the next quarter.

You can gather MID-QUARTER FEEDBACK by distributing a short questionnaire to your students around midterms. The intent of mid-quarter feedback is to provide you with information reflecting student opinion about specific aspects of your section, e.g., clarity of presentation, relevance of material, willingness to respond to questions. These brief surveys can also provide some global reactions to the course in general. By reviewing these questionnaires right away, you may be able to make changes that will affect the students from whom the feedback was obtained.

Another very different way to evaluate your success as a TA is to utilize the campus wide TA Development Program's VIDEOTAPE AND CONSULTATION SERVICE. This service allows you to be videotaped while teaching. You can then gain a new perspective on your in-class performance by viewing your tape. By viewing your tape, you will be able to gain a "student's eye" view of your teaching and, at best, after receiving feedback on your tape you may be able to make changes in your teaching strategies. To arrange a videotaping, call TV Services x4346.

---