Tuesday, April 17, 2018

Grading True/False Questions

A student just told me that they had gotten a true-false wrong, but knew the answer. They had marked it "True" and then written in the margin, "It's true, but you made a typo: it should be x/y" (or whatever the math had been— I didn't understand the question well enough to know what she was talking about). So, as instructor what do you do with that?

Mark it correct of course. The correction demonstrates the knowledge we were testing for, and even if they missed the entire point of how True-False is supposed to work, we're ultimately trying to determine their knowledge of course objectives, not their understanding of test-format. So that's just funny, but deserves a mark.

Indeed, I would argue that all True-False should work that way. Good T/F design* includes students explaining why the F options are false. That not only catches the sort of problem above, but ensures that students choosing F aren't just guessing, and that they know the right answer. There may also be more than one way to make a false statement true...that's all okay, as long as it demonstrates the students knowledge (or lack thereof).

T/F Ice freezes at 0 degrees C

That's true; but 'false' and adding "at sea level" is also correct and tells you even more about the student's knowledge. Both answers should get the mark.

*Okay, that's an oxymoron: almost all T/F are useless and to be avoided, but there are a few special circumstances—memorization of definitions, names, or dates, as if any of that is ever useful—where they can be used successfully.

Saturday, March 10, 2018

Found on Twitter

Found on Dr. Alex Couros (@courosa March 10, 2018) Twitter feed with the comment "Well played".

I deeply appreciate any instructor who grades the answer as presented rather than only the expected replies. Full marks here for both the student and the professor!

Friday, March 9, 2018

Teaching Excellence

Only slightly off topic of student assessment is the question of how one evaluates good teaching. Just as with student assessment, defining one's objectives and how to measure progress toward those goals is in fact a highly political activity. Why this learning objective and not that one; why is this particular standard considered inadequate, satisfactory, or excellent; why this group's (students? peers? parents? administrators? stakeholders? taxpayers?) perceptions of quality rather than some others? Do we measure excellent teaching against student expectation, student learning, student engagement, student enjoyment, student self-fulfillment; or by employer needs and expectations, graduate employment figures, graduate life chances; by political socialization or active citizenship; critical thinking or ideological conformity; or societal arts and culture, inventiveness, entrepreneurialism, the reproduction or elimination of poverty and injustice... You get the idea.

I'm very pleased to have "Excellence for what? Policy Development and the Discourse of the Purpose of Higher Education," appear as a chapter in the just-released Routledge collection, Global Perspectives on Teaching Excellence. The collection is basically a reaction to recent legislation in the UK that attempted to measure and mandate teaching excellence in higher education. My wife and I wrote a critique using my discourse analysis model of the purpose of higher education applied to the new legislation to suggest that the government's definition of 'excellence' might be somewhat problematic from the perspective of students and learning.

Monday, January 22, 2018

Dyslexia and assessment.

Here is a good video on the experience of dyslexia from the student perspective.

I'm often shocked by how poorly many teachers adapt assessments to deal with disabilities, such as dyslexia. I hosted a panel on "Dyslexia, dysgraphia and the writing experience" at a writer's conference in August, and the room was filled with parents openly weeping over how their dyslexic child had been treated in the school system. The parents were immensely relieved to see a panel of writers and editors talking about how they had managed to not just overcome/deal with their disability, but actually become successful readers, writers and editors in spite of what happened in their schooling.

I keep running into teachers who don't know what dyslexia is or how to deal with it, even though research suggests it affects about 15% of the population. They're always surprised when I tell them that the reason this child can't spell is that they are dyslexic or dysgraphic.

"I just thought they were being lazy,not making the effort to learn how to spell the word."

Well, "lazy" is better than "stupid", I suppose, which is how the peer group labels poor readers/spellers.

"Why are you making this child read out loud in front of their peers? What is the purpose of this public humiliation?" I will ask them.

"Well, you've got to read out loud. It's in the curriculum."

"Okay, but have them do that at recess or after class in private. Assessments are supposed to be confidential."

"Really? But this course objective says, 'to an audience'".

"Okay, but then give them the reading ahead of time so they can decipher it at home, learn it as a recitation if necessary, so they are able to be successful. Do not make them do a cold reading where so you set them up for a public failure."

"I don't know. That seems to be giving them an unfair advantage, somehow."

And so on, endlessly. No variation from the routine, the easy; no sense that accommodations are necessary to level the playing field. Often hints that the teacher secretly believes the child really is stupid, because how hard is it to read this very elementary book? Most troublesome are the teachers that, having assigned the child to the lowest reading group (which is of no help because they are still dyslexic and now surrounded by peers who act out with various forms of antisocial behavior because they are all being subjected to the daily humiliation of being forced to attempt reading aloud; and the reading material is so far below grade level as to kill anyone's motivation to read) keep them in that group for social studies ("well, that's mostly reading too, right?") and math and art ("Well, they might as well stay in the same group all day, right? It's just easier for everyone") even though the research is conclusive that streaming isn't useful for any student, and in the long run, not even a benefit to the teacher since it leads to more behavior and control issues.

And then there was the teacher with the fill-in-the-blank worksheet. Read the chapter, fill in the (apparently random--not even key words) blanks. When I questioned why she was making this dyslexic child read and fill in the blanks when the student (a) couldn't read and (b) couldn't write, she conceded that that might be a problem.

"I'll give the child the answer sheet. That way they won't have to read the text or spell the words."

But then, why make them do it at all? What was the possible purpose, the learning outcome, of laborously transferring random words from one sheet to another, one letter at a time? How is that an authentic assessment? How exactly did that teach this student the content or the spelling, when they couldn't actually read the sheet?

"But what do you want me to do? How can I teach the course if they don't fill in the worksheet?"

If a student teacher had said that to me, I would have failed them on the spot. Worksheets are your only teaching strategy? You're done now! But it wasn't a student I was dealing with, so okay, how to explain that they had confused means and goals. The worksheet is supposed to help students achieve a learning outcome--but somewhere along the line, completing the sheet had become the goal for this teacher, a meaningless daily productivity that the teacher could point to to say, "my students are working and learning". But of course they weren't learning at all.

Well,not the content, anyway. They were learning that they hated that teacher's subject, that school is inherently boring, that work is tedious. If the purpose of schooling is to condition graduates to the tedium of the adult workplace, conditioning them to tolerate a 9-5 routine of pointless, alienating labour, then yeah, this teacher was doing just fine. But I don't think that's why we chose to become teachers.

And don't get me started on some teachers' insistence on teaching cursive, when no adult under thirty ever uses cursive writing; and dysgraphic kids literally can't do it. It is exactly like demanding the kid in a wheelchair compete in the 100 meter race or fail phys. ed., but somehow, nothing deters these adults from pointlessly torturing and humiliating children with these learning disabilities.

Schools as we know them were built around reading and writing. I think that's still important, but I'm confused why audiobooks and instructional video and the rest are not also considered valid if they help nonreaders achieve learning objectives. Why pride of place to just that one medium of print? Why steamroller over the 15% of kids who can't read or write by insisting that that is the only way to complete assignments? It's assessment abuse.

Find other ways to find out what the student knows and can do. That is the point of any assessment. If a student cannot do the default assignment, find some other way to assess their knowledge and skills. The assessment is the means to the goal, not the goal. It's that simple.

Tuesday, October 17, 2017

Bringing Notes to the Examination

A meme downloaded from "S**t Academics Say" Oct17, 2017:

There are several points to be made here.

First, good on the examiner for recognizing that was on them, not the student. I detest when an instructor makes a mistake and then tells a student, "well, that's not what I meant!" Unless mind reading is criteria in your psychic examination, it is inappropriate to hold students accountable for the instructor's intentions. If they got the literal answer to one's poorly worded question or followed one's poorly worded instructions literally, they get the mark or behaviour.

Second, you have to love this student! That is out of the box thinking! Exactly what every discipline needs more of.

Third, open book examinations are actually harder. Okay, if it's closed book for everyone else and open book for you, that does give one a slight advantage, but not as much as most people assume. There is relatively little knowledge that we need to memorize, as opposed to understanding and having available to us on our computers or google or etc. In most disciplines, one ends up memorize the important facts because that's easier than looking them up every time. If the fact is one you have to look up, it is by definition one that did not come up enough to be worth memorizing. Exams should be testing understanding, not just rote memorization. If the exam is testing higher orders of cognition (analysis, synthesis, evaluation and so on) then making information available to students makes for a better test of skills than the usual rote memorization that many exams seem to be geared towards.

Thursday, April 13, 2017

Science Fairs

My brother-in-law, who is an excellent physics teacher, spent a lot of years founding, running and judging science fairs. Science fairs, done correctly, are an excellent learning tool.

But um...

I feel the need to provide an alternative, student-eye view of science fairs:

I think my brother-in-law's science fairs worked better because, in part, they were not compulsory, but intended for those who were actually interested in science and in participating. (The other part is that my brother-in-law is kind of an outstanding teacher, so, you know, don't necessarily try this at home, or your school, unless you actually care as much as my brother-in-law.)

Monday, April 10, 2017

Group Work

Pretty much my experience of group work: good students who want a good mark have to carry the unmotivated because they do not want to take the hit on their grades, while there are essentially no consequences for one or two students doing nothing.

There is a lot of lip service given to using group work to teach collaborative skills, but the cartoon reflects what students actually learn; i.e., that group work is a fraud. As I have stated before in this blog, group work can only contribute to learning if relevant theory on group dynamics and instruction in practical collaborative skills have been explicitly taught and assessed, either in the current course or a prerequisite. Even then, successful group work requires considerable preparation by the instructor to carefully structure tasks, distribute the workload equitably, and have clear and thoughtful assessment. I have seen this used appropriately in a few rare programs, but the true purpose of the vast majority of group work assigned is merely to reduce the instructor's marking load. Bah humbug!

Given motivated students and topics students care about, small group (in-class, ungraded) discussion can work well, provided students are given structured activities (e.g., a series of questions to discuss) and a clear time limit (to stay focused).

Any other group work that requires students to get together outside of class discriminates against students with long commutes, single moms, students working their way through college, mature students with elder care issues, minority students whose family or cultural expectations prohibit their participation outside of school hours, or etc.

Group work for marks where the group processes are not directly observed and assessed by the instructor make it almost impossible to assign grades fairly.

Where groups are allowed to chose themselves, 'A' students will strive to join together, excluding not just 'B' and 'C' students, but also anyone with a visible disability, member of a lower social class, any member of a cultural minority, and so on. Group work where group members are not assigned by the instructor are therefore inevitably sexist, racist, classist, abilist, and so on. Instructors who do not assign students to groups must recognize that the primary purpose of group work in their class is to ensure the reproduction of the current social order rather than to certify achievement based on talent and effort.

Assigning students to work together in groups ensures that A students will be frustrated by having to do a greater share of the work to ensure a minimally acceptable grade and/or accept a significant 'hit' to their grade point average. Assigning students to work groups without adequate training in group collaboration skills and appropriate checks and balances on workload distribution and task assignment ensures group conflict, and an end product that is greater mess than the sum of the flaws introduced by each individual.

I'm not a fan.