Friday, November 20, 2015

Is Your Grading Biased? Beware the Halo Effect

Humans are prone to taking mental short cuts, it's part of the physiology of our brains. We are exposed to so much information throughout our lives and from day-to-day that it is necessary for our brain to utilize mental short-cuts (i.e., heuristics) to help sort information and to determine what should grab one's attention. Heuristics are mental guidelines you create from your experience to help you reach an answer more quickly than by algorithm searching through every possible answer. The only problem is that this quick thinking can also bias cognition and decision-making, and grading is no exception.

The Halo effect is one form of heuristic bias that impacts everyone's decision making. It happens when we find good qualities in an individual and, through experience, incorporate those good qualities as part of our perception of the person. This biases the way we perceive their actions, whether good or bad. For instance, attractive individuals are often perceived to be more intelligent, talented, and generally good more often than individuals of average attractiveness.

The Halo effect can produce biases in grading just like any other area of judgment. In a previous post we discussed briefly how faculty will frequently grade students based on their personal characteristics and past performance rather than solely on their performance on the work in question. Faculty may begin reading an assignment, stop and think, 'so-and-so usually does great work' and allow that judgment to alter the way they grade the assignment.

This bias in judgment and decision-making process is exactly what researchers found at work in a study examining the Halo effect in subjective student assessments. The researchers randomly assigned faculty to observe a student perform either very poorly or very well on an oral presentation, and then graded the student on a written assignment. When grading the exact same student on the exact same writing assignment, faculty members gave substantially higher grades when the students had done well in a presentation before being graded and lower grades when the students had done poorly before being graded. Other than giving either a poor or good oral performance, the student was the same and the quality of the written assignment was the same. And yet the grades were different. Prior experience with students will bias the way faculty members grade students on future assessments, but by being aware of this "Halo effect", faculty can more effectively protect against bias. Making student assessments anonymous is another effective was to reduce biased grading.

Wednesday, November 18, 2015

Making Office Hours Required for Students: The Best Bad Idea You've Ever Heard

It is well known that students who build working relationships with their professors are
more successful both in college and after graduation. One of the best ways to cultivate that relationship is the simple step of coming to a professor's office hours.

Despite the fact that office hours are theoretically beneficial for students, very few students seem to take advantage of this valuable opportunity. Office hours should be the time students are receiving much-needed mentoring that can lead to increased student engagement, leading to better performance in the course and ultimately to higher graduation rates.

Should faculty then make office hours mandatory for students? The logic behind this move is simple. Much is lost in the translation of course material and knowledge from the professor to the student during the lecture because it lacks personal and motivational factors that are typical in more personal conversations. This effect is compounded when grades and feedback are given only through writing, whether digital or analog. Making a personal connection has always been, and is still, extremely helpful to the learning process. Although practical ways to implement required office hours depend tremendously on course size, the idea that students benefit from engaging with their professors on an individual level about course topics is wholly unsurprising and is worth pursuing.

Let us know your feedback. Do you require office visits for your students? What would be some advantages or disadvantages?

Tuesday, November 17, 2015

TurnItIn Provides a Valuable Service, But Jeopardizes Students' Rights

Realistically, if you're a professor who needs to ensure students aren't cheating on their papers, is a true time-saver with few alternatives. TurnItIn is a tremendous boon for professors who simply do not have time to painstakingly check their students' written work for plagiarism against the millions of other scholastic documents on the internet. In an insightful article published last month, one professor praises TurnItIn for its incredible efficiency and criticizes it for its monopolistic hold on students' original works.

The issue hinges on what TurnItIn does with all of those student papers it receives. It keeps them, archives them, and continues to use them for plagiarism checking. Whereas this is more of an issue in principle rather than reality, the fact remains that students are required to give over their intellectual products to a company that will use them to make a profit. Some may find that to be a bit unsettling, although there is no apparent harm that comes from it. After a 2007 lawsuit, a district court determined that TurnItIn is not breaking any copyright laws in their practices. Despite the fact that it is not technically unlawful, TurnItIn does influence the rights of students to be in control of their intellectual property.

As term papers, research manuscripts, and other long written assignments pile up, it is hard to argue against the benefits of using a site like TurnItIn to protect academic integrity and ensure the quality of students' educations. But there is a hidden cost of student ownership that, although practically innocuous (for now), may trouble the ethical constitutions of some.

Friday, November 13, 2015

For Ph.D.s Working Outside Academia, Incorporation Holds Benefits

Occupational prospects for Ph.D. holding scientists and researchers have become increasingly sparse in recent years. In 2012 around 10% of scientists with a Ph.D. were unemployed. For many highly-educated individuals who have been unable to find careers as a university faculty, working as a freelance researcher, consultant, grant writer, or scientific worker outside of academia has been the best available option. Regardless of employment status, Ph.D.s may stand to benefit by incorporating themselves into a limited liability company.

According to Andrew Thaler, Ph.D., incorporation provides a more efficient way to manage finances and make services more appealing to outside sources of work/funding. As a "single-owner S Corp limited liability company (LLC)", for instance, personal wealth is better protected from lawsuits. In addition, there are a number of tax benefits afforded to a LLC that are not available to an individual person. Perhaps more importantly, forming a corporate organization allows a given Ph.D. to be more competitive for research grants and other opportunities for which they may otherwise be overlooked. Funding organizations are less hesitant to pay large amounts in the form of a grant or other vector to a professional consulting organization than they are to an individual. Organizations or corporations appear to be more accountable and therefore less risky in terms of investment. Additionally, the perception of being the leader of an organization, rather than a lone, semi-unemployed freelancer, is more appealing to those who can offer opportunities to further your career.

Incorporating can serve to legitimize the functions and services that are likely to produce greater opportunity for career growth and success. Futures look positive for more Ph.D.s to seek incorporation regardless of their current employment status.

Wednesday, November 4, 2015

AACU Encourages Civic Learning, Provides Practical Ways to Implement It

The American Association of Colleges and Universities (AACU) has made available a free publication on practical ways faculty can better produce civic learning. The publication, called Civic Prompts, addresses a goal that has been part of the American education system since its inception: to instill the values of democracy in American college students and inspire and enable them to take democratic action and further democratic process. In recent years, the American higher education system has been charged by United States leadership to define how each discipline can contribute to public well-being and how to incorporate civic learning into coursework.

Civic Prompts helps to provide faculty with resources to encourage civic learning and generate students who will use their degrees to benefit the nation as a whole and advance democracy worldwide. Some prompts include, "What are some big issues that are common to your disciplinary domain that lend themselves to civic inquiries and/or actions?", "What kinds of assignments generate a line of civic questioning or civic actions within the context of your disciplinary or interdisciplinary course?", and "What are some civic pedagogies suited to your disciplinary domain?". The AACU hopes priming faculty with these prompts will make civic learning more routine across the disciplines of higher education.

Tuesday, November 3, 2015

Study Suggests Student Self-Assessment Is an Untapped Resource

RM Tamrakar;
Students are constantly being academically assessed,usually by professors, in the form of exams and quizzes. Students are rarely given the opportunity, however, to assess their own work. Researchers found that students who voluntarily self-assess become better able to make accurate judgement about their own work. A recent study showed that, as students continued to voluntarily self-assess their work, the judgments they made about their work became more closely aligned with those of tutors who were experts in assessment for the particular subject.

What does this imply for higher education faculty? It looks like it is a good idea to allow your students to self-assess more often. Although this study evaluated voluntary (self-selected) student self-assessment and not the effects of student self-assessment overall, it is theoretically likely that self-assessment would improve students' academic judgments of their own work regardless of whether they have chosen to self-assess on their own. Asking students to assess their own work in class or on assignments gives them a chance to develop assessment skills that they would otherwise have no opportunity to practice.

Self-assessment practice pays dividends post-graduation, when evaluation of one's own task performance does not come until after the performance is completed and cannot be taken back. Being able to accurately assess the quality of one's own work before putting it into action is critical in the real-world, where there are few second chances. Better self-assessment in college work could save professors more time, as students are better able to determine what areas need improvement before turning in completed work. 

Thursday, October 29, 2015

When Assessing and Correcting Personal Bias, Slow and Steady Wins

The first step in eliminating bias is acknowledging that a bias exist. However, few faculty members, or any other profession, are willing to acknowledge their own biases in the classroom. Although progress can come from large-scale change that trickles down into individual lives and everyday interaction, equally beneficial change can work from the bottom-up. Taking small steps on a manageable time scale to reduce our own biases can make all the difference.

Here are a few tips to becoming a little more equitable everyday.
1) Faculty are often tempted to grade students based on personal characteristics and their work, rather than solely on the quality of the work. Like many other mental shortcuts used for judgement and decision-making, making assumptions about another person saves massive amounts of time and are often accurate enough. Faculty may begin reading an assignment, stop and think, "So-and-so usually does great work," and, thus, skim the rest of the assignment with their mind already made up on what type of grade to give. However, when it comes to grading, using our first impression of a person is genuinely unfair. Students benefit when faculty take the time to thoroughly assess work for the work itself.

2) Whereas a faculty member may not be consciously biased, every human is biased on an unconscious level. If you don't believe that, follow this link and take the Race Implicit Attitude Test (IAT), Age IAT, or other available IAT. It's likely that you have no desire to act on your biases, whether unconscious or conscious, and simply becoming aware that you may be accidentally adjusting student performance evaluations or classroom experiences allows you to correct for biases.

3) Follow-up with yourself to continually evaluate how you are working to reduce your own bias.

4) Listen actively and empathetically to each individual with whom you speak. This allows you to look past surface-level characteristics and focus on content, substance, and quality.

5) Seek progress, not perfection. In other words, don't expect to ever fully eliminate bias, whether that's based on gender, race, age, or education level, because it is impossible to do so. But it's worth attempting to reduce as much as possible. By doing so, you are making a more fair and inclusive environment for your students.