Last week Occupy Math looked with deep skepticism at standardized tests. This week, a related problem: are our math tests — from in class to the scholastic aptitude tests (SATs) — testing math or something else like language proficiency? Occupy Math thanks Catherine Emerson for pointing to this topic and Peter Ashlock for serving as Occupy Math’s research staff on this one. The key point is this.
Math tests often test language proficiency and cultural norms, not math.
On a college admissions test like the SATs, this materializes as a bias toward wealth and membership in the dominant culture, which is an example of structural racism, racism built into the very structure of society. In a high school enriched with immigrants, this effect has (not can: has) resulted in reduced funding for schools that need added funding to help immigrants adapt and become productive citizens sooner. This latter observation is another fruit of Occupy Math’s attendance at the Canadian Math Education Study Group.
There were four teachers at the conference, one of whom worked in an immigrant-enriched school. The standard Canadian assessments of math skill require that the person taking them be highly proficient in English or French (two versions). The tests forbid the person administering them to speak with the students at all beyond interactions like “yes, you may go to the restroom”. Imagine how a 6th grader from Guatemala or Uganda feels when they are asked to take a very important test that makes no sense at all and the teacher just says “I’m not allowed to explain”. Forcing the teacher to obviously betray his or her students is not a good teaching technique!
The result is that the Canadian math-skills assessment says the school is doing a bad job based on its student’s lack of English or French.
A school’s ranking on this test is public. This completely unfair assessment causes parents with a choice — which a lot of Canadian parents have — to take their sons and daughters elsewhere. This then increases the density of immigrant children, whose parents are less likely to be tied into the school’s rankings and so are less likely to make a decision based on them. This impacts enrollment, funding, and reputation. This is, in fact, a mechanism by which schools can be ruined in a slow chain of falling dominoes. Again – this is a situation reported to Occupy Math by a front-line teacher.
Let’s move beyond Canada.
An article published by Reuters is part of a series on the SATs. Occupy Math thinks the article worth reading in full, but here is the headline.
Internal documents show the makers of the new SAT knew the test was overloaded with wordy math problems — a hurdle that could reinforce race and income disparities. The College Board went ahead with the exam anyway.
The way you phrase a math problem can make all the difference. While math is universal truth, as soon as someone writes a math problem, it gets all tangled up in language. From the student who, when told to “Find x“, circled the letter x to the hypothetical immigrant child asked a complex question about cars leaving places they’ve never heard of in directions they use other words for, there are many issues with how we choose to present math problems. I’m going to ask Bill Watterson for some help explaining the sense of this one, through his character Tracer Bullet.
What is the effect of the SATs choosing to present their math questions in wordy form?
- The SAT test is excellent at predicting income,
- It is bad at predicting grades and academic success, and
- It is specifically biased towards up-checking white people, not only rich people.
Given the criminally irresponsible failure of American states to fund universities, it may be that an admissions test that ignores academic potential but accurately detects money is exactly what is needed at this point in history but, in the name of all that is holy, Occupy Math hopes not. This phenomenon is not limited to Illinois (that’s the last link). Occupy Math left Iowa because his employer broke numerous promises — and they did this because they were dying the death of a thousand budget cuts. Twelve years later this is still going on.
This thoughtful and concise discussion of the problem, even if the student is language proficient, is worth reading. Occupy Math has assigned very badly phrased problems here and there over the decades and has tried to learn from his mistakes. He has nothing like the problem of the school teacher told she may not help those for whom the language of the test is a second language (or not known at all).
This sort of idiocy is an example of the law of unintended consequences.
Occupy Math thinks a full-semester course on case studies of attempts to solve problems that had unintended consequences should be part of every citizen’s education. This is difficult because the course belongs in the interstices between civics, sociology, and mathematics. It has no natural home, a topic that Occupy Math has discussed before. The reason we need this training is that we do the same stupid thing over and over. Something bad happens. We pass a rule, regulation, or law intended to stop or prevent the bad thing. To our shocked surprise, the new rule has all sorts of unintended consequences. Another reason that math is the right of all free people is because:
The core math skill of logical reasoning reduces unintended consequences.
One of the first things Occupy Math has to teach his students is how to explain math in English. This is needed in this first-year, third-year, fourth-year, and graduate courses in this very Fall of 2016. Part of the problem is that most of them cannot explain how to tie their shoes in English – there is a real deficit in writing skills across the board. During office hours one of Occupy Math’s students in abstract algebra said “I can get the concepts, but I need help getting the words right.” He’s trying to write out mathematical proofs, which is not easy. Occupy Math is very happy he knows his language skills are deficient – but why on earth do third year math majors not already have adequate language skills?
How on earth is “ignore language skills” the default response to the problem?
Part of the reason is that STEM majors get almost no help learning to explain clearly. Mass lectures prevent graded writing assignments and helpful feedback. In addition, somehow, the ability to write clearly is not valued by science students. Maybe their idea of scientists comes from TV shows? Every professor Occupy Math knows who supervises graduate students views this lack as a critical, disabling one or (worse) a reason not to accept a student. Math isn’t easy to explain clearly. Good word problems are hard to write even for people you share a language with. This is an important skill and it should be taught!
Let’s return to the core topic of today’s blog. Why is Canada enforcing a policy that violently discriminates against immigrants and actively harms the schools they attend? Because they want to make their math skills assessment fair, ensuring that teachers do not cook the results by helping their students. The discrimination is a perfect example of an unintended consequence. Occupy Math’s question is: why don’t people passing rules take ten minutes to think about what their new rule will do besides maybe solve the current problem?
Rules intended to combat injustice or prevent bad behavior are often drafted in a haze of self-righteous anger that is the enemy of reflection and reason.
During the last government in Canada, Prime Minister Stephen Harper reluctantly passed an economic stimulus package. He correctly chose infrastructure improvement as the place to put the money. The stimulus required that the money be spent quickly so that a rapid stimulus would result. In Occupy Math’s home town of Guelph and elsewhere, the result was disastrous. So many road repairs happened at the same time that many small businesses were completely inaccessible to most of their customers. Occupy Math’s count has 22 small businesses including a Belgian chocolate shop driven to bankruptcy by this “stimulus” package because it was structured without trying to assess possible unintended consequences.
Did we learn? No, we did not. The current Liberal government of Ontario just came up with a (25 year overdue) infusion of funds to renovate the MacNaughton building where Occupy Math’s office is. These funds must be spent quickly, leaving only one summer during the renovation period. This building houses a good chunk of the chemistry department including multi-million dollar equipment. Two years of planning is standard for a renovation in this kind of situation. Instead we are being asked to work from home next summer. I feel a little weird about asking my graduate students to meet at my house, but the alternative proposed by my employer is the atrium of the science complex where there is no white board, lots of noise, and where there are often no tables available. Once again we stand at the corner of Fiasco and No Planning!
Occupy Math hopes he has made the case for worrying about how language and math interact. We can surely improve the quality and fairness of our current tests and assessments by consulting with our colleagues in the language arts. The issue of unintended consequences is an incredibly important one and Occupy Math would love to hear about your experiences in this area. Occupy Math notes also that reader feedback and suggestions led to this post. Please comment or tweet!
I hope to see you here again,
University of Guelph,
Department of Mathematics and Statistics