There has been a furore in England about an exam question which made people think. Many students did not take kindly to it, and complained it was unfair. They felt angry and frustrated, and wanted the pass mark to be lowered. The examining body Edexcel explained that they had designed it to test “the full range of abilities” (which may count as a rare public admission that students vary in ability). In more precise terms one could say that the examiners were trying to distinguish between students who could solve individual mathematical problems one operation at a time, and those (a minority) who, without any specific guidance, could integrate the individual steps into a full solution to a novel problem.
http://www.bbc.co.uk/news/education-33017299
Readers of this blog will know that this sounds very familiar: it is the shift at around IQ 115-120 from being restricted to specific examples and written instructions, to being able to gather information and make inferences: the level at which students have to think for themselves.
In the former case each step has to be rehearsed, but tends to be seen as a task on its own. In the latter case students realise that they are being given tools they can apply in novel circumstances: they learn general principles, and apply them generally. In a nutshell, in designing this question the examiners were searching for A and A* students, or in my terms, Tribe 5.
http://drjamesthompson.blogspot.co.uk/2013/12/the-7-tribes-of-intellect.html
Readers of this blog will also know that I try to confess my errors as quickly as possible, so you should know that I did not get the problem right. I will give you the problem and ask you to solve it, just for your own fun. I know that my readers are persons of distinction, and will have a go and keep their workings, and not immediately jump to the BBC link provided to find the school answer. Take a separate piece of paper, which you mark with the date and time, and then draw a line underneath at the end of your solution. Use Registrar’s ink, which darkens with age, and lasts at least 500 years.
I then go on my the main task: I want to simplify the problem so that it contains no algebra, and can thus be used as a puzzle for the general public. I regard myself as a disciple of Gerd Gigerenzer, and so this is intended to be a pace or two in his footsteps.
There are n sweets in a bag.
6 of the sweets are orange.
The rest of the sweets are yellow.
Hannah takes at random a sweet from the bag.
She eats the sweet.
Hannah then takes at random another sweet from the bag.
She eats the sweet.
The probability that Hannah eats two orange sweets is 1/3
(a) Show that n^2 - n - 90 = 0
(b) Solve n^2 - n - 90 = 0 to find the value of n
Once you have solved that, look at the slightly simpler version:
There are an unknown number of sweets in a bag.
6 of the sweets are orange.
The rest of the sweets are yellow.
Without looking in the bag, Hannah takes a sweet from the bag.
She eats the sweet.
Without looking in the bag, Hannah then takes another sweet from the bag.
She eats the sweet.
The probability that Hannah, just by chance, got out two orange sweets from the bag (took one orange sweet out which wasn’t put back in the bag, then took another orange sweet out which wasn’t put back in the bag) is 1 in 3.
(a) Show what you think are the number of orange and yellow sweets before Hannah eats any sweets, and then after she eats the first sweet and the second sweet.
(b) Work out how many sweets were in the bag to begin with.
My reworking of the problem is to remove the algebra. I know that this is a maths exam, and that knowing algebra is part of maths, and that it might help you to solve this problem, and many other problems. However, I want to avoid people being frightened off by algebraic notation and thus not realise that they might be able to solve it by using natural frequencies. In that vein, I also got rid of and explain “random”. I use “1 in 3” because people might find that more easy than a fraction, which some adults do not understand. In a major simplification, I am also reminding people that the sweets, once eaten, do not go back into the bag (which I forgot in the second phase of calculation when attempting a quick and lazy calculation myself).
What I am attempting to do is to remove any of the “surface” distractors and put the problem into its most practical and essential form. This allows us to judge whether the difficulty lies in the format of the question, on in the irreducible complexity of the arguments to be considered. Pace Gigerenzer, most doctors fail probability questions when they are couched in percentages and symbolic logic, but solve them easily when they are presented in natural frequencies, say as patients out of 1000 people in the population.
Here are some explanations as to why some question forms mislead, but do not necessarily tell us much about underlying complexity.
http://drjamesthompson.blogspot.co.uk/2014/02/the-many-headed-hydra-of-alternate.html
I notice that the official accounts dare not mention the notion of intelligence, which is a pity. The OECD funded Pisa study also avoid the topic. Some critics argue that children are not being taught properly, and that maths education needs to be improved. Of course, that may well be true, but some ideas are hard to grasp, even when we make them as clear as we possibly can. An individual’s intelligence level is found at the point where problem complexity defeats problem solution. Something has to give, and it is usually the problem solver.
http://drjamesthompson.blogspot.co.uk/2013/12/oecd-children-become-oecd-adults.html
What do you think of a) the first version as an exam question and b) the simplified version?
Anyone want to test the two versions on the general public?
P.S. I showed the original problem to an esteemed person of my acquaintance, and she gave me the correct answer in 20 seconds. A fluke, no doubt. I expect I may be reminded about it from time to time.
For those who have done or like combinatorics problems, this can be done mentally. It is the framing of the problem, i.e., pattern recognition how to define the 1/3 probability. There is no trickery that I could see that misleads. So if students have not learnt combinations (taking 2 at a time) or not learnt probability, this can be hard. Yes, they need to know algebra too, although given than n is an integer the correct value can be found by inspection (quadratic formula not required). The problem becomes more complicated if one interprets the sequencing to be somehow important (oh, she ate it!) etc. and analyzing it step by step rather than see it as a selection of 2 sweets from the lot. I guess expertise helps in choosing the right frames. Given 2 individuals with similar fluid intelligence, the person with prior experience or expertise will choose the easy framing naturally. In a sense, thats what expertise is all about.
ReplyDeleteThough I do enjoy combinatorics from time to time, I could not have done this in my head. Too many details, not enough RAM.
DeleteI also found the simplified version to be more confusing than the original, which I did on paper in a little less than a minute. Even got the correct answer without screwing up the algebra.
I also think that how much a student may have been exposed to these sorts of combinatorial/probability problems will have everything to do with how easy the answer comes for them.
DeleteIf a student has had a course that gets into such issues, then he has probably seen many problems just like this. The first question such a student will ask is: is the sampling with or without replacement? Well, if Hannah ate the candy, that's not hard to figure.
After that it's just the most trivial kind of algebra.
I couldn't (or wouldn't) myself do it in my head -- or at least I wouldn't much trust my calculations, because I'm generally terrible at calculation.
If a student hasn't seen these sorts of problems, then it would take quite a few more IQ points to pull out a solution. Each step and consideration would have to be thought out from scratch, and that is definitely a far harder business.
Ditto all.
DeleteTrial and error approach (faster):
ReplyDeletehttps://twitter.com/KirkegaardEmil/status/607250113589702656
This would probably be the method the less mathematically knowledgeable but smart would use. First intuit that the number is likely fairly small. Second, try some numbers.
6/7*5/6 = .71 (too high)
6/8*5/7 = .54 (still too high)
6/9*5/8 = .42 (almost)
6/10*5/9 = .33 (bingo)
Analytic solution:
(6 / n) * (5 / (n - 1)) = 1/3
solve n
n = 10 (or -9, but we ignore the negative solution as nonsense).
Steps
(6 / n) * (5 / (n - 1)) = 1/3
multiply LHS
30/(n^2 - n) = 1/3
multiply by 3
90/(n^2 - n) = 1
multiply by (n^2 - n)
90 = n^2 - n
then we can see that the result is 10, because 10 * 10 = 100, 100-10 = 90.
solving it completely is somewhat difficult, requires using the discriminant formula for a second order polynomial
subtract 90
0 = n^2 - n - 90
apply discriminant formula
d = b^2 − 4ac, where a = 1, b = -1, c = -90
d = (-1)^2 − 4 * 1 * -90
d = 1 − 4 * 1 * -90 = 361
then use the formula to find x
x = (−b ± sqrt(d)) / (2 * a)
we want the positive solution so
x = (−b + sqrt(d)) / (2 * a)
insert in numbers
x = (1 + sqrt(361)) / (2 * 1)
simplify
x = (1 + 19) / 2
again
x = 10
I congratulate the question setter on his amusing allusion to the fact that the only Smartie that you can recognise the colour of by taste alone is the orange one.
ReplyDelete"the shift at around IQ 115-120 from being restricted to specific examples and written instructions, to being able to gather information and make inferences: the level at which students have to think for themselves." At my secondary school we were told that admission to the top two streams required an IQ of 118.
For what little it's worth, I found the original phrasing superior, in the sense that it didn't make me doze off. Sorry, Doc.
Darn - I clicked the link before I read the paragraph below, which stated not to click the link.
ReplyDeleteI don't get it. This is a simple question,
ReplyDeleteOnce the they tell the probability is 1/3, you know that 6 is greater than 1/2 the candies, because 1/3 is larger than 1/4. So the total must be 11 or less. There are only 5 possibly answers. There is no trick. This is grade school math.
The rather low pass rate suggests otherwise. Problem solving cases like these are much harder for people, despite the math being relatively simple (by brute forcing the few plausible options).
DeleteAs Charles Murray noted in his book, Real Education:
"In short, just about every reader understands from personal and vicarious life experiences what below average means for bodily-kinesthetic, musical, interpersonal, and intrapersonal ability, and for the aspects of spatial ability associated with hand-eye coordination and visual apprehension. You may think you also know what below average means for linguistic ability, logical-mathematical ability, and spatial abilities associated with mental visualization because you know you are better at some of these intellectual tasks than at others. But here you are probably mistaken. It is safe to say that a majority of readers have little experience with what it means to be below average in any of the components of academic ability.
The first basis for this statement is that I know you have reached the second chapter of a nonfiction book on a public policy issue, which means you are probably well above average in academic ability— not because getting to the second chapter of this book requires that you be especially bright, but because people with below-average academic ability hardly ever choose to read books like this." (pp. 32-33)
You probably didn't go to an average grade school, and you probably didn't hang out with average children. This question is definitely not grade school stuff. I'd hazard the guess that a large fraction of European adults would fail this question.
Yes - most people would definitely fail this question.
DeleteMy IQ is only 120 and I thought that was incredibly easy. It was basically a middling-level SAT Math question.
ReplyDeleteWell, my IQ is well north of 120, but I didn't understand the question at first, and not because the math was so advanced or because it takes a high IQ to spot the connection.
ReplyDeleteI puzzled over it for a while, because it's been a number of years since I've taught combinatorics. They used to always be part of algebra 2, but once I realized that the kids didn't use them in pre-calc, I dropped them.
It's not the algebra that's the problem. It's misleading test question in that people who know combinatorics might not be able to see the underlying math topic. So if you're testing for IQ that's one thing, and it won't be completely reliable because it's not in a puzzle book but in a math test, where even high IQ people would be trying to see the underlying math to work.
If you're testing for people who understand both combinatorics and quadratics, then notice of the people offering comments here, one of them completely failed to answer part a, which means it's not "simple", and the other apparently missed that the equation factors, and I'd knock off a point for anyone who didn't notice that, or who moved the 90 over to the other side to guess and check.
0 = n^2 - n - 90
0=(n-10)(n+9)
As written, part a comes off as a complete non sequitur. If Pearson was actually trying to make it harder by confusing the tester, that's pretty pointless.
a) Express the probability that Hannah chose 2 orange sweets in terms of n.
b) Use this equation to determine the number of sweets in the bag.
c) Use the solution to show that n^2 -n-90 = 0
That makes it fairer in that you've appropriately established it as a probability question. Then you can also eliminate the people who guess and check.
And this is not a middling SAT math question, which would have been: Hannah had 10 piece of candy in a bag, 6 of which were orange. Hannah picked one randomly for a snack after lunch, and then gave another, picked at random, to a friend. What is the probability that both candies were orange?
Which is a much easier question.
Dear Ed Realist, Thank you for your intervention, which I appreciate, and I am also grateful for your simplified version, which is better than mine.
DeleteI'm pretty sure combinations with repetitions aren't on the SAT math (at least not SAT I).
DeleteI guess the original was so easy because I skipped the words and went straight to the algebra. The words require you to know combinations which aren't tested on the SAT.
Deletenot very helpful aside:
ReplyDeletethe publisher would have merely computed the item difficulty level (% passing), & the correlation of the item with total score.
& if the publisher is anal retentive enough they'll visually inspect the data to make sure an item's not acting "weird" -- such as a quarter of people who are IQ-ish 115-120 pass, then hardly anybody passes who are 120-130, then a bunch more who are >130-ish pass (IQ estimates here are just proxies for how many std. deviations above the mean people's total scores are.)
publishers can be fooled:
some items are difficult due to their obtuseness, rather than measuring what they're supposed to measure (& being a "good" difficult item). then the publisher thinks, "hey, whatever, it fills a gap between item difficulty levels, giving the test fairly equal gradients between item difficulty levels - cool!" (& the item's not biased against any group, performs the same for different groups in rank order of difficulty, bla bla bla).
sadly, the publishers are loath to let others play with their item level data (you probably have to work there to play with their data:)
Thanks for your very informed comment
DeleteEgalitarian ideology by scholastic people, now by ''everybody ''with'' iq above 110 can do it''. Ad nauseabund.
ReplyDeleteSantoculto
Definitely prefered the first version. The second version is different, in that it puts the entire problem into a) and has b) become an addition problem, at most. An addition problem that is actually implied by a), otherwise, how can you get at the right number of yellow sweets?
ReplyDeleteFor my English daughter these exams are four years in the future and in middling state schools she has already been taught conditional probability and enough algebra. She likes little bites of mathematics as much as sweets, so eagerly reasoned and wriggled her way to the correct answer to part (a). The standard quadratic recipe was printed at the front of the exam paper for part (b) so that isn't interesting.
ReplyDeleteYes she's a Tribe 5 inferer on your terms, but there was also some related practice at school a couple of months ago courtesy of Venn diagrams where the more difficult questions typically require a voluntary decision to use algebra.
That exam paper is aimed at something like the 40th percentile upwards, so this question was bound to offend a substantial number of children. However, I suspect what threw some of them (and subsequently many adults) is the direction i.e. "Here is the answer. Now show why".
As an aside I'm dismayed by these quite high stakes exams because in attempting to test such a wide ability range the "A/A*" is determined by a very small number of questions. Humans err, even Tribe 5.
Yes, Tribe 5 humans err, so the lack of sufficient items at that level is a problem. A two stage selection procedure might be more valid.
DeleteNice problem, readily accessible to undergraduates in an introductory statistics course who are learning about joint and conditional probability. I'm stealing it.
ReplyDeleteThis is stuff that could be taught in American high schools, maybe even junior high, but isn't.