Recently I posted some findings about sex differences in the public understanding of science. I criticized the Pew quiz for having items which were far too easy, and proposed a few harder items, on vaccinations and the expanding universe.
https://drjamesthompson.blogspot.co.uk/2016/09/are-science-quizzes-scientific.html
Before I could refine questions on those two subjects, a reader reminded me of a delightful program in which Harvard graduates in 1987 were asked “why do the seasons happen?”
https://www.youtube.com/watch?v=p0wk4qG2mIg
I tried to put this particular question into the very simple and very restricted format of a Twitter poll. Of course my followers are not a random selection of the public (see below). I was reaching out to an elite. Of course I know I should look for samples which are representative and also sizeable. Of course, of course. 303 non-random respondents are not enough, though possibly more than in many social psychology papers.
However, my intention was to try to create one science item in a science quiz, which can later be improved. I know that an open-ended question is far better, because respondents are not prompted in any way. I made the Twitter question as simple as I could, but probably should have stuck to “why do seasons happen?”. However, physicist Roy Bishop asked his students in 1993 “What makes summer hotter than winter?” Yes, he was in Canada, so he had an geographic interest in the topic
http://adsabs.harvard.edu/full/1993JRASC..87..346B
Given 4 reply options, respondents know one of them must be correct, which makes their task easier. I found it hard to make the three other options equally plausible. You may be able to do better. As it is, the quiz may have taught people some science, which was not my intention. My very brief “correct” answer is insufficient as a full explanation, but will have to do for the time being.
So, a majority got it right, and only a minority went for the traditional, but wrong, answer. Only 2% went for the silly answers.
Comments please.
Disclaimer: my Twitter followers are 75% male. 57% are in the 18-35 years age range. 68% of followers are interested in science news. In brief, their scores ought to be high.
But the trees give more shade. It's all a mystery.
ReplyDeleteCovary for evergreens
DeleteHere's a set of options:
ReplyDeletea) Earth's revolution moves it closer to Sun
b) Earth's rotation shifts it closer toward Sun
c) Earth's revolution shifts its orientation to Sun
d) Earth's precession tilts its axis toward Sun
Frank, you are dealing with the General public: revolution, rotation, orientation, precession are all too difficult. (Thanks anyway).
ReplyDeleteNot necessarily a problem. It makes the item harder because it's a combination of a vocabulary test and an astronomy knowledge test.
DeleteBest kept apart, tested separately.
DeleteA person either knows or doesn't. But I am ever amazed at just how much people forget. I recall waiting to see the principal after a fight in the 7th grade and overhearing all the adults in the nearby room trying to find one among them who could remember how do to simple fractions. No one could.
ReplyDeleteI presume the adults had never had to deal with fractions, so forgetting was a rational allocation of effort. You should have put your head round the door and seen if they could work out the cost of carpeting a room, given only the dimensions of the room and the cost per square yard of carpet. Next time, perhaps!
ReplyDeletehttp://matematiikkalehtisolmu.fi/2005/erik/KivTarEng.html
ReplyDelete"""polytechnics (universities of applied sciences) and technical universities ... only 35 percent of the 2400 tested students have been able to do an elementary problem where a fraction is subtracted from another fraction and the difference is divided by an integer."""
dux.ie
Fascinating. Turku rings a bell. Perhaps I landed there once on the way to Vaasa. Anyway, there are two problems here. The first is teaching well. The second is teaching what. Finland does well on PISA but the Finns are clever enough to "teach to the test" and may be distorting their efforts so as to get good results. My own education stressed fractions and elementary algebra, but I cannot say it has influenced my life. If I had been taught statistics, life might have been different. Thanks for this very interesting reference.
ReplyDelete"If I had been taught statistics, life might have been different."
ReplyDeleteMy school maths had far too much geometry, too little linear algebra, and neither probability nor statistics. When school maths was made "more modern" much of the change was, alas, just silliness: set theory, but with no purpose, that sort of thing.
I remember the lurch into set theory, which was a preposterous excursion into a topic of no relevance to the usual requirements of most students. For once, I think that employers might talk to educationalists and explain what sorts of maths skills they need. This for a culture in which many adults cannot calculate the cost of carpeting a room, given only the dimension and the cost per square yard.
DeleteThe ACS (American Chemical Society) does a pretty good job of setting up multiple choice questions. You have to really know chemistry to pass their exams. For your summer question, you might eliminate the silly answers and offer:
ReplyDeleteA. Earth moves closer to sun.
B. Tilt of earths axis.
C. All of the above.
D. None of the above.
It works better when there are 3 plausible answers. The test taker must really know the subject in order to avoid the ambiguity.
My comment is, your question was a simple common sense test.
ReplyDeleteThe 3 other options were all to implausible, and respondents understand #3 must be the right pick even if they knew nothing about the axis movement (as was my case).
You know the other options were too implausible, 14% did not.
ReplyDelete