Based on your findings, which theory about alien thickness seems most valid or most accurate?

Seventh-grade science students with flexible ethics: you’ve come to the right place!

Every so often I look at the search terms that led people to this blog. Most of them are what you would expect, but one of the top search terms, one that keeps showing up again and again and again, is “based on your findings, which theory about alien thickness seems most valid or most accurate?”

I feel like at some point I must have mentioned the words “aliens” and “theory” or “thickness” together by accident, and that started it off. Then at one point I commented on it, just wrote a paragraph on how weird it was that I keep getting these alien thickness people, and since that caused the entire phrase to be on my blog in one piece, it opened the floodgates and now I can’t stop getting curious alien thickness theorists.

So today I finally decided to figure out what was going on, and Google led me a 7th grade science class at Madisonville Junior High School, Los Angeles. As best I can tell, this class’s teacher gives her students a homework assignment that includes various questions on genetics and ecology, most of which make sense.

[EDIT: or it may be a national/statewide curriculum, which only that teacher has put online. That would explain the large number of search terms better than a single class would]

But on question 25, it suddenly jumps to a question about alien thickness which is completely inexplicable by the terms of everything that has come before:

25. Based on your findings, which theory about alien thickness seems most valid or most accurate?
A. Alien thickness is mainly affected by sunlight. The thickness may be used as a way to shield out the sunlight.
B. Alien thickness is mainly affected by temperature. This may be because aliens become dehydrated (lose water) at higher temperatures and become thinner.
C. Alien thickness is mainly affected by temperature. The greater thickness at lower temperatures may be used as a way to stay warmer in colder weather.
D. Alien thickness seems to change at random. It does not seem to be affected

Although there are a few other questions that deal with a changing population of thick and thin aliens, none of them give any information on sunlight, temperature, or hydration status. So I’m not surprised that what, by the numbers, has to be every single student in her class decides to Google the question to try to find the answer online.

I looked around myself and eventually found this web gizmo, where you adjust water, temperature, and sunlight to a group of little aliens and it tells you about their changing phenotypes. This has got to be the source of the test question, but it’s not mentioned on the test and judging by student confusion they don’t know about it. Maybe it’s mentioned in class one day and then it ends up on a worksheet a month later and nobody remembers it anymore? In any case, dozens of students keep entering it into Google and ending up at my blog.

And this is bad because the blog entries where I mention how weird it is that I’m getting all these alien thickness people also mention lots of things that are really really inappropriate for seventh graders, yet likely seventh-graders are finding them through alien-thickness-motivated confusion. So my new plan is to direct them all here, to answer their question, and let them go on their way and maybe finish their homework a little quicker. I don’t feel bad about this since any question that gets an entire class trying to cheat en masse must be pretty flawed and since homework is mostly bad for kids anyway.

So, seventh-graders! Still with me? Let’s talk alien thickness!

Start by going to the gizmo and adjust the different sliders from lowest to highest one-at-a-time, while watching the bar graph measuring alien thickness. You will notice that adjusting the water slider from highest to lowest doesn’t change thickness. Likewise, adjusting the sunlight slider from highest to lowest doesn’t change thickness. But adjusting the temperature slider while holding the other two constant does change thickness. So we conclude that thickness probably depends on temperature.

So now we can eliminate all the answers except B and C, the ones that say that alien thickness is affected by temperature. How do we distinguish between these two?

Well, B says that temperature only affects aliens indirectly, through its effect on dehydration. But if that were true, we would expect preventing the aliens from getting dehydrated to remove the effect of temperature. But this doesn’t happen – no matter how high the water slider is, moving the temperature slider still causes the aliens to shift from thick to thin. So the effect of temperature doesn’t depend on hydration.

Armed with this knowledge it should be pretty simple to pick the correct answer through process of elimination.

Let’s move on to question 26:

26. Based on the data you found, about how many of the 100 aliens would become thin if the temperature were 35°C?
A. fewer than 10
B. about 50
C. about 80
D. more than 90

You notice that at temperature 20 degrees, about fifty aliens are thin. At 25, about seventy aliens are thin. And at 30, about eighty-eight aliens are thin. The take-home point is that the higher the temperature, the more aliens we expect to be thin. So at 35 degrees, we would expect more aliens to be thin than the eighty-eight who are thin at 30 degrees. Which option best reflects that expectation?

So there’s your answer. But there’s a more important meta-point here. Your teacher wouldn’t include a nonsensical question on the worksheet, so clearly in 26 she expects you to be able to calculate alien thickness based on temperature. So just by reading 26, you know the answer to 25 is one that says thickness is based on temperature. So you can eliminate A and D and be left with a 50% chance of getting it right. And without looking at the original data, you can conclude that it’s probably not B, since it says only temperature matters but the explanation implies that water and hydration status matter as well. So really, even if your teacher forgot to link you to the gizmo thing, you should be able to guess the right answer based on test-taking skills alone.

A story from my own life – my first month of medical residency, my schedule was extremely disorganized and I ended up starting a class they day they were having their final exam. This exam happened to be on the treatment of radioactivity-related injuries, a field of medicine I was unaware existed until that moment. Because of inconsistent answers, clues in other questions, and basic common sense, I was able to guess well and ended up getting a B- (the class average was a C).

My point is, test-taking skills matter.

I haven’t gotten any Google search queries asking about any of the other questions, so I’m going to assume you’ve got all of those down. Good job, seventh-grade science students!

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

56 Responses to Based on your findings, which theory about alien thickness seems most valid or most accurate?

  1. Daniel says:

    > homework is mostly bad for kids anyway.

    If kids are going to be forced to do homework for many years to come, is it harmful to let them know it’s mostly bad for them, like deconverting someone who is physically prevented from leaving their church?

    • Anonymous says:

      I would tell a student that homework is good, and doing homework leads to better performance in class and on exams, because they are going to have to do homework anyway, and maybe this will serve as some sort of placebo effect.

    • Kaminiwa says:

      I think most kids already assume homework is bad. The smart kids can already do Google research to confirm this, and for everyone else I don’t think Slate Star Codex is really influential enough to be much of a bias – “But moooooom, that blogger who talks about alien thickness said homework was bad for me!”

    • Decius says:

      If children are going to be exposed to secondhand smoke anyway, is it harmful to let them know that it’s bad for them?

      • Daniel says:

        Children can take actions to minimize their exposure to smoke, like not standing right next to their parents while they smoke. It even goes hand in hand with deciding whether to smoke themselves! And once they become adults, they will have good control over their exposure. So we should tell them that smoke is bad for them.

        Whereas they have no control over having to do homework. And once they become adults they will never have to do homework again even if they think it’s good for them.

        • Error says:

          Well, they have no control over having to do it, but they do have control over doing it. I used to exercise this control often, to my academic detriment.

          (also, homework is a sucking black hole of damnation and despair, and the sooner it dies, the better.)

    • Michael Edward Vassar says:

      I think that there are pretty good priors against lying out of duty towards those one is lying to.

    • misha says:

      Certainly a large part of my depression and loss of interest in school was realizing homework was a useless waste of everybody’s time, but I don’t think it’ll ever go away if we pretend it’s not to make diligent schoolchildren happier.

  2. St. Rev says:

    Because of inconsistent answers, clues in other questions, and basic common sense, I was able to guess well

    Teaching this skill is (or was when I taught for them, anyway) basically Princeton Review’s business model for standardized test prep courses.

    • Scott Alexander says:

      It confuses me that even standardized tests where professionals are supposedly paid to come up with good questions still have these little “tells”, unless it’s a conspiracy between the standardized test industry and the test-taking-skills tutoring industry.

      • gwern says:

        What makes you think that noticing and being able to exploit ‘tells’ doesn’t simply load on what they’re trying to measure? 🙂

        • St. Rev says:

          Independently developed exploitation skill is probably substantially g-loaded. Problem is it’s moderately easy to teach to kids who’d never come up with it themselves.

          • gwern says:

            I’d agree the teachability of tricks does reduce the loading, but it’s still going to be hard. I mean, look at the OP examples: they’re pretty much harder than just answering the question normally. And the more questions you’re pitting against each other to tease out consistent sets of answers to guess on, the more demanding the task becomes. You can teach someone the very limited set of rules and transformations that go into a matrix test (matrix tests can be automatically generated, even), but they’re still not going to hit the ceiling on the RAPM unless they were pretty smart to begin with.

        • St. Rev says:

          1) Standardized tests have to do a good job differentiating takers at all levels of aptitude. That means every taker is going to encounter a different mix of easy problems, challenging problems, and impossible problems. All you need to do to raise a student’s score is improve their ability to exploit at their level of challenge–a much narrower task.

          Mind you, that’s on a paper test. A lot of electronic exams work on an adaptive algorithm that quickly calibrates to the taker’s aptitude–i.e. if you keep getting problems right it keeps raising the difficulty until you’re scoring ~50%. Not sure how that changes the strategy, if at all.

          2) Most standardized tests aren’t tests of knowledge. They’re very rigidly constrained in how they can be constructed, and those constraints make gaming them more tractable–exploting them is more about syntactical structure than semantic content.

      • Douglas Knight says:

        For some classes of tells, this may be relevant, though it doesn’t cover the case of comparing questions.

      • St. Rev says:

        Designing robust standardized tests is EXTREMELY hard.

        Some of the tells are deliberate; there’s a whole class of answers called ‘distractors’ that are designed to be more appealing than the true answers to students who don’t understand the question. Consider e.g.:

        duck: bird ::

        c) water: egg

        That’s pretty clearly an incorrect answer if you have any ability at all to formulate analogiess–but it’s semantically appealing on a superficial level. Someone who’s lost is likely to pick it.

        The testing companies modified their approach to this kind of thing precisely because the test prep companies exploited the hell out of it; it’s why they took analogies off the SAT. Not sure what the state of the art is these days.

        • Douglas Knight says:

          Do you have a source for the history of SAT analogies and why they were removed?

        • St. Rev says:

          Uh I was teaching prep courses around the time it was happening?

        • Douglas Knight says:

          Yeah, I googled before asking the question and didn’t get anything out of that column. Do you think there is any value in reading it? The author quotes the college board, insinuates that you shouldn’t believe them and doesn’t actually say anything. Did you read it?

        • St. Rev says:

          I read it. I cite it only because it pins down a date.

        • Decius says:

          I thought they took analogies off because they basically measure “Does this student know these vocabulary words?”

          Hell, I thought that’s what they were intended to measure! (“I know a duck is a type of bird, but I don’t know which of these fancy words is a subset of these other fancy words”)

        • St. Rev says:

          Well, ETS doesn’t give a detailed public description of their decision process–that would just make the tests even more gameable. But testing vocabulary is an important component of PSAT/SAT/GRE/etc. Analogies were one of their tools for doing that, but not the only one.

        • St. Rev says:

          nb: I said above that these exams aren’t intended as tests of knowledge–vocabulary and some basic mathematical formulae are the main exceptions to that.

  3. Sniffnoy says:

    A note: Link to the original assignment is broken (missing http://).

    While I claim to know just why the alien thickness thing may have come up on your blog, my suspicion is that it is probably related to the Ebborians from LessWrong.

    • Sniffnoy says:

      Oh, nevermind, going by this old entry I see you’ve already noticed this. I mean, I should have expected that, but since you didn’t explicitly mention it here…

      • Scott Alexander says:

        Didn’t want to link to that because inappropriate for seventh-graders, and in fact I wrote this to keep the seventh-graders away from my *next* search terms post which I was planning for today before this issue distracted me.

        As far as I can tell, I never mentioned the Ebborians on this blog before that reference.

        • Sniffnoy says:

          Oh, oops. Didn’t realize that. Sorry.

          As far as I can tell, I never mentioned the Ebborians on this blog before that reference.

          Yeah, that’s odd. Could it have come up in the comments?

          (Also, missing “can’t” in my initial comment makes it sound very strange…)

        • Watercressed says:

          Telling seventh graders that they shouldn’t read the rest of the blog because it’s inappropriate seems counterproductive.

        • Alexander Stanislaw says:

          I agree with Watercressed, if I was trying to get rid of seventh graders, I would give them the answers first, then give them the explanation, then put everything else after.

  4. ciil says:

    How old is the average 7th-grader? Around 14? Do you really feel this blog contains so much age-inappropriate content for that age-group? I mean, science, reason and medicine (and I can’t remember you posting so much as a gory picture of some flesh-eating bacteria infected wound or anything) aren’t really age-inappropriate for any age, right?

    • Anonymous says:

      I wish I had read this blog at 14.

    • Gunlord says:

      Some of the commentary can get a bit heated…while there’s nothing NSFW, I’m not certain debates over various genocides (such as that 7-kill stele thing) are what most teachers would consider ideal for younguns.

      • nydwracu says:

        They teach WW2 around that age, don’t they?

        Then again, when I was 14 I spent a lot of time researching the question of what motivated spree shooters, so I may not be the best judge of these things.

    • Scott Alexander says:

      Tomorrow-ish I will post the post that was going to include this discussion (a list of the search terms that lead people to this blog) and that should answer your question.

    • ESRogs says:

      > How old is the average 7th-grader? Around 14?

      12-13

    • a person says:

      14 was when my friends and I discovered 4chan. The idea that a fourteen year old could be “too young” for anything seems kind of quaint to me.

  5. Harvey says:

    Good one this time. I noticed the type of stuff you’re talking about around high school too. It seems its pretty difficult to write a test asking questions without giving away some of the answers in the later questions. So, most any english or science test wasn’t too bad by default. Math tests, of course, were still hard since it didn’t matter what the answers were to the last question, and you had to show work and stuff.

    Also, you still had to take tests as a resident? Christ.

  6. Harold says:

    This was a really nice thing of you to do. You’re a good person Scott.

  7. Miranda says:

    A story from my own life – my first month of medical residency, my schedule was extremely disorganized and I ended up starting a class they day they were having their final exam. This exam happened to be on the treatment of radioactivity-related injuries, a field of medicine I was unaware existed until that moment. Because of inconsistent answers, clues in other questions, and basic common sense, I was able to guess well and ended up getting a B- (the class average was a C).
    This is why I’m better at tests than reality. Reality doesn’t tell you which of a million bits of information to look at.

    • Scott Alexander says:

      Indeed. I dread the day there is a nuclear detonation and someone thinks “Wait! Scott got a good grade on the radioactivity medicine test! We should make him deal with this!”

  8. Error says:

    “Because of inconsistent answers, clues in other questions, and basic common sense, I was able to guess well and ended up getting a B-”

    I can top this. I once manged a B on a final exam essay about a book I had never read, using only information contained in the question.

    • Tab Atkins says:

      Me too! Back when I was a high school senior I didn’t have a 7th period, so I’d sometimes drop in on my wife’s last class, Biology 2. (The teacher liked both of us, and I was quiet.) I never paid any attention in class, usually reading instead, and only did it about a third of the time anyway, so I missed large chunks of important information even if I was absorbing things unconsciously.

      On test days, she wouldn’t let me stay unless I took the test too, for fear of me being distracting to the other students, and I usually would because I was bored. I ended up taking a good percentage of the class’s tests through the year, and based on those grades, would have passed with a solid C. I regularly beat a few of the students who actually believed they had a chance of getting into/through medical school.

  9. JRM says:

    I took a chemistry test in high school for an award from the American Chemistry Society. It was 50 multiple choice questions which got increasingly hard. I knew the first three. And I had a good idea on question five. And then I just tried to figure it out (this page-sized molecule has five of the same large parts – let’s look for five-based prefixes when we’re calling it something.) A friend of mine said he knew most of the answers, so I was confident I was toast. Then I beat him.

    There are tests with few tells. The LSAT, which is just a non-mathy IQ test, is very good at avoiding tells most of the time. I’d guess a lot of SSC readers would smite it anyway, but on merit rather than just raw test-taking skills. (Unlike most pre-graduate tests, the LSAT requires no subject matter knowledge at all.)

    I wonder how much the good test writers get paid. It strikes me as something where you need to intrinsically understand the test-radar skills, unless that’s what you want to test.

  10. JRM says:

    Oh, and this is a seventh-grade assignment? I don’t think you can criticize Ms. Blankenship’s teaching methodologies. I am duly impressed. Wow.

  11. Nous-Suits says:

    Ms Blankenship seems to have an email address listed on the Madisonville Junior High website. Would it be appropriate to offer a complementary link? If any of her students do come across this post and find their teacher and school mentioned by name, it’ll probably get back to her sooner or later.

    Unless Scott’s already gotten in touch, which seems entirely plausible

  12. Douglas Knight says:

    one that keeps showing up again and again and again

    Not just last week? So not just Mrs Blankenship’s class?

    • Vaniver says:

      If there is a steady stream of them, it is likely that the homework assignment is produced by the same company that produces the web gizmo, and Mrs. Blankenship is just one of the consumers.

    • Randy M says:

      Wonder if the search terms are plotting by time. Perhaps there is a spike every year at alien-thickness assignment time.

  13. Anonymous says:

    On the subject of homework that *actually* needs Googling… that I had to do today…

    The widely-used MasteringPhysics homework system (following the modern customer/student-hostile trend of tying access to graded homework to buying a new textbook) has the following malformed question:

    A demolition crew uses dynamite to blow an old building apart. Debris from the explosion flies off in all directions, and is later found at distances up to a distance (your number may vary) from the explosion. Find the maximum speed at which debris was blown outward by the explosion. Neglect air resistance.

    Obviously, if you assume that gravity doesn’t scale with distance (because that’s how Freshman Physics Gravity works), there’s no actual answer–just keep raising the angle closer to straight up, and you can get arbitrarily high speeds. A Yahoo Answers guy suggested that the question might really want the *minimum*; this indeed was graded as “correct” for me.

    • Anonymous says:

      …oooor never mind… I guess we’re assuming a large enough set of debris that all angles were used for the maximum-speed debris pieces, in which case using a 45-degree angle makes sense.

  14. Pingback: More Search Terms That Have Led People To This Blog | Slate Star Codex

  15. Pingback: Aliens and Analytics: A Meditation on Using the Internet Well | All Across Learning