[This post is about the 2018 SSC Survey. If you’ve read at least one blog post here before, please take the survey if you haven’t already. Please don’t read on until you’ve taken it, since this could bias your results.]
I’m preregistering my hypotheses for the survey this year. So far I’ve glanced at Google’s bar graphs for each individual question but haven’t started exploring relationships yet, so I’m not cheating too badly. I’ll still look for things I haven’t preregistered, but I’ll admit they’re preliminary results only. This is the stuff I’ve been thinking about beforehand and will be taking more seriously:
1. I plan to replicate the general thrust of last year’s results reported in Can We Link Perception And Cognition on the sample of new people who didn’t take the survey last year. In particular, I’m expecting that weirder, more autistic, more liberal, more schizophrenic, and more transgender people will be more likely to display unusual patterns of perception (hollowness or ambiguity) in the Hollow Mask illusion. I expect this to become much more obvious since I’ve included three examples of the illusion this year including one that seems to give a wider diversity of results.
1a. I plan to replicate the results from last year that people who were better at noticing duplicate “the’s” are more likely to display unusual patterns of perception on the Hollow Mask illusion.
2. I plan to conceptually replicate Mitchell et al’s study showing that autistic people are less susceptible to the Shepherd Table Illusion.
3. I plan to conceptually replicate Caparos et al’s study showing that politically further-right people are more likely to use global processing on a Navon task (eg when there’s an H made of tiny Es, they see the H more than the Es).
4. I plan to investigate a general construct of “first sight and second thoughts” that involves people being better able to see what’s actually there, and less susceptible to illusions, priors, stereotypes, and assumptions. This will involve correlations between the two Duplicate Thes illusions, the Hollow Mask illusion, the Shepherd Table illusion, the Cookies illusion, the Parentheses palindrome, the Map riddle, the Surgeon riddle, the Switched Answers task, the Cognitive Reflection test, and the Wason task.
4a. If I can figure out how to get a common factor out of all of these, I plan to see if it’s the same thing I’m looking at in 1, and how it relates to the same groups.
4b. Whether this relates to a general willingness to believe strange or unpopular things. Check vs. AI risk concern and HBD support.
5. I plan to investigate a general construct of “ambiguity tolerance” that involves people being okay with a superposition of different conflicting ideas. This will involve correlations between ambiguous results on the Hollow Mask illusion, the Spinning Dancer illusion and the Squares-Circles illusion, and with answers to the questions from the Tolerance Of Ambiguity and Tolerance of Uncertainty scales.
5a. Whether perceptual ambiguity relates to cognitive ambiguity. I want to check whether people with high ambiguity tolerance on the optical illusions are more likely to say their political opponents have some good points, are less likely to say their political opponents are evil, and are less likely to say the existing political system is justifiable. Also if they’re more likely to enjoy puns.
5b. To what degree this is the same construct as (1), and is stronger among the same demographic groups.
5c. I also want to see if people with high ambiguity tolerance give less extreme answers on questions in general. I’ll probably use Ambition, Social Status, Romantic Life, and Morality for this, just because these seem like complicated questions there’s no obvious right answer to.
5d. I plan to confirm previous studies showing low ambiguity tolerance correlates with conservative philosophy; check vs. Political Spectrum 1-10. I predict that this will be stronger for populists than for “business conservatives”, so I expect the low ambiguity correlation will be weak for generic conservatives, stronger for Trump supporters, strongest for people who identify as alt-right.
6. I plan to investigate whether autistic people are more likely to give process-centered rather than person-centered answers to the two political categorization questions (categorizing Nazis, categorizing civil disobedience on gay marriage). That is, neurotypical people will be more likely to categorize based on which side wins, and autistic people will be more likely to categorize based on what procedures were followed (eg violence, civil disobedience).
6a. I also want to investigate how these correlate with political views. I may end up controlling for this as a confounder in (6) above.
6b. This is a totally wild out-of-left field idea, but I suppose I should check how these relate to the Navon figures since they’re both about categorization.
7. I plan to confirm or disprove, once and for all, whether our community has more older siblings. For lack of a fancier way to do this, I’ll take the set of all people who have exactly one sibling, and see what percent of them are older vs. younger. If it’s significantly above 50% older, I’m going to interpret this as a birth order effect. I’ll do the same with the set of people who have two siblings, three siblings, etc, and combine them all for a final determination. Half-siblings will be ignored. If you have any problems with this methodology, tell me now.
7a. If I find we’re disproportionately older, try to use subgroups to figure out where the effect is stronger or weaker, to try to find exactly what’s going on. For example, are Less Wrongers more older-skewed than SSC readers in general?
7b. Birth order by autism, Openness, and IQ/SAT.
7c. One traditional birth-order claim is that younger children are more rebellious, so check birth order vs. people who think system needs to be fine-tuned or destroyed.
8. I plan to conceptually replicate studies showing that the more older brothers (but not younger brothers, or older or younger sisters) you have, the more likely you are to be gay.
8b. See if this predicts anything else: bisexuality, transgender, gender non-conformity, political leftism, autism, possibly ‘first sight and second thoughts’, possibly ‘ambiguity tolerance’.
9. I plan to see whether people with ADHD are more likely to prefer the buzzing city aesthetic to the quiet village aesthetic, more likely to rate themselves as more risk-taking, and more likely to describe themselves as ambitious.
10. I plan to investigate the hypothesis about sexual harassment mentioned here: that it’s higher in gender imbalanced industries only due to potential-perpetrator-to-victim ratio. I predict that in relatively gender imbalanced industries (in terms of survey categories, all three Computers fields, Finance, Physics, and Mathematics) compared to relatively gender-balanced industries (Health Care, Psychology, Art, Law, Biology), a higher percent of women will report being harassed at work, but the percent of men reporting harassing at work will remain the same.
10b. I predict that the more people identify with social justice, and the more positively they feel about feminism, the more likely they are to report both being harassed and harassing others, due to more awareness and lower threshold to report. I predict poor social skills and autism spectrum will predict more likely to say one is a harasser, due to causing unintentional offense. I predict people who are harassed more at work will also be harassed more outside of work.
11. A long time ago, I randomized people into groups and made them read articles on AI risk to see how it changed their minds. The effect mostly persisted after one month. Since those groups were randomized by birth date, and I asked respondents their birthdates, I plan to see if those effects continue to persist after a year.
These are mostly conceptual descriptions of what I’m going to do rather than algorithmic descriptions of exactly how I’m going to process the data. Part of that is that a lot of this involves statistical techniques at the limits of my abilities and I’m going to have to see if I can actually do them. Most important, I would like to learn enough about factor analysis to actually check for a General Factor Of First Sight/Second Thoughts, and a General Factor of Ambiguity Tolerance. If I have them, I’d like to use them to see if they correlate with the other things I’m wondering if they correlate with. If I can’t make this work or beg someone else to do it for me, I’ll just eyeball the correlations between individual questions, see which ones are highest, and maybe take an average of those questions or something.
Mostly I won’t be doing anything fancy or with too many branching paths to the data, but I plan to operationalize autism in two ways. First, a scale where professional diagnosis equals 3, self-diagnosis equals 2, family member equals 1, and no personal/family history equals 0. Second, the Autism Spectrum Quotient test I made people take at the bottom of the survey. I’m not at all confident these will correlate more than a weak amount, but I’ll try it and see. I might also try some kind of average of the two measures. Since there are a few things I expect to be correlated with autism – mathematical careers, bad response to clothing tags, poor social skills – I might check to see whether the first measure, the second measure, or the combination does a better job of predicting these, and stick with whichever one does. I’ll try not to base which measure I use on any of the variables I’m actually testing.
One thing to keep in mind is that those of us who have been around here for a while are more likely to notice ‘the the’ simply because it is a particular technique we are expecting you to use. It’s almost become habit that at any point that I see a line ending with ‘the’ that I explicitly check the beginning of the first line for duplicates.
Scott can only analyse casual/infrequent readers if he wants.
I’ve been reading SSC for quite a while, and I still missed at least one duplicated “the”. So although the effect may be weaker in long-time readers, I don’t think it will be completely absent.
As a counter point:
I’m well aware of the “the the” thing, but I found that while I had the desire to notice them, I had very little motivation to do so. I didn’t want to spend an unreasonable amount of time taking the survey, and I found that reading with the focus that would be required to reliably spot “the the” wasn’t worth the effort.
I think it’s the same for me – I know about the trick and can easily detect it if I am in “attentive reading with intent to find problems” (e.g. the way one would read program code that is known to be buggy or text that needs to be corrected), but this mode of reading is slower and requires more mental resources. Thus, I don’t use it unless I have a reason to. If that means Scott can trick me over and over with the the the trick, so be it, I don’t really care.
Thirded, or whatever the relevant number is. I’m a good copy editor, and confident that I would notice this or things like it were I motivated to do so, but I wasn’t, so I didn’t.
Another one of the regular readers who would have been paying attention for the trick if warned beforehand, but was instead reading for speed not detail and got caught.
So the bait still works even on the older fishies in the SSC pond!
Yeah, the survey was really long so I was going through it pretty quickly and missed both “the the”s (in the second I was just scanning for keywords)
Another thing about the “the” thing — I suspect that a lot of people feel bad admitting to having been “tricked” and will lie to protect their ego. So peoples’ answers on the “the” question may correlate with the answers to other questions for the simple reason that both questions have a tendency to invite lies.
I think in general there is a decent chunk of the population which will consistently tell little lies simply to avoid giving an answer to a question which they think might make them look bad in some way. Or which they think might undermine their identity in some way. As an attorney, I occasionally make use of this phenomenon when cross-examining hostile witnesses.
I actually really doubt that (and I am usually on board with explaining things via people lying a lot). If IQ self reporting, which is a far bigger deal for ego, is fairly accurate, then self-reporting about having noticed the the should be, too. Moreover, it’s a pretty hard thing to lie to yourself about, because if you didn’t notice you didn’t notice – you can’t tweak that. And finally, in the last survey, only a very small percentage claimed to have noticed them, so clearly the people lying weren’t a big percentage.
I have never spotted any of the the the’s and I don’t see any significant loss of ego in admitting that… I see a bigger loss in admitting that I wasn’t capable of having the dancer change directions and not having noticed the circles, but still not much.
That’s quite a big “if.” What’s the evidence that IQ self reporting is fairly accurate?
There is no evidence of this. It seems to be assumed.
I’m not really sure why.
Well, actually the evidence seems to be to the contrary. https://www.researchgate.net/publication/247676963_Self-Report_Measures_of_Intelligence_Are_They_Useful_as_Proxy_IQ_Tests
And presumably there the test subjects knew that there was some minimal degree of accountability for their claims.
By the way, I’m pretty confident — based on my general observations and knowledge of human nature — that virtually all self-reporting is unreliable. If there is the slightest bit of ego involvement, social status at stake, or other incentive, a lot of people will exaggerate, twist, and outright lie.
This is especially true in situations where there is little or no accountability, i.e. the person thinks that there is no way to verify what he is saying.
With all due respect, that report is not talking about self-reporting of IQ… it is talking about the self-reporting of intelligence. In other words, in some studies, people are asked “How intelligent do you think you are?” In single question assessments like this, the correlation between their assessment and their actual IQ is extremely low (as reported in the paper you posted.)
This is not at all what Scott is doing in his survey. What he is doing is asking you to report your IQ as reported on some test you took somewhere. He’s not asking you to guess at your intelligence level… he is asking you to self-report your IQ.
My argument is that in an anonymous survey where people are motivated to want to be part of the “in group” of smart readers of this blog… there is a tendency to over-report their IQ. In fact, I wouldn’t be surprised if people who have never taken an IQ test report their IQ and pick an appropriately “smart sounding” number.
If you look at the data from previous years, the reported IQ is extremely high and skewed in an odd way given it is presumably sampling from the right tail of a normal distribution. The data I looked at last year would have to have been an extremely unusual sample given where in the population distribution the sample presumably came from.
Maybe this readership really is that unusual. Or maybe people don’t self-report all that well.
There are other issues with this measure including the fact that not all IQs are equal (for example, Cattell is scored with a standard deviation of 24, not 15) so the numbers by themselves aren’t very meaningful.
But people in the rational community like to talk about IQ so it is what it is.
I bet there’ll be a correlation between the “the the” question and the ambiguity measure, as those who dislike ambiguity would read the question and answers before they look at the image.
There are a number of internal threats to validity of this type of study. I’ve always been kind of curious whether this is intended to be a real study or just a “throw it against the wall and see what’s interesting” kind of thing. The fact that Scott is pre-registering seems to indicate he feels this is a real study… so I don’t think I am arguing a straw man here.
Compensatory rivalry is probably one of the biggest ones here… most readers of this blog know what the anticipated or exciting outcomes would be and there is also a strong bias to be part of certain in-groups (read: be in the high-IQ group).
Of course selection bias can’t be ignored… only SSC readers will participate and even then only a subset (I didn’t take the survey for instance.)
Then you have repeated testing effects (the one you mentioned) along with a list of other issues.
Of course this isn’t a purely experimental study and naturally you can’t control for every threat to validity in a study but there are a number of issues with the way this data is presented to the community. It is interesting and valuable and absolutely worthwhile. I’m not sure it should be taken with quite the level of breath-taking, gospel-reading, knowingly-head-nodding wisdom that happens every year.
But, it’s fun anyway. And yep, I’ll read the results… because they are interesting.
I’m doing a statistics-focused economics PhD and I’d be happy to help with the data analysis. Let me know if you want methodological guidance or help writing code.
Under #7, you say “Half-siblings will be ignored.” I double-checked the survey and it doesn’t say one way or the other whether half-siblings should be ignored when answering “How many (older brothers/younger brothers/older sisters/younger sisters) do you have?”
Noticed that too. I reported them as 0.5 just for fun (Scott, you can throw those out), but most people presumably reported them either as 1 or 0. Really they should be 1 for maternal half-sibling and 0 for paternal. Not sure if enough people have half-siblings to skew it.
I have 3 half siblings, and 2 sorta-kinda-adopted, and just reported them all without caveats. I didn’t notice any options or instructions that would have clarified how to report these.
Only if respondents knew in advance answers would be used to study in utero effect — if they thought it would be about nurture effects they answered 1 for half-siblings they were raised with and 0 for ones who were raised apart, etc.
I have 3 half-siblings, a maternal sister I was raised with and two paternal sisters I barely even know. I answered 1. I figured that was the spirit of the question.
I wrote out “three full and two half”, or words to that effect, which is accurate. I didn’t mention that the halves are paternal and much younger (there are 27 years between me and my younger half-brother, 18 between the youngest full and the older half).
All 4 of my siblings are half siblings, split among the two sides. I did not notice any disclaimer about half siblings so I reported them as full.
All my siblings are half-siblings, and I counted them as full siblings because the survey didn’t specify.
Not sure if you’re open to suggestions, but I recommend looking for a correlation between “gender thoughts” and both IQ and autism. Queer/”type two” trans women seem to score higher on these characteristics, and I’m wondering whether it’s a filter effect (e.g. people don’t transition unless they’re smart enough that they know they can make it afterwards) or whether these things are also correlated with more “gender thoughts”. I have reasons to think that it’s the latter, since certain autism-heavy high-IQ communities seem to score a lot higher on “gender thoughts” too. But I’m not sure.
There’s been a bunch of links lately about correlation between autism and trans.
My personal observation has been that ‘cis, I guess, but genderfeels/thoughts’ is far more common in autistic AFAB people but I never really seen the same thing in AMAB autistic people.
I tend to do gender surveys and I’ve found plenty of “cis, I guess, but genderfeels/thoughts” in autistic AMAB people. It’d be nice to examine with a real autism test, though, since I’ve only tested with autism diagnoses.
I would consider autism diagnoses to be a better call than some check the boxes test.
The problem is that autism may be underdiagnosed. E.g. only 16.6% of the participants in the 2017 Slate Star Codex survey reported being autistic (and that’s counting the people who said that they hadn’t been diagnosed, but suspected they might be autistic), which sounds too low to me. (Am I being crazy here? I feel like this place should be Autism Central…)
@tailcalled
I personally think that a lot of people find the border between spectrum diagnoses such as autism and certain basic personality types to be very blurry, even though they aren’t.
For autism specifically it’s unfortunate that (to use Jungian nomenclature) the introverted types in general, and the introverted thinking types in particular, seem to be a strong Asperger’s stereotype.
I don’t know if this is your issue.
The US rate is <2%, so 16.6% is Autism Central.
But is 16.6% Autism Central enough? I guess we’ll find out…
If we’re suggesting predictions, I predict that formally-diagnosed autistic trans people have lower scores on the autism tests than formally-diagnosed autistic cis people, and/or same with suspected autistic people.
(My hypothesis is that being trans causes something that looks like autism—if that’s the case, then it seems likely that there are differences in autistic traits compared to types of autism that don’t relate to gender. That said, this is far from perfect; e.g., as someone mentioned below the tests might be biased towards male autistic people, and I’m assuming the tests are based on what cis autistic people look like which might not be the case.)
This is a point of criticism s I have of the test. There are over 100 trillion ways to reach the cutoff 26 score, and I can post a known cognitive bias or a function of how answer depends on (none of stereotypical traits) for quite a few of the questions.
This is then related to how the set of traits is not used consistently either in the literature. Under capatilistic incentives, looks like it can devolve into “dudes weird, slap a diagnosis and see me next week”
I don’t even feel like autism is very well understood at this point. The whole idea of an “autism spectrum” implies that high functioning conditions like Asperger’s would be quantitatively different from autism, and have the same deficits in a weaker form, whereas it seems as though it is qualitatively different and things like language acquisition can actually be quicker than in normal children. Are high functioning autism and asperger’s really the same thing? They’ve been folded together in DSM-V, but if this was a mistake then it might mess up the gender correlations.
“4. I plan to investigate a general construct of “first sight and second thoughts” that involves people being better able to see what’s actually there, and less susceptible to illusions, priors, stereotypes, and assumptions. This will involve correlations between the two Duplicate Thes illusions, the Hollow Mask illusion, the Shepherd Table illusion, the Cookies illusion, the Parentheses palindrome, the Map riddle, the Surgeon riddle, the Switched Answers task, the Cognitive Reflection test, and the Wason task.
”
You (implied) worded that you wanted first sight, and *not* second thoughts on some of them. People not knowing which one you wanted can throw off the answers. Is this a visual glitch test, or a second-thoughts look back rationality test? Even though I knew I could answer correctly on some of them, your wording was
“How do the apparent?”
With *a* definition of apparent being
“seeming real or true, but not necessarily so.”
iiinnn that case, sure. Give the correct incorrect answer….or the incorrect correct answer? What?
My process. First thought “looks like that” immediate thought afterwards ” wait but rationality tells me to DO THIS” third thought “the wording…wait?”
Naq V’z cerggl fher gur fcvaavat qnapre vyyhfvba vf n “fhttrfgvovyvgl” vyyhfvba. V jnf abg noyr gb trg vg gb ghea gur bgure qverpgvba…naq oernxvat vg qbja senzr ol senzr, vg pyrneyl tbrf va whfg bar, gubhtu gur nez sbe nobhg n sbhegu bs vg vf shaxl.
Kxn drkd msbmvoc docd sc ofox wybo mvokbvi k docd dy coo gry sc sxmvsxon dy vso yx droco drsxqc.
…vz qrsraqvat zl “ab pvepyrf” bofreingvba(abj gung gurl unir orra cbvagrq bhg) ol fnlvat gung vgf whfg n onq erfbyhgvba nccebkvzngvba gb n pvepyr….
Bssvpvnyyl, v’z abg jebat.
If you couldn’t see the msbmvoc, then you should read this comment to the previous post about the survey.
ETA: Apparently I was posting to a cached copy of this page that didn’t have your reply to yourself, so you probably don’t need this anymore.
A problem is knowing that questions where there is no answer happen all the time in psychology…so when it comes to visual illusion questions in this context I have no way of knowing how long I should look for the answer!
I expect the percentage of men reporting sexual harassment or assault at work will be approximately half the percentage of women reporting the same.
The really
Culture Warringinteresting thing there will be the men reporting “no” to “Have you ever sexually harassed anyone?” because Harvey may say “I was only offering a friendly neck massage!” while Lupita says “This made me very uncomfortable”.Gentlemen, did you take off your trousers uninvited? 🙂
I would have answered yes to: “Have you ever made someone of the sex/gender you are attracted to uncomfortable due to actions based on that attraction.” Because I am sure I have.
I don’t think this rose to the level of sexual harassment.
/uncomfortable
By some definitions, any unwanted sexual advance may constitute harassment. So, e.g. asking someone out for coffee could count as harassment, according to the laws in your state and/or the internal rules of your organization.
But does a smoldering look count?
https://www.youtube.com/watch?v=UYhTLSs7jrE
That sounds an abuse of the term: harassment in normal logic has to have an element of _repetition_ in it. Like, someone ringing my door and trying to sell something is not harassing me. If they come back tomorrow and try to sell something else is no harassment either. If I tell them don’t come back and don’t try to sell anything again, and they still do, that is harassment.
https://www.thefreedictionary.com/harassment
“Synonyms: harass, harry, hound, badger, pester, plague
These verbs mean to trouble persistently or incessantly. “
I think based on the Weinstein example, as long as the trousers remained in place and fastened, it didn’t rise to “harassment” 🙂
I’d imagine those trousers would go out of the window in an instant and then I would be really embarrassed recovering them in my undies.
Re: the ‘the the’ thing and related stuff – I find my brain works differently depending on whether I’m trying to *read* the thing or just noticing it, if that makes sense? I will notice abnormalities and errors in stuff other people don’t notice due to the ‘snapping into the thing you expect’ effect most of the time – but never when I’m actively reading the thing. Poss related: ADD? In terms of ‘I am doing the task ‘reading’ and have no spare memory for the ‘noticing errors’ background process.
IMHO that Autism scale you’re using is a particularly bad one, overly focused on sorta stereotypical boy-nerd-who-likes-trains kinda autism, so given that I’ve come to expect that you know what you’re talking about in psych I found it odd that you’d picked it. Makes sense that you don’t expect it to correlate all that well with diagnosis. Are there not any you think are better?
Somewhere on this blog, a link testing lots of diagnosis like that effectively chalked up close to half the variation in choice of the doctor…aka luck. How good of a correlation after that can you expect?
Your link is broken; you want this link instead.
I found it odd that you’d picked it
If it’s the most commonly used one (or historically so) and a lot of studies have been done using it, that gives Scott a body of research to compare the survey results against. It may be a poor choice (I agree that it’s very gender roles bound and a bit stuck in the times of ‘boys like guns, girls like dolls’) but half a loaf is better than no bread?
For #7, you need to control for age. In a family with one child aged 20 and another aged 10, only the older is probably taking your survey. My sister is 15, and she doesn’t read SSC. Off the top of my head, two ways you can do it:
1. Throw out everyone below 30. This is pretty bad, and might skew things another way (50-year-olds read blogs less than 30-year-olds).
2. Look at the age distribution curve and the average gap between siblings in N-sibling families, and use that to normalize by the looking at the older-sibling proportion in each age group.
This second one requires a bit of math, which I’m always happy to help with. On that note, if you release the data to the readers at the same time you get it, a lot of us will do the analyses independently which will make the results a lot more robust.
The 15-year-old sister example doesn’t prove that we should expect an a priori correlation between SSC readership (mediated by age) and younger/older sibling. Indeed, in an equilibrium population society where birth order and life expectancy are independent, there is necessarily no correlation between birth order and age. If this seems unintuitive, consider that it is also the case that very few 100-year-olds read SSC, and thus there are one or several age brackets in which the younger sibling is the one more likely to be reading SSC (e.g. 30-year-old vs 40-year-old).
Of course, we are not in an equilibrium population society, and I think this means there is a weak effect in the direction you suggest. If population were decreasing, the effect would be in the opposite direction.
After doing the math more carefully, the first half of what I wrote is right but the second half is wrong. The dynamics don’t depend just on whether the population is increasing or decreasing but also on the specific dynamics of the population change. If the population is increasing exponentially then there is still no correlation between birth order and age but surprisingly, in absolute terms more people have younger siblings than older siblings (I should clarify that I am including siblings that are not alive (either dead or yet to be born/conceived), though most people don’t have these). The mathematical argument:
If the population is increasing exponentially, then the number of births in year n is proportional to x^n for some x>1. The number of births in year n of people with exactly 1 sibling who is 1 year younger than them is also proportional to x^n. Say the first number is ax^n and the second is bx^n. Then the number of births in year n of people with exactly 1 sibling who is 1 year older than them is equal to bx^{n-1}, since it is the same as the number of people born in year n-1 with 1 younger sibling. So b/a of people have a 1-year-younger sibling, but only b/xa people have a 1-year-older sibling. So out of people who have a 1-year-different sibling, x/1+x are the older sibling and 1/1+x are the younger sibling, and this is independent of the year of birth. For an n-year different sibling, it is x^n/1+x^n and 1/1+x^n.
Statistics suggest the world has about 1.4% population growth per year, which would imply x=1.014 (this is based on a doubling time of 50 years). Even if the average siblings are 10 years apart then this only gives x^n/1+x^n=0.535 (i.e. 53.5% of siblings are older), so it is not nearly enough to explain the effect sizes seen in Unnamed’s comment below (however it is in the right direction).
Of course, the world doesn’t have an exactly exponentially increasing population, so there could be some correlation between birth order and age. But the effect will be even smaller than the absolute birth order discrepancy calculated above (and significantly smaller if the population increase is close to exponential), so it probably doesn’t need to be controlled for. I’m not sure what direction the effect should be in, it depends on which direction the deviation from exponentiality is in.
You’re right, I confused myself with the math and agree that there shouldn’t be a bias towards older siblings by default. I didn’t even think of looking at deviations from exponential growth, so at least one good thing came from my confusion: I learned something from you 🙂
The birth order methodology you describe seems likely to be confounded by the fact that e.g. people older than 20 are more likely to read SSC than people younger than 5, so if you’re 25 and have a 5 year old younger sibling, there will be a birth order effect for boring reasons.
I think that controlling for age should *approximately* solve this. It won’t solve it exactly–for instance, if fertility increases over time, then a random 20-year-old will be more likely to have a younger sibling than an older sibling, which means that at a population level there will be more older-sibling 20yo’s than younger-sibling 20yo’s which will skew the results even after controlling for age. But that effect seems small enough that maybe you can ignore it?
I have a twin sibling. This means that I have 0 older siblings, and 0 younger siblings. Did you intentionally leave out twins and other forms of multiple birth? Should I answer based on birth order even though the difference is 45 minutes instead of at least 10 months?
I would have counted a twin as 0.5 older siblings and 0.5 younger siblings.
I had the same problem. “Fractional” siblings weren’t clearly an option, and anyway, would be ambiguous with “half brother” or “half sister”.
For next year’s survey, it might also be neat to ask about identical/fraternal multiples, and same-sex vs. opposite-sex fraternal pairs. Another thing would be to ask about maternal/paternal age at birth.
Clearly for twins, Scott should tweak the survey question to “Are you the Evil Twin or the Good Twin?” (We can check their honesty by comparing the answer with the one for the dominant hand – lefties are the Evil Twins).
“I plan to investigate the hypothesis about sexual harassment mentioned here: ”
I was sexually harassed in a summer job immediately following high school. This is not the industry I am in now and is not the industry I indicated on the survey.
My survey identification “random” identifier has the word “funk” in it.
By the by: I have worked in a total of at least 3 or 4 industries using the broadest possible definition of industry (5+ using more narrow definitions). Many of your respondents are likely the same.
I’m an actor, working in that capacity in film and theatre, which are at best arguably the same industry, and public sector recruitment, which certainly isn’t. I’ve also previously worked in theatre as a producer, director, production manager and writer, in risk management as an analyst, editor and web producer, for a TV systems company as a marketing manager, copy writer and field technician, and managed the refit of an office building to accommodate a new call centre, in the charity sector as a telephone fundraiser and for a ladieswear manufacturer/retailer in customer services. I’ve neither perpetrated nor suffered sexual assault or harassment in any of those jobs, but my response of “artist” (IIRC) to the job question may not encapsulate my professional experience all that well…
I’m an actor, working in that capacity in film and theatre, which are at best arguably the same industry, and public sector recruitment
Do you do any of those training videos where it’s “Hi, I’m Joe, a porter working in BigCity Hospital and definitely not a professional actor whose face you have seen before in other roles”? 🙂
(I had to watch one of these recently for work and I spent most of the time the earnest little “now, this is how you notice the signs of Bad Things In The Workplace and report them” scene was playing running an internal monologue critiquing the casting choices rather than Watching And Taking Note).
No, although I certainly would if someone offered me the gig. What I’ve done a ton of is pretending (in person) to be a prisoner or other person one might commonly encounter in a prison* for the benefit of people applying for jobs as prison officers. Also some corporate negotiation training, which inexplicably involved pretending to be the foreign secretary of a fictional nation, again in person.
*Usually a prisoner, though. Each candidate will have around 4 10 minute role play scenarios as part of the interview process. Typically 1-2 of those will be prisoners, 1 someone visiting a prisoner, and 0-2 people working in the prison in other capacities (chaplain, catering manager, you name it). 1-2 will be white males. Exactly one of them will be very angry and shouty and need calming down; that one will always be one of the white males, and usually a prisoner, presumably to avoid associating negative stereotypes with members of minorities.
Oh, I think I’ve heard of that in training medical students! They get people to pretend to have a particular ailment, teach them the symptoms, teach them what the student should be looking for, and teach them how to grade on (e.g.) were they sympathetic, brusque, explained what was going on, etc.
Similar idea, although the prison stuff is more heavily scripted than the medical ones, from what I understand, and assessment is done by a third party assessor, via either a live video feed or recording (and 10% of the recordings re-evaluated by someone higher up the foodchain for quality control purposes). I don’t actually think it’s a very effective test of the qualities it’s supposed to identify for most candidates, because it’s so formulaic most of them are able to just work out the game and play it, revealing next to nothing about themselves. The medical stuff is probably better, because it isn’t required to be as uniform or “fair”.
Another thing here is that the type of work someone does is not necessarily reflective of their workplace – think e.g. legal, HR, marketing in a tech company, or an IT support person in a law firm.
I interpreted the question as asking about the type of work I do rather than the type of company I work for, because I assumed the information of interest is the professional makeup of the community. But this makes the answer less useful for the purpose of getting info about industries. (Though not useless, since of course people’s jobs and industries should be highly correlated, and since e.g. a tech lawyer might still be predominantly surrounded by other lawyers in their job if their company’s legal department is sufficiently large.)
On the sexual harassment question, I don’t know if you did it the way you did for simplicity, but if you were trying for randomized response, you needed to make sure that all responses can be randomly selected; for example, “If the seconds ends in 0, respond ‘Prefer not to say’; if the seconds ends in 1, respond ‘Yes’; if the seconds ends in 2, respond ‘No’.” In this case, anyone saying “Yes” (or “No”, for that matter) has plausible deniability; as it stands, any “Yes” response is an admission of guilt (or a lie). If you employed RR, you could even potentially leave out the “Prefer not to say” option, relying on the plausible deniability built-in to the survey.
You also need to be careful in future surveys if you ask this question again, since you are able to correlate responses, and 1/100 of taking the question at exactly the right moment is a lot more suspect than a 1/10 chance.
I still don’t understand what that “number of seconds” caveat was intended to do.
I think it was to help the kind of people who take surveys on here and may be coming from Less Wrong; rationalists (especially those also trying to be Effective Altruists) seem to have a higher than usual proportion of people suffering from scrupulosity, and someone who does not want to answer the question but is struggling with guilt over “I have to answer, I can’t skip this, and I have to answer honestly” will get an out if the question says “You can skip this if the second hand is at this number”.
Meanwhile any Catholics taking the survey can go “Nah, I’ll lie about this and then go to Confession later*” 🙂
*No, theologically you can’t do this, anecdote for humorous purposes only!
But what if you also confess that you did that?
— Father, I lied on a survey, and I also justified it to myself by planning to go to confession today, and I justified that by going to confession, and after that I didn’t think about it explicitly, I just said a mental ‘etcetera ad infinitum’.
— My child, you must say 4 Hail Marys for the lie, 2 for justifying the lie to yourself, 1 for justifying that, etcetera ad infinitum.
— So 8 Hail Marys?
— I don’t know, I’m no good at math; I only went into the priesthood instead of clinical psychology because I couldn’t pass the required statistics course.
But what if you also confess that you did that?
Too late, you go straight to Hell, as witness the unimpeachable theology of Dante 🙂
You can’t say “I’ll commit the sin, then go straight to confession” and you can’t say “I’ll ignore this sin until the next time I remember it in confession whenever I go again” because that’s pretending to forget a sin and not genuinely forgetting it, and you can’t say “Never mind, I simply won’t confess this one!” because that then is unconfessed, unrepented and unshriven sin.
OK, seriously, I know that unrepented sin is unforgiven, no matter how many Hail Marys you say afterwards. But I think that it’s awesome that Dante’s devils are trained in logic!
This is a good idea, but some people who should answer Yes based on the clock may still be wary of doing so, which will skew the results even more.
Regarding the Shepard illusion, I only just now realized I forgot to mention that the way you asked it was really ambiguous. You asked us to compare the table’s lengths but didn’t say which dimension that was. I ultimately guessed you wanted us to compare the longer dimension of one table to the longer dimension of the other, but other people could easily have interpreted it differently. I have to wonder to what extent you’ll actually get anything useful out of that question.
Seconding being confused about this question. I was only confident what was being asked because I’d seen the illusion before, and knew what the trick was.
In addition, I wasn’t sure whether to answer the question based on my visual perception, or based on what I thought was true. It is a common illusion, and the answer is easily checkable by measuring. My train of thought for this question was:
A) The tables look different lengths, but
B) I’ve seen this illusion before, and they were the same length.
C) Further, as a meta-fact about optical illusions, when two things that are similar but different looking are shown, they almost always turn out to be the same. Otherwise, why would we be asked about them.
D) What if in this case, the objects are actually different, to test whether we are answering based on initial perception, or based on the meta knowledge of B) or C).
So I measured the objects, and found them to be the same.
This is all in contrast to the spinning masks, where it was clear that the question was about perception, rather than about the truth of what the object in question is like.
I hadn’t seen this particular illusion but I have seen similar ones, so I knew this was an illusion. But I answered based on what it looked like, not on “I know they’re both the same length”.
Similar thought process here. Despite my feeling pretty sure the parallelograms were the same or almost the same length, I chose to answer with what my raw perception seemed to give me, which was the other extreme at “twice as long”. Wording of the question could’ve been less ambiguous (unless this wording is The Standard Wording?), but I hope I gave what you were looking for.
By definition, the word ‘length’ is: “the greater of two or the greatest of three dimensions of a body”.
Seriously? The word is used in multiple ways. What’s in the dictionary isn’t very relevant here. I dunno how much saying “go read the Sequences” is frowned upon here at SSC, but… go read A Human’s Guide to Words, you know?
That’s an interesting guide, and I’ve saved it for future reference, but in the context of the dimensions of a table, ‘length’ is not an ambiguous term for me.
I’m curious, if someone gave you a tape measure and asked you to measure the length, the width, and the height of the table, would it really be ambiguous as to which axis was meant by each term?
Or if someone asked: “The table is 24″ x 36″. How long is the table?” Would you find that question ambiguous?
I’m on Otzi Ozbjorn’s side. I have read that sequence, and I agree that consulting a dictionary is normally a sign that something has gone wrong in a discussion, but I have never encountered ambiguity with the word “length”. (With the exception of buildings that are taller than any other dimension. In this case I don’t use the word “length” at all.)
There are definitely contexts where “length” does not or may not mean the longest dimension of an object, e.g. the length of an aircraft is the dimension aligned with the normal direction of motion of the aircraft even when this is smaller than the wingspan. But “length = greatest not-otherwise-defined” dimension is a pretty consistent default usage.
I wrote:
Of course, actually looking up the puzzle now, there isn’t a higher aspect ratio table, but I still think the ambiguity in language is there. Perhaps that’s what he was trying to get at with whatever the options were about 25% longer, 50% longer, or whatever. Maybe one of them maps uniquely to, “I just compared the y-axes in pixel space rather than trying to rotate and compare distances in my head.”
There’s also the fact that it asked which table was longer; the 2D representations may be the same length, but the table depicted on the left is almost certainly longer.
Indeed, I answered based on the image being a 2D representation of 3D reality and then what the length of the 3D tables are.
Perspective doesn’t appear to be used in the drawings (unless the two tables have very odd shapes), though point of view is.
We’re used to seeing perspective, even when it isn’t there. The angle at which the table is tilted seems to be identical in both images, but this means that the longer dimension (length) goes further from the perpendicular in the first image than the shorter dimension does in the second image.
I do wonder if some of the things you are looking to correlate are natural traits of human psychology vary based on circumstance.
For example, while my political views have not changed, my answer to that Nazi question is very different than it would have been two years ago, because circumstances are different.
7: We already confirmed once that rationalists are disproportionately likely to be first-born. What would make this time more “for all”? The results from the 2012 LW Survey:
Potential problem with H’s and E’s – on my shit Australian internet the image loaded so slowly the H was not fully rendered at first sight.
Yeah, any signal about H’s and E’s is probably going to be drowned out by the ridiculous amount of noise from browser choice, web vs. mobile, connection speed, and just about everything else that affects how large an image is when it opens and how it renders. Sorry, Scott.
You might want to do some evil sub-group analysis on these things, because I noticed several assumptions you make that are US or at least Anglosphere-specific phenomena and us foreigners will just show up as noise.
Blame any faulty data or dodgy-looking statistical analysis on the Evil Foreigners messing up the True-Blue American survey! 🙂
This comment exists to point out that the questions on your survey cause a selection effect among responders. I started answering, but quickly abandoned it.
Ditto.
I don’t normally give “me too” posts, but in this case I wanted to make the statistic available. Perhaps we can have a surey about whether people are willing to fill out surveys?
Seconding this. I finished answering on the half-way at the psychological part. At first I thought I could go thru this part but I didn’t.
(‘+1’
I answered a small number of questions, but decided against answering many of the questions included. If I’d had a ‘I don’t give a crap about this topic’-option (or something similar), I’d have used it quite often while answering the questions included. It’s quite likely that the ‘question response rates’ (the proportion of people who answer a particular question) will be potentially informative). (I hate to tell a successful blogger how to do things – I’m not the guy to ask, I’m implicitly trying to limit the number of readers of my blog – but if this was my blog, I’d definitely pay attention to large variations in response rates across the questions included in the survey – the people who are *not* answering specific questions are potentially providing important feedback, and it’d be stupid not to take this into account while analyzing the data.)
Regarding siblings age: after you have done that analisys it will be interesting to compare it with those who have zero siblings. My guess is that only children will look roughly similar to eldest-children, but who can really say?
I could not find my unique identifier string from last year, even after looking through emails to myself and saved documents from around this time last year, as well as going through last year’s published results csv file and finding myself in it. I expect a lot of other people with executive dysfunction issues will have a similar problem, so filtering out to just people taking the survey for the first time by looking for results from previous years that share an identifier string is probably going to affect eg conscientiousness scores.
Isn’t this the first year with an identification string?
4. I plan to investigate a general construct of “first sight and second thoughts” that involves people being better able to see what’s actually there, and less susceptible to illusions, priors, stereotypes, and assumptions.
So if I see chemical reactions instead of a child, I’m being rational?
A.I researchers won’t take this assumption well.
“7. I plan to confirm or disprove, once and for all, whether our community has more older siblings.”
Was anyone else confused by the wording of this? I thought “That’s strange, I thought the question was whether we have more younger siblings.” I didn’t even realize the ambiguity until I read the next paragraph. I had taken “our community has more older siblings” to mean “the people in our community have more older siblings”, but you meant it as “our community contains more people who are older siblings”.
Also, I agree with a previous commenter about the methodology for linking sexual harassment at work to industry type. Probably most people have worked in more than one industry, especially if you count casual and summer jobs, and if someone has been harassed or harassed others, it may well not be in the industry where they currently work. Or if someone was harassed at work but by customers rather than co-workers, the gender ratio of employees in the industry itself is irrelevant (for example, I’d expect waitresses to be harassed at work quite a bit).
@Scott Alexander:
If you plan to replicate one of Blanchard’s famous theories (the older brothers effect on homosexuality), why not investigate also his other, more controversial one, about the 2 types of MTF transsexuals? You have the data to do this – last year there were 80 MTF transsexuals among survey respondents.
If Blanchard & Bailey are right, then you’ll have 2 very different groups among MTF transsexual respondents, with significantly different results everywhere where there is a pronounced difference between men and women (ex: working in IT & STEM, atheism, libertarians, things vs. people etc.), with androphile MTFs (called “homosexual transsexuals” by Blanchard & Bailey) being somewhat closer to women and lesbian/bisexual/asexual MTF (called “autogynephile transsexuals” by Blanchard & Bailey) being somewhat closer to men.
You can also investigate if the supposed link between autism and transsexuals is true for both types or only for one of them (Bailey describes “autogynephile transsexuals” as frequently nerdy, so I suppose that their theory would predict that only the lesbian/bisexual/asexual MTFs would have an unusual rate of autism). If that is so, then I suppose the hollow mask lack of illusion (formerly observed by you in autists and transsexuals) will probably be also true only for lesbian/bisexual/asexual MTFs.
Also, according to Blanchard, only the androphile (“homosexual”) MTF would be affected by fraternal birth order effect. Maybe it would be interesting to compare the results of cis homosexual males with the 2 groups of MTF (if there are 2 groups).
Merry Christmas to everybody that celebrates Christmas!
The rationalist community is a high-IQ community with masculine nerdy interests. This means that it’s unlikely to attract enough HSTSs for them to be measurable.
That could be a problem indeed. But there were 221 homosexuals in the 2017 survey, so maybe there are a few so-called “homosexual transsexuals” too.
… maybe. I doubt it though; it is specifically the most feminine homosexual AMABs who end up as HSTSs, and I’d expect those in the rationalist community to be unusually masculine.
In addition, the rationalist community probably has such a high rate of autogynephiles that the HSTS minority is going to be statistically undetectable.
IIRC the B-B ‘homosexual transsexual’ wouldn’t show up on the survey as a homosexual – she’d show up as a straight (trans) woman. So the number of gay people doesn’t help you here.
Yes, but the number of feminine homosexual men allow us to make predictions about how many HSTSs there are in the sample, since they share their etiology.
I looked into the 2017 survey results (the Excel file) and among the ~ 80 MTF transsexuals there were only 7 androphiles (“heterosexual” in the survey, suposed “homosexual transsexuals” in Blanchard-Bailey terminology). 2 were agnostic, 5 were atheist (and not spiritual). One was a mathematician, 2 were biologists, 4 were computer programmers. 2 were diagnosed with autism, 1 had autist relatives, 4 had no autism and no relatives with it. All of them saw the Einstein mask “normally”, not hollow (had the illusion).
These data apparently contradict Blanchard-Bailey hypothesis – but maybe it’s because of the special, unrepresentative characteristics of this “rationalist community” with “masculine nerdy interests”, as tailcalled said? The cis women seem mostly atheistic, too.
You should note that some autogynephilic trans women are likely exclusively attracted to men, through what’s known as meta-attraction or pseudo-androphilia.
Basically, they are sufficiently autogynephilic that their autogynephilia overshadows their allogynephilic (allo = attraction to others, i.e. not auto) sexuality and makes them not attracted to women (like the “asexual” AGPs). In addition, they experience meta-attraction, i.e. attraction to being women who have sex with men, which in a sort of autogynephilic way makes them attracted to men – but not in the same sense as the HSTSs.
Eh, that term is too loaded. Homosexual vs non homosexual transitioners work better.
If you have the typical non- homosexual male image-sexual response heuristic, then autogynephile is implied to some extent.
Do hot gay guys like looking in the mirror? For seriously.
If this was the case, then most straight men should be somewhat autogynephilic, no? (And most straight women somewhat autoandrophilic?)
Idk. That term seems loaded and…too reductionist, in relation to how it’s commonly used. So I just don’t like using it.
Just….if a person with geno-typical XY visual sexual response is inclined to switch genders, some of the autogynrphilic traits are reasonablly deduced.
I took the survey on my tablet with a phone nearby. I did not have access to a clock with seconds. I quickly checked the default clock/timer/alarm apps on both, no big clock with seconds shown was available. I answered the sexual assault questions seriously.
https://time.is/
My own hypothesises:
* American readers are going to be notably more right/libertarian than other nations. 80%
I got to section 3, and went off to take the other surveys. After the 3rd one, my browser exploded, and lost everything. At least I remember my autism score. Maybe I can come back and start again.
I have the impression that most people consider personal experience rather more definitive and motivational than some dry study, so I’ve been wondering lately how much causality might run in the other direction. If you have direct experience (particularly negative experience) with those sorts of issues (harrassment, -isms, pay gap, etc), maybe you’re more likely to want to do something about them.
In which case even by well-defined objective standards, there would be higher rates among these communities than among the general public.
Of course. Libertarianism is non-existent outside North America.
Libertarianism seems to me to be a specifically North American brand of classical liberalism. There are ideological proponents of laissez-faire economics, and strictly limited central government in other industrialized democracies, but none go as far as US style libertarians.
My suspicion is that it a big part of it has to do with the history if the colonization of the American west. For a time in the second half of the nineteenth century the US could solve almost any social problem by endlessly exporting the poor to vast virgin lands west of the Mississippi. This not only delayed greatly the development of a socialist movement in the US, it created a national myth of “rugged individualism”, and an almost religious faith in a permanently open frontier.
You really do hear people on the American right say that the poor could solve their own problems
if they just had the grit to go out and stake their own claim, …somehow.
weirder, more autistic, more liberal, more schizophrenic, and more transgender people
Our brethren/sistern/othern! 🙂
Sounds like a fascinating project and I’m anticipating seeing if you manage to extract any kind of orderly result out of all this or run screaming mad into the night, whichever happens first.
Not arguing about the methodology, but as a matter of interest, why ignore half-siblings? I suppose this might lean more towards biasing the result to older siblings, as “my parents divorced then when mother/father remarried, they had a new baby and that’s my younger half-sib”, but isn’t there an equal chance of respondents to the survey being “I am the younger child of a second/third/whatever marriage”? And you probably aren’t making this obvious mistake, but I’ve seen it often enough in my work when people are completing forms and aren’t sure what to call their blended families: are you excluding not alone half-siblings (child of one but not two common parents with you) but also step-siblings, adoptive siblings and foster siblings? (People, you would not believe how confused parents get about if their child and their other child are half-siblings, step-siblings, plain siblings, or some other relationship).
Since those groups were randomized by birth date, and I asked respondents their birthdates, I plan to see if those effects continue to persist after a year.
And I’ve confused your nice clean data, because as I mentioned in another comment, I screwed up reading the question and gave the last digit of my birth year not the last digit of my birth day. Apologies! I don’t know if it’s any help now to say I put down 3 when it should have been 8?
That stingy dozen of hypotheses reads like the basis for many lifetimes of research. Don’t you risk torturing the data? That said, the hypotheses, themselves, were far more interesting and insightful than I would expect any data driven results to be. The right questions do not always have crisp answers.
The Surgeon Riddle result is going to be really good, as I’m forecasting it will show a cleavage between the older participants on SSC who remember the original riddle (or at least the underlying attitudes upon which it relies) from the 70s/80s, and the younger ones who were all “but what does the dead parent being the father have to do with the surgeon being the boy’s mother/second gay dad?”
Fun thing relating to that riddle: Linguistic differences between mother tongues can seriously affect this. If you translate the riddle to German, for example, you *must* gender the profession, and the default for ‘unknown’ is indeed masculine. If you do that translation and then ask yourself the riddle, I suspect it’s not as easy to get to the ‘mother’ answer as it otherwise might be, since even though you translated *to* German as “unknown”, the translated riddle has a masculine connotation in your head, regardless of your actual expectations (or lack thereof) about the gender prevalence in certain professions.
(This is obviously only true if your thought processes are German. I speak German and was born in Germany, but my thought processes aren’t always German. I don’t remember what mode I was in while answering this riddle, but since I’d heard the riddle before it didn’t matter. 😛 But it might nonetheless be fun comparing the country of origin (weak predictor of language in this day and age, but I don’t remember there being a language question) with the answers to the riddle.)
I see Bugmaster made a comparable observation a bit further down the page: https://slatestarcodex.com/2017/12/25/preregistration-of-hypotheses-for-the-ssc-survey/#comment-581672
(And in a funny coincidence I wrote “Bugmeister” before I wrote “Bugmaster”.)
Yeah, those who remember the original riddle will have chosen “ignore me, I’ve seen this riddle before”.
On birth order: younger siblings are more likely to have had older parents, and thus higher genetic load. So that is a potential confound for birth order per se, presumably one that comes up in actual birth order studies, although I don’t really know that literature.
I feel like I gave enough data that the computer could generate a detailed personality profile, complete with a list of problems I’m likely to be having and suggestions on how to fix them.
I feel like a “maybe/unsure” option for the sexual harassment questions would be a good idea, and might give interesting results.
Does anyone think anyone as socially callous and terrible as Weinstein would *ever* respond yes on one of these? Might as well admit to murder. This could easily end up lizardmans analysis,or as defunct as any analysis of “did you do Terri ble thing X? Please admit to it on our anonymous survey!”
That’s what makes it interesting, as I said above. A Weinstein-type will be all “sexual harassment? no, never! I merely was very friendly to some attractive young ladies, and offered to give them massages to put them at their ease, after all we’re in a profession where casual physical intimacy is commonplace”. So there will be guys/gals who harassed others and genuinely don’t see what they did as harassment.
What the split will be like is going to be the fun part: out of those who say “yes I’ve been harassed/no I’ve never harassed anyone”, what’s the male:female ratio? Of those who say “no” to all four? Of those who say “yes” to all four?
Yeah. You’re just going to get guys with a poor application of Game Theory and then felt bad about it answering yes, and not the actual worst.
I am puzzled by the whole thing as in my experience attractive young ladies I would be interested in don’t just work in offices. They work as waitresses, cosmeticians, hairdressers etc. but not in business. Average age of office women in my experience tends to be 35 and overweight, and generally not too attractive. I mean, to work in the office one needs a degree, and if I go to my local university, say, Bremen, the female students already look a whole lot less attractive than the hairdresser in the hair salon.
I don’t mean it in a bad way, I mean you can say the same about men. Go to a uni, or an office requiring a degree the selection of men is generally less attractive there. The sexy man is typically a motorbike repairman. The male version of the female hairdresser.
I mean, the whole unversity and office drone machine tends to suck both feminity and masculinity out of people. We just become genderless Professional People. Because there is nothing feminine or masculine about using Microsoft Excel.
And less educated jobs are simply more deeply gendered, there is an obvious feminity to being a hairdresser and working on making people beautiful and it is similar for the motorbike repairman.
I thought it is commonly accepted! That we trade much of our sex and gender in in return of a well paying career, this is the pact with the Devil professionals make. And this means no flirting, and also no harassment. As that would require taking off the mask of the gray-suit Professional and being a real human, good or bad, but real.
Pulchritude has not been lacking at the offices I’ve worked at recently. I’m too old, married, and unattractive to take advantage, but that’s another story. Hairdressers are a fairly mixed bunch; some good looking ones, but a lot of very overweight to obese ones. I get low-end haircuts though; maybe the higher-end hairdressers are more consistently attractive.
(Note that I work in New York City, which has a reputation for an excess of single women; this probably has an effect)
That’s the pact, but it’s not with the Devil, only the Bureaucracy; it’s signed only in ink, not blood. And people break the pact all the time. Rules or no you get a bunch of young and reasonably attractive people of mixed gender in the same office, you’re going to get flirting, unless all of them are hopeless introverts. And it only takes one sociopath to get harassment.
Maybe not the way you do it…
For what it’s worth, the thing to do would be:
1. Write the preregistration post, hash it, and publish the hash.
2. Publish the survey.
3. Close the survey, analyze the data, and publish the results along with the preregistration post.
Attentive readers can still guess at what some of the questions are getting at, but there’s no pleasing everybody.
Anyone reading SSC for any length of time will know what many of the questions were getting at.
And SSC readers will be the ones taking the survey.
Conundrum indeed.
I had trouble with the same question this year and last year, and it’s the minimum wage question. I agonized over how to answer and in the end, I don’t actually remember how I did.
I’m 100% in favor of UBI, and IF we have UBI, I’m 100% in favor of eliminating the minimum wage. In the absence of UBI though, I believe having a minimum wage is better than not having one.
I answered in the affirmative to the UBI question, and the minimum wage was the very next one, so I couldn’t decide whether the the minimum wage question was independent or not.
My ideal world definitely has no minimum wage, but if you put me in charge of making this one decision, “should we get rid of the minimum wage,” I would say no. It makes this question very hard to answer, and ultimately whichever answer I gave (I genuinely don’t remember) is a bad answer for almost anything you’re trying to do with this data.
Same here, except I just answered “no minimum wage, yes ubi.” I hope Scott sees this and remembers that these questions should be treated as one.
If I understand the math well enough, things are actually in reverse: you want minimum wage only if you have UBI, and you definitely don’t want minimum wage without UBI.
Minimum wage is helping incumbents – those who already have jobs – while hurting the unemployed, who have less chance of getting a job. It’s also helping privileged classes and hurting the outcasts – if you have halitosis, or the wrong ethnicity, or the wrong age or sex, you’re less likely to be hired because you’re already being hired at above-market salary, so the employer has a large selection to pick from.
Remember, with the minimum wage the state is not guaranteeing this income to everybody – is saying that if you can’t get hired at this much, you’re not allowed to have a job.
I disagree. A minimum wage causes a serious market distortion in the presence of a UBI, but corrects for a market distortion without one, because of the difference in the bargaining power of labor.
If work is necessary for survival, then labor has very little choice but to accept what is offered, but with a UBI, labor can use the choice not to work as leverage to demand more accurate pricing. This is why I would actually expect wages for most things to actually rise in the presence of UBI, especially jobs that are dangerous, monotonous, or otherwise distasteful. Without one, employers can keep wages artificially low, and minimum wage is a necessary (if highly inefficient) corrective for that power imbalance and its accompanying mispricing of the value of labor.
“If food is necessary for survival, then consumers have very little choice but to accept what price is offered…”
The same argument?
I don’t think it’s a market distortion that someone really wants to sell their labor and the buyers need is somewhat less urgent. And in modern America with the welfare state, disability, etc. there is some slack in any case.
I also don’t get the claim of “artificially low”. The employers offer what it takes to get the kind of employee they need, right? You’re not positing collusion?
The worker’s part of the negotiation is that if the wage is not sufficient they will look for another job. Distasteful jobs, such as dangerous jobs, are generally higher-compensated and not impacted by the minimum wage.
Most minimum wage jobs are held for a short period of time and/or by teenagers and retired people. For most people it’s the lowest rung on the ladder so you really jeopardize peoples’ future by making it more difficult for them to get that entry level job. We’re seen minimum wage devastate teen employment (although part of that is rising prosperity that makes jobs less necessary for teens). But especially among the lower skilled it’s not a good thing to take that first rung away.
Collective bargaining is a thing invented long ago, fixing the labor bargaining problem. That is, the labor bargaining problem is that walking away from the deal means only 1 worker less for the employer, but no income at all for the employee. Collective bargaining means walking away from the deal means no income for all of the employees, but the factory effective shuts down. These are roughly equally bad outcomes fixing the problem, making both sides about equally likely to compromise.
> walking away from the deal means only 1 worker less for the employer, but no income at all for the employee
Alternatively, you could say that the employee can get a comparable job soon enough, and a worse job immediately, but the employer is constrained to hire only someone properly trained, or spend significant time and money on instruction.
Collective bargaining went out of fashion mostly because it’s a lot less necessary. And when it is, things aren’t great for different reasons (bad economy, government employer etc)
> but corrects for a market distortion without one
I’m sorry, but I don’t understand. I think you’re assuming everybody is employed?
If you have minimum wage, vulnerable categories do not have a job. If you have minimum wage and no UBI, vulnerable categories are at the mercy of a complex, inefficient and very sticky welfare system.
If you do not have minimum wage, there’s a lot more employment and people can get at lest the equivalent of an UBI (which won’t be great in any scenario).
For the Surgeon riddle, you might want to check the correlation with the survey taker’s native language. Some non-English languages have gendered words; for example, the word “surgeon” in Russian has a male gender. A female surgeon would still be referred to by the same male-gendered word in Russian; but still, a native Russian speaker may be primed to think of e.g. surgeons as male, cats as female (although there’s a special word for “tomcat”), and trees as neutral.
…for example, the word “surgeon” in Russian has a male gender.
Same in French, same in German, etc.
Maybe it is not a coincidence that “surgeon” is usually masculine.
I thought that my native tongue had influenced me, but after seeing so many native speakers of English make the same mistake, I am now much more doubtful that this is a language issue.
Well, in the US, surgeons are over 80% male overall, and ER doctors are over 75% male. More senior/experienced surgeons are more likely to be male than junior surgeons. Some surgical specialties are even more skewed, such as orthopedic surgeons (96% male) and thoracic surgeons (95% male)– which, given the setup of the problem, would be more likely to be the type of surgeon called to the case. So it’s not primarily a linguistic quirk, to assume that a surgeon is a male.
You remind me of some real and fictional examples of situations that one hypothetical person might claim are examples of the Sapir-Whorf Menace while another hypothetical person might point out that they’re basically just local language accurately describing local reality:
– in Brandon Sanderson’s Stormlight series, a particular country determines nobility by the shade of a person’s eyes. If your eyes were dark and become light, you become an aristocrat. The term for “noble” in the local language is “lighteyes”. When a foreign storyteller is describing customs in a different country to commoners from Alethkar, he glosses a foreign term for nobles as “they are lighteyes, except their eyes are not light”, prompting the confused response “how can you be a lighteyes without light eyes?”
– I have personally encountered a close analogy in English: a Chinese person asked me once what the English word for 姓 was. Embarrassingly, the American word for this concept is “last name”, despite the fact that this part of the name comes first in Asian countries.
– Chinese languages don’t mark gender on anything. The syntax doesn’t call for it and it isn’t normal practice. That said, the common word for “whore”, 妓女, is overtly female.
Regarding ‘姓’, you can at least translate it as ‘surname’ or ‘family name’.
Regarding the lack of gendering, even the gendered pronouns ‘他’ and ‘她’ were only distinguished after contact with Europeans, and they continue to be pronounced the same (at least in the commonly spoken languages, such as Mandarin and Cantonese). Along with the introduction of European punctuation marks, this always seemed like a silly accommodation to me.
You could translate 姓 as “surname” or “family name”, but those are not current words for the concept. If an American wants to refer to this concept, they will use the word “last name” 100% of the time (within measurement error).
This makes “surname” and “family name” pretty dubious as answers to the question “how do you say 姓 in English?”
I understand your point, but this statement is false.
@Toby Bartels
Here is a list of government forms: https://www.opm.gov/forms/standard-forms/
I clicked on the first one, and as expected it had written at the top: “Name (last, first, middle)”.
OK, I’ll take your word for it; let me know when you’ve gotten through 100% or reached the limits of your measurement ability.
Meanwhile, I’ll keep saying ‘surname’ and ‘given name’ to my students when I need to clarify what their names are. Let me know if you want to make observations.
The important point is that translating it as “surname” or “family name” will mislead the Chinese person far more often than translating it as “last name”.
Your students, assuming they’re mostly natives, will have no problem internally translating “surname” or “family name” to “last name”.
Well, I hope that wasn’t Michael Watts’s point, because I certainly don’t agree with that! Saying ‘last name’ is exactly the sort of thing that might lead to confusion, because they might think that I mean one thing when I really mean another. Whereas, if I say ‘surname’ or ‘family name’ (or ‘given name’ or ‘personal name’ for the other meaning), then if they don’t know what I mean, then at least they’ll know that they don’t know what I mean.
The pertinent point in my mind is that they not get confused when faced with forms which assume that “last name” is the family name/surname.
I’m generally responding to your 6:59 PM post (as to why the translation of that character should not be “surname” or “family name”), not your response to Michael Watts’ post.
Yes, for that purpose, ‘last name’ is the right word to use. (I might argue that the English word best translated from ‘姓’ is ‘surname’, but the English word best translated to ‘姓’ is ‘last name’. However that probably won’t hold up on all situations.) The best choice depends on context.
Given that you’re testing 23 hypotheses I expect that one will turn out to be p <0.05 by chance. If only one does then I’ll not believe the result, but of course I’ll believe things that are much more significant.
If you believe any of them simply because of a p-value, then I would argue you are interpreting the data and conclusions wrong.
ASA Statement on Statistical Significance and p-values
Wouldn’t announcing your results to the poll respondents ahead of time risk influencing their answers? For instance, say I’m a conservative who feels strongly that conservatives are better at seeing the forest for the trees, and take a lot of personal pride in this – wouldn’t I be more likely to take careful note to be on the lookout for the “HE” illusion, and make sure to respond that I see the “H” first, even if in actuality I did in fact notice the E’s before the overall H?
Not to mention that this gives those who enjoy messing with survey results (or their takers) on principle all the ammo they need to debunk your various hypotheses .
Of course, maybe you’re aware of this, and on Yomi-level 2, so some of your professed hypotheses are in fact the opposite of what you’re actually expecting to see… 🙂
Still, I think a better way to pre-register (and this extends to survey takers in general – the field as a whole seems to be locked in a quite naive mindset vis-a-vis the human capacity for “test-taker’s rebellion” and other such cognitive shenanigans) would be to post a md5 hash of your hypotheses ahead of time, but keep the actual text under wraps until the conclusion of the experiment. Then those interested can check the actual text against the time-stamped md5 to confirm your honesty.
Well, Scott can presumably check how many surveys were completed before and after he published the pre-registration post, and if the “after” surveys all show the kind of tilted responses you suggest, he can take that data into account as well (viz. “X percent of people are lying liars!”)
Having the “before” answers to compare against the “after” answers will give some kind of guide as to what the ‘raw’ data should look like without “I’m going to answer this to tilt it in the direction that Scott was looking for”.
#7 The question was do I have an older sibling. I said no, because he died. I am not sure that it would be statistically significant, but if question was put in other way, I would say yes. I think it is nice idea to change the question in future surveys.
I don’t know if I’ll run the numbers again, but I expect the curves for charity vs politics and charity vs religion to look about the same as when I ran them on the 2014 data.
Similarly for the strong negative correlation between polyamory and mental health found in 2017’s results.
Ambiguity tolerance -> higher likelihood poly?
I wrote:
I’ll be fascinated to discover if my linguistic pedantry actually predicts my right-leaning-ness.
It”s too late now but I wish you had gone deeper on the sexual harassment question. I am a man who personally answered yes to the question “have you been sexually harassed,” only because I once had a girlfriend who did pressure/harass me into having sex with her on multiple occasions. Based on that question alone, I would bet things are pretty similar for each gender. But if, say, the question was how many people I’ve been harassed by, or if sexual harassment has had a significant negative effect on my life, I bet the results would be quite different. In other words, my guess is that men and women both experience sexual harassment at about the same rate, but that the frequency of harassment is much higher for women, and that it impacts the average woman’s life more negatively than it does the average man’s. I haven’t seen you reference any studies that make that distinction.
“I plan to investigate whether autistic people are more likely to give process-centered rather than person-centered answers to the two political categorization questions (categorizing Nazis, categorizing civil disobedience on gay marriage). That is, neurotypical people will be more likely to categorize based on which side wins, and autistic people will be more likely to categorize based on what procedures were followed (eg violence, civil disobedience).”
But who does win? The nazi one presupposed that nazis beating up minorities means nazis “win” when the result is the opposite of winning, as even they usually have enough brain matter to understand. The gay marriage one presupposed that if a clerk printed a marriage license for a gay couple in a place where gay marriage was illegal the result would be that they are married, when all they’d really get is an invalid document and fifteen minutes of fame.
all they’d really get is an invalid document and fifteen minutes of fame
Which is all a lot of the cases in the early days wanted, it seemed to me. They wanted that fifteen minutes of fame to garner publicity to help drive the “but joe and bob are so nice, they look like such a lovely couple, why shouldn’t they be able to get married?” change of attitude among the straights.
Like the cake-baking case where, even though their out-of-state marriage was not legally recognised in the state, so as far as the law where they lived was concerned they were two single people cohabiting not a married couple, they still took and won a case against a baker refusing to bake them a cake to celebrate their ‘marriage’. They didn’t get a valid marriage out of that, but the aid towards the impetus to change the law was the thing they really wanted and what they got.
Moreover, I think there is an assumption there that all the people who want one side to win, they want the same side. Since in modern language “nazi” mostly means “evil”, and “beating up innocent people” is something obviously wrong, the question is formulated in a way that it is nearly impossible to want the nazi to win. But the other one is less clear-cut, some people do want the gays to lose and don’t care much about the procedure.
Maybe Scott is aware of this and formulate this exactly for this reason: that if someone wants both the nazis and the gays to lose, then they may be something like a person-centered conservative?
I am really hoping to see analysis of enthusiasm for puns vs autism score. It’s, um, relevant to my interests.
u dog
i wouldn’t have done this survey if i knew i was gonna be used for a correlation study
And this is why we need the IRB!
The SSC IRB is mysteriously rubber-stampish when it comes to Scott’s studies
Scott,
I think you really don’t understand ADHD people. My psychiatrist told me every single case of diagnosing someone with ADHD in their adulthood also came with depression, every one, because the world gives you enough shit calling you lazy, get yo shit together son etc. if they don’t find it out in your childhood. And does enjoying the buzz, being ambitious and enjoying risk sounds like a depressed profile to you? Don’t buy into the XKCD “look there is something cool shiny there!” stereotype of adult ADHD. Since it tends to come with depression, it is more like attention directed INWARD, ruminating, rumbling thoughts, being lost in thoughs, more like zoning out and listening inwards, to the inner monologue, not chasing shiny things outside. So actually it is the opposite, I like the idea of the village where the inner carousel of thoughts chasing each other could relax.
I thinkl your impressions are muddled by perhaps belonging to a group where childhood ADHD is very effectively noticed and medicated. It was clearly not so in e.g. East Germany in 1980, but not even in West Germany. We tend to find out in our adulthood when we began critically analyzing ourselves.
I don’t really understand the Caparos study. I am pretty right wing, yet in my mind the E *is* a letter, while the H shape just *looks like* a letter. From my angle it is a test between real things and appearances. I also don’t think it is cognitive rigidity, you should control that for IQ, from my angle rightwingitude comes from the sheer doubt of the common oppressor-oppressed narratives, from thinking the world is more ambiguous than that. High IQ right wingers simply think everything we were thought as a black and white fight between good and evil was ambiguous and even the South or Hitler were not simply evil, they had their own perspective. I think today complete utter moral relativism, cynicism, nihilism would make you a right-winger because you would fail to morally condemn bigots. But actually in this case I display rigidity by thinking the E letter IS a letter, while the H shape is just a bunch of letters looking like a letter but not IS so.
Would like to predict a move to the right wing in the SSC readership, tho I am lazy enough to not want to search to see if the old surveys took this data.
Just a note: I’m a twin. When answering the question “How many older brothers” I put 0.5. When answering “how many younger brothers” I put 1.5.
Unless double-barreled vaginas exist, there has to be a birth order between twins, making one technically older.
Only if you’re counting relative age at time of birth and not time of conception or implantation. I favor the last given IVF.
Some people have been counting stepsiblings and half-siblings (even on the father’s side); there are also potentially adoptive and foster siblings. So while equal birth times would be an amazing coincidence here, they are possible. (More likely, although probably still pretty rare, are cases where the precise order is unknown due to imprecision in the known birth times or even dates.)
I have been reading this blog for several years but have never commented before. I enjoy the blog because I gain some insight as to the issues involved in setting up experiments and then analyzing the data. Scott clearly cares about his work and his patients, and I appreciate and respect this. I would not take the time to read the blog if I detected an attitude of contempt.
I am writing to describe what I consider an oversight in the 2018 survey: Of the over 20 choices for occupation, there are no choices for technical positions that do not require a degree, or for the skilled trades, or for low or mid level service positions, or for laborers.
It appeared that Scott was not interested in surveying people without a formal education until I saw the choices of “none” and “high school” for the question asking for the highest degree earned. Did the occupations listed above not occur to him as he was thinking up the categories for “occupation”? If so, why? There are plenty of us everywhere! Are we invisible, or does he assume that none of us will read the blog or take the survey?
Whatever the reason, I think that his data would be of higher quality if there were a broader selection of occupation types. Of course, I can accept that he may interested only in people who are highly intelligent and educated (another thing: is it common for people to know their IQ? How do they find out what it is?)
In the US many school districts test all of the students with an IQ test prior to the grade in which the gifted program starts. Others use group ability tests such as the COGAT with some correlation to IQ. Many other students who are at least “bright” will take an IQ test to see if they qualify for a gifted program.
If you have taken an IQ test for K-12 school just call up your high school and ask to see your permanent record. They may need time to pull it out of long-term storage (if it’s on paper), but your IQ test results will be in it.
Note that childhood IQ tests only correlate somewhat to adult IQ.
As an adult, you can likely take an IQ test after enrolling in college and seeking services at the psychological center. This will likely cost a few hundred dollars, but is a discount compared to taking an IQ test through a regular psychologist.
I’d like an option for periodical commenting. I common very frequently for short bursts and then leave for a while.
I’m the same. I just rounded down to “I only lurk”
Its a pity you didn’t include methylphenidate in the survey since it fills the role of Adderall i.e. study aid and ADHD medication in sweden, and likely europe.
A few points of feedback to the survey:
– Unless you are interested specifically in polyamory and polyamory only, I think “Prefer non-monogamous” would be a better phrasing for the relationship style question as it covers the range outside of monogamy (eg open, swinging, etc). Or further split it up – monogamous, non-monogamous, polyamorous. ‘Other’ doesn’t convey much information.
– Regarding the area of work, I gave my current area, but my answer to the sexual harrassment questions pertained to when I worked in a different area. To the extent that this occurs it will skew results re: hypothesis 10.
– I really wish you would separate SJ and feminism. Sure the latter is very amorphous and open to individual interpretation, but it is certainly separate from SJ. Everyday Feminism != feminism.
You list a lot of comparisons for preregistration. Especially since many questions contain more than one thing you want to compair (“I’ll probably use Ambition, Social Status, Romantic Life, and Morality for this”). I don’t know how you want to combine these, but I am worried you will still have a lot of comparisons in the end (except if you make a more algorithmic preregistration later and it has less comparisons). If that is the case there can still be a lot of false positives. I think it is then important to use the data in such a way that you deal with this: listing the results of all comparisons and if you use a specific p-value later compensating for multiple comparizons. (But I am not entirely sure since I am no expert (yet))
You raise a good point. The family-wise error rate on some of these questions is probably fairly high. For example there are a number of hypotheses related specifically to birth-order (5 or 6 maybe?) so there should be a correction (Bonferroni or similar) to account for multiple hypotheses about the same data.
Simply pre-registering isn’t enough to mitigate against data dredging… statistically there is a need to control for false positives as you say. Good call.