Stuffing up the Dan Ariely Survey
Stuffing up the Dan Ariely Survey
Earlier in the week I asked Bristlemouth readers a couple of unrelated questions:
1. What are the last two numbers of your mobile phone number?
2. What percentage of countries in the United Nations are African?
We received 218 responses to our little poll and we split the responses up into those with a phone number than ends between 50 and 99 and those with a phone number between 00 and 49. The results look like this:
|
Last two digits of phone number |
Average guess for number of African countries in United Nations |
|
00-49 |
23.53 |
|
50-99 |
23.83 |
For those who prefer a visual representation, here’s a graph:
Looks pretty random, doesn’t it? That’s because it is. There is no statistical correlation between the two.
It wasn’t supposed to work out that way. The idea was that, not knowing the answer to question two, Bristlemouth readers ‘anchor’ themselves to the answer they had given to question one, and we’d be able to see a correlation between phone numbers and the estimated percentage of African countries in the UN. Then I’d turn around and tell you how important this concept is in the stockmarket, where we are faced with a lot of uncertainty and a lot of mostly random, useless information on which to anchor ourselves.
If you’re a QBE shareholder, write down what you think the shares are worth. Now write down your average purchase price. Is the answer to the second question influencing your perception of the first? For most of us, I’d guess so.
Unfortunately, I stuffed the experiment up. During our interview, Ariely explained that you need to create a high level of uncertainty. In this case, most of you thought you could take a decent stab at the answer and didn’t need to anchor. The average response, almost 24% was close to the correct answer, 29% (at least the experiment provided some more evidence for James Surowiecki’s The Wisdom of Crowds). What I should have done is, instead of asking for a number, ask you whether the percentage was higher or lower than 29%. That would have created a lot more uncertainty and, in theory at least, a lot more anchoring.
Sorry about that. The good news is that Ariely and his team want to conduct investor-specific experiments and I’ve volunteered us to be guinea pigs. I’m sure his experiments will be much better than mine.
Custom Search 2
Comments
Maybe Bristlemouth readers are a lot smarter than you expect, aware of the behavioural biases which beset investors (and everyone else) and were simply able to dissasociate the two questions from each other. Maybe.
Steve, I understand the purpose of the survey, but am struggling to comprehend the chart titled 'Survey Results'. Would it not make better sense to have the last two digits of one's phone number on the x-axis (horizontal), and % African nations in the UN on the y-axis (vertical)?
That would resemble the legend in the chart.
Instead it appears the chart depicts the relationship between sample number (218 respondents on the x-axis) and phone number (y-axis). Perhaps that is why there is no apparent correlation! Or am I missing something?
P.S. Will Bristlemouth be posting a record of the interview with Ariely?
I'll definitely post a link to it - we'll open it up to the world.
The chart gives you the two data points for each entry (sorted from lowest phone number to highest phone number). So the red dot is the phone number, then there's a green dot associated with that number (they are aligned vertically). If there was correlation you would see the green dots rising with the red dots.
We could have just plotted question one against question two, it would have given the same result.
I'd agree with Justin. I don't think we were the ideal guinea pigs for an anchoring experiment when it is something you and the guys at II highlight quite a bit as something to be aware of.
I guess the secret is that most of the respondents KNEW their mobile phone number. :-)
Hi Steve,
I would have thought the problem was that the 2 questions were completely unrelated - my phone number would have no affect on what I thought the % of nations was. However, asked what price I paid for QBE shares, followed by what price I now value them at, probably would affect my judgement as both questions concern a $ value for QBE shares.
Cheers,
Shaun
I found another problem with this survey. I don't have a mobile phone. Nonetheless I completed it, using my landline, as I thought the voice of the conscientious objector should be heard! I think the percentage question was one where a lot of people, much like the II of old, could get it roughly right rather than precisely wrong. And most, I think could recognize an anchoring trap. Recognizing a value trap - that's the hard bit.
I think your point is right about Surowiecki who said 'collective decisions are most likely to be good ones when they are made up by people with diverse opinion, reaching individual conclusions relying primarily on their private information’. Asking the last two digits didn't impact any of the criteria necessary for an informed group decision.
Generally I find there is far too much emphasis given to the view that people are stupid, especially crowds. The market isn't stupid either if those conditions for a collective decision are in place.
Post new comment