How wrong is David Brooks?

(written by lawrence, however indented passages are often quotes)

David Brooks is sloppy in his use of sociology:

In this case, Mr. Brooks has taken his science from the work of Richard E. Nisbett, as described in his 2003 book The Geography of Thought: How Asians and Westerners Think Differently and Why, and in many papers, some of which are cited below. I was familiar with some of this work, which has linguistic aspects, and so I traced Brooks’ assertions to their sources. And even I, a hardened Brooks-checker, was surprised to find how careless his account of the research is. The relation between Brooks’ column and the facts inspired me to model my discussion after the Radio Yerevan jokes that arose in the Soviet Union as a way to mock the pathetically transparent spin of the Soviet media:

Question to Radio Yerevan: Is it correct that Grigori Grigorievich Grigoriev won a luxury car at the All-Union Championship in Moscow?

Answer: In principle, yes. But first of all it was not Grigori Grigorievich Grigoriev, but Vassili Vassilievich Vassiliev; second, it was not at the All-Union Championship in Moscow, but at a Collective Farm Sports Festival in Smolensk; third, it was not a car, but a bicycle; and fourth he didn’t win it, but rather it was stolen from him.

Question to Language Log: Is it correct that if you show an American an image of a fish tank, the American will usually describe the biggest fish in the tank and what it is doing, while if you ask a Chinese person to describe a fish tank, the Chinese will usually describe the context in which the fish swim?
Answer: In principle, yes. But first of all, it wasn’t a representative sample of Americans, it was undergraduates in a psychology course at the University of Michigan; and second, it wasn’t Chinese, it was undergraduates in a psychology course at Kyoto University in Japan; and third, it wasn’t a fish tank, it was 10 20-second animated vignettes of underwater scenes; and fourth, the Americans didn’t mention the “focal fish” more often than the Japanese, they mentioned them less often.

The research in question was reported in T. Masuda and R.E. Nisbett, “Attending holistically vs. analytically: Comparing the context sensitivity of Japanese and Americans”, J. Pers. Soc. Psychol. 81:922–934, 2001.

The subjects were 36 Americans at the University of Michigan and 41 Japanese at Kyoto University, who “participated in the experiments as a course requirement”. The subjects each watched 10 animated vignettes of underwater scenes, where “Each scene was characterized by having ‘focal fish,’which were large and had salient colors and shapes, moving in front of a complicated scene”.

After a vignette was presented twice, the subject was asked “What did you see in the animation? Please describe it, taking as much as 2 min.” The subjects’ oral responses were recorded, transcribed and coded.

Source