Quack quack!

Quack quack!

This is rather exciting. The excellent David Colquhoun of DC Science is coming up to Auld Reekie to give a talk on pseudoscience and quackery. If anyone is around Edinburgh on the 20th March I thoroughly recommend you pop along to Appleton Tower and hear him.

For those who’ve yet to hear of him, you should stop what you’re doing now and visit his webpage. David Colquhoun (described by Ben Goldacre as “The funniest man I know”) regularly blogs in his typical sardonic fashion on the onset of fake medicine and the “Endarkenment”. This is going to be brilliant.

Mmm...brains, anyone?

Mmm...brains, anyone?

Opening up my Guardian today, I was pleased to read their report on the Which? Magazine survey focusing on “Brain Training” games. You know the sort; the user is given a simple repetitive task to do, such as adding up numbers, and over time it will ‘improve’ your brain. Except that it won’t really. Well, not much more then any other activity, such as doing a crossword. That won’t set you back £30 for one of their games as well.

The Which? report found, with the help of several neuroscientists, that there was no peer-reviewed evidence to back up any of the manufacturer’s claims that they will improve performance or stave off dementia and other mental illnesses. The same level of improvement could have easily been attained through playing other computer games such as Tetris.

Fair play to the Guardian for being the only major British newspaper to report this, especially as they have a page on their jobs website sponsored by Lumosity in which one can, erm, ‘Train their Brain’ in order to “Improve basic cognitive abilities including memory and processing speed.”

'Train your brain' on the Guardian's own website

'Train your brain' on the Guardian's own website

Lumosity have been mentioned a lot in connection with this report, as it is their very own Michael Scanlon who offered a rebuttal to their findings. “We would never say Lumosity is proven to improve day-to-day living, but there is more and more evidence it does. We have actually conducted our own clinical trials to measure effectiveness of the product”, he said using his own mouth.

It’s interesting that he said “We would never say Lumosity is proven to improve day-to-day living”, considering that in the picture above it explicitly claims that their games can improve memory (note the meaningless graph on the right-hand side) and ‘processing speed’, whatever the hell that is. Their own website is full of PR guff on how their games ‘take care of your brain’ (and presumably dry your tears when you’re feeling down as well).

The news article also lists him as a ‘neuroscientist’, which is repeated ad nauseum in other reports on this story. Again this is interesting as this factoid isn’t listed on Lumosity’s own website, and the only references to his neuroscience qualifications was this page, which lists him as an alumnus of Stanford University. So it appears he has the credentials, but I find it a bit weird calling someone a neuroscientist when they haven’t recently performed or published any research. Especially research into their own wondergames.

That’s not entirely true. Dig deeper and you will find this small internal study which appears to back up the companies’ claims that their games will makes you smarter. Except that it’s flawed to hell and back (which is presumably why it’s unpublished in any decent scientific journal). They took around 20 people and split them into two groups; one that performed simple visual and memory tests without playing their games (the so-called ‘control group’) and another which did whilst playing Lumosity games in between. So what went wrong?

1) Few participants. They only looked at 23 people, with 14 placed in the trained group and nine placed in the control group (and one of them dropped out). This is a tiny number to make up a scientific study with. There will naturally be so much random noise that it would be hard to pinpoint down any changes between groups. It’s not exactly hard to hire people to play games for a study, so there’s really no excuse.

2) Lack of a decent control. The only ‘control’ group did not play anything between taking tests. So how can the investigators be sure that it is the games themselves causing any benefit, or just the fact that a group of people were performing a simple task to increase awareness? A third group in which volunteers did crosswords or played Tetris in between tests would have sorted this out. Of course, this might show that Lumosity games are not very good, which would make for a horrendous marketing campaign, wouldn’t it?

3) Cooked results. One of their claims is that ‘memory span’ was increased after playing their games. Was it though? Their graph was presented as follows:

graffThe study made a big deal that in the trained group there is a ‘statistically significant’ increase in memory capacity after playing these games. That is, the increase was very unlikely to happen by random. What about the control group though? The scores attained there are, in essence, exactly the same as the trained group. So these games don’t make a bit of difference to memory performance, in this regard. In fact I wouldn’t be suprised that just getting used to the tests lead to a naturally increased score, just by becoming accustomed to it.

Sigh. I’m getting tired now or marketing guff and the pointless word ‘Lumosity’. If you really want to mildly stretch your brain, why not do a crossword? The Guardian have a massive archive for you to try. (I have to linked to them as I do feel mildly guilty with the earlier comments; being a long-haired liberal, it is still my newspaper of choice.)

Finally, for more brain myths debunked, why not watch online the University of Edinburgh’s Christmas lecture, given by the excellent Sergio Della Sala? OK so it may take ages to load and still relies on Quick Time as opposed to Flash, but it’s a great way to learn more about why these marketing ideas are actually more likely to turn your brain to goo.

I feel it ironic that my own personal vision of hell consists of a young-earth creationist lecturing me until the end of eternity. That’s what it felt like, sitting in the lecture presided over by Kirk Durston on why “Everything you know about intelligent design was wrong.” Drone drone, waffle waffle, glib glab; he went on, unable to focus on a point when he needed to. As far as talks in bullshit go, this one was particularly soul-destroying.

Bullshit it was though, and there’s so much of the stinking mess to cut through that I don’t know where to start. There was one constant feature which I’ve alluded to before; the fact that Kirk had to resort to blinding the audience with condescending remarks and scientific lighting instead of pure, logical argument.

Kirk Durston (an apologist for genocide, remember) displays all the hallmarks of the pseudoscientist; he invoked technical terms when they were not needed and was in control of the whole talk. Questions had to be reserved until the end when naturally everyone was kicked out of the room so he does not have to be embarrassed in front of an audience. He doesn’t appear to have much faith in his work, y’see.

Alarm bells were ringing from the start; he discussed how Intelligent Design was not creationism (despite the presence of the Wedge Document, outlining how Intelligent Design is a tool to “defeat [scientific] materialism”, and that Durston is the director of the evangelical New Scholars Society) and how he had a method to ‘test’ for ID. He even had the gall to discuss how most secular scientists confuse science with philosophy in their work. I think his irony meter broke upon hearing that one. (Incidentally at this point I wrote in my notes “NURSE! FETCH THE LAUNDINUM!” as the pain was so great. He wasn’t even past the introduction yet.)

Most of the never-ending lecture was on how to test for ID, which runs roughly as follows; use some maths to show that the probability of something evolving is very low, and then assume that therefore the was ‘intelligence’ involved in its creation (but it’s not creationism theory, oh no). The equation he decides to use is a real-life one presented by Robert Hazel and his colleagues.

In the world of genuine science, Hazel’s paper presents a method in the fascinating area of artificial life, in determining how useful a generated genome is. Say one generated a genome that consists of a series of bases, a set of G, C, T and A which make up all life on Earth. Hazel’s formula asks, “How many rearrangements of this base, or variants of genomes of a certain size, can produce a specific protein?”

The example given in Hazel’s paper itself is the phrase “Fire on Main Street”, which can be represented in a series of short sentences of the same length. So the phrases “FIREONMAIN”, “DANGERFIRE” and “BURNINGNOW” all fit the bill. The theory applies to proteins, since groups of three bases on the genome code for a specific protein.

Phew. Got that? Good. How can this be used and abused by Kirk? By applying it to the wrong context, and deliberately choosing figures that makes something seem unlikely. This is a perfect example of ‘post-hoc’ reasoning; choosing one’s conclusion, then cooking data so as to support it. He even said, “Our aim is to test for intelligent design” with no apparent qualms whatsoever.

So how does Kirk Durston, the apologist for genocide, manage this? First, by not measuring any genomes, but by applying it to the number of unique species that have ever existed. Which is completely meaningless as many of those species contain the same properties and equalities in their genomes. Humans and chimpanzees have very similar genomes as one, er, evolved from the latter. Sorry guys.

Then there was the measure of how many of these can code for a specific, useful protein. Considering the sheer sizes of genomes, how does he calculate this? Erm, by setting it to one. As there’s only one protein he’s looking at, so that must be the well-designed one. Never mind the other possible proteins that could carry out a specific task. See what I mean about fixing the result?

In the end, he ends up with a false, very low probability so can magic up his conclusion. He doesn’t even manage this part very well; his threshold for a protein being ‘unlikely’ to evolve is “less then 50%”. Which means that if even in his phony calculation, if a protein has a near 50/50 chance of evolving ‘by chance’ then it must have been intelligently designed. By the same logic, the outcome of a coin toss is so unlikely that it must has been determined by some external intelligence.

As you can hopefully see, the argument is arsegravy of the highest order. As well as being mathematically inaccurate, it’s also biologically flawed as it ignores selection of fittest genomes, which forms a major part of all evolution processes. It is, in essence, a rehash of the argument that genomes cannot come together by chance, which is a major statistical fallacy.

There was more but I don’t think you need a further diatribe to be told that it was complete nonsense. He even wheeled out the recent New Scientist cover stating that “DARWIN WAS WRONG” as evidence! Oh, and did I mention that he’s an apologist for genocide?

Despite what some of its proponents may claim, intelligent design is no more then creationism given a wash and some new clothes. Its also boring, anti-intellectual, pseudoscientific rubbish whose sole raison d’etre is that it picks holes in existing evolution theory, and aims to fill those with mystical gibberish. Intelligent design? I think that one breaks the trade descriptions act.