Space City Skeptics

The Official Blog of the Houston Skeptic Society

Posts Tagged ‘CAM

How to Design a Positive Study: Meditation for Childhood ADHD…..

with 63 comments

Several news sources and blogs have recently reported on a study looking into the benefit of transcendental meditation (TM) in children diagnosed with ADHD. A January 5th report from Reuters Health, a news service which claims to be “internationally recognized as unbiased, authoritative, timely and dependable, with the reputation for quality that one expects from a Reuters company“, actually serves as a perfect example of how not to cover science or health news. With a skeptical mindset, a few minutes of spare time and an internet connection, I was easily able to discover the dubious reality behind this “landmark” research.

The study, led by “cognitive learning specialist” Sarina J. Grosswald, involved the instruction in TM techniques of 10 students previously diagnosed with ADHD and enrolled in a private school for children with learning disabilities. These students were followed over three months, at the end of which they were evaluated for improvement in a number of areas.  According to the Reuters’ piece,

After three months, Grosswald and her colleagues found, the students reported lower stress and anxiety levels, while their ADHD symptoms also improved, based on questionnaires given to teachers and parents.”

Impressive. For those who are confused, TM is a form of meditation developed by Maharishi Mahesh Yogi in the 1950’s which involves the repetition of a meaningless sound, or mantra, while sitting quietly with eyes shut. This allows the practitioner to quiet the mind and discover the “source of thought”. The mind and body are then able to achieve a state of calm and, according to proponents, reap myriad medical benefits far beyond that of simply increased peace of mind. Believers in TM have also been known to claim that an extension of the technique can lead to fighting crime with their minds and flying. I’m not kidding.

This is a pilot study published in a online education journal edited by graduate students, something which does no inspire confidence in me. I do not think that a reputable peer-reviewed medical journal would have accpeted it. At least the authors do admit in the discussion section that it would be inappropriate to make claims regarding a cause and effect relationship between TM and any improvement in ADHD symptoms based on these results. That doesn’t stop them from making bold statements regarding the benefit of TM, however, as I will soon get to. 

The flaws in this study are numerous. The number of subjects is too small, there is no control group and it isn’t blinded. The study reveals that some of the children are on medication but it does not take into account the possibility of recent changes in medical therapy, or improved compliance while on the study. It is based purely on self-report and subjective questionairres and there is very high liklihood that a placebo effect could have been the sole responsible factor in the subjects’ apparent improvements. The authors then call for larger and better designed studies, something which I don’t think is justified for these reasons, but my problem with this study, and concerns regarding the credulous take by the media, go much deeper than what I’ve already explained.

What led me to dig deeper after reading the Reuters’ report was the following quote:

The effect was much greater than we expected,” lead researcher Sarina J. Grosswald, a cognitive learning specialist in Arlington, Virginia, said in a written statement.” 

I wondered why the researcher had expected an effect and hypothesized that there may be a connection between the researchers and TM more significant than academic curiosity. I was quickly able to discover that Grosswald is a hardcore believer in TM. Just read this quote by Grosswald from a website called Ask The Doctors, which provides a forum for specialists to answer questions related to TM and health:

The TM technique is the exact opposite of harmful. It reduces your risk of getting serious chronic health problems like hypertension, diabetes, and heart disease. In fact, research on the Transcendental Meditation program shows that people who practice it go to the doctor about 50% less than the general population. And if they are in the hospital for some reason, their hospital stay is 50% shorter, on average. For some conditions, the need for medical care is as much as 87% less for TM meditators. Practicing the Transcendental Meditation technique is one of the best things you can do for your health.”

She clearly does not come across as an unbiased investigator. In reading her other responses on the “Ask The Doctors” website, and especially after listening to a 16-minute talk she gave in 2005, when this research actually took place, which is posted on a Maharishi Vedic Education Development Corporation sponsored website that focuses on “ADHD, the Mind and the Transcendental Meditation technique” called Insights in Health, it is obvious that she is a true believer. That doesn’t prove the research is bogus by itself, but it is a red flag.

A more concerning red flag, and one which was also discovered in the talk given by Grosswald, is the fact that the 10 children involved in the study may have been coached. In the last few minutes of the presentation, Grosswald presents clips of the children meeting with the TM proponents prior to the initiation of the study, where it appears that they are told what the expected outcome of the trial is, that their symptoms will improve with TM. 

Not only are these kids aged 11 to 14 being told what the expected outcome of the study is by study investigators, the headmaster of the school, Linda Handy, can be seen at the very end of the video discussing how amazing the technique is and how it will change the students’ futures for the better. I am forced to question whether the teachers, whose evaluations of the study subjects’ behavior and performance are an integral component of the study conclusions of positive effect, might have been hesitant to give a negative evaluation when their boss is clearly also a true believer.

In the acknowledgments section, The authors thank the Abramson Family Foundation for funding and the Institute for Community Enrichment for support. The Abramson Family Foundation is an organization which believes that TM can help students achieve the full potential of their brain.

The Abramson Family Foundation has been funding research on Transcendental Meditation and providing scholarships for students to learn the technique for the past 20 years. This has been a rewarding investment in the youth of our nation. Here is a common sense approach—a sound and scientific way—of fulfilling the purpose of education, which is to create intelligent, dynamic, happy, healthy and successful human beings.”

Of note, Grosswald sits on the Board of Advisors for the foundation. Joining her on the Board is none other than the  school headmaster Linda Handy. Whatever doubt I had in my lack of enthusiasm for this study fell by the wayside upon that discovery. While it may not have been intentional on the part of the study authors, the school headmaster, or the pro-TM funding organizations, this study was designed in a way that coulnd’t possibly yield anything other than a positive result. And calls for further study, especially with public funds in addition to the over twenty million already spent by the NIH on TM, are unwarranted.

It wasn’t difficult to look at this study and see that the claims being made by TM supporters aren’t valid. It wasn’t even that hard to uncover the connections between the investigators, the school where the study was conducted and pro-TM organizations. Yet I was unable to find one news report that displayed even the slightest amount of critical thinking, instead reading like press releases from TM believers. The current state of science and health reporting is rather depressing, and I don’t see things improving any time soon as more and more dedicated science writers are falling prey to the poor economy.

Written by skepticpedi

January 7, 2009 at 1:33 pm

Update on Parade Magazine Article Criticism: Dr. Liponis Responds…..

with 10 comments

In my last post, I harshly critiqued the recent Parade Magazine article on alternative medicine by Dr. Mark Liponis. I was delighted to find that I was not the only skeptic to tackle the poorly researched claim by Dr. Liponis that acupuncture, biofeedback and meditation “have scientific backing and have passed the litmus test of rigorous medical inquiry.” I enjoyed reading Dr. Steven Salzberg’s post on the subject, which I’ve linked to above.

Both Dr. Salzberg and I received nearly identical responses from Dr. Liponis attempting to rebut the concerns raised in our respective posts. Dr. Salzberg did an excellent job responding, taking each of Dr. Liponis’ points to task seperately. I recommend you read his response as he goes into more detail regarding the specific studies. I want to focus a little more on the errors in logic committed. Here is the comment on my post by Dr. Liponis, with my comments interspersed:

“I suppose everyone is entitled to their opinions…
But just because we don’t understand something doesn’t mean it’s not true.

Dr. Liponis claimed that three alternative therapies worked as if it were fact. In no way did he come off as if merely stating an opinion. My statements is based on the medical literature, an understanding of which would not lead any reasonable person to believe that acupuncture has, to beat a dead horse, “passed the litmus test of rigorous medical inquiry.”

There is little doubt that a lack of full understanding of a phenomenon does not equate to that something not being true. It would be equally valid for me to state, however, that just because we don’t understand something doesn’t mean that it is true. These are true statements but in the context of attempting to support a claim are nothing but arguments from ignorance. 

Here’s some more research to support the “patently ridiculous” concepts:

In the 2005 study mentioned by you (Cardini et al., BJOG 112(6): 743-747) the authors state: “The study was interrupted when 123 participants had been recruited (46% of the planned sample). Intermediate data monitoring revealed a high number of treatment interruptions. “… “They do not support either the effectiveness or the ineffectiveness of moxibustion in correcting fetal breech presentation.” http://tinyurl.com/6hcdmp

I’m not clear what Dr. Liponis’ point is exactly. This reaks of a biased author attempting to save face but perhaps it is merely a researcher stating the obvious. No negative study can completely prove that a treatment does not work in all circumstances. But taking into account the extreme implausibility of using toe acupuncture to reposition a malpositioned fetus, a negative study that is better designed and controlled comes pretty close in my opinion. And it raises the question of whether or not precious research dollars should go towards further studies looking into this.

In a systematic meta-review of 65 citations including six RCTs (randomized controlled trials) authors of this April 2008 review conclude: Our results suggest that acupuncture-type interventions on BL 67 are effective in correcting breech presentation compared to expectant management. Here’s the source from pubmed: http://tinyurl.com/6pvhks

Meta-analyses are notorious for their ability to thoroughly misrepresent reality. There is an old saying regarding these unwieldy monstrosities of “Garbage In, Garbage Out”. Taking a large number of small, poorly designed studies and using statistical analysis to attempt to amplify them results in what amounts to one large, poorly designed study. In this case, you have a biased journal of poor quality and a group of studies with bad methodology. But no meta-analysis is more meaningful than one large and well designed RCT. And a comprehensive review of the literature, as are reported in the Cochrane database, tend to be much more accurate as well.

A review by Cochrane database in April 2005 (3 RCTs involving 597 women) concluded: “Moxibustion may be beneficial in reducing the need for ECV (external cephalic version), and decreasing the use of ocytocin” although numbers of participants precluded statistical analysis. http://tinyurl.com/56kbky

And speaking of Cochrane reviews, when I read the conclusion of this one I once again began to doubt whether or not Dr. Liponis actually did any research into these topics that wasn’t effected by confirmation bias. If I weren’t such an damn optimist, I’d swear he was being purposefully deceitful. The conclusion of the review comes across as decidely more reasonable when read as a whole rather than carved up: “There is insufficient evidence to support the use of moxibustion to correct a breech presentation. Moxibustion may be beneficial in reducing the need for ECV, and decreasing the use of ocytocin, however there is a need for well-designed randomised controlled trials to evaluate moxibustion for breech presentation which report on clinically relevant outcomes as well as the safety of the intervention.”

An April 2004 study of 240 women published in the Journal of Maternal Fetal and Neonatal Medicine concluded; “Acupuncture plus moxibustion is more effective than observation in revolving fetuses in breech presentation. Such a method appears to be a valid option for women willing to experience a natural birth.” Here’s the pubmed reference: http://tinyurl.com/6fq7w9

Was it published on April 1st? This is yet another study done with poor methodology, no placebo and questionable blinding.

I have no idea how it really works, it just seems to work, somehow. Also, as a physician (MD) who sees patients and works with acupuncturists I have also seen many patients helped by acupuncture. I don’t presume to understand how it works.

These statements make it abundantly clear where Dr. Liponis is coming from. He is a believer in acupuncture and doesn’t care about what the science shows. When all is said and done,  science is irrelevant unless it supports, or at least appears to as in this case, his stance. I wonder, would Dr. Liponis stop recommending acupuncture at any point based on scientific evidence that it was ineffective. I doubt it.  It is extremely easy for even the brightest of people to be fooled by our inate weakness in interpreting the world around us. Anecdotal evidence is the weakest form of evidence, and notice I didn’t say it was devoid of all value, simply because our brains are so prone to bias. We are at the mercy of entities such as the placebo effect, the desire to please, the thrill of a novel therapy, regression to the mean, errors in logic and sometimes outright fraud. Taking these concerns, and countless others, into account is why scientific medicine has been so successful.

More:
http://www.usatoday.com/news/health/2008-02-07-acupuncture-fertility_N.htm

Do you believe it’s unreasonable for a woman to try acupuncture/moxibustion when faced with the alternative of a C-section or manual manipulation of the baby at the time of delivery?”

I do not think it unreasonable for an adult to try acupuncture/moxibustion if they are doing so based on what they perceive to be accurate information. Unfortunately, the public is generally woefully naive when it comes to science and medicine and often depends on sources of information such as the article written by Dr. Liponis. I would think it unreasonable were a patient to seek out any therapy knowing that it has zero prior-plausibility and is not supported by the medical literature.

Written by skepticpedi

December 16, 2008 at 1:30 pm

More Alternative Medicine in the News…..

with 10 comments

December has seen a big spike in reports on alternative medicine, in particular the practice of acupuncture. Last week, a number of outlets ineptly covered the release of the most current numbers on CAM usage, choosing to simply regurgitate press releases from the National Center for Complementary and Alternative Medicine rather than practice actual journalism. Prior to that, the official journal of the American Academy of Pediatrics had published a blatantly pro-CAM article on pediatric use of alternative medicine in which the authors boldly provided the following gem:

“Formal evaluation has suggested that the quality of RCTs of CAM is as good as that of RCTs of conventional medicine, and the quality of systematic reviews of CAM exceeds that of systematic reviews of conventional medicine.”

This notable diversion from reality was followed up by a healthy, and not at all unexpected, dose of praising anecdotal n-of-1 studies and then blaming difficulty obtaining IRB approval of CAM studies on ignorance of just how wonderful the stuff is.  On Saturday, I was made aware of the habitually woo-friendly U.S. military’s foray into “battlefield acupuncture“. And yesterday, as I sat down to peruse the Sunday paper I happened upon the following article on “Alternative Therapies That Really Work” in Parade Magazine.

Now I am painfully aware of the reality that Parade Magazine is not a medical journal. It isn’t even a popular science magazine like Discover or Scientific American. Its articles barely even qualify as reporting at all. Occassionally entertaining fluff, from cover to cover, just about sums it up.  But it is a source of information that millions of people read each and every week, many of whom very likely consider what they read within to be reliable. And it may be reliable when it comes to the latest news on which actresses still wear fur or who readers think is the hottest celebrity in 2008 (still Jennifer Connelly, always Jennifer Connelly), but it is horrendous when it comes to coverage of medical topics.

It was 2.5 years ago, in June of 2006, when Dr. Isadore Rosenfeld wrote a typical article on acupuncture for Parade. In it, after cherry picking a few positive and negative studies to mention, he cautioned readers that more studies are needed before a conclusion on efficacy could be made. Sound reasonable right? He then included an absurd anecdote, one which he has used before in prior articles and one which was effectively handled by Gary Posner and Wally Sampson 10 years ago:

 “My personal experience with acupuncture helps me keep an open mind. In 1978, I was invited to China to witness an open-heart procedure on a young woman. She remained wide awake and smiling throughout the operation even though the only anesthesia administered was an acupuncture needle placed in her ear.”

Naturally this significantly hampered any chance of most readers taking home the only reasonable conclusion which could be made from the article, that acupuncture had not been proven by appropriate investigation to be effective. It still hasn’t. In fact, the position that acupuncture has no real effect, and not that it hasn’t been proven either way, is well supported by the trend in the literature of increasingly negative results in studies of  increasingly improved design and control.

With the help of a number of “experts”, like the aforementioned Dr. Isadore Rosenfeld and Dr. Mark Liponis, Parade has mishandled the subject of alternative medicine for years. The current example is yesterday’s one page article by Dr. Liponis, who according to his profile,

“…..has always had an interest in holistic health and wellness, and continues to expand his expertise in integrative medicine as Corporate Medical Director of Canyon Ranch.”

I suspect that he is more interested in expanding his bank account by taking advantage of the public credulity more so than his expertise. There certainly is not a hint of any any regarding acupuncture, one of the three alternative modalities claimed to work in the article. Here is what Liponis has to say about it:

What it is: Acupuncture is a traditional Chinese practice involving the placement of very skinny, sterile needles into the skin at specific points located along “energy meridians.”
How it works: Eastern philosophy says that acupuncture affects the flow of qi (pronounced “chee”), or energy, through the energy meridians. Western science reasons that the needles interact with our nervous system, triggering the release of hormonelike chemicals that affect our mood, perception of pain, and immune response. 
What it’s good for: In a 2004 study, acupuncture was shown to be helpful in reducing pain due to knee arthritis. It also could be beneficial for sufferers of post-traumatic stress disorder. And when used along with in vitro fertilization, it may be effective in increasing the odds of success in female conception. Stimulating an acupuncture point in the toe even may help correct the breech position of babies in the last trimester and allow more women to avoid C-sections, according to a study in the Journal of the American Medical Association.

First off, invoking the terminology “western science” is a huge red flag that you will soon be venturing off the well-trodden path of critical thinking on a subject. The scientific method is the same regardless of where you live and this kind of terminology serves the underhanded purpose of attempting to establish a double-standard when it comes to evaluating therapies. Usually when it is mentioned it isn’t long before someone is claiming that it isn’t the right way to study [insert unproven therapy]. Furthermore, some scientists may express the opinion that apparant effects of acupuncture are due to the release of a chemicals within the body, but many would argue that this is putting the cart before the horse. If there is no proven effect, what is the point of attempting to establish a mechanism.

As far as the cherry picked studies chosen by Liponis go, well I think calling them cherry picked pretty much makes my point. Looking at three studies, at least two of which are of very poor quality, is insufficient to make a claim that acupuncture works. Enjoy Steven Novella’s dismantling of the claims of acupuncture’s effectiveness in improving IVF success rates back in November. The study on acupuncture and the repositioning of breech fetuses is equally dubious. It is an unblinded and uncontrolled  study. All three of those aspects make studies on acupuncture very suspect. Virtually all studies on acupuncture coming out of China are positive and should be approached with appropriate skepticism. When replication in a non-Chinese population was attempted by the same study author in 2005, no effect was found. This isn’t suprising as it is a patently ridiculous concept with zero prior-plausibility. Dr. Liponis very likely did not do any research when preparing his article as this information was extremely easy to find.

Written by skepticpedi

December 15, 2008 at 12:16 pm

Another Side to Latest Survey on CAM Use…..

with 6 comments

Geek Goddess beat me to posting about the Washington Post article on the widespread use of so-called alternative medicine published today along with versions by several other news sources, and while I agree that the reporter handled the subject better than in the average fluff piece, it was by no means appropriate coverage.  The numbers, which have actually been out for over two months, actually paint a picture very different than that of the quoted CAM proponents, in particular Richard L. Nahin of the National Institutes of Health’s National Center for Complementary and Alternative Medicine.  As broken down in typical expert fashion by Steve Novella at Neurologica, the numbers reflect not an increase in usage of CAM, but no real change in the perecentage of Americans seeking out alternative medicine over the past twenty years. 

The most glaring lack of journalistic effort comes in the form of the apparent acceptance of a complete misrepresentation of what alternative medicine actually is. Novella nails it as usual in his post, revealing that,

“Also, the numbers reveal the hollowness of the CAM label – what does CAM really mean? The numbers are inflated by including items that are not necessarily out of the mainstream.”

Many of the modalities included in the survey do not necessarily deserve the title of alternative medicine. Entities like yoga, massage, and other techniques geared towards relaxation or treatment of some musculoskeletal complaints do not alternative medicine make. Potentially a very large percentage of the survey is made up of people using legitimate therapies. Novella reveals,

“If you factor out these modalities (and also legitimate use of nutrition), the numbers remaining are all quite low – in the single digits, and not significantly increasing. Only 1.8% of the population used homeopathy, 1.4 acupuncture, 0.3 naturopathy, and 0.5 energy healing. For these hard-core CAM modalities usage is still marginal and not really changing.”

One aspect of this issue that Dr. Novella did not address in his post, and not because he isn’t aware of it, is that of herbal therapies being included in the blanket category of alternative medicine, although not isolated specifically in this survey (ayurvedic and naturopathic medicine make generous use of herbs and supplements). Herbs and supplements are not alternative medicine, they are drugs. They are drugs which have either not been studied for safety and efficacy or have been studied and did not stand up to the scrutiny of proper scientific investigation. Many medications with true efficacy have been derived from plants but this is not evidence that alternative medicine in general is effective, despite what proponents may claim. The fact that digitalis came from foxglove in no way supports homeopathy, acupuncture, healing touch, or any other bogus therapy. Unfortunately, the current state of affairs is such that even the hallowed halls of many major academic medical centers are being infiltrated by such quackery in good part because of just that kind of flawed logic. The media interpretation of this survey, largely just regurgitation of NCCAM press releases, is another example of propaganda that will serve to elevate CAM to an undeserved legitimacy in the public eye.

Written by skepticpedi

December 11, 2008 at 10:13 pm