Stimulated by Lee et al 2023.[1]
APs – acupuncture points
KIOM – Korean Institute of Oriental Medicine
NMA – network meta-analysis
IF – impact factor
EBM – evidence based medicine
SATV – sham acupuncture therapy verum point
AT – acupuncture therapy
SATS – sham acupuncture therapy sham point
VAS – visual analogue scale
RMDQ – Roland-Morris Disability Questionnaire
NRS – numerical rating scalekey to acronyms
This week I had no choice! No sooner had this paper come out, than a young professorial colleague (Ari) sent me a sample blog post on this paper written by ChatGPT in the style of the BMAS Blog. I wasn’t going to highlight the paper because I had inherent physiological concerns about the implication of the conclusion.
ChatGPT was very positive about the paper and certainly picked up my rather casual style, but clearly didn’t have the same physiological concerns as the real me.
This paper comes from the team at KIOM led by Myeong Soo Lee. They have already published a couple of NMAs on a similar theme that I highlighted previously: [2,3]
This one is a bit different…
The conclusion in the journal JAMA Network Open (IF 13.8) is as follows:
In this NMA, sham acupuncture needling at the same points as those in acupuncture was not a true placebo control for assessing the efficacy of acupuncture for cLBP and might underestimate the outcome of acupuncture in clinical settings.
“… is not a true placebo control…” must be music to the ears of all of us who have struggled with the prevalent EBM interpretations that acupuncture has minimal effects over placebo, and anything I say below is not intended to dull that music.
When I read the results of a new piece of research, I always think of the physiological backdrop and whether or not the conclusion makes sense to me in this scene. In this case, the result made me raise a questioning eyebrow and mutter a quiet but elongated ‘really’.
What is my problem then? Well, even us so-called experts are not much good at finding APs.[4] The size of the area targeted often exceeds that of a single receptive field in skin (50 to 100mm). This is often much farther than the location of sham APs from the theoretical real equivalent. Hence, in terms of point location, there is unlikely to be any consistent physiological difference between verum points and nearby sham points, apart from the way in which they are stimulated.
We are more familiar with the comparison of penetrating versus non-penetrating shams. This clearly makes physiological sense, although we run into the problem of reducing the size of the acupuncture effect from baseline (due to the use of sham devices in the real acupuncture groups, see the blog links above) whilst increasing the difference when measured against sham (the only outcome that rigid EBM advocates are interested in). The biggest data set on this comparison comes from the Vickers IPDM update: . In case there is any confusion with the reference, the paper was published online in 2017 and then made it into an issue in 2018.[5]
So, I took a look at the data, and immediately my eye was drawn to the slim line connecting SATV with AT in the network diagrams. There was only one trial making up this part of the network – Cherkin et al’s “Toothpick acupuncture better than drugs” trial.[6,7] By comparison the line joining SATS to AT for the pain outcome includes 7 trials.
Next, I had a look at the results in the two big trials that contributed to the indirect comparison of SATV with SATS – Haake et al and Cherkin et al. Unfortunately, they use entirely different outcomes. Indeed, there was no VAS pain in either – the primary pooled outcome used in the NMA. The German paper used the Von Korff Chronic Pain Grade Scale – Von Korff for short. This includes has a pain outcome that is reported as a 0 to 100 score, but it is derived from three 0 to 10 NRS. The US paper used the RMDQ and a 0 to 10 NRS of bothersomeness.
Now, in meta-analyses a variety of different outcomes can be combined provided that they roughly measure the same thing. In this case, I suspected that the NRS of bothersomeness was included within the VAS pain in the NMA. If in general patients rate bigger changes in bothersomeness than they do in pain intensity following a course of acupuncture, this would not be a problem in a standard two-way meta-analysis, since both sides get a slightly inflated effect. However, in a network with a skinny arm, as we have here, the pooled effect for AT is likely to drag down the AT result in the theoretically inflated skinny arm (Cherkin et al). This leaves the only estimate determining the SATV node sitting relatively higher than everything else, since pooled estimates usually put AT above SATS and both above waiting list.
The other way to counter the perils of skinny arms, is to include as much data as possible for balance. In this case, the two big sham controlled trials also had usual care controls, but this data was not included. I don’t blame the authors for this, because usual care can vary from quite intensive guideline-based conventional care (as in Haake et al) to benign neglect (as in Cherkin et al).
So, to sum up, I worry that the difference between SATV and SATS is inflated by the inclusion of a subtly different outcome on a flimsy stalk of the network, and in reality, there is unlikely to be much difference between them. But drawing attention to the fact that any form of sham acupuncture is not a true placebo is a worthy result, and I just hope that none of the anti-acupuncture EBM zealots read as far as this.
References
1 Lee B, Kwon C-Y, Lee HW, et al. Needling Point Location Used in Sham Acupuncture for Chronic Nonspecific Low Back Pain: A Systematic Review and Network Meta-Analysis. JAMA Netw Open 2023;6:e2332452. doi:10.1001/jamanetworkopen.2023.32452
2 Lee B, Kim T-H, Birch S, et al. Comparative effectiveness of acupuncture in sham-controlled trials for knee osteoarthritis: A systematic review and network meta-analysis. Front Med 2022;9:1061878. doi:10.3389/fmed.2022.1061878
3 Kim T-H, Lee MS, Alraek T, et al. Acupuncture in sham device controlled trials may not be as effective as acupuncture in the real world: a preliminary network meta-analysis of studies of acupuncture for hot flashes in menopausal women. Acupunct Med 2020;38:37–44. doi:10.1136/acupmed-2018-011671
4 Bäumler PI, Simang M, Kramer S, et al. Acupuncture point localization varies among acupuncturists. Forsch Komplementarmed 2012;19:31–7. doi:10.1159/000335825
5 Vickers AJ, Vertosick EA, Lewith G, et al. Acupuncture for Chronic Pain: Update of an Individual Patient Data Meta-Analysis. J Pain 2018;19:455–74. doi:10.1016/j.jpain.2017.11.005
6 Cherkin DC, Sherman KJ, Avins AL, et al. A Randomized Trial Comparing Acupuncture, Simulated Acupuncture, and Usual Care for Chronic Low Back Pain. Arch Intern Med 2009;169:858. doi:10.1001/archinternmed.2009.65
7 Toothpick acupuncture better than drugs. Mirror.co.uk. 2009. (accessed 11 Sep 2023).
