- FAMRI Center
Italian e-cig study does not support the conclusion that e-cigarettes stimulate smoking cessation
A paper entitled "EffiCiencyand Safety of an eLectronic cigAreTte (ECLAT) as Tobacco Cigarettes Substitute: A Prospective 12-Month Randomized Control Design Study" was just published in PLoS One that is being interpreted as indicating that e-cigarettes lead to cessation of conventional cigarettes among people who are not planning to quit smoking regular cigarettes.
For the reasons discussed below, this is not an appropriate interpretation of the results in this study.
This study involved randomizing people in Italy who said they were using conventional cigarettes and not interested in quitting to three groups: one receiving e-cigarettes with higher nicotine (Group A), one receiving e-cigarettes with a lower level of nicotine (Group B), and a third group receiving zero nicotine e-cigarettes (Group C, which the paper considered the “control” group). Based on comparing quit rates of conventional cigarettes one year later, the authors concluded that “in smokers not intending to quit, the use of e-cigarettes, with or without nicotine, decreased consumption and elicited enduring tobacco abstinence.” This conclusion is not supported by the data in the paper for two reasons.
First, and most important, despite the fact that the title describes the paper as a “randomized control design,” there is not a control group of people who were not using e-cigarettes that would allow assessment of spontaneous quit rates. By not having a true control group that would account for spontaneous quitting without using e-cigarettes one cannot say anything about whether e-cigarettes affected quitting.
This is a very important point because, as noted in my textbook Primer of Biostatistics (7ed, McGraw-Hill, 2012, p. 250), “To reach meaningful conclusions about the efficacy of some treatment, one must compare the results obtained in the individuals who receive the treatment with an appropriate control group that is identical to the treatment group in all respects except the treatment. Clinical studies often fail to include adequate controls. This omission generally biases the study in favor of the treatment.”
Second, there are issues with the statistical analysis which, when corrected, eliminate the reported statistically significant results.
The authors state that “At week 52 quitters were 22/200 (11.0%) in Groups A-B [the two groups of nicotine e-cigarette users combined] and 4/100 (4.0%) in Group C [the zero nicotine e-cigarette users] (p = 0.04), which is the basis for the “enduring abstinence” conclusion. The authors based this conclusion on the fact that a chi-square test of a 2 x 2 contingency table (smoking or not smoking vs nicotine or non-nicotine e-cigarettes) reached statistical significance (p = 0.04, which is less than 0.05, the cutoff for conventional statistical significance). The problem is that the authors failed to include the required Yates correction* in their calculation of the chi-square test statistic and associated p value. Recalculating the test properly yields p = 0.07, which is no longer statistically significant. Thus, the correct conclusion is that there is no statistically significant difference between the nicotine and non-nicotine e-cigarettes.
Probably a more appropriate comparison – which follows the experimental design – would be to treat the data as a 2 x 3 contingency table (smoking or not smoking vs the three different kinds of e-cigarettes). The chi-square analysis of the 3 x 2 contingency table yields p = 0.08, which is even further from statistical significance.
Thus, combining the fact that there is not a non-e-cigarette control group and correcting the statistics means that the appropriate conclusion to draw about quitting smoking based on these data is that the level of nicotine in the e-cigarette (including zero nicotine) has no detectable effect on quitting smoking conventional cigarettes.
These data cannot be used to support any statement about the efficacy of e-cigarettes for stimulating smoking cessation one way or the other.
*NOTE ON THE YATES CORRECTION (from Primer of Biostatistics, page 84): “… when analyzing 2 × 2 contingency tables, the value of chi-square computed [from the data] and the theoretical chi-square distribution leads to P values that are smaller than they ought to be. Thus, the results are biased toward concluding that the treatment had an effect when the evidence does not support such a conclusion. The mathematical reason for this problem has to do with the fact that the theoretical chi-square distribution is continuous whereas the set of all possible values that the chi-square test statistic can take on is not. To obtain values of the test statistic that are more compatible with the critical values computed from the theoretical chi-square distribution [for a 2 x 2 contingency table], apply the Yates correction (or continuity correction) to compute a corrected chi-square test statistic …. This correction slightly reduces the value of chi-square associated with the contingency table and compensates for the mathematical problem just described.”
I have posted essentially the same comment (without some of the introductory explanation) as a comment on the paper on the PLoS One website.