{"id":16379,"date":"2023-03-28T17:34:28","date_gmt":"2023-03-28T17:34:28","guid":{"rendered":"https:\/\/drugprevent.org.uk\/ppp\/?p=16379"},"modified":"2023-06-06T19:07:59","modified_gmt":"2023-06-06T19:07:59","slug":"a-study-questioning-the-evidence-for-safe-injection-sites-has-been-retracted","status":"publish","type":"post","link":"https:\/\/drugprevent.org.uk\/ppp\/2023\/03\/a-study-questioning-the-evidence-for-safe-injection-sites-has-been-retracted\/","title":{"rendered":"A Study Questioning The Evidence For Safe Injection Sites Has Been Retracted"},"content":{"rendered":"

A meta-analysis reviewing the evidence on safe injection sites has been retracted due to \u201cmethodological weaknesses.\u201d<\/span><\/p>\n

Update (September 27, 2018):\u00a0The study, published by the\u00a0International Journal of Drug Policy<\/em>, has been\u00a0retracted by the journal<\/a>\u00a0due to \u201cmethodological weaknesses.\u201d As such, it should no longer be taken seriously. What follows is Vox\u2019s original piece on the study.<\/strong><\/span><\/p>\n

In response to the\u00a0opioid epidemic<\/a>, several cities, from New York City to Seattle, are considering a controversial policy: allowing spaces where people can, under supervision, inject heroin and use other drugs. The idea is that if people are going to use drugs anyway, there might as well be places where those using drugs can be supervised in case something goes wrong.<\/span><\/p>\n

\u201cAfter a rigorous review of similar efforts across the world, and after careful consideration of public health and safety expert views, we believe overdose prevention centers will save lives and get more New Yorkers into the treatment they need to beat this deadly addiction,\u201d New York City Mayor Bill de Blasio\u00a0said in a statement<\/a>\u00a0earlier this year.<\/span><\/p>\n

But a new study has found that these places, known as supervised drug consumption sites, safe injection sites, and\u00a0many other names<\/a>, may not be as effective at preventing overdose deaths and other drug-related problems as once thought. According to a\u00a0new review of the research<\/a>\u00a0published in the\u00a0International Journal of Drug Policy,<\/em>\u00a0safe consumption sites appear to have only a small favorable relation to drug-related crimes but no significant effect on several other outcomes, including overdose mortality and syringe sharing.<\/span><\/p>\n

\u201cThe contrast between the claims that are being made and what the evidence actually says\u201d stuck out to Keith Humphreys, a drug policy expert at Stanford University who was not involved in the review. The new research review\u2019s results, he said, \u201care fairly disappointing.\u201d<\/span><\/p>\n

In the past, experts, advocates, and journalists (including myself<\/a>) have said that supervised consumption sites have a lot of evidence supporting them \u2014 pointing to past reviews of the research that concluded the sites are effective in several areas. But this latest review of the research is more rigorous than those done before it, and it detected little to no effect from supervised consumption sites in the best studies the researchers could find.<\/span><\/p>\n

That is not to definitively say that supervised consumption sites don\u2019t work; it\u2019s more that we simply don\u2019t know yet. One of the problems the review found is that the research is seriously lacking in this area. Out of the dozens of studies on the topic they found, the researchers concluded that only eight were rigorous and transparent enough to include in the review. With such a small pool of studies included, it\u2019s possible \u2014 maybe even likely \u2014 that these few studies were in some ways biased, so future research could produce entirely different findings.<\/span><\/p>\n

As a result, several experts who support supervised consumption sites said that the new review of the research is fundamentally flawed. \u201cThey excluded, almost systematically, a lot of the studies that had demonstrated benefits on the metrics that they have selected,\u201d Leo Beletsky, a professor of law and health sciences at Northeastern University, told me.<\/span><\/p>\n

The review does not show that supervised consumption sites lead to, as detractors claim, more drug use and crime. In fact, the findings speak against that, if anything, as the sites appear to be linked to slightly lower drug-related crime.<\/span><\/p>\n

But the review indicates that the sites are not as evidence-based as supporters often claim, and more research is needed to reach hard conclusions about supervised consumption sites one way or the other.<\/span><\/p>\n

What the new review of the research found<\/span><\/h3>\n

The\u00a0new review of the research<\/a>, from Tom May, Trevor Bennett, and Katy Holloway at the University of South Wales in the UK, was a standard meta-analysis. The researchers first searched for previous studies on supervised drug consumption sites, pulling out 40, most of which looked at sites in Vancouver, Canada, and Sydney, Australia.<\/span><\/p>\n

They then tried to weed out the weaker studies \u2014 meaning, in scientific terms, those that didn\u2019t provide fully replicable data and those that didn\u2019t have a comparison group. That left them with eight studies total.<\/span><\/p>\n

The researchers then looked through the eight studies to measure the possible effects of the sites on several outcomes, including ambulance attendances relating to opioid-related events, overdose mortality, drug-related crime, borrowing or sharing syringes and injecting equipment, and problematic heroin use or injection.<\/span><\/p>\n

Ultimately, the researchers concluded that supervised consumption sites had no significant effect on most outcomes. The sites only had a small favorable relation with drug-related crimes, and a small unfavorable association to problematic heroin use or injection.<\/span><\/p>\n

The unfavorable result, however, does not necessarily mean that supervised consumption sites lead to more problematic heroin use or injection. By their very nature, these sites are built for people who are using heroin in a problematic way \u2014 that\u2019s why these people may need such an intervention and supervision in the first place. In other words, the finding may only speak to the existing population that supervised consumption sites attract.<\/span><\/p>\n

The researchers noted as much: Supervised consumption sites \u201chave been found to attract the most problematic heroin users.\u201d They went on: \u201cThis might influence outcomes as a result of comparing pre-existing risk behaviours and related health harms with less serious behaviours among the non-[site] group.\u201d<\/span><\/p>\n

Rebecca Goldin, director of\u00a0STATS.org<\/a>, said this reflects a common problem in this kind of research: \u201cNo meta-analysis can overcome systematic bias or problems that occur with the body of literature it incorporates. To mind, a risk in this particular literature is that a higher risk population is taking part in [supervised consumption sites], resulting in a diminished effect in the assessment.\u201d<\/span><\/p>\n

Still, as the first meta-analysis of supervised consumption sites to look at a more thorough list of outcomes, it presents disappointing findings \u2014 suggesting that these sites may have little to no impact overall.<\/span><\/p>\n

There are limitations in the review. The researchers might have missed some potentially strong studies, particularly those that weren\u2019t in English and didn\u2019t provide fully replicable data. The researchers also acknowledged that \u201cthere were relatively few studies suitable for meta-analysis,\u201d and once the body of research grows, it could lead to different conclusions.<\/span><\/p>\n

The review contradicts past research<\/span><\/h3>\n

The new review\u2019s conclusions also sharply contradict previous reviews of the research.<\/span><\/p>\n

For example:\u00a0Drawing on more than a decade of studies<\/a>, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) in 2018 concluded that safe injection sites led to \u201csafer use for clients\u201d and \u201cwider health and public order benefits.\u201d Among those benefits: reductions in risky behavior that can lead to HIV or hepatitis C transmission, drops in drug-related deaths and emergency service call-outs related to overdoses, and greater uptake in drug addiction treatment, including\u00a0highly effective medications for opioid addiction<\/a>.<\/span><\/p>\n

But EMCDDA\u2019s review wasn\u2019t a traditional meta-analysis, so it wasn\u2019t as rigorous or selective in what studies \u2014 and what\u00a0quality<\/em>\u00a0of studies \u2014 were included in the review. That allowed EMCDDA to include more studies, but many of those studies may have been of poor quality.<\/span><\/p>\n

For Humphreys, the new review is more reliable than EMCDDA\u2019s look at the research. As he put it, \u201cIf you impose even a modest methodological bar, and then those [studies\u2019] effects go away, to me that\u2019s worrisome.\u201d<\/span><\/p>\n

Beletsky pushed back \u2014 pointing out that the review of the evidence only looked at eight studies, none of which were randomized controlled trials. \u201cThat signals in and of itself that they\u2019re not literally looking at the full picture,\u201d Beletsky said. That\u2019s why he favors the systematic reviews that have been done in the past and included far more studies, such as EMCDDA\u2019s.<\/span><\/p>\n

The eight studies, though, were meant to be the best that the researchers could find. The studies that were excluded were those for which the researchers couldn\u2019t get full data sets and which didn\u2019t have comparison groups \u2014 fairly big methodological gaps.<\/span><\/p>\n

This is typical in meta-analyses: The ideal is randomized controlled trials. But if none exist, researchers start looking at other kinds of studies, while maintaining some level of rigor, to tease out the evidence that is available.<\/span><\/p>\n

David Wilson, a criminologist at George Mason University, said the review \u201cis a solid meta-analysis and adheres to basic practice standards for this type of study.\u201d But he took issue with one of the models the researchers used, and felt they could have paid more attention to publication bias.<\/span><\/p>\n

Others were more critical. Michael Lavine, a statistician at the University of Massachusetts Amherst, acknowledged that the methodology in the meta-analysis is \u201cvery common in medical and social science research.\u201d But he called that methodology \u201cbad,\u201d and warned that the outcome measure it uses \u2014 known as\u00a0\u201codds ratio\u201d<\/a>\u00a0\u2014 doesn\u2019t tell us how many people are helped by supervised consumption sites.<\/span><\/p>\n

Regina Nuzzo, a statistician at Gallaudet University, echoed Lavine\u2019s concerns. She also emphasized that not only did the review analyze just eight studies, but that those eight studies only looked at four supervised consumption sites total \u2014 which she said is \u201ca bit like double-dipping.\u201d<\/span><\/p>\n

Another way to look at this, though, is not that this review is flawed, but that the underlying research is flawed \u2014 if these truly are the eight most rigorous studies in the field \u2014 and, as a result, the research can\u2019t give us much information about the effectiveness of supervised consumption sites.<\/span><\/p>\n

\u201cIf you are an advocate, you could say correctly that if we assume these are effective, we do not have sufficient information to confidently overturn that presumption,\u201d Humphreys said. \u201cBut it\u2019s equally true if you took another view \u2014 just look at it as a cold, scientific question \u2014 you could say we also don\u2019t have the evidence to overturn the presumption that these don\u2019t make any difference.\u201d<\/span><\/p>\n

The potential problem: supervised consumption sites may not scale well<\/span><\/h3>\n

One thing supervised consumption sites do is reverse overdoses \u2014 thousands over the years, by some advocates\u2019 estimates. That\u2019s why the sites\u2019 staff have\u00a0naloxone<\/a>, the opioid overdose antidote, and oxygen tanks on-site. So how could it possibly be that the sites don\u2019t reduce overdose mortality, perhaps the most important metric in an\u00a0increasingly deadly opioid crisis<\/a>, when they\u2019re reversing all these overdoses?<\/span><\/p>\n

Part of it, Humphreys suggested, is most overdoses are not fatal. It\u2019s also possible that the sites may enable more drug use, leading to more overdose deaths even as others are stopped \u2014 although the there\u2019s no good evidence to support this possibility.<\/span><\/p>\n

The bigger problem, though, seems to be that supervised consumption sites may not have enough reach to have a significant impact.<\/span><\/p>\n

The review of the research speaks to this point, noting that \u201cfacilities are limited in the number of users they can accommodate.\u201d Consider that Vancouver, for example, was previously estimated to have about 5,000 people who inject drugs. A site that can hold at most a dozen or so people at a time and is closed for some parts of the day is simply not going to have much of a reach in such a large population \u2014 servicing, the review suggested, \u201ca small fraction of users each day.\u201d<\/span><\/p>\n

That\u2019s made worse by further restrictions on who supervised consumption sites will accept. They often won\u2019t, for example, allow people to share drugs or assist each other in injecting. So people who share drugs or need assistance from others will simply use elsewhere \u2014 in the streets, at home, in a motel, wherever. That further limits these sites\u2019 reach.<\/span><\/p>\n

Beletsky agreed: \u201cIt\u2019s not surprising to me that the population-level impact is limited because the capacity of these facilities is limited in terms of hours, throughput of people, and so forth.\u201d<\/span><\/p>\n

Humphreys guessed that the likely truth is supervised consumption sites work \u201creally little.\u201d It\u2019s not that they don\u2019t have any effect, but that the effect is likely so small that it\u2019s not going to be picked up at a population level by the research.<\/span><\/p>\n

To this end, some advocates are trying to expand the reach of supervised consumption sites. In Canada, for instance, activists\u00a0have deployed<\/a>\u00a0more mobile pop-up sites that can reach communities where a fully staffed building may not always be needed or available.<\/span><\/p>\n

Another point, made by Beletsky, is perhaps single supervised consumption sites\u00a0aren\u2019t<\/em>\u00a0supposed to have big effects on a population scale. Maybe it\u2019s fine if the sites just help a limited group of people who need them.<\/span><\/p>\n

But by scaling them up through other means \u2014 like pop-up sites \u2014 you may start seeing a broader community effect, Beletsky argued. \u201cThus far, these interventions have been limited,\u201d he said. \u201cThey\u2019ve been mired in legal and political battles. They\u2019ve been artificially suppressed. They could be doing a lot more.\u201d<\/span><\/p>\n

There are plenty of evidence-backed solutions to the opioid epidemic<\/span><\/p>\n

Despite the disappointing results for supervised consumption sites, Humphreys said that he\u2019s not discouraged about the country\u2019s ability to fight the opioid epidemic. \u201cWe have plenty of other things that we know, with much more confidence, that work,\u201d he explained.<\/span><\/p>\n

At the top of those other things is treatment \u2014 specifically,\u00a0medications like methadone and buprenorphine<\/a>. There is decades of evidence behind these medications, showing that they reduce the mortality rate among opioid addiction patients by\u00a0half<\/a>\u00a0or\u00a0more<\/a>\u00a0and keep people in treatment\u00a0better than other approaches<\/a>. When France relaxed restrictions on doctors prescribing buprenorphine in response to its own opioid crisis in 1995, the number of people in treatment rose and overdose deaths\u00a0fell by 79 percent<\/a>\u00a0over the following four years.<\/span><\/p>\n

But these medications, and addiction treatment in general, remain largely inaccessible in the US. A 2016 surgeon general report\u00a0concluded<\/a>\u00a0that only 10 percent of people with a substance use disorder get specialty treatment, in large part due to a lack of affordable and accessible treatment options. And even when treatment is available,\u00a0other federal data<\/a>\u00a0suggests that fewer than half of treatment facilities offer opioid addiction medications.<\/span><\/p>\n

Sticking exclusively to the realm of harm reduction, the US could do a lot more there too. Consider needle exchanges, where people can pick up sterile syringes and trade in used needles. The\u00a0decades<\/a>\u00a0of\u00a0research<\/a>\u00a0show needle exchanges\u00a0combat the spread of bloodborne diseases<\/a>\u00a0like hepatitis C and HIV,\u00a0cut<\/a>\u00a0down<\/a>\u00a0on the number of needles thrown out in public spaces, and\u00a0link more people<\/a>\u00a0to\u00a0treatment<\/a>\u00a0\u2014 all\u00a0without enabling more drug use<\/a>.<\/span><\/p>\n

Yet needle exchanges remain scarce in the US, as Josh Katz\u00a0reported for the New York Times<\/a>: \u201cAccording to the\u00a0North American Syringe Exchange Network<\/a>, 333 such programs operate across the country, up from 204 in 2013. In Australia, a country with less than a tenth as many people, there are more than 3,000.\u201d<\/span><\/p>\n

Even some more innovative, controversial solutions appear to have more evidence than supervised consumption facilities. Humphreys said that the evidence behind\u00a0prescription heroin sites<\/a>, as one example, is \u201cmuch stronger.\u201d<\/span><\/p>\n

The idea behind prescription heroin sites: Some people with opioid addiction are going to use heroin no matter what. For whatever reason, traditional therapies just aren\u2019t going to work for them \u2014 just like some treatments for, say, heart disease or cancer don\u2019t work for some patients. So if that happens, it\u2019s better to give them a safe source of the drug they\u2019re seeking and a safe place to inject it, rather than letting them pick it up on the street \u2014 laced with who knows what \u2014 and possibly overdose without medical supervision.<\/span><\/p>\n

Researchers\u00a0credit<\/a>\u00a0the European prescription heroin programs with better health outcomes, reductions in drug-related crimes, and improvements in social functioning, such as stabilized housing and employment.\u00a0Canadian<\/a>\u00a0studies<\/a>\u00a0also deemed prescription heroin effective for treating heavy heroin use. A\u00a0review of the research<\/a>\u00a0\u2014 which included randomized controlled trials from Switzerland, the Netherlands, Spain, Germany, Canada, and the UK \u2014 reached similar conclusions, noting sharp drops in street heroin use among people in the treatment.<\/span><\/p>\n

There is no prescription heroin program in the US.<\/span><\/p>\n

Humphreys argued that, in a world with limited financial resources and finite political and cultural capital, governments should try to first support the more evidence-based approaches than those with less.<\/span><\/p>\n

\u201cShould you have a culture war over something that barely engages the population and at most has a teeny effect when we still have people who can\u2019t get methadone and buprenorphine, which have a whopping effect and can engage a huge number of people?\u201d Humphreys said. \u201cFor me, that would be an obvious decision.\u201d<\/span><\/p>\n

Beletsky rejected the idea that we have to choose between different approaches, arguing that safe consumption sites can complement other interventions in the opioid epidemic.<\/span><\/p>\n

\u201cWe should be doing all those things that you mentioned,\u201d Beletsky said. \u201cBut there are challenges in reaching some of the most at-need populations who can benefit from those interventions. And I think that safe consumption facilities provide a platform for reaching those folks.\u201d He added, \u201cSafe consumption facilities really operate as a low-threshold doorway for people who typically will not seek care in other settings.\u201d<\/span><\/p>\n

For example, someone who uses heroin may have had bad experiences with the criminal justice system or health care system in the past. That may make him skeptical of going to these institutions \u2014 or any other official institutions \u2014 for help. A supervised consumption site, though, can be different, since it\u2019s an environment in which people are less judgmental about drug use. If the people running supervised consumption sites take advantage of this, they could use their better stature with people who use heroin to guide them to treatment and recovery.<\/span><\/p>\n

But there\u2019s no strong evidence to support the sites as an effective intervention for getting people into treatment and recovery \u2014 given that the new review of the research found no good studies that adequately evaluated for this.<\/span><\/p>\n

That goes back to the core problem: There is a lot out there about supervised consumption sites that certainly seems promising, even intuitive. But until the empirical research backs it up, pouring time and money into this kind of intervention may not be as evidence-based as people think.<\/span><\/p>\n

Source: A study questioning the evidence for safe injection sites has been retracted – Vox<\/a> August 2018<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

A meta-analysis reviewing the evidence on safe injection sites has been retracted due to \u201cmethodological weaknesses.\u201d Update (September 27, 2018):\u00a0The study, published by the\u00a0International Journal of Drug Policy, has been\u00a0retracted by the journal\u00a0due to \u201cmethodological weaknesses.\u201d As such, it should no longer be taken seriously. What follows is Vox\u2019s original piece on the study. In […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[67,14],"tags":[],"class_list":["post-16379","post","type-post","status-publish","format-standard","hentry","category-needle-exchange","category-social-affairs"],"_links":{"self":[{"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/posts\/16379","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/comments?post=16379"}],"version-history":[{"count":0,"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/posts\/16379\/revisions"}],"wp:attachment":[{"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/media?parent=16379"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/categories?post=16379"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/drugprevent.org.uk\/ppp\/wp-json\/wp\/v2\/tags?post=16379"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}