Development Safety Update Reports (DSURs): simplifying periodic safety reporting

The US Food and Drug Administration (FDA) in August proposed a new guideline asking the sponsors of clinical trials to submit annual development safety update reports (DSURs). The guideline, titled “E2F Development Safety Update Report,” is in draft form and describes the format, content, and timing of a DSUR. It specifies that a DSUR update the status of the clinical trial, summarise the sponsor’s understanding and management of identified and potential risks, describe new safety concerns that could affect the protection of trial subjects, and examine whether the information collected in the previous year accords with current knowledge of the product’s safety.

DSURs would be required for investigational drugs, including biologicals, with or without marketing approval, and whether or not the clinical trials are being conducted by commercial or non-commercial sponsors. A DSUR would have to be submitted within 60 days of the DSUR data lock-point, determined by the date of the sponsor’s first authorisation to conduct a clinical trial in any country (the “Development International Birth Date”).

The guideline instructs sponsors to focus on data from interventional trials. However, it also advises the inclusion of other findings that may have a bearing on the safety of trial subjects. Such information could include findings of non-clinical trials, as well as clinical trials conducted by the sponsor’s development partners and non-interventional or compassionate-use studies.

The FDA draft guideline matches one developed by the International Conference on Harmonisation (ICH).

Will DSURs create more work for the sponsors of clinical trials? Apparently not, at least in the long run. First, the guideline follows a standard format that has been developed for submission in the three ICH regulatory regions (the United States , the European Union, and Japan ). A report produced in accordance with the guideline could be submitted simultaneously in all three regions. Second, the DSUR would replace some reports that are currently required, such as the IND Annual Report. For already marketed products, some of the information required for the DSUR may be provided in the periodic safety update report (PSUR), on which the DSUR is patterned.

The draft guideline may be obtained online at www.regulations.gov or www.fda.gov/cber/guidelines.htm. The FDA has solicited comment on the draft. To be useful, comments should be submitted by 3 November 2008.

Increased calls for publishing negative clinical trial data

A negative result is defined as one that shows evidence of absent (or no) effect of an intervention; it is not due to absence of evidence, i.e, inconclusive results due to an underpowered study.

Traditionally, negative clinical results were considered ‘uninteresting’ and so were not published, or were not submitted for publication. In experience as a biochemist, a result was not ‘good’ or ‘bad’, ‘positive’ or ‘negative’; it simply was. But a doctoral student could not obtain a PhD with only negative results, published or not. So, if we only strive for positive results, is this really rigorous science?

Dr Trisha Groves, Deputy Editor of the British Medical Journal , says that ‘we’re keen to publish negative studies when they illustrate important points. Specifically, our policy is that if a research question is important, original, and relevant to the decision making of our general medical readership, and if that question is answered with the right study design and sufficient power, the answer should be published, whether it’s positive {or} negative. We often publish negative results’. In fact, two of the BMJ’s top 10 research papers published in 2006–2007 were negative studies (‘top’ being defined by a combination of cites, hits, downloads, letters, pick-ups in evidence-based medicine journals, email alerting services, and media coverage).

How do negative clinical trial results affect economic analyses? Are they even used? One assumes that negative clinical trial data affect HE outcomes (e.g, assumptions made about the study population or efficacy of study drug, utilization rates, rates of adverse effects).

Christopher Carswell MSc, MRPharmS, Editor of PharmacoEconomics , feels that the effect of limited published clinical trials with negative results on HE research ‘is a very interesting research question, which to my knowledge has not been investigated and would probably be context specific. It undoubtedly leads to biased estimates of cost effectiveness but by how much and whether it is enough to affect decision making is an interesting question’.

When asked whether health economists actively look for negative clinical trial results in their economic analyses, Carswell says, ‘If so, I have rarely seen authors make an effort to search for anything other than major published clinical trials, which I find very disappointing. A related point – rarely do authors employ formal methods to combine evidence from disparate sources, e.g, meta-analyses, mixed treatment comparisons, meta-regression – which is also disappointing’.

Both editors agree that negative clinical trial results are an important part of decision-making – for both efficacy and cost-effectiveness. As Christopher Carswell notes, ‘I don’t like the term “negative [result]” as I think such studies should still be viewed in a positive light – they have shown a technology is not cost effective which helps with future decision making. In other words, we have added to the evidence base, which is not a negative thing’.

Outsourcing clinical trials

Writing in Pharmatech a couple of years ago, Dr Faiz Kermani observed that “many clinical trials are plagued by delays and setbacks that can cost pharmaceutical companies millions of dollars in missed sales, making the maximisation of R & D efforts a top priority.”

It is not surprising, said Dr. Kermani, who at that time was a marketing executive at Chiltern International in California, that outsourcing clinical trials has become a popular option for cutting costs and minimising time. Contract research organisations (CROs) account for about 20% of the pharmaceutical and biotech R&D budget, he wrote.

Today, Kermani works for Health Interactions in London as an account manager. He believes that if anything the trend towards outsourcing has become even more pronounced. He told HOC that in 2004, the latest year for which figures are available, leading CROs managed nearly 23,000 I–IV studies at 152,000 sites worldwide. Moreover, he says, clinical development projects in which CROs were greatly involved were submitted to the FDA more than 30 days closer to the projected submission date than were projects with less CRO involvement.

Kermani notes, too, that the international picture is changing… with much greater use of emerging market countries in Eastern Europe, Latin America, India and China. Citing a CMR International study in the UK, he says that Ukraine has had a 700% surge in its usage for patient recruitment between 2002 and 2005, while he says that the UK Trade and Investment Biotech Scoping Mission report of 2007 found that clinical trials can be conducted in China for about 10% of costs in Western countries. Finally, McKinsey has estimated that by 2010 the pharma industry will spend some $1 to $1.5 million on clinical trials in India alone.

In fact, countries now compete vigorously for conducting clinical trial application reviews. Canada is one country with a low time of approximately one month (by contrast, China takes eight months and India three and a half) and promotes its advantages in a glossy brochure. The Canadian cost advantage, it states, is 22.4% over costs in the US. And in a breakdown of annual costs of clinical trials management by country, Canada showed a profit before tax of $924,000, compared with the US at $157,000. Among the CROs doing business in Canada are Covance, Parexel, and Quintiles.

The Canadian Consul and Senior Trade Commissioner in Philadelphia, David B. Weiner, notes that Canada is competing for clinical trial business not only in the United States, but also globally. Says he: “Canada offers a number of advantages for clinical trial research. Canada is highly competitive with the US in the areas of clinical trial quality, efficiency, and costs. Some of our cost advantages include lower clinical trial recruitment costs and lower per-nurse labour costs. Most routine diagnostic procedure costs for clinical trial patients are covered through provincial health programmes. In addition, Canada’s generous R&D tax incentive programme can drastically reduce the overall costs of clinical research conducted in the country.”

Understanding study design in comparing drug studies

For a variety of reasons, pharmaceutical manufacturers often do not conduct head-to-head studies of their drugs versus the competition. On the other hand, head-to-head studies are the ultimate choice for people who have to make comparative assessments of drugs.

In the absence of head-to-head studies, decision makers generally have to make do by trying to compare competing therapies by looking at studies that have the most similarities (patient population, disease characteristics, treatment plan, etc). An important consideration in comparing studies is study design, and in particular, the methods used for analysis of the primary end points.

A common method for analysis is the last observation carried forward (LOCF) method. For subjects who for whatever reason drop out of a clinical study, the last-measured value for a variable such as response to treatment is carried forward and assumed to be valid for the last scheduled measurement in the study.

An alternative is the non-responder imputation (NRI) method. In this method, which is used for dichotomous (“yes or no”) or categorical variables, if a subject drops out of a study, that subject is assumed to be a non-responder, regardless of whether or not the subject was responding to treatment at the time of dropout.

Why is the method of analysis important? It can significantly affect the apparent result of a study. In the LOCF method, for example, the last observation is often the best observed for subjects who drop out because they cannot tolerate the therapy. This biases the result in favour of the active treatment.

In contrast, NRI is a conservative method that avoids this bias. NRI may, in fact, underestimate the efficacy of a therapy, for example by categorising as a non-responder a subject who may have dropped out simply because he moved or couldn’t make scheduled study visits. Because NRI is conservative, it is gaining respect as an analytical method. In fact, according to Okamoto et al*, the US Food and Drug Administration sees LOCF as no longer acceptable for some analyses, favouring conservative methods of imputing missing values.

NRI is, in fact, one of several methods of imputation, including worst observation carried forward (WOCF), group mean imputation (GMI), placebo mean imputation (PMI), and imputation based on the reason for discontinuation (IDUR). Furthermore, conservative methods and LOCF can be used appropriately in the same study. (NRI is not appropriate for continuous variables.)
The point for those who evaluate drugs and therapies is this: when comparing studies, know which analytical methods are being used, and understand how the choice of analytical method can affect how study results look.

*Okamoto A, Wang J, Mohanty S. Rescue behavior and imputation strategies in analgesic studies (PowerPoint presentation). Accessed 01 October 2007.

Observational studies: answering real-life questions about healthcare practice

This is the first in a series of four articles about observational studies. It’s our impression that observational studies (also known as “naturalistic studies”) are becoming more and more important to healthcare authorities, and it behoves us all to understand more about the uses and abuses of this type of research.

So we have created this series to 1) describe what they are, 2) show how they differ from randomised controlled trials in the way data are collected, 3) have a brief look at how they should be conducted, and 4) describe how to interpret the results.

“Science is built up of facts, as a house is built of stones; but an accumulation of facts
is no more science than a heap of stones is a house.”
Henri Poincare (mathematician, 1854–1912) Science and Hypothesis, 1905

Part 1 – What are observational studies?

While randomised, controlled clinical trials (RCTs) are the cornerstone of the drug development process, they cannot replicate actual clinical practices. Observational studies help close the evidence gap by providing insights into real life situations, and thus aid our understanding of how both patients and their clinicians manage healthcare problems.

Observational studies are characterised by the lack of intervention when treatment decisions are made; the treatments are administered as they would be in normal clinical practice, and information is collected regarding the outcomes of those treatments. This means that switching therapies midway through the treatment can be common – patients are not restricted to a particular drug therapy. Some observational studies are carried out retrospectively using existing databases of patient data, but the most robust and useful type of study is carried out prospectively; i.e. the study design is decided, then the patients are enrolled.

Observational studies can also collect data on outcomes important to patients that may not be included in RCTs, for example:

• quality of life;

• satisfaction with treatment;

• out-of-pocket expenses;

• activities of daily living/independence;

• employment status/time off work;

• social inclusion;

• impact on careers; and

• longer term outcomes (can be over many years).

Patients and their concerns are central to the study, and unlike RCTs, patients are active partners in both their treatments and the study. Typically, patient reported outcomes (PROs) are integral to the study design.

The table below shows where observational studies tend to fit on the hierarchy of evidence established by Bandolier.

Levels of evidence

Level 1 is the highest; i.e. considered the most robust. Levels of evidence system adapted from Bandolier.

While RCTS answer questions such as “Is medicine efficacious?” or “Is a treatment safe and tolerable?” these answers are often provided in highly controlled settings, which may mean that the findings are not easily translated to actual clinical practice.

So while RCTs can provide definitive answers in specific circumstances and populations, other evidence is required to answer more far-reaching questions now posed by healthcare authorities, such as

• How will this treatment work in actual clinical practice?

• What happens if the patient has co-existing conditions?

• Will treatment be taken according to the dosage regimen prescribed?

Large, well-designed prospective observational studies help provide answers to these all-important questions about medicine use in the real world.