Data program focuses on facilitating the coordinated evolution of global data systems to more effectively reduce uncertainty in drug development and clinical decision-making.

The New Drug Development Paradigms Program (NEWDIGS) of the MIT Center for Biomedical Innovation has launched a program focused on driving greater value from healthcare data for patients, regulators, payers, and physicians. The first meeting brought together international experts from academia, government, information technology, and the healthcare sector. On the demand side, regulators, payers, providers, patients, and pharmaceutical firms described their needs for better real world information on safety, efficacy and effectiveness. On the supply side, managers of public and private sector data initiatives in the US, Europe and Canada described how they pool, protect and analyze healthcare data and identified problems with standards, data access, data security, and analytic methods. Finally, all participants joined in formulating a research agenda with associated pilot programs to develop the tools, methods and policies needed to leverage real world evidence for better decision-making.

The inaugural meeting of the MIT NEWDIGS Data Program was kicked off by Gigi Hirsch, Executive Director of MIT NEWDIGS. NEWDIGS is a “think and do” tank focused on delivering new, better, affordable therapeutics to the right patients faster. MIT NEWDIGS has been at the vanguard of adaptive licensing (also known as adaptive pathways), a way to bring medicines to patients through licensing, pricing and reimbursement conditions that are based on evidence generated by patient responses to a therapy over its lifecycle. This approach was developed in partnership with key stakeholders, and the European Medicines Agency (EMA) has put in place a pilot project to further evaluate the adaptive pathways approach using actual therapies.[1]

The workshop referenced three important paradigm shifts currently unfolding in biomedical and healthcare innovation, which are converging in a way that is forcing advancements in the world of health data that many believe are long overdue. These change drivers include:

  • Increasing pressures to accelerate access to new and better treatments for patients in need. Central to this trend (and including adaptive pathways) is an evolution from the binary pre-market/post-market to a lifespan approach to the reduction of uncertainty related to the safety, efficacy, and effectiveness of new drugs.
  • The advancement of translational and “precision” medicine,” associated with erosion of the traditional blockbuster model of innovation toward one that targets small subpopulations and individually tailored treatments.
  • Growing demands from reimbursement/coverage decision-makers for quantifiable measures of relative effectiveness and value of new drugs, rapidly becoming as significant a source of risk and uncertainty in innovation as has traditionally been the case for regulatory market approvals.

The goal of the MIT NEWDIGS Data Program is to improve the capacity of local, regional and global data systems to optimize decisions on safety, efficacy and effectiveness over the life cycle of new therapies. The workshop was designed to achieve three main objectives: (1) to explore the need for international cooperation on development and use of data and data systems; (2) to define the domains that need to be addressed for successful cooperation, taking into account lessons learned from the successful design and implementation of current distributed data networks; and (3) to develop a plan of action for MIT NEWDIGS.

Framing the Need and Opportunity

A series of panel discussions involving various stakeholders focused on identifying the gaps and opportunities that might be addressed through better international cooperation in data system design and use. This session kicked off with perspectives from MIT NEWDIGS’ Canadian collaborators and focused on ways to harness databases in Canada for the benefit of payers and regulators. According to David Lee, Director of Legislative and Regulatory Modernization of Health Canada, recent post marketing legislation will require the advancement of better methodologies for life-cycle evaluation. He proposed that the collection of accurate, quality data is vital and the structural needs to facilitate that data need to be defined, “because if we don’t have good data, we’re already at a loss to make the correct decision.”

Brian O’Rourke of the Canadian Agency for Drugs and Technologies in Health observed that in many countries, national HTA agencies make non-binding recommendations to the payers regarding the clinical effectiveness and cost effectiveness of a particular technology. Payers use the HTA recommendations as a key component in their decision-making process. There is still some skepticism in the global payer community regarding the utility of adaptive pathways, as there are significant hurdles to overcome in our ability to generate real world evidence in the post-market space.  In many jurisdictions, the generation of evidence is also complicated by regional divisions of authority of payers.  Furthermore, payers and regulators will need to accept and be able to analyze data originating outside of the traditional randomized clinical trial (RCT) process.  “Where do we get that data to evaluate?” he asked.

The workshop then moved to European perspectives. The European Medicines Agency’s Hans-Georg Eichler noted that many new therapies being tested in Europe had undergone multi-regional RCTs previously and would have some type of evidence trail. He asked, “How can we make best use of that data?” The fact that many treatment signals are context-dependent, using data obtained from testing around the world without a control, will require the ability to identify and separate a context-dependent event from a drug based event in order to be able to identify the value of the therapy. There will be a need to validate the evidence from the existing data set and establish if that drug is effective, but that comes with a litany of issues regarding the potential for bias, and the quantity and type of data required depending on that specific situation. “This will all need to be investigated and sorted out,” Eichler said. Variability too is contextual, particularly with respect to health economics. It is crucial to understand the origin and driver of the variability to determine effectiveness.

“Most people are probably not in equipoise with regard to choosing alternative comparative treatments, they are probably in cluelessness with regard to the limitations of the data and its quality,”

Robert O’Neill, FDA


Robert O’Neill of the FDA began by noting that scientific skill sets are getting better all the time. Even so, credible and valid inferences on causal association are difficult to glean from observational data. He continued, it is generally agreed that comparative effectiveness research has not caught up to the examples that exist in post-approval safety such as the groundbreaking work being done by the U.S. Mini Sentinel initiative[2]. He contends that there is a group of opinion leaders who feel that the use of observational data is a long way away from being able to make any inferences that can be trusted as evidence. “Most people are probably not in equipoise with regard to choosing alternative comparative treatments, they are probably in cluelessness with regard to the limitations of the data and its quality,” O’Neill noted, regarding current observational data practices.

However, there was general agreement from attendees that projects being run and funded by the FDA, IMI and EMA to investigate the usage of observational data as evidence are an excellent basis of greater international collaboration; and the more everyone discusses what is happening, the less duplication there will be. To quote O’Neill, “we need to figure out who or what is the ‘mother ship’ organizing all this ‘stuff’, so there aren’t any missed opportunities, and further, how to create these international observational data partnerships in a way that they develop workable evidence?”

“We need to figure out who or what is the ‘mother ship’ organizing all this ‘stuff’, so there aren’t any missed opportunities, and further, how to create these international observational data partnerships in a way that they develop workable evidence?”

Robert O’Neill, FDA

Naomi Aronson, Executive Director, Clinical Evaluation, Innovation and Policy, Blue Cross and Blue Shield Association, noted that global collaboration with patient data records would be an enormous gain to control costs, simply on the basis of the raw numbers related to market size for development. This situation was noted as being particularly acute in the realm of orphan conditions, where small numbers of localized patients with unmet medical needs were increasingly translating into high development costs per unit with serious long term implications related to affordability and sustainability. Aronson quoted, “FDA’s 2014 pipeline had 41 new drugs, 14 of which were for orphan conditions.”

The data implications of orphan diseases and drugs, the issues of interaction effects, subpopulations and combo-therapies are becoming a norm. The world is one where the development strategy is for a single therapy, yet the usage is not. The data needs to be there, in a usable way, to address the reality of today’s prescribing. Some types of data are not currently captured anywhere in the system. Further, given the FDA’s increasing ratio of approvals of orphan drugs in 2014, the workshop raised the point that an international data strategy should prepare all regulators globally for “orphanization.” In addition, the need for more and better effectiveness data was highlighted by the payer representatives on the panel.

Pamela Gavin, COO of the National Organization for Rare Disorders (NORD) contends that there is a need for more collaboration with patients, highlighting that “patients should have access to solutions, even if they are unable to participate at local level.”

“Is patient involvement and engagement the answer to the collection of data?” asked Duane Schulthess of Vital Transformation. He quoted a recent research using data from patient communities showing that 73% of the patients were willing to participate in clinical trials as subjects and 83% were dealing with a chronic condition for which there was no cure. “We need to embrace the fact that patients will have opinions,” said Robyn Lim, Senior Science Advisor with Health Canada. A majority of the workshop participants agreed that informed patients need to be part of the process and their input should be considered in decision-making done on their behalf by other stakeholders.

“We are interested in knowing what works for our patients.”

Chester Good, CHERP


 Good Chester-sq

Chester (Bernie) Good, Core Investigator, VA Center for Health Equity Research and Promotion (CHERP), Professor of Medicine and Pharmacy, University of Pittsburgh, and Chair of the Medical Advisory Panel for Pharmacy Benefits Management for the Department of Veterans Affairs, provided a detailed overview of a real world research project investigating adverse events associated with anticoagulation therapies. According to him, research using real world databases could help determine the root cause of safety signals associated with these drugs. Regarding the all too frequent gap between efficacy and effectiveness, Good commented “we are interested in knowing what works for our patients.” As patients have a high incidence of comorbidities, this provider system is also reaching out internationally to foster greater collaboration with partners. He went on to explain that greater international collaboration would provide a pool of interesting research data to inform continuous improvement in clinical care, such as, for example, potential approaches to intractable issues around comorbidities. He also highlighted the need to generate and communicate information faster among international partners.


“It’s spooky if you see exactly the same result everywhere, and it’s frustrating when you see different results everywhere”

Sebastian Schneeweiss, Harvard Medical School

“The pharmaceutical industry is trying to understand how to better harness and generate greater and more clinically meaningful evidence in the drug development process utilizing multiple sources.” Maha Radhakrishnan, Biogen Idec, Vice President, US Medical, said the data could help us learn about the disease, outcomes that will matter to all stakeholders and will help identify the clinically meaningful endpoints that will also matter in the real world.” Anne-Virginie Eggimann from bluebird bio said, “There’s far more room for international collaboration and communication, but it is a significant challenge to have the required feedback from the numerous stakeholders and create a consensus on the data needed and the evidence required.” Actual interpretation of the data, even when the systems are operational, can be equally vexing with regards to variation in findings across different databases and populations. “It’s spooky if you see exactly the same result everywhere, and it’s frustrating when you see different results everywhere, remarked Sebastian Schneeweiss of Harvard Medical School.

Learning from Other Data Sharing Initiatives

The need for data and information for drug development is currently being addressed through a number of collaborative initiatives in the EU, many of which are funded through the Innovative Medicines Initiative, IMI. IMI-GetReal ( is a three-year initiative that aims to incorporate real-world data (RWD) in drug development and assessment involving multiple stakeholders in Europe. Amr Makady, Assistant Project Manager for IMI-GetReal at the National Health Care Institute in the Netherlands, presented results from one the initiative’s projects during the workshop. The aim of the project was to assess the policies and perspectives of stakeholders regarding use of RWD for drug development and relative effectiveness assessment. The conclusion from this project was that increased stakeholder collaboration is needed to develop common understanding regarding terminology and relevance of real-world evidence. The project also recommended the development of governance mechanisms for collection of data, use of evidence, guidance on tools, methodologies, and strategies. Several participants asked if there were a way for jurisdictions outside of the EU get involved in this work.   Although the current plan and formal collaborators for the GetReal project are set, future IMI proposals will likely expand upon the GetReal project findings and formal international collaboration may be feasible.

An overview of various activities in Europe was provided by Peter Arlett, Head of Pharmacovigilance EMA , including the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP), a network of excellence in pharmacoepidemiology (147 centres, 22 networks, 51 data sources) governed by a shared code of conduct. It provides guidance on epidemiology methods and serves as a public register for epidemiology studies in Europe. ENCePP aims to improve the quality, ease, speed, transparency and reliability of post-authorization evidence feeding into regulatory decision-making. Another example given was an EU pilot program aiming to develop and test an EU collaborative framework for patient registries that would facilitate the collection and analysis of high quality data on the efficacy and safety of medicinal products in the healthcare setting, in order to confirm their benefit-risk profile. Further mapping, linking and sharing between existing initiatives and collaborations would be beneficial from a public health perspective, but can be challenging when considering the diverse stakeholders, broad range of data sources, as well as legal, financial and political constraints.

Jeff Brown, Assistant Professor, Department of Population Medicine, HMS & Harvard Pilgrim Health Care Institute, outlined current activities of US Mini-Sentinel and PCORnet, and how these could inform best practice for future initiatives in data sharing. Funded by a grant from the FDA, Mini-Sentinel is a pilot project to monitor the post approval safety of FDA-regulated medical products using electronic healthcare data from multiple sources, where collaborating Institutions provide access to data as well as scientific and organizational expertise. FDA’s Sentinel Initiative is exploring a variety of approaches for improving the Agency’s ability to quickly identify and assess safety issues. Currently running in 22 partners with 178 million covered lives, Mini-Sentinel harnesses a distributed computing network based on reusable tools. While many electronic health record systems require data mapping beyond the scope of most institutional IT teams, Mini-Sentinel has gained a reputation of a ‘light-touch’ implementation, as its core analytics require only 18 lab and vital sign metrics. This approach to pharmacovigilance was shown to provide a huge gain in efficiency, with one example of Mini-Sentinel running a safety signal verification in five months for $250,000, as opposed to a normal verification of adverse signals taking up to two years at a cost of $2 million. There was some discussion about the global scalability of Mini-Sentinel, a point of potential interest from some MIT NEWDIGS collaborators who are seeking practical and cost-effective approaches to address pressing challenges in their jurisdictions.


In a common data model, relational structure provides an analysis-ready data platform incorporating electronic health record (EHR) and claims-type data. (From the presentation “Learning from other data sharing initiatives” by Jeff Brown, HMS & Harvard Pilgrim Health Institute).


William Crown, Chief Scientific Officer, Optum Labs, presented a first of its kind open, collaborative research and innovation center designed to harness real world evidence to accelerate innovation for improved patient care and patient value. Optum Labs and its collaborators target the “80% of the clinical content still sitting in notes,” having developed unique extraction tools and methodologies in twelve diseases to convert medical notes into integrated data sets. A case study was described where third-party marketing data was linked to electronic health records and was able to provide unique insights into demographic information on diabetic patients. The tools allow for the ability to explore a specific patient’s cost profile and impact across a range of medical codes using visual heat maps by stratified disease codes, providing practitioners and payers the ability to obtain hypothesis-free discovery on numerous complex medical conditions. Additionally, a Community Health Measure Platform addressing the question, “how does a community/network compare on key actionable aspects of cost and health care use?” could be used to generate community health efficiency ratings.

Summary and Conclusions

In summary, key issues identified for consideration in this workshop include the following:

Through a matrix organization supported by expert teams, NEWDIGS offers a scalable collaboration platform to achieve past and future goals.


  • Data: Evidence generation will increasingly demand that we leverage a broad range of types of data (e.g, administrative/payer claims; electronic medical records; patient and product registries, among others). In addition, there are emerging types of data (e.g., continuous monitoring from mobile applications, social media, etc.) that may offer new kinds of actionable knowledge if integrated strategically with others kinds of data. These data types lack interoperability and standards for their integrated use, and embody varying degrees of quality, accuracy, and completeness. In most cases, these data were originally created for other purposes, so it is unclear how “fit for purpose” they are in this emerging context.
  • Data access: There is a need to generate actionable evidence through the interrogation of disparate and distributed data sources across traditional organizational and geographic boundaries. There are a number of emerging governance, business and technology-enabled process models taking shape around these challenges but there is a need to more systematically extract lessons learned and best practice models from these. We are also likely to see an unprecedented need for international cooperation in data system design and use as orphan and ultra-orphan drug development become increasingly prevalent.
  • Advanced analytic methods and study designs: Increasingly the world of new drug development, as well as the optimization of value from real world use of products on the market, requires that we advance our understanding of how to collect and analyze real world data. This will require that we advance to a world where the right mix of RCTs, observational studies and pragmatic trials are integrated in ways that more effectively accelerate the delivery of targeted treatments to patients in need while also bridging the current efficacy-effectiveness gap. While there are a number of collaborative initiatives underway to advance aspects of this challenge, they are unfolding in fragmented ways. If we are to successfully address such complex but critical challenges as, for example, causal inference, it will be essential that we evaluate emerging methods in pilot projects to better understand how to optimize their integrated use across the lifespan of actual products under real world use.
  • The evolution of stakeholder roles and engagement models: including
    • Much more active participation of patients in providing data, in regulatory decision-making, and in sophisticated grass roots organization of disease-focused communities – with muscle and financial resources.
    • Increased focused on coordination and potentially even alignment of some aspects of decision-making between regulators, HTAs, and payers in the interest of optimizing efficiencies and sustainability of innovation.
  • Legal issues: related to intellectual property, patient privacy, and cyber security.

A strategic portfolio of activities has already been launched in the MIT Data Program to address some of these points, including:

  • Ongoing development and refinement of an open access Global Question Bank (GQB) that provides insights into specific questions that different stakeholders have across the innovation lifecycle, as well as the types of data required, what existing systems are workable or fit for purpose, and function as an organizing structure for defining and advancing best practices in the use of existing and emerging data types.
  • The application and contextualization of the GQB to focused case studies to inform the development and validation of a structured open access “Readiness Assessment Framework” that supports the identification of gaps and opportunities for improvement within specific data systems and target disease areas.
  • Real world pilot projects designed to address the identified areas for improvement. These pilots, currently under discussion with several global jurisdictions, will be designed and executed through local multi-stakeholder teams, with MIT NEWDIGS playing a leadership role in accelerating shared learning, advancing best practices, fostering targeted collaborative innovation, and measuring impact.
  • The establishment of several critical cross-functional Expert Teams to help advance knowledge, methods and tools as issues arise in the pilot activities. Examples include the need for an Analytic Methods team to focus on causal inference and evidentiary standards for informed decision-making associated with the management and progressive reduction of uncertainty, as well as a Legal Team to focus on data access, data protection related to both intellectual property and patient privacy, and cybersecurity.
  • Joint activities with the MIT NEWDIGS Janus Program, which provides a multi-stakeholder process and tool set for facilitating and quantifying adaptive pathways design scenarios[3]. Janus provides a powerful emerging platform for evaluating the potential convergent impact of a range of study designs, evidence streams, analytic methods, and post market conditions for a given R&D asset on tradeoff decisions made by key stakeholders. Janus builds on proven methodologies from the initial phase of MIT NEWDIGS’ work on adaptive pathways,[4] [5]enhanced for greater rigor, scalability, and quantification.

The next steps for MIT NEWDIGS outlined above will be further refined incorporating input from the workshop.


MIT NEWDIGS members can access additional information about the event, i.e. presentations, via the MIT NEWDIGS members online portal.

Members Only Portal

download the pdf of the full report

For more information on MIT NEWDIGS and to find out how you can get involved:
MIT Center for Biomedical Innovation
77 Massachusetts Avenue (E19-604)
 Cambridge, MA 02139-4307
Phone: 617-324-9640
Fax: 617-253-0687





[3] M.R. Trusheim et al. The Janus initiative: A multi-stakeholder process and tool set for facilitating and quantifying Adaptive Licensing discussions. Health Policy and Technology (2014) 3, 241-247.

[4] Lynn Baird and Gigi Hirsch. Adaptive licensing: creating a safe haven for discussions. Scrip Regulatory Affairs (September 2013), 10-11.

[5] LG Baird et al. Comparison of Stakeholder Metrics for Traditional and Adaptive Development and Licensing Approaches to Drug Development. Therapeutic Innovation & Regulatory Science (May 2013), 1-11.