Data, Reporting, and Evaluation

 
The types of data involved with the National Diabetes Prevention Program (National DPP) lifestyle change program and the processes and protocols by which that data is shared among parties are similar to other data and data-sharing processes used by payers (e.g., state Medicaid agencies, Medicaid managed care organizations (MCOs), and commercial plans) in their normal business operations with each other and with medical providers. These data and other sources can also be used in the evaluation of the National DPP lifestyle change program and associated pilots.

This page includes the following sections:

  1. Types of Data
  2. Data Process Flows
  3. Data Security and Regulatory Compliance
  4. Data Sharing and Ownership
  5. Evaluation

Types of Data

Payers, CDC-recognized organizations, and third-party organizations, if used, will need to establish procedures to exchange the following data:

  • Medicaid eligibility information
  • Program enrollee contact information
  • CDC-recognized organization encounter data
  • Claims data
  • Cost data

CDC-recognized organizations maintain participant data such as attendance, weight, minutes of physical activity, etc., as required by the CDC’s Diabetes Prevention Recognition Program (DPRP), which sets the standards for CDC recognition and serves as a neutral quality assurance function to assure quality and fidelity to scientific evidence. CDC-recognized organizations can consider entering into an umbrella hub arrangement (UHA) to help aggregate DPRP data and streamline business and administrative duties. For more information, visit the Umbrella Hub Arrangements pages of the Coverage Toolkit.

Note: Only entities that elect to become CDC-recognized organizations are required to submit deidentified data to CDC.


Data Process Flows

The following are the key process flows/data exchange frameworks that the relevant parties will need to put in place to facilitate program participation, ensure an appropriate reimbursement framework is established between CDC-recognized organizations and applicable payers, and support program evaluation:

Medicaid Agencies


  1. Beneficiaries are enrolled in Medicaid and eligible for services at the time they are furnished;
  2. Physician group and hospital may provide electronic health record data to identify enrollees for outreach efforts (for more information, see Screening & Identification);
  3. To the extent required to be documented by the state, maintain evidence that the participant is eligible to take part in the National DPP lifestyle change program;
  4. CDC-recognized organization submission of a claim or invoice and encounter data to the state Medicaid agency for reimbursement and evaluation purposes;
  5. CDC-recognized organization collection of and submission to CDC of required program evaluation data elements for purposes of receiving “pending,” “preliminary,” or “full recognition” designation status. (For more information, see DPRP requirements);
  6. Experience has shown that provider referrals and sharing National DPP lifestyle change program results with a participant’s primary care provider can increase program enrollment and retention.

Medicaid MCOs


  1. CDC-recognized organization access Medicaid eligibility data from the state Medicaid agency (e.g., via the eligibility verification system) to confirm that an individual eligible for the National DPP lifestyle change program is enrolled in Medicaid.
  2. Physician group and hospitals can provide evidence of program eligibility as part of a referral and/or outreach effort to a CDC-recognized organization. Data sharing agreements may be required if lists of potential participants are going to the CDC-recognized organization directly for outreach purposes (for more information, see Program Delivery: Screening & Identification).
  3. CDC-recognized organization submission of a claim or invoice to the Medicaid MCO (or third party organization, if used) for reimbursement and evaluation purposes.
  4. CDC-recognized organization or Medicaid MCO submission of a claim or invoice and encounter data as a way to track quality improvements, meet program integrity, or ensure compliance in case of audit.
  5. CDC-recognized organization collection of and submission to CDC of required program evaluation data elements for purposes of receiving “pending,” “preliminary,” and “full recognition” National DPP lifestyle change program designation status (for more information, see DPRP requirements).
  6. Medicaid MCO submission of program cost data to state Medicaid agency for purposes of refining health plan rates.
  7. Experience has shown that provider referrals and sharing National DPP lifestyle change program results with a participant’s primary care provider can increase program enrollment and retention.

Commercial Plans


  1. Commercial plan submission of member lists to CDC-recognized organizations, generated based on an analysis of historical claims data, to identify enrollees for outreach efforts.
  2. Physician group and hospital submission of electronic health record (EHR) data to a commercial plan or CDC-recognized organization to identify enrollees for outreach efforts. (For more information, see Program Delivery: Screening & Identification)
  3. CDC-recognized organization submission of a claim or invoice and encounter data to the commercial plan, employer, or third party organization for reimbursement and evaluation purposes.
  4. CDC-recognized organization collection, and submission to CDC, of required program evaluation data elements for purposes of receiving pending, preliminary, and full recognition. (For more information, see DPRP requirements)
  5. Experience has shown that provider referrals and sharing National DPP lifestyle change program results with a participant’s primary care provider can increase program enrollment and retention.

 

Data Security and Regulatory Compliance

Payers will want to ensure that CDC-recognized organizations with which they work have the capacity to meet all statutory and regulatory requirements pertaining to privacy and data security. At a basic level, CDC-recognized organizations will need to be able to ensure the privacy and confidentiality of the data their program participants will be sharing with them.
 

Business Associate Agreement (BAA) or Data Use Agreement (DUA)

As noted above, CDC-recognized organizations will need to comply with HIPAA, the HITECH Act, and all applicable state privacy and data security statutes and relevant regulatory requirements issued by state insurance commissioners or other state regulatory authorities.

Payers will enter into agreements with the CDC-recognized organizations they contract with to ensure data security and regulatory compliance. A sample BAA developed by the American Medical Association (AMA) can be accessed here.

A BAA or DUA will likely be used and may include the following elements:

Read more

 

Data Sharing and Ownership

It is critical that all parties have a clear understanding around National DPP lifestyle change program data sharing and ownership needs and that all agreements pertaining to data sharing and ownership are reflected in any relevant contracts, business associate agreements, and memoranda of understanding. This is especially key for CDC-recognized organizations.

CDC-recognized organizations are required to share deidentified program data with the CDC’s DPRP to maintain recognition. They do not need a data-sharing agreement with the CDC to provide this information. They may also need to share data with their parent organization, a third-party program evaluator, or other entity on a case-by-case basis. For data to be shared with groups other than the CDC, data sharing agreements may be required.


Evaluation

Developing a comprehensive evaluation plan will support efforts to improve the National DPP lifestyle change program and can be done in any context, such as during a pilot, when the program is a fully covered Medicaid benefit, or within a commercial context. The data collected during an evaluation can be used for many purposes, including to guide benefit design, measure changes over time against a baseline, and establish accountability for the use of resources.

When developing an evaluation plan, it is important to identify the questions of interest, how needed information will be collected, and how the results will be shared and utilized. Answering these questions will help the evaluation team define roles and responsibilities as they relate to the evaluation and ultimately set the evaluation up for success.

Potential Evaluation Questions

When implementing an evaluation of the National DPP lifestyle change program, it is important to start by defining the questions to be answered. Potential evaluation questions include:

  • How were the delivery models implemented and what factors may have influenced implementation?
  • How many (and what proportion) of the state’s Medicaid population diagnosed with or at risk for prediabetes were engaged, enrolled, retained, and completed the National DPP lifestyle change program?
  • How did delivery programs retain Medicaid participants? What were the factors associated with retention?
  • Did Medicaid participants achieve the expected outcomes to meet the standards of the DPRP? Which participants were most likely to achieve these outcomes? What benefits did participants experience through participation in the program? What were the social/behavioral outcomes?

Measures for Consideration

In 2022-2023, grantees participating in the Medicaid Beneficiary Enrollment Project (MBEP)* focused on the following measures for evaluation related to advancing access to the National DPP lifestyle change program to Medicaid populations:

  • Total adult Medicaid beneficiaries in the state
  • Total Medicaid beneficiaries in the state eligible for the National DPP lifestyle change program
  • Number of eligible referrals
  • Referral sources
  • Referral to enrollment conversion rates/source conversion
  • Number of Medicaid beneficiaries enrolled

The following graphic provides examples of how the measures listed above can guide the selection or implementation of supporting measures within an evaluation of the National DPP lifestyle change program.

Hover over the green plus signs on the graphic below to see the supporting measures.

*MBEP was an opportunity for states to receive funding and group-based technical assistance from CDC Division of Diabetes Translation and NACDD to increase access to and enrollment in the National DPP lifestyle change program for Medicaid beneficiaries.

Data Considerations

Data Sources and Collection Methods

There are many ways to evaluate a program or pilot, and the best approach will be determined by your question(s) and the available data. The data sources described in the Types of Data section of this page could be utilized as key data sources when evaluating the National DPP lifestyle change program benefit or pilot. Additional data sources for evaluation could include primary data collected through surveys, interviews, or focus groups.

When determining appropriate data collection methods, consider the complexity associated with each method and the resources and staffing available to carry out the evaluation. Below are examples of data collection methods and how they may be used to evaluate the National DPP lifestyle change program.

Data Collection Method Potential Use
Pre/post participant survey
  • Assess participant satisfaction
  • Assess participant lifestyle changes (healthy eating, activity, etc.)
Clinical data monitoring
  • Assess participant outcomes (weight loss and A1c)
Survey of participating CDC-recognized organizations
  • Gather data on CDC-recognized organizations

Timing

When planning an evaluation for the National DPP lifestyle change program, it is important to consider what indicators can reasonably be measured in a given timeframe. It can take time to demonstrate the impact of the National DPP lifestyle change program. For example, demonstrating return on investment (ROI) often takes at least three years. It can take up to six months alone for Medicaid claims to become available for evaluation purposes. As a result, it is important for stakeholders to understand the timeline of the evaluation from the beginning. It may also be beneficial to build in indicators or measurements that can be looked at periodically throughout the evaluation process, such as referrals to the program, enrollment in each session, or number of cohorts.

For more information about the ROI for the National DPP lifestyle change program, please see the Cost and Value page of the Coverage Toolkit.

Evaluation Examples

Medicaid Coverage for the National DPP Demonstration Project

Incorporating Multiple Evaluation Methods

During the Medicaid Coverage for the National DPP Demonstration Project, NACDD worked with Research Triangle Institute (RTI) International to evaluate the following:

  • Process for Medicaid coverage and delivery of the National DPP lifestyle change program in Maryland and Oregon
  • Cost of the different delivery models
  • Enrollment, engagement, and retention strategies
  • Participant outcomes for demonstration participants

The multimethod evaluation of the Medicaid Coverage for the National DPP Demonstration Project included:

  • Program implementation surveys, cost surveys and interviews with states, MCOs in Maryland, coordinated care organizations (CCOs) in Oregon, and CDC-recognized organization staff
  • Focus groups with Lifestyle Coaches
  • Pre/post telephone surveys with Medicaid beneficiary participants

The executive summary of the evaluation report was released on 1/10/19. Learn more about the evaluation of the Medicaid Coverage for the National DPP Demonstration Project here.

Michigan

Utilizing Evaluation Results

As part of Michigan Department of Health and Human Services’ (MDHHS) MCO pilot of the National DPP lifestyle change program, MDHHS collects data to evaluate each CDC-recognized organization receiving 1815 funding through the pilot. MDHHS collects a variety of data including:

  • Cost per participant to provide the program
  • Enrollment in the program
  • Attendance at sessions
  • Weight loss across participants

These data are utilized to evaluate whether participating organizations are meeting MDHHS requirements, which were modeled after the current DPRP standards.

After obtaining needed data, MDHHS’ timeline for evaluation prompts them to give technical assistance to CDC-recognized organizations that are not meeting DPRP standards. If CDC-recognized organizations are not reaching DPRP standards halfway through the contract year, MDHHS works with them over the course of the year to create a performance improvement plan. If at the end of the year the CDC-recognized organization has still not achieved the standards, pilot reimbursement is decreased incrementally until funding ends. From MDHHS’ perspective, this process of utilizing evaluation data to drive technical assistance creates a realistic model for pay-for-performance that builds capacity for CDC-recognized organizations.

To learn more, see Michigan’s State Story of Medicaid Coverage on the Coverage Toolkit.

Montana

Evaluating Multiple Aspects of the National DPP Lifestyle Change Program

Montana is a state that has conducted rigorous evaluation on various aspects of the National DPP lifestyle change program since implementing Medicaid coverage. Montana’s evaluation studied the following:

  • Program adaptations for the Medicaid population
  • Effectiveness of the program for individuals with disabilities
  • Effects of financial incentives for program participants
  • Differences in program effectiveness based on age
  • Effectiveness of telehealth delivery of the program to rural populations

To learn more, visit Montana’s State Story of Medicaid Coverage on the Coverage Toolkit.

Content Updated: January 30, 2023