Data, Reporting, and Evaluation
The types of data involved with the National Diabetes Prevention Program (National DPP) lifestyle change program and the processes and protocols by which that data is shared among parties are similar to other data and data-sharing processes used by payers (e.g., state Medicaid agencies, Medicaid managed care organizations (MCOs), and commercial plans) in their normal business operations with each other and with medical providers. These data and other sources can also be used in the evaluation of the National DPP lifestyle change program and associated pilots.
This page includes the following sections:
- Types of Data
- Data Process Flows
- Data Security and Regulatory Compliance
- Data Sharing and Ownership
- Evaluation
Types of Data
Payers, CDC-recognized organizations, and third-party organizations, if used, will need to establish procedures to exchange the following data:
- Medicaid eligibility information
- Program enrollee contact information
- CDC-recognized organization encounter data
- Claims data
- Cost data
CDC-recognized organizations maintain participant data such as attendance, weight, minutes of physical activity, etc., as required by the CDC’s Diabetes Prevention Recognition Program (DPRP), which sets the standards for CDC recognition and serves as a neutral quality assurance function to assure quality and fidelity to scientific evidence. CDC-recognized organizations can consider entering into an umbrella hub arrangement (UHA) to help aggregate DPRP data and streamline business and administrative duties. For more information, visit the Umbrella Hub Arrangements pages of the Coverage Toolkit.
Note: Only entities that elect to become CDC-recognized organizations are required to submit deidentified data to CDC.
Data Process Flows
The following are the key process flows/data exchange frameworks that the relevant parties will need to put in place to facilitate program participation, ensure an appropriate reimbursement framework is established between CDC-recognized organizations and applicable payers, and support program evaluation:
- Medicaid Agencies
- Medicaid MCOs
- Commercial Plans
Medicaid Agencies
- Beneficiaries are enrolled in Medicaid and eligible for services at the time they are furnished;
- Physician group and hospital may provide electronic health record data to identify enrollees for outreach efforts (for more information, see Screening & Identification);
- To the extent required to be documented by the state, maintain evidence that the participant is eligible to take part in the National DPP lifestyle change program;
- CDC-recognized organization submission of a claim or invoice and encounter data to the state Medicaid agency for reimbursement and evaluation purposes;
- CDC-recognized organization collection of and submission to CDC of required program evaluation data elements for purposes of receiving “pending,” “preliminary,” or “full recognition” designation status. (For more information, see DPRP requirements);
- Experience has shown that provider referrals and sharing National DPP lifestyle change program results with a participant’s primary care provider can increase program enrollment and retention.
Medicaid MCOs
Commercial Plans
Data Security and Regulatory Compliance
Payers will want to ensure that CDC-recognized organizations with which they work have the capacity to meet all statutory and regulatory requirements pertaining to privacy and data security. At a basic level, CDC-recognized organizations will need to be able to ensure the privacy and confidentiality of the data their program participants will be sharing with them.
Business Associate Agreement (BAA) or Data Use Agreement (DUA)
As noted above, CDC-recognized organizations will need to comply with HIPAA, the HITECH Act, and all applicable state privacy and data security statutes and relevant regulatory requirements issued by state insurance commissioners or other state regulatory authorities.
Payers will enter into agreements with the CDC-recognized organizations they contract with to ensure data security and regulatory compliance. A sample BAA developed by the American Medical Association (AMA) can be accessed here.
A BAA or DUA will likely be used and may include the following elements:
- Permitted and prohibited uses of Protected Health Information (PHI) and nonpublic personal financial information: outlines that uses of PHI and nonpublic personal financial information must comply with all applicable privacy and security laws, including HIPAA.
- Obligations for privacy and security breaches: outlines that a CDC-recognized organization may have obligations, such as reporting obligations, if there is a privacy or security breach. Examples of breaches include information systems being exposed to a virus or worm, an individual using company data through unauthorized access, an attack compromising a server, or unauthorized access or disclosure of PHI.
- Obligations upon termination: outlines obligations upon the conclusion of a BAA, including the return or destruction of PHI or continued protection of PHI.
- Required security controls: required security controls may include (a(n)):
- Information security program – such as written policies for security and the identity of the individual responsible for enforcement of the security program.
- Audit plan – may include who can complete an audit and how frequently the audit must be conducted.
- Approved encryption – required use of approved encryption for the transfer of confidential information to and from the Medicaid MCO and to and from third-parties.
- Network and systems security programs/tools – required use of security programs such as an industry standard malware detection program, an intrusion detection or prevention system, and firewalls that separate networks containing confidential information from public networks. Medicaid MCOs may also require third-party annual penetration testing of both internal and external systems.
- Data destruction agreement – outlines the type of data that must be destroyed, the circumstances under which destruction is required, and the method of destruction required.
- Physical and system controls – may include required use of endpoint protection for remote access of confidential information, keeping operating systems updated, safeguarding hard copies with a clean desk policy, and/or retaining visitor logs for the facility.
- Controls on workforce members accessing information – examples include background checks prior to providing employee access to confidential information, only providing access to employees who have a legitimate need to use the information as part of their job responsibilities, using IDs and passwords to access confidential information, and providing security awareness trainings prior to granting employees access to confidential information.
- Cloud storage controls – additional controls may be necessary if data is stored using a cloud-based technology.
- Business continuity and disaster recovery plan – plans outlining the critical information an organization needs to continue operating during an unplanned event or disaster needs be documented and tested regularly.
- Incident response plan – to be documented and tested regularly.
Data Sharing and Ownership
It is critical that all parties have a clear understanding around National DPP lifestyle change program data sharing and ownership needs and that all agreements pertaining to data sharing and ownership are reflected in any relevant contracts, business associate agreements, and memoranda of understanding. This is especially key for CDC-recognized organizations.
CDC-recognized organizations are required to share deidentified program data with the CDC’s DPRP to maintain recognition. They do not need a data-sharing agreement with the CDC to provide this information. They may also need to share data with their parent organization, a third-party program evaluator, or other entity on a case-by-case basis. For data to be shared with groups other than the CDC, data sharing agreements may be required.
Evaluation
Developing a comprehensive evaluation plan will support efforts to improve the National DPP lifestyle change program and can be done in any context, such as during a pilot, when the program is a fully covered Medicaid benefit, or within a commercial context. The data collected during an evaluation can be used for many purposes, including to guide benefit design, measure changes over time against a baseline, and establish accountability for the use of resources.
To learn more about evaluating efforts to support the National DPP lifestyle change program, please watch this video (start at 18 minutes and 33 seconds).
When developing an evaluation plan, it is important to identify the questions of interest, how needed information will be collected, and how the results will be shared and utilized. Answering these questions will help the evaluation team define roles and responsibilities as they relate to the evaluation and ultimately set the evaluation up for success.
Potential Evaluation Questions
When implementing an evaluation of the National DPP lifestyle change program, it is important to start by defining the questions to be answered. Potential evaluation questions include:
- How were the delivery models implemented and what factors may have influenced implementation?
- How many (and what proportion) of the state’s Medicaid population diagnosed with or at risk for prediabetes were engaged, enrolled, retained, and completed the National DPP lifestyle change program?
- How did delivery programs retain Medicaid participants? What were the factors associated with retention?
- Did Medicaid participants achieve the expected outcomes to meet the standards of the DPRP? Which participants were most likely to achieve these outcomes? What benefits did participants experience through participation in the program? What were the social/behavioral outcomes?
Measures for Consideration
In 2022-2023, grantees participating in the Medicaid Beneficiary Enrollment Project (MBEP)* focused on the following measures for evaluation related to advancing access to the National DPP lifestyle change program to Medicaid populations:
- Total adult Medicaid beneficiaries in the state
- Total Medicaid beneficiaries in the state eligible for the National DPP lifestyle change program
- Number of eligible referrals
- Referral sources
- Referral to enrollment conversion rates/source conversion
- Number of Medicaid beneficiaries enrolled
The following graphic provides examples of how the measures listed above can guide the selection or implementation of supporting measures within an evaluation of the National DPP lifestyle change program.
Hover over the green plus signs on the graphic below to see the supporting measures.
*MBEP was an opportunity for states to receive funding and group-based technical assistance from CDC Division of Diabetes Translation and NACDD to increase access to and enrollment in the National DPP lifestyle change program for Medicaid beneficiaries.
Data Considerations
Data Sources and Collection Methods
There are many ways to evaluate a program or pilot, and the best approach will be determined by your question(s) and the available data. The data sources described in the Types of Data section of this page could be utilized as key data sources when evaluating the National DPP lifestyle change program benefit or pilot. Additional data sources for evaluation could include primary data collected through surveys, interviews, or focus groups.
When determining appropriate data collection methods, consider the complexity associated with each method and the resources and staffing available to carry out the evaluation. Below are examples of data collection methods and how they may be used to evaluate the National DPP lifestyle change program.
Data Collection Method | Potential Use |
Pre/post participant survey |
|
Clinical data monitoring |
|
Survey of participating CDC-recognized organizations |
|
Timing
When planning an evaluation for the National DPP lifestyle change program, it is important to consider what indicators can reasonably be measured in a given timeframe. It can take time to demonstrate the impact of the National DPP lifestyle change program. For example, demonstrating return on investment (ROI) often takes at least three years. It can take up to six months alone for Medicaid claims to become available for evaluation purposes. As a result, it is important for stakeholders to understand the timeline of the evaluation from the beginning. It may also be beneficial to build in indicators or measurements that can be looked at periodically throughout the evaluation process, such as referrals to the program, enrollment in each session, or number of cohorts.
For more information about the ROI for the National DPP lifestyle change program, please see the Cost and Value page of the Coverage Toolkit.
Evaluation Examples
- Medicaid Coverage for the National DPP Demonstration Project
- Michigan
- Montana
- State Data
Medicaid Coverage for the National DPP Demonstration Project
Incorporating Multiple Evaluation Methods
During the Medicaid Coverage for the National DPP Demonstration Project, NACDD worked with Research Triangle Institute (RTI) International to evaluate the following:
- Process for Medicaid coverage and delivery of the National DPP lifestyle change program in Maryland and Oregon
- Cost of the different delivery models
- Enrollment, engagement, and retention strategies
- Participant outcomes for demonstration participants
The multimethod evaluation of the Medicaid Coverage for the National DPP Demonstration Project included:
- Program implementation surveys, cost surveys and interviews with states, MCOs in Maryland, coordinated care organizations (CCOs) in Oregon, and CDC-recognized organization staff
- Focus groups with Lifestyle Coaches
- Pre/post telephone surveys with Medicaid beneficiary participants
The executive summary of the evaluation report was released on 1/10/19. Learn more about the evaluation of the Medicaid Coverage for the National DPP Demonstration Project here.