The Need
A primary responsibility of the Department is ensuring the health, wellness, and safety of our members. As such, we must implement quality standards and maintain strict oversight of provider agencies. The Department proposes a series of projects to develop quality frameworks and oversight requirements, resulting in anticipated cost savings, provider accountability, and improved quality outcomes.
Initiative 8.06. - PACE Licensure
Within the PACE program, the Department will develop quality standards by establishing a PACE licensure type to ensure appropriate oversight and compliance. The Department will establish a PACE audit structure including fee cost, resource needs, timeline, survey elements, corrective action plan templates, reporting requirements, valid sample size, appeal process, performance measures, and interview questions. The Department will also develop a system to record and capture incident reviews, complaints, survey results, and reports. This will require the Department to submit amendments to the State Plan and Program Agreements with each PACE Organization.
Initiative 8.04. - Pay-for-Performance for PACE
The Department will identify key performance measures to incorporate into a pay-for-performance methodology within the PACE capitation payments. The percentage for each performance measure will be identified and the monitoring processes and reporting requirements will be outlined. The appeals process and contractual language will also be developed.
✅Initiative 8.08. - CMS Quality Metrics - Completed
Key Project Achievements and/or Activities:
ARPA project 8.08 HCBS Quality Dashboards have been created to meet a need of transparency, accessibility, and information distribution. Quality Dashboards were created with the assistance of stakeholder feedback, an accessibility team review, data team review, and HCPF staff direction. These dashboards contain member experience survey response information. The information has previously been gathered and shared in a lengthy annual report. This report was not easily digestible and was not accessible. These dashboards have been created to be interactive and show the information from these surveys in one space. The format of these dashboards should allow for members, community members, organizations, and other interested stakeholders to view the feedback gathered through HCBS member experience surveys and see the information in an accessible and interactive format. All of the information gathered for these dashboards is housed in a database that protects the privacy of those surveyed.
Also developed to accompany the dashboards is an accessible pdf version of the dashboard. This format will allow the information to be accessed through keyboard navigation and screen reader software.
Summary / Project Outcome:
HCPF participates in conducting Member experience surveys each year, called National Core Indicators (NCI) surveys. These surveys are completed by members who are enrolled in HCBS waivers. There are 3 versions of the NCI surveys:
NCI-AD: Aging and Disabilities HCBS waiver participants
NCI-IDD: Intellectual and Developmental Disabilities HCBS waiver participants
State of the Workforce: Direct Care Workforce providers
In addition to the NCI surveys, HCPF also conducts CWS: Children’s Waiver Satisfaction Surveys. These surveys were developed in collaboration with vendor Vital Research. It is a Colorado specific survey to gather feedback from families whose children are enrolled in HCBS Children’s waivers.
The purpose of these surveys is to understand member experiences with HCBS, address gaps to improve programs, ensure quality of services, and gather required HCBS Measure Set Data. The survey responses and outcomes are currently reported annually, but these reports are lengthy and contain a lot of industry information and jargon, making the reports less digestible for members, families, and other community members.
ARPA project 8.08 has created accessible, user-friendly HCBS Quality Dashboards to share the NCI survey member experience responses and State of the Workforce provider responses. These dashboards are interactive, filterable by defined service area, year, and waiver. These dashboards will house the data for the previous 3 years of NCI survey responses and provide a tool to pull reports, recognize gaps in services, and provide valuable information regarding the overall health of CO HCBS services. HCPF hopes to continue adding data for subsequent survey cycles to help evaluate trends over time. We look forward to sharing these dashboards on the HCPF website in the near future.
✅Initiative 8.11. - Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) Benefits Training - Completed
(previously named Quality Measures and Benefits Training)
Key Project Achievements and/or Activities:
The project developed comprehensive training tools designed to enhance the understanding and utilization of Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) benefits within the Medicaid population, including children and youth with special health care needs (CYSHCN) and youth in any of the child welfare programs. These tools also serve to bridge the gap between EPSDT services and other programs, ensuring better integration of care for beneficiaries.
Summary / Project Outcome:
Training Development and Implementation
The group designed a series of 5 training modules that focus on EPSDT benefits, federal performance metrics, and their intersection with waiver services under HCBS.
Policy and Program Adjustments
Through the EPSDT analysis, HCPF identified areas where policies such as the CMS requirement that services be labeled as “free” could be strengthened or adjusted to better serve both Medicaid and CYSHCN populations. These insights were integrated into the training content, further contributing to policy refinement and continuous improvement of service delivery. The training supports compliance with the federal requirements of integrating EPSDT and waiver services as outlined in the CMS Part V Manual.
Educational Outreach and Member Engagement
To engage both members and providers, HCPF created an infomercial, now housed on our YouTube Channel, that was used as a push notification to introduce EPSDT services. This resource serves as an introductory tool to raise awareness of the benefits available under the EPSDT program, ensuring that members and providers have an accessible and concise overview of the services offered.
Sustainability and Accessibility
The EPSDT training modules were designed using adult learning principles, ensuring that the content is accessible, easy to understand, and adaptable to a wide audience, and available in English and Spanish. The training materials were made available on Department websites. This sustainability mechanism allows for continuous access to ensure compliance with evolving federal requirements and performance metrics.
Federal Compliance and Quality Performance Measures
The training and analysis process directly supports HCPF's efforts to meet federal compliance standards regarding the EPSDT program.
External Facing Reports/Websites:
Toolkits in English and Spanish
Child Welfare (CW) Explainer YouTube Playlist
EPSDT Parent/Caregiver YouTube Playlist
✅Initiative 8.02. - Provider Oversight - Completed
Key Project Achievements and/or Activities:
The project’s external vendor performed the following activities to further inform work to streamline provider enrollment processes:
Surveyed 13 state agencies regarding Home and Community Based Services (HCBS) service delivery and processes for licensing and Medicaid certification
Identified inconsistencies between the Code of Colorado Regulations and HCBS Final Rule requirements that contributed to provider enrollment difficulties
Through system user testing, recommended clearer guidelines and fixes within the interChange system for the improvement of the overall enrollment application process
Key Achievement #1 - Creation of a shared database to eliminate the isolated data maintained in other systems used by HCPF and Colorado Department of Public Health and Environment (CDPHE). The platform allows agencies to track provider enrollment and survey status.
Key Achievement #2 - Development of a training module that provides comprehensive guidance for new and revalidating provider enrollment. The training module includes a training manual, online training video, and electronic test/certification.
Key Achievement #3 – A comprehensive recommendation schedule was generated to resolve perceived inconsistencies in the Medical Assistance Act Section 7.000 Concerning Case
Management, Member Rights and Responsibilities Home and Community Based Services and Provider Agency Requirements.
Summary / Project Outcome:
In 2017, the Department engaged an external vendor to review interagency provider enrollment processes for areas of alignment, to improve data sharing, and reduce agency survey wait times. The vendor produced a report outlining several high-level recommendations to streamline the provider enrollment process between HCPF and CDPHE. The Provider Oversight project provided an opportunity to reevaluate previous findings by acknowledging changes in regulation, addressing the impacts of COVID-19 on agency health compliance, and updating current systems utilized by each department.
Regulatory Review - The vendor thoroughly reviewed state regulations and HCBS Settings Rule as they applied to provider enrollment and recertification. Recognizing that contradictions and inconsistencies between the rules were often confusing and impacting providers’ ability to fully comply, draft revisions to existing regulations that address inconsistencies related to person-centeredness were developed, removing compliance barriers and reducing unnecessary delays.
Data Sharing Capability - HCPF and CDPHE faced challenges in managing and sharing data due to maintaining separate systems; Gainwell and COHFI, respectively. A new database was developed utilizing Caspio which allows tracking of provider status and survey metrics, between the agencies. This also allows the agencies the ability to communicate within the system and upload relevant documents, streamlining the process by eliminating need for email communication and its inherent inefficiencies. Additionally, this platform has expansion capabilities for future shared database needs.
Provider Training Improvements - The existing online training materials for enrolling and revalidating providers were in need of updating, lacked appropriate guidance and support, and hosted an inefficient quiz/certification process. In collaboration between HCPF and CPDHE, the vendor developed a comprehensive online training program detailing the entire process to include, certification, enrollment, recertification, and revalidation. The dynamic training program now allows providers to access appropriate content depending on where they are in the process. The online training contains system demonstrations, links to additional resources, quizzes to test knowledge, and contact information for additional technical assistance.
✅ Initiative 8.05. - Pay-for-Performance for Home Health & HCBS - Completed
(combined 8.05 Pay for Performance for Home Health & 8.03 Pay for Performance for HCBS)
Key Project Achievements and/or Activities:
HCPF completed research and analysis that will guide future efforts in pursuing a pay-for-performance model in long-term home health and Residential HCBS. Through internal and external stakeholder interviews, a literature review, national scan, and actuarial analyses, HCPF received recommendations for a series of quality measures to employ in unison with a pay-for-performance payment model.
Summary / Project Outcome:
The team received a menu of potential metrics that could be used for pay for performance in long-term home health and Residential HCBS. If HCPF pursues pay-for-performance models in long-term home health or Residential HCBS services, they will finalize the selection of metrics with stakeholders.
✅Initiative 8.10. - Criminal Justice Partnership - Completed
Key Project Achievements and/or Activities:
One of the most significant achievements of ARPA project 8.10 is the relationships that were built and the resulting partnerships with the community, state and local government, and teams across offices at HCPF. These partnerships lay the foundation for successfully planning and implementing Medicaid reentry services in the years to come. We had strong participation in stakeholder engagement efforts and brought key stakeholders together to form the HCPF Criminal and Juvenile Justice Collaborative. Information gathered through research and stakeholder engagement was used in the development of a strategic plana and completion of the Medicaid & Criminal Justice Gap Analysis which will guide future work. Additional information is available on the Health First Colorado and Criminal Justice Involved Population webpage.
Summary / Project Outcome:
Serving the justice-involved population is an ongoing priority and will be continued through the Medicaid Reentry Services project.
External Facing Reports/Websites:
Initiative 8.01. - Provider Scorecards
To support quality performance, the Department will establish metrics and develop public-facing provider scorecards. Scorecards can be used to identify providers that may need more intense oversight and to help consumers and their families make choices about their care. Providers with continuously low scores could face additional corrective action.
The Department will create provider and CMA scorecards and will add the scorecards to the provider search tool. Applicable performance measures will also be included in the scorecard. The Department will develop metrics and a weighting algorithm incorporating provider input. Providers should understand metrics and underlying data sources and believe that scores accurately and meaningfully represent care quality. Provider input and buy-in can help the Department develop a better methodology, promote higher quality data collection, and encourage providers to improve performance based on findings. The Department will continue to update these scorecards moving forward.
✅Initiative 8.07 - eConsult to Improve Quality - Completed
Key Project Achievements and/or Activities:
Provider enrollment and revalidation now includes self-attested cultural competencies such as organization affiliations, language proficiency, American Sign Language proficiency, and telehealth availability. In the future, this information will flow to the eConsult vendor to better inform the eConsult process.
Additionally, research was conducted to better understand how primary care providers, including Federally Qualified Health Centers (FQHCs) are currently managing disability competent care. This research sparked some interest in this area and a convening of stakeholders yielded some interest in continuing a discussion beyond the project.
Summary / Project Outcome:
This project added additional features to the provider enrollment/revalidation process to allow providers to self report competencies like languages spoken and proficiency levels. In the future, a report will be available to the eConsult vendor, case management agencies, and regional accountable entities so competencies can be searched to provide additional information about providers.
Disability competency at primary care provider offices and Federally Qualified Health Centers (FQHC) in Colorado was researched. The main barriers to giving and getting disability competent care fall into three areas — limited training, money, and preparation. A workgroup was convened to discuss findings and recommendations. This advisory group will continue after the American Rescue Plan Act (ARPA) period to support efforts around disability competent care and access in Colorado.
✅ Initiative 8.09. - Waiver Quality Expansion - Completed
Key Project Activities and/or Achievements:
ARPA 8.09 expanded the waiver quality surveys administered by the Department of Health Care Policy and Financing (HCPF). HCPF partnered with an independent firm, Vital Research, to assist in the development and administration of two surveys, the Children's Waiver Satisfaction Survey (CWSS) and the Case Management Satisfaction Survey (CMSS).
Summary / Project Outcome:
Given the importance of case managers in delivering services to residents of the state of Colorado, ARPA 8.09 generated the Case Management Satisfaction Survey to obtain a better understanding of case manager's satisfaction with compensation, benefits, career advancement, training, and overall fulfillment in their positions. In addition, HCPF hoped this understanding would aid in recruitment and retention of case managers throughout Case Management Redesign. The final survey included 34 items, where case managers were asked close-ended items about their overall engagement and satisfaction in their roles. In addition, an open-ended item for additional comments was included.
A total of 1,040 potential respondents received a survey link and three reminders to complete the survey via email. Data collection began on 10/27/2022 and was concluded on 11/21/2022. Of the 1,040 invitations that were emailed to complete the survey, a total of 580 responses were received for a response rate of 56%. Using a 95% confidence interval, the margin of error was +/- 2.7%.
Key findings from the Case Management Satisfaction Survey:
- Most respondents had worked for their current agency for one to two years (31%) and 67% had worked as a case manager for three or more years.
- Nearly 80% of case managers are satisfied with their job and 73% are satisfied with their wages and benefits. Yet based on the open-ended comments, compensation was noted most often as an area that is a source of dissatisfaction.
- Case managers are fulfilled in their jobs. Case managers think that the work they do is important (99%), they feel a sense of accomplishment (91%) and they find their job enjoyable (82%). Yet, half of all case managers feel their work environment is very stressful and four in 10 feels emotionally drained from work. This is supported by open-ended comments where burnout, stress and morale were mentioned as contributing to case manager mental health issues.
- Case managers feel they enjoy a good amount of autonomy in their job. Case managers have lots of flexibility (94%), they feel their ideas and suggestions are valued (79%) and they believe their current work environment empowers them to accomplish their work in an effective manner (79%). On the other hand, almost two thirds (63%) of case managers believe they have too much responsibility and not enough authority.
- When case managers were asked how likely they would be to leave their job in the next year, 37% indicated that it was somewhat or very likely. In addition, case managers are also not likely to recommend this type of work to a family, friend, or colleague.
- Over 90% of case managers like the people they work with (99%), feel respected by their co-workers (97%) and supervisor (96%) and believe their supervisor is someone they can trust (94%). Case managers also feel their work is appreciated (83%) and they are respected by upper management and leadership personnel (78%). On the other hand, nearly a third (31%) do not believe they receive adequate resources and materials to execute their assignments, which is supported by workload expectation comments that were mentioned 19% of the time by case managers.
The creation of the Children's Waiver Satisfaction Survey (CWSS) was an effort to expand on the National Core Indicator’s Children’s Family Survey (NCI-CFS) which primarily focused on Colorado’s Children’s Extensive Services (CES) waiver. The new children’s survey received feedback from families and members enrolled on the four children’s waivers, which included the Children's with Life-Limiting Illness (CLLI) waiver, Children's Extensive Services (CES) waiver, Children's Habilitation Residential Program (CHRP) waiver, and Children's Home and Community Based Services (HCBS) waiver. This survey aimed to establish insights into member experience, satisfaction, and report of care needs versus the actual services received. The data will be utilized to recommend changes to the waiver programs and will serve as baseline data to track outcomes of changes in policies and programs in the future.
The project produced two children’s waiver survey instruments, one for members under the age of 18, and one for members 18+. Surveys consisted of 48 items for those under age 16 and 52 items for 16-17-year-olds related to the following topics:
- Accessing + Understanding Child’s Waiver Program
- Case Management + Service Planning
- Child’s Services + Supports
- Family Needs + Supports
- Transition Needs for Young Adults (only answered by those whose children were age 16-17)
- Health Insurance + Other Benefits
- Demographics
The surveys launched in August 2023 and data collection closed in October 2023. 5,259 families enrolled on the Home and Community Based Services (HCBS) children's waivers were sent surveys via mail, and 2,392 families received the survey via email in addition to the mailed survey. 941 surveys were completed and used for analysis; 573 were received by mail, and 361 were received online. Using a 95% confidence interval, the margin of error was +/- 2.9%.
Key findings from the Children's Waiver Satisfaction Survey:
- Generally, results for case management were positive. 90% of respondents reported case managers were responsive to their requests. 89% of respondents were very satisfied or somewhat satisfied with their case manager. 98% of respondents reported their case manager respects their family’s choices and opinions. 46% of families did not experience any turnover in their case manager during the year prior to when surveys were completed.
- All respondents were asked if they had received a copy of their child’s service plan. While 68% of respondents had received a copy of the service plan, 32% had not or did not know if they had received a copy.
- All survey respondents were asked to rate the overall quality of the services and supports their family receives on their waiver program. Eighty-three percent of respondents rated the quality of their services and supports as either excellent or good. Just 17% of respondents felt their services and supports were fair or poor.