In January 2017, Centers for Medicare and Medicaid Services (CMS) implemented the Merit-based Incentive Payment System (MIPS), a new option for physicians to participate in the CMS Quality Payment Program (QPP) that aims to improve the quality of medical care by offering quality-based payments to physicians.
MIPS participation was relatively rare early in the program, and only reached 5–15 percent of eligible physicians by the end of 2017. This resulted in a class imbalance problem: one class of observations in the data (non-participating physicians) was heavily outnumbered by another class (participating physicians). Our team studied the latest development in both data-level and algorithm-level methods to correct for class imbalance problems, and conducted an empirical evaluation of available predictive algorithms.
Initial uses of the findings from this task include: (1) understanding how the multi-dimensional prediction of program participation can be improved with machine learning, (2) identifying the multi-dimensional combination of physician characteristics that predict participation with the highest accuracy, and (3) informing the improvement of program design, such as optimizing the eligibility criteria and rate of downward adjustment for non-participants.
While correcting for class imbalance problems, our team studied the prediction results generated by both conventional regression methods, such as logistic and Probit Regressions, and machine learning methods, such as Kernel Ridge Regression, Naïve Bayes, k-Nearest Neighbor, Support Vector Machines, Decision Trees, and Random Forests. We then compared these models under a comprehensive list of performance measures including precision, recall, accuracy, FF1-score, area-under-curve (AUC), and log-loss.
Skills and Experience
IMPAQ houses experts fully versed in both conventional and sophisticated machine learning algorithms (e.g., economists, data scientists, statisticians, mathematicians), as well as healthcare policy experts attuned to CMS' payment innovations.
The IMPAQ team for this project consists of Senior Researchers, Economists, and Data Analysts with expertise in statistical and machine learning methods, data programming, and QPP MIPS.