Risk Stratification

This affinity group includes @NEVHC @LifeLong @CMC. Please feel free to share your group’s progress and updates below!

Hi PHLN Affinity Group
NEVHC is using the NACHC Risk Stratification Tool from i2i Tracks which uses the Counting Chronic Conditions methodology.
We have generated the report for our Diabetic Patients for all sites and by Health Center. I am attaching our results. I was surprised to see so many of our Diabetic Patients fell into the lower risk tiers, 1 and 2 (kind of happy about that) and very few in the highest tier 4. Our first step is to work with our teams to validate the assigned risk score using feedback from providers and care team staff.
Do you want to respond to this forum and/or set up a call?DM Risk Stratification by Site.xlsx (10.0 KB)

1 Like

Sounds like you are off to a great start! I’m not familiar with that particular tool. Is it based exclusively on number of conditions, or does can you add more risk factors? If the former, it seems like a good way to identify patients for further screening, the way Petaluma uses their tool to refer patients to CHWs for PRAPARE. We are thinking about a similar process. We have free access to ACG risk scores, so we may use those scores (which predict future utilization based on demographic factors, past utilization, diagnoses, and medications) as a starting point and add SDOH factors to assess eligibility for case management. We will also be looking at a bunch of other organizations’ risk stratification tools (Axis Community Health, Cal Optima, MemorialCare, Petaluma).

1 Like

Yes, our risk stratification tool is based exclusively on counting chronic diseases. It was not my first choice, but the one that seems to make sense at this time. We will be validating this tool (we cannot add more risk factors) However, it has been shown to be associated with high annual costs as well as persistence in high costs and was sited in AJMC Risk Stratification Methods for identifying patients for care coordination Sept 17, 2013. The CCC tool is based on the total sum of selected comorbid conditions and is publicly available Agency for Healthcare Research and Quality’s Clinical Classification of Software. By the way, we are testing Eagle Dream and utilized the ACG risk score. We were surprised that both for peds and adults, our risk categories were shaped like a “diamond” rather than the pyramid me expected. We didn’t know where to go with this information. I would be very interested in learning about other risk stratification tools, and will let you know the results of our validation efforts.

1 Like

That sounds like a good place to start. We are in a similar place (starting with what we have available, even if it’s not ideal). Did you find that your results from Eagle Dream (with ACG) are similar to what you found with CCC?

We are meeting with Axis next month and soon will start the type of analysis that you’ve already done with your diabetic patients. We’ll share what we learn as we get further along. In the meantime, I’m attaching descriptions of the other risk stratification tools I mentioned.1.1.3MemorialCare_PatientIdentificationRiskStratification.pdf (395.8 KB)
1.2.2Humboldt_DomainsScoringLevels.pdf (191.3 KB)
Cal Optima risk stratification tool.pdf (233.3 KB)

1 Like

NACHC Risk Score DM Patients.docx (13.7 KB)

I am attaching what we found for risk scores for our DM patients using CCC. It is VERY different than the Eagle Dream (with ACG) Very different. using CCC, we clearly came out with a Triangle with very few patients at the highest risk. This is what we expected. With ACG, we came up with a Diamond (even for our peds population). Their methodology is very different and the results are different. They spent a fair amount of time explaining it to us, and it sort of made sense, but we still don’t know how to develop a care coordination model using a Diamond. I will look at the other tools you sent, thanks. And I will share our data validation of the CCC tool with you.

1 Like

Hi everyone,

Based on the doodle poll, Thursday, June 20 from 10am-11am works the best overall for our next risk stratification affinity group meeting. As a reminder, it would be * ideal * to have at least 2 individuals from each team participate, and more are welcome.

A couple of quick reminders:

  • I’m sending out a calendar invite to everyone on this email, but of course decline if you can’t make it or have delegated attendance to others in your group.
  • We’ll be using Zoom and I’ll be using my camera, and it would be great if others (as you are able) plan to use yours too so we can see each other.

We’ll be joined on the call by:

  • Sarah Campbell, Improvement Advisor at La Clinica in Medford. She can share more about La Clinica’s process, about how the methodology came together - who was involved, how was it tested, how did you come up with diagnoses to include, etc.
  • CCI’s ED Michael Rothman. Michael built risk adjustment models for The Permanente Medical Group (TPMG).

As a reminder, here are the slides from La Clinica’s presentation on risk stratification & care coordination from the site visit: https://www.careinnovations.org/wp-content/uploads/Risk-Stratification-Care-Coordination_La-Clinica_Site-Visit-10.16.18.pdf

In advance of the call, please send any questions or topics you’d like to discuss with Sarah or Michael to Megan (mobrien@careinnovations.org) by June 17 .

Thanks all!

@NEVHC @LifeLong @CMC @meaghan_cci @michael_cci

Hi everyone! Thanks for a great call last week! A special thank you to Sarah from La Clinica and @michael_cci for joining us.

You can view the rough notes from the call here. Since I didn’t record the call I took as verbatim notes as possible (please excuse any typos or things I may have written down wrong).

Some key takeaways/advice I’ve pulled out from the notes are:

  • Remember that the tool is just a tool; a guide. Showing staff that you trust and listen to their thoughts around patient categorization/risk assignment helps staff see that their opinions/knowledge of the patients matter.

  • This is not about fancy statistics or getting the number exactly right analytically. The priority is to develop ways to define patient groups and match intervention(s) to the patients that work. It could be a number or it could be a signal word that signals an intervention to address gaps in care.

  • If your tool just says everyone is high risk, then it’s not helping you figure out where you can put the work in with a subset of patients.

  • Huddles are a good way to talk about patients coming in the following week that need care plans.

  • As with anything else - do PDSAs/small tests of change. Make a plan of how to do tiny adjustments.

Next steps:

  • Connect via Sarah to La Clinica’s Epic expert (LifeLong)
  • See if there’s a regular meeting schedule that works for everyone

Possible future discussion topics:

  • Capturing SDOH and incorporating SDOH into risk scores

  • Stratifying risk for behavioral health

  • Connecting ED utilization data

  • Topics beyond the tool: care coordination, staffing, care plans, measuring patient satisfaction and engagement, care team approach to this work

@NEVHC @CMC @LifeLong @megan_cci

Here are some thoughts and guidance from @michael_cci:

1. Goal Clarity. It is critical to identify the problem or goal for risk stratification. Risk stratification isn’t a clear enough goal, it needs to be more specific. It might be to “identify higher cost patients who we can help by both improving well-being and reducing total cost.” Or it might be to “distribute predicted primary care workload evenly across primary care panels.”

  • It is VERY important to be as specific as possible about the goals because different goals drive different approaches
  • A 2x2 matrix can help if they are focusing on segmentation: total cost (or however the clinic is defining cost on one dimension) and “confidence that we know what to do to decrease cost” on the other dimension.

2. Dependent variable clarity. - this is VERY important

  • If you are trying to balance work among PCPs, you should focus on a “future PCP work at our health centers” dependent variable.
  • If you are trying to reduce total cost with a health plan partner, you should focus on a health plan total cost dependent variable
  • The off-the-shelf risk adjustment scores (ACG, Charleston) are usually predicting total resource use or mortality. These might or might not be the right dependent variable for you.
  • You can use your own dependent variable (total cost from your health plan, or total visits at your health center) and then use the risk score (from ACG or Charleston) as an independent variable.

3. Dependent variable selection .

  • You should test relationships between independent variables and dependent variables statistically (and make sure those are stable relationships by reserving part of the data set for validation and testing the relationship over different time periods).
  • It’s not good to just assume a relationship or credit for a factor

4. Selecting Risk Factors
Selecting risk factors (or independent variables in a regression model) should be based on two approaches:

  • What makes clinical or business sense (face validity) and what has a potential logical relationship with the dependent variable (thing you are trying to predict)?

  • What statistically predicts the dependent variable. Simple regressions (rather than really fancy ones like Cerner or Hopkins might do) can be used to test the fit of different variables. Look at what the model predicts and see if it is right (is it predicting true positives?); and, look at some patients who you BELIEVE are high risk and see if the model also rates them as high risk.

5. Evaluation of interventions

  • It is very easy to falsely believe that a care management intervention focused on “high risk” cases is effective. On average, almost all currently high cost groups will become less expensive in the future. If you are testing an intervention with a high cost/high risk group, it is good to reserve a randomly selected part of that group and then do a “difference of difference” analysis to see if the intervention groups’s costs or other outcomes improved more than the control group. The control group’s costs will decline a lot without any intervention.

@NEVHC @CMC @LifeLong @megan_cci

Hi all @NEVHC @CMC @LifeLong,
Here’s follow up from our July call and a variety of resources I’ve pulled together that relate to topics you brought up on the last call: