How to Pressure-Test a Benefits Vendor's Savings Claim

Key Takeaways

• Most benefits vendor savings claims aren't wrong so much as impossible to check, because vendors rarely show how they got to their number.
• Six questions about how a vendor calculated its savings, from the comparison group to the fees, show whether a claim is worth trusting.
• Garner pairs provider-quality analytics with savings measured in an independent Aon study, and buyers should hold that data to the same six questions as any other vendor’s.

A savings number from a vendor is a claim, not a result. Every vendor in a benefits evaluation brings one, and they rarely calculate it the same way, so the numbers can’t be directly compared.

No agreed standard governs how vendors calculate savings, and the common methods leave so much room for choice that a program with no real effect can still produce a number that looks defensible. So the number a vendor shows you matters less than how it was calculated. But six questions will show you how any savings claim was calculated, and you don’t need an actuary to ask them.

Why most benefits vendor savings claims can't be verified

When a vendor reports its own results, it makes four decisions that can swing the final number:

• Who the program gets compared against
• How long to measure
• Which costs count as avoided
• How the analysis is run

 Most vendors make those choices in their own favor. They want to look good, their competitors are doing the same, and no one asks them to show how they got their number. All you see is the final claim, not the choices behind it. So the check falls to you.

Employer coalitions treat this as a buyer’s problem, and the National Alliance of Healthcare Purchaser Coalitions has written about independent validation as a way for employers to confirm or refute what vendors claim. Skipping the check costs more every year that healthcare costs keep rising.

The six questions to ask about any savings claim

Ask these six questions about any vendor’s savings claim. Each one is followed by the answer you want to hear and the answer that should give you pause.

1. Who was the comparison group?

This is the most important question, and vendors often answer it with a comparison that favors their program. Employees who sign up for a benefits program tend to be more motivated, better at making sense of health information, and already more involved in their own care than those who don't. So when a vendor compares members with non-members, the savings include those differences in motivation and behavior, and the vendor is claiming credit for people who would have spent less anyway.

What to ask: Who did you compare our members against, and how did you build that group?

A strong answer: The vendor built a matched cohort built before the program started, using traits such as age, geography, and diagnosed conditions. Comparing everyone eligible against a similar population also works, because it doesn't depend on who chose to take part in the program.

A weak answer: The vendor says members who used the program spent less than members who didn’t. Another warning sign is a vendor that chose its comparison group after seeing how members did. That lets it pick whichever group makes the savings look biggest.

2. Is the number gross or net of fees?

A program can produce real savings and still come out close to even once the vendor's own fees are subtracted.

What to ask: What were the savings before your fees, what were they after, and can you show how you arrived at your final figure?

A strong answer: The vendor shows both figures in dollars, the fee schedule behind them, and any other costs the program adds, such as incentives the employer funds.

A weak answer: The vendor quotes a savings percentage or a return-on-investment multiple, such as 4 to 1, without showing the fees or the dollar amounts it was based on.

3. Over what time period was it measured?

Members who have an expensive year usually have a cheaper one next, whether or not anything changed. Actuaries call this regression to the mean. A program that enrolls its highest-cost members and measures them for a few months can record that natural drop as its own result.

What to ask: How long was the measurement period, and what happened in the years after the first one?

A strong answer: The vendor shows results for each year, measured the same way every year. A multi-year contract only pays off if the savings last, and only several years of results can show that.

A weak answer: The vendor shows savings from a single quarter, or a first-year figure with no later years.

4. How large was the population?

Small groups produce dramatic results in both directions. A vendor with dozens of clients will have a few clients whose numbers look spectacular, and those are the ones that turn into case studies.

What to ask: How many members were in the program group, and how many were in the comparison group? Don't accept the vendor's total membership as an answer.

A strong answer: The vendor gives you the number of members in both groups, the number of employers included, and who was left out and why.

A weak answer: The vendor quotes a percentage without saying how many members it's based on.

5. Who ran the analysis?

Think of evidence as a ladder. A vendor's own analysis sits at the bottom rung, followed by a third-party review the vendor commissioned. Then an independent study, and at the top, peer-reviewed publication. A commissioned study can still be useful, though. What matters is whether the analyst controlled the method and whether the method is published. Some vendors also tout a certification, such as one from the Validation Institute, which reviews a vendor's reported results and awards a credential if the claims check out. That’s a useful signal, but a badge doesn’t show how the number was calculated, so ask to see the details either way.

What to ask: Who ran the analysis, who paid for it, and can we read the full report?

A strong answer: The vendor names the analyst, states plainly who commissioned the work, and gives you a report you can hand to your own consultant.

A weak answer: The vendor describes a study but won't share it.

6. What counts as a saving?

Vendors can claim savings in three different ways. Cost avoidance counts procedures that never happened, lower unit prices cut what each service costs, and a drop in total spend means the plan paid less overall. Only the last one shows up when you add up what the plan paid. A vendor can lower the price of an MRI while total spend stays flat because more MRIs get ordered.

What to ask: Which line on our financials should change if the program works, and by how much?

A strong answer: The vendor points to a specific line, such as medical spend per member per year, and gives a number you can check against your own claims at renewal.

A weak answer: The vendor claims savings from avoided procedures that were never scheduled.

How the same claims data produces three different answers

Take a hypothetical employer with 10,000 members, average medical spend of $6,000 per member per year, and a program that costs $60 per member per year. The figures are illustrative, but vendors commonly report savings each of these ways.

Only the third row shows real savings because it compares similar people over a full year and subtracts the vendor's fee. The first row mostly reflects who chose to join, since those members were already spending 20% less before the program started. The second row mostly reflects regression to the mean, because high-cost members tend to cost less the next year anyway. The program is the same in every row. Only the method changed.

What a credible savings claim looks like

A credible savings claim comes with a written report, not just a slide. A strong report includes an analysis run by someone other than the vendor, a matched comparison group with the matching criteria written down, results for more than one year, savings stated after fees, and a method your own actuary or consultant can check. Not every vendor will have all five, so count what you can get and ask about what's missing.

Garner built its evidence to meet this standard, and it should be held to the same six questions as any other vendor. Garner identifies the best-performing doctors in an employer's existing network and helps cover employees' out-of-pocket costs when they see one, so its results should show up in total medical spend rather than in estimates of avoided care. In a Garner-commissioned analysis, Aon compared 5,583 Garner-eligible members from three employers who used medical services in both the year before and the year after implementation, with a control group from a multi-employer claims database, matched on geography, demographics, and clinical comorbidities. In the Aon study, Garner-eligible members had 7.4% lower medical spend than the control group in the first year after implementation, or $345 per member per year on average. Aon's figures cover medical claims only and do not include Garner's fees, employer-funded incentives, or pharmacy costs.

How to run this evaluation without an actuary

A benefits vendor evaluation doesn't need an analytics team, but it does need the right documents in writing. In your RFP, ask every vendor to provide these four items:

• The full methodology document
• The definition of the comparison group
• Results for every year measured
• The fee schedule, shown next to the savings figure

Advisors can take the review further for their clients. Ask your advisor or consultant to run three checks on every proposal. Was the comparison group built before the program started? Do the savings survive once fees and employer-funded incentives come out? And does the population in the study look like yours in size, age, and health?

How a vendor responds when you ask tells you something too. Vendors with a solid method usually hand it over readily, while those that offer only a summary deck may be hiding a number that won't hold up under scrutiny.

Ask better questions, buy better outcomes

A savings number you can't check is just a number, however big it is. Ask every vendor the same six questions, including the ones you already work with and like. Advisors can use the same list with their clients.

Run these questions on us. We'll walk your team or your advisor through the full methodology behind our savings figures, including what the Aon study did and didn't measure.

FAQs

How do you evaluate a benefits vendor's savings claim?

In a benefits vendor evaluation, start with how a savings claim was calculated before you look at how large it is. Find out who the program was compared against, whether savings are net of fees, how long results were measured, how many members were included, and who ran the analysis. A claim that holds up comes with a written methodology your consultant can check.

What is a matched-cohort study in healthcare benefits?

A matched-cohort study compares members in a benefits program with a separate group of people who resemble them on measurable traits, such as age, gender, geography, and diagnosed conditions, but who didn't have the program. Because each member is paired with a similar person, spending differences are more likely to come from the program than from who chose to enroll.

What's the difference between gross and net savings in a vendor proposal?

Gross savings are the drop in medical spend a vendor credits to its program, while net savings are what's left after subtracting the vendor's fees and any costs the program adds, such as employer-funded incentives. A program with solid gross savings can land close to break-even once fees come out, so ask for both numbers.

Why do vendor savings claims vary so much between vendors?

Vendor savings claims differ so much because each vendor chooses its own comparison group, measurement window, population, and definition of a saving. Those choices can turn the same claims data into very different numbers, so two figures can't be compared until the methods match. Ask each vendor to show its method in writing, and compare the numbers only after you know what each one measures.

What should employers ask for during a benefits vendor RFP?

During a benefits vendor RFP, employers should ask for the full methodology document, the definition of the comparison group, results for every year measured, the number of members in each group, and the fee schedule alongside the savings figure. A vendor's willingness to share them is useful information on its own.