NAHQ Annual Report 2011: What’s Not Being Said About the CPHQ Program

A couple of weeks ago, I received a copy of the National Association for Healthcare Quality (NAHQ) 2011 Annual Report. But I’ve been too busy with work to review it until today.

As with many annual reports, this had far too much marketese. This finding was well within expectation and, therefore, did not disappoint. However, the same cannot be said of the section on CPHQ Certification on Page 17 of the report:

I bet the last bar (the rightmost one) caught your attention. It did mine.

So let’s examine the data a little further. From 2010 to 2011, the total number of CPHQs had risen from 7099 to 7810 – that’s a difference of 711 CPHQs (people who were certified) or a 10% rise from the 7099 CPHQs at the end of 2010. I don’t know about you, but that seems like a dramatic rise, particularly when you take into account the fairly flat numbers in the preceding few years. The graph is deceptive because the scale on the vertical axis starts from 6200 instead of zero (as it should).

For a discussion of why this is a no-no when presenting data (let alone the fact that the audience consists mainly of quality professionals), read Ezra Klein’s “Lies, damn lies, and the ‘Y’ axis” and/or Page 2 of this document entitled “Misleading Graphs and Statistics.”

To put things in their proper perspective, I redrew the chart above using the same data but correcting the scale on the vertical axis:

Ah, much better! We can now see a much more accurate picture. After several years of plateaued (read: negligible) annual growth in CPHQs, there was a 10% rise between 2010 and 2011. I think anyone will be impressed by a 10% growth over a 1-year period. I was.

The next question: what contributed to the significant and sudden growth? Here is where things get hairy. The explanation is given on Page 17 in the annual report (above the chart):

The CPHQ program continued its upward trend in 2011, welcoming more new CPHQs and enjoying a strong renewal rate. The program enjoyed a 10% growth rate over the past year. This growth comes from two factors: a 40% increase in candidates sitting for the CPHQ exam and a strong 87% recertification rate. Enhanced online information and access, as well as increased communication with certificants, is reflected in this growth.

Firstly, let’s get something straight: the “upward trend” before 2011 was negligible, as explained earlier in this article.

Secondly, the two factors given to explain the 10% growth do not add up. Just as I did last year when I reviewed the 2010 NAHQ Annual Report (which showed inaccurate numbers in a bar chart) I whipped out my electronic calculator to confirm or debunk my suspicions the numbers and/or explanatory text are wrong.

In case you were not aware, CPHQ recertification follows a 2-year cycle and is due on December 31 of the year the current certification expires. Therefore, if you were certified in September 2012 (this month), you will only need to recertify by December 31, 2014.

Just by looking at the data and text in past annual reports, you will be able to estimate the number of CPHQs who recertified in 2011. A conservative estimate of CPHQs who were due to recertify was 7080 (2009 figure). The number is probably much lower, i.e. less than 4000, but let’s assume it’s 7080 for now. In fact, these figures are extremely conservative if you take into account what was written in the 2008 NAHQ Annual Report [PDF – 5.66 MB] (Page 13):

Recertification: For the expiration date of December 31, 2008, there were 3,375 CPHQs due to recertify.

An 87% recertification rate means that 6160 (i.e. 87% of 7080) CPHQs recertified. (By the way, that also means 920 out of 7080 CPHQs did not recertify… Why?)

If there were 7810 CPHQs in 2011, and if 6160 of them were recertified CPHQs, then it means 1650 (7810 minus 6160) CPHQs had just passed the CPHQ exam in 2011, i.e. there were 1650 new CPHQs (!?!). Now I’ve been around long enough to know that that number (1650) cannot be real – it’s too good to be true!

If the recertification rate was 100% in 2011, 7810 CPHQs in 2011 would mean that 711 candidates passed the exam (as opposed to sitting the exam). In 2010, 547 candidates passed the same exam. Seven hundred and eleven (711) new CPHQs in 2011 represents a 30% increase from 2010 to 2011! It’s not impossible but, in the context of lackluster growth in recent years, a 30% change will surely raise eyebrows. Coincidentally, assuming a 100% recertification in 2011 (which I am certain is impossible) and a 30% increase in the number of certificants in 2011 (which is possible) will be consistent with the bar chart above, i.e. a difference of 711 CPHQs between 2010 and 2011. And even if only 2000 CPHQs needed to recertify in 2011 and “only” 13% did not do so, that would mean we had to see an additional 260 fresh certificants in 2011 – a 47.5% increase compared to 2010 – just to keep the total number of CPHQs constant.

So let’s take a look at the other “factor” that was given as the reason for the 10% growth in CPHQ numbers – “a 40% increase in candidates sitting for the CPHQ exam”. In the 2010 NAHQ Annual Report, the total number of people taking the exam was 811. Of these candidates, 547 passed. A “40% increase in candidates sitting the CPHQ exam” is equivalent to only 1135 (1.4 × 811) people. And not all of them would have passed. In fact, the historical annual pass rate is only around 68.5%. Even if there was a typo error and the 2011 Annual Report meant to say a 40% increase in the number of people who passed the CPHQ exam, that would only contribute 766 CPHQs to the 2011 statistic. Seven hundred and sixty-six (766) new CPHQs will not make up for those CPHQs who did not recertify, and I’m absolutely sure there were more than 55 (766 minus 711) of them in 2011.

Bottom line: the numbers do not add up. Your next question is probably, “Where is the error?” The simple answer is “I don’t know.” But here are some possibilities:

• The total number of CPHQs in 2011 is not 7810 as stated in the chart. This seems the most likely since the numbers have been flat for several years. By this, I do not mean to imply there was no growth in CPHQs. In fact, based on the demand for my CPHQ exam preparation services, as well as the strong growth in demand of the Teh & Associates online CPHQ exam preparation products, I would be extremely surprised if there was no increase in CPHQ numbers in 2011. I also note from comparing the 2010 and 2011 annual Reports, “expenditure” on certification activities were more than $110,000 in 2010 compared to 2011. Increase in marketing in 2010 may be part of the reason for an increase in CPHQs in 2011.
• The explanation for the 10% growth in total CPHQs between 2010 and 2011 is incomplete.
• The explanation for the 10% growth in total CPHQs between 2010 and 2011 is inaccurate.
• All of the above.

In any case, the situation is troubling. Particularly in the context of an annual report of a quality organization.

When presented with surprising data, an intelligent person won’t take them at face value. He/She will ask if the data are valid and what the most likely reasons for the deviation from expectation are. Is there something missing? It’s this last question that makes this situation most interesting, given what’s been happening with the CPHQ program in the last 18 months or so.

What was the relative growth in US and international candidates in 2011 compared to 2010? (I expect relatively stronger growth in international candidates.)

What was the relative growth in US and international certificants in 2011 compared to 2010? (I expect relatively stronger results in international candidates.)

Was there a sudden and dramatic increase in the number of persons who attempted the CPHQ exam because of the announcement that a new exam content outline would be introduced in 2012?

Is the NAHQ/HQCC withholding other information that may not reflect well on decisions they made in the last two years?

Given the paucity of information that makes sense, the following quote by the late management guru Peter Drucker is pertinent:

The most important thing in communication is to hear what isn’t being said.

— Peter F. Drucker (1909–2005)