Quality Control

‘Benchmarking’ is crucial to
developing high performing,
sustainable healthcare systems

By Simon Leary,
PwC’s Middle East Health Industries Leader
John Middleton,
PwC’s Middle East Health Industries
Account Manager

High performing hospitals, primary care clinics, rehabilitation centres, regulators, pharmaceutical suppliers and health ministry’s constantly ask themselves: How well are we performing? What does excellence look like? Where are the variations in our performance? Are we continuously improving? How are we performing in comparison with our peers? The asking of these questions reflects a common drive to compare themselves to their peers and indentify and pursue excellence.

Indeed, it is only by answering these questions that: a hospital knows its current level of performance in terms of achieving cost effective quality healthcare outcomes; a ministry or insurer knows how effectively it is commissioning care and serving the needs of the population; a regulator knows whether it is effectively regulating the quality of service provision and, importantly, patients know what their healthcare options are. The answers lie with benchmarking. Benchmarking allows current performance to be compared to local, regional and international best practice, it gives insights into what excellence looks like and gives you a push by showing how it can be achieved based on others successes.

One example of the power of benchmarking, cited by Atul Gawande a prominent US surgeon, lies with the advancement of Cystic Fibrosis (CF) care over the last 10 years in the US. A decade ago CF patients faced an average life expectancy of 33 years. At the time, CF care providers were heavily regulated, requiring highly specialised facilities and clinical teams, following similar clinical protocols, and as a result were widely regarded to be performing at the same high level. However, driven by the pleas of stand-out CF clinicians and a couple of outstanding facilities, this widely held belief was called into question, resulting in a benchmarking exercise that compared a number of CF providers across a range of outcome metrics.

It was revealed that while the average life expectancy across all the providers was 33 years, there were a small number achieving an average life expectancy of 47 years. Differences were similarly large across the suite of outcome metrics, regardless of risk adjustment, and they shocked the CF community. Following a heated debate it was decided that there should be transparent benchmarking of outcome metrics across CF providers.

What followed is a remarkable testimony to the power of benchmarking. Those providers who achieved “below average” outcomes, many of whom who up until this point had believed they were no worse than average in a narrow range, were given a large incentive to improve. The transparency of the benchmarking allowed them to visit those providers who were excelling and learn from their successes, while the high performing facilities, buoyed by their success, were determined to maintain their position ahead of the curve. Insurers were better informed of the care they were commissioning and regulators realised the power of effective outcome metrics. Finally, and perhaps most importantly, was the impact on patients. Far from the low performing providers being stripped of all their patients, many patients and their families actually reinforced providers’ determination to improve care by accepting their promises to improve and holding them accountable if they did not. This culture has led to many CF sufferers in the US now living well into their 50s/60s.

However, the CF example is often cited, given its simple outcome metrics and small specialised field. In order to garner benefits/ insights on a larger scale, such as across general hospitals, regional healthcare models and national systems, a real understanding of how to run effective benchmarking exercises is required. For every success story, such as the field of CF, there are numerous examples where benchmark studies/exercises have proved to be far from helpful.


The first key issue is to decide what you want to benchmark and what is the exam question you are trying to answer? There are hundreds of different possible metrics, whether you are a provider or a regulator within the healthcare industry. Benchmarks can cover clinical outcomes, financial performance, service demand based on demographics, organisational structures, optimum workforce, models of care, service utilisation and so on. The right metrics have to be chosen, based on what you are trying to achieve, what data you are currently and/or planning to collect, and what local, regional and international metrics are available to benchmark against.

Following on from this, it is crucial to decide who/what you want to benchmark against. It is a common failing of benchmarking exercises to compare incomparable facilities, organisations, systems and populations. At the macro level it is important to ensure that you benchmark against an organisation facing similar system wide challenges. For example comparing organisations operating under the mainly private insurance system of the US and the NHS in the UK could be misleading as they are facing different national system based pressures. Similarly at the individual level, it is important to compare like for like. Benchmarking the cost of service provision between a primary care clinic and an academic medical centre is of limited use due to different cost bases and types of services provided. Benchmarking is only credible, useful and a driver for improvement when the two organisations face similar outlooks and goals on the macro and individual basis. A good example of where benchmarking fell down due to a lack of standardised metrics and overwhelming local differences is a national community benchmarking exercise that was recently carried out across a European national health system. The differences in regional population need due to demography and socio-economic factors, combined with a national lack of community level financial and clinical outcome data made the exercise untenable. The results were quickly picked apart by providers as they asked the obvious question – how can you compare us with them when we are facing totally different circumstances!

Another crucial aspect of a successful benchmarking is engaging the right individuals and organisations. As a quality control factor it is vitally important. What might look like a good clinical outcome metric to a national regulator may be far from it when viewed by a clinician. This is the same at whatever level you are running the benchmarking. Benchmarking relies on being credible and evidenced, whether from the point of view of a financier, clinician, provider, regulator or pharmaceutical company etc. Without the engagement of those you are benchmarking, the exercise can quickly lose all credibility – this is particularly true when looking at clinical outcomes. Such metrics require clinical involvement to ensure that the metrics are applicable, accurately recordable and useful.


The final critical element of a benchmarking exercise is how you engage the right stakeholders and the levers you develop to ensure buy-in and belief in the exercise. While national benchmarking exercises can carry the weight of regulatory requirements and accreditation checklists, there are examples of where informal benchmarking has achieved great success. Dr Devi Shetty is a prominent surgeon who has set himself the task of performing open heart surgery for under a US$1,000 in India. He is going about this exercise by engaging global clinicians in the challenge, relying on their informal but professional competitiveness to drive an exercise that is comparing different procedures based on clinical and financial metrics. Such a benchmarking exercise, while not on a national scale, shows the power of engagement and the use of levers outside national regulations. Coming back to the CF example, much progress was achieved through clinicians informally comparing their results, driven by a professional yearning for improvement mixed with the embarrassment of being identified as one who did not want to partake in a patient benefiting initiative.

Benchmarking is key to encouraging continuous improvement across healthcare industries and developing the building blocks for developing a health system based on international best practice. At every level it is important, whether you are managing a single operating theatre or creating a national model of care. However, for it to be a driving force for improvement, it has to be applicable, credible and evidenced with secure stakeholder buy-in, developed levers (whether informal or formal), data based and focused on achieving the local or national vision. Benchmarking for benchmarking sake is not useful and can even be damaging to operational and strategic performance by diverting attention away from what really matters. When done properly it is a crucial driver of excellence and sustainability, which all organisations should carry out periodically and act upon the results.

Published with permission from PwC.
© 2012 PricewaterhouseCoopers


PwC firms provide industry-focused assurance, tax and advisory services to enhance value for their clients. More than 161,000 people in 154 countries in firms across the PwC network share their thinking, experience and solutions to develop fresh perspectives and practical advice. Established in the Middle East for 40 years, PwC has offices in Bahrain, Egypt, Iraq, Jordan, Kuwait, Lebanon, Libya, Oman, Palestine, Qatar, Saudi Arabia and the United Arab Emirates, with around 2,500 people. Find out more at pwc.com/middle-east. “PwC” is the brand under which member firms of PricewaterhouseCoopers International Limited (PwCIL) operate and provide services. Together, these firms form the PwC network. Each firm in the network is a separate legal entity and does not act as agent of PwCIL or any other member firm. PwCIL does not provide any services to clients.

 Date of upload: 26th Sep 2012


                                               Copyright © 2012 MiddleEastHealthMag.com. All Rights Reserved.