The GLC speaks to experienced NFP director and former charity analyst, Justine Jarvinen, about the challenges of measuring performance in not-for-profit organisations.
Assessing the performance of not-for-profit organisations can be challenging. The Governance Leadership Centre recently spoke to Justine Jarvinen GAICD about this important topic. Justine has substantial expertise in the not-for-profit and charitable sectors. Her work has shaped the public debate and best practice on charity impact and reporting.
Governance Leadership Centre (GLC): How should NFP boards assess the performance of their organisation?
Justine Jarvinen (JJ): For a not-for-profit, success is having an impact. Every not-for-profit has a mission and success is achieving that mission. It might be finding a cure if you’re a medical charity, protecting vulnerable habitat if you’re an environmental NGO, or connecting people with jobs through proven employment pathways for social enterprises.
I believe that a charity’s ultimate success is that it can ‘shut up shop’; that is, it has either fulfilled its mission, or has proven what works such that its activities become mainstream and government funded. (Of course, not all NFPs are charities with a long term ‘end’ goal. For example, the mission of a membership association or club may be to serve its members on an ongoing basis.)
Achieving a mission is rarely straightforward and usually takes a lot of time and many steps along the way. Long term outcomes or impact may be lofty ambitions such as improving society, the environment or the economy. But the path to long term impact might well be via short-term outcomes (e.g. improving awareness, attitudes and knowledge) and medium-term outcomes (e.g. improving policies, practices and behaviours). Measuring short-term and intermediate outcomes can demonstrate that an organisation is making progress.
…NFPs should report on their impact as a way to ‘pay back’ people’s trust and investment: why should anyone donate time or money to an organisation that can’t demonstrate it makes a difference?
Boards should acknowledge that success is different in the short, medium and long term, and use that to guide impact measurement.
GLC: Why should NFPs measure and report on their impact? What benefits can it bring if it’s done effectively?
JJ: NFPs that measure their impact report a range of benefits. These include:
- improving services by better understanding what works and what doesn’t;
- obtaining more funding from donors;
- motivating and inspiring staff; and
- influencing the debate about what works and contributing to sector-wide improvements.
I’m also of the opinion that NFPs should report on their impact as a way to ‘pay back’ people’s trust and investment: why should anyone donate time or money to an organisation that can’t demonstrate it makes a difference?
Performance measurement data can also give a voice to the vulnerable people that the organisation is trying to help.
GLC: It is often said that it is difficult for NFPs to effectively measure and report on their performance – do you agree with this? How should organisations get started?
JJ: Yes, the main barriers to measuring impact are a lack of funding or resources, not having the right skills, not knowing what to measure, and not knowing how to measure it. My former employer, New Philanthropy Capital (NPC) in the UK, suggests 4 pillars to build an effective measurement framework.
The first pillar is to map the organisation’s theory of change – this is a hypothesis that shows how the organisation’s current activities will create long-term impact. There are several ways to represent a theory of change; my favourite is a logic flow diagram, showing how your activities can lead to intermediate outcomes and then to achieving the mission.
The second pillar is to prioritise what is measured and to focus on what matters. NPC says to measure outcomes that the organisation directly influences, are important to its mission, are not too expensive to measure, and will provide credible data.
The third pillar is to choose an appropriate level of evidence and be pragmatic. Some measurements of impact are straightforward, such as before/after surveys, interviews and feedback forms. Economic analyses, such as Social Return on Investment, are more difficult. More complex measurements include longitudinal studies, control groups and randomised trials.
It can be helpful to look at the accepted minimum in a sector (randomized control trials in medicine v. statistical correlations in education); use short term feedback if the organisation is piloting and trialing programs and needs feedback loops; and use longer term feedback if the organisation has a stable program that it wants to measure.
The fourth pillar is to select appropriate sources and tools, and to try not to reinvent the wheel. Collect data from people that the organisation interacts with, such as beneficiaries, staff, practitioners, policy makers and funders. Quantitative data could come from surveys, psychometric scales, casework records and statutory data. Qualitative data could come from interviews, observations, focus groups and anecdotes.
GLC: What are some non-financial metrics that NFP boards commonly use to track and measure their organisation’s performance?
JJ: If outcomes and impact are what needs to get done, then outcomes and impact are what needs to be measured.
Some NFPs start and stop their measurement efforts at inputs. Inputs are what we invest: time invested, money raised, equipment and facilities bought, staff hired, and brand awareness etc. For example, let’s take a charity that provides a before-school breakfast club for disadvantaged children who might not otherwise have breakfast before school. Inputs could include how much money was donated or how much food was provided by corporate partners.
Some NFPs measure success by benchmarking outputs: the amount they spend delivering services or the number of people using their services.
Outputs are what we do and who we reach: activities, programs and beneficiaries. For our breakfast club charity, outputs might be the number of students who ate breakfast.
Measuring inputs and outputs is helpful but not sufficient. It is important to consider outcomes and impact. For our breakfast club charity, outcomes could be that more children attended school each day because they turned up for breakfast and then stayed the entire day. Or that the students had better concentration levels after breakfast than previously on an empty stomach. Impact is that those children who attended the breakfast club have higher literacy rates, go on to complete schooling, or go on to employment opportunities.
The principle ‘do no harm’ is also important. For example, short-term mentoring can leave vulnerable children worse off than before. Measuring inputs and outputs wouldn’t necessarily uncover whether harm is done, but measuring outcomes and impact could pick this up.
GLC: What are some financial performance metrics that NFP boards commonly use?
JJ: NFPs often run on tight budgets, and it is important to monitor financial resilience and resources. Monitoring profit and loss, cashflow and balance sheet reporting is obviously essential. It’s particularly important to understand whether your NFP can withstand temporary or systemic fluctuations in income. NFP boards should assess the types and variety of their income, avoid over-reliance on a single source of income, and properly account for restricted funding (to particular projects). I like to keep an eye on what the unit costs of service delivery are, whether we are recovering the costs of delivering each service, and that we are getting a decent return from each of our fundraising activities.
NPC has published a pragmatic report about assessing what makes a good charity.
GLC: What is an example of a NFP that has good performance measurement processes/systems?
JJ: A UK domestic violence charity that I analysed ran accredited training programs for advocates. They measured and communicated their results so well that they brought about tangible changes in the UK justice system that have since been replicated in other countries.
GLC: What is an example of a NFP that has good performance reporting practices?
JJ: I’m director of a social enterprise called Pollinate Energy, which provides life-changing products to people living in India and Nepal, trains and supports a salesforce of micro-entrepreneurs, and runs fellowships about operating a social enterprise. The organisation undertakes robust and comprehensive monitoring and evaluation of each of these strands of work. We conduct baseline surveys at the start of new activities, and collect quantitative as well as qualitative information from our various stakeholders. We produce annual reports and biennial impact assessment reports, and have an ‘Our Impact’ page on our website.
GLC: What can directors do to help the performance reporting practices of NFPs?
JJ: Directors can play a supportive role in measuring impact. This might include defining and communicating success, supporting and empowering the leadership team to develop a theory of change, and helping removing barriers to measurement (lack of funds, skills, know-how). It is also important for directors to engage with stakeholders so they understand what the organisation is measuring, why, and the benefits it will bring.
About the author
Justine Jarvinen GAICD is currently Chair of Wattwatchers, a non-executive director of Milton Corporation and Pollinate Energy, and Chief Operating Officer of the UNSW Energy Institute. Justine holds a first class honours degree in Chemical Engineering and a Graduate Diploma in Applied Finance and Investment.
Already a member?
Login to view this content