[Special Feature: 20 Years of NPOs] Social Impact Assessment of NPOs

2018/11/05

In recent years, "Social Impact Assessment" has become a topic of active discussion within non-profit organization networks in Japan. Social Impact Assessment is a method for evaluating how projects implemented for social purposes have actually influenced society. In particular, combined with the discussion regarding the utilization of dormant bank accounts as funds (which will be described later), the issue of how to evaluate social projects implemented by NPOs and other organizations has become a major point of debate.

The initiatives implied by the term Social Impact Assessment are by no means new. For example, in the medical world, there is a long history of verifying which therapies or drug administrations are effective. In the realm of policy, "policy evaluation" saw many initiatives in the United States in the 1960s and has been introduced in Japan since the 1990s. Furthermore, there is a long history of program evaluation initiatives in the fields of education and international development.

In modern Japan, where the population peaked in 2008 and has since turned toward decline due to the low birthrate and aging population—with 45 million people expected to disappear over the next 50 years—it is clear that the need for welfare services will increase while public funds in administrative budgets become tighter. As seen in discussions regarding long-term care insurance, a situation has begun where social service levels are being lowered and the living standards of citizens are regressing if public projects are conducted in the same way as before. To prevent such a situation, "social productivity" that effectively solves social issues is required. To achieve this, initiatives to constantly measure and improve the productivity of social services are necessary.

However, despite these expectations for Social Impact Assessment, the practice of social evaluation is accompanied by technical difficulties. In corporate management, profitability can be calculated by the ratio of costs, sales, and the resulting profit. One can compare the productivity of one business with another, or conduct time-series or departmental analyses to consider measures for improving productivity.

On the other hand, the value evaluation of social services requires various considerations. This is because the results of social projects such as education and welfare are medium- to long-term, and the majority are difficult to evaluate quantitatively. The results of primary education manifest 10 or 20 years later, and because there are many external factors, the causal relationship cannot be easily identified. Furthermore, it is difficult to judge what the results of welfare services that guarantee minimum social needs, such as public assistance, actually are.

In response to these challenges, initiatives have been undertaken to standardize social value evaluation methods, particularly in the UK and the US. Specifically, in the field of quantitative evaluation, there are several different methods, one of which is SROI (Social Return on Investment).

REDF (Roberts Enterprise Development Fund), located in San Francisco, USA, was a foundation that provided financial and non-financial support to non-profit organizations primarily tackling homelessness issues, but it faced the challenge of how to evaluate the results of the organizations it supported. Therefore, in the 1990s, the then Executive Director Jed Emerson developed SROI based on grants from investment funds. SROI is a framework that measures the return on investment of activities by performing quantitative evaluation through monetary value conversion not only for the results of activities (outputs) but also for the outcomes.

For example, in homelessness support activities, it captures outcomes such as "what kind of jobs did the homeless people get as a result of receiving support?", "what is the average wage?", and "how much was the effect of reduced public assistance?" rather than just the activity results of "how many homeless people were provided with services?" It then calculates cost-effectiveness by contrasting these with the costs of program implementation. It was an ambitious attempt to measure return on investment through monetary value conversion by quantitatively evaluating outcomes rather than just "how much of the project was implemented."

These SROI methods have spread as they meet the needs of funders (grant-making foundations and social investors) who want to measure the cost-effectiveness of social projects, while simultaneously meeting the needs of social entrepreneurs who want to acquire more funds by increasing service productivity. In 2008, the SROI Network was established in the UK, and as of 2018, its branches have been established in more than 20 countries, showing a certain level of expansion.

This approach to quantitative Social Impact Assessment grew particularly alongside the increase in social investment in social projects. For example, Acumen, established in New York in 2001, is said to be a pioneer of social investment funds and is a fund that makes equity investments in social enterprises providing sanitation and infrastructure projects, mainly in developing countries. Looking at the world as a whole, such social investment has grown to $228 billion (approx. 25 trillion yen) over the past 20 years (estimate by Global Impact Investing Network: 2018). Within financial markets, it has grown as the ESG investment market, with a global market size of $22.89 trillion (2,541 trillion yen) (Global Sustainable Investment Alliance survey: 2016), and 136 trillion yen in Japan (Japan Sustainable Investment Forum survey: 2017).

Social investment is characterized by making investment decisions based on evaluation axes that take social impact into account in addition to normal risk and return. This approach to investment that emphasizes sociality in financial markets is also part of a social trend toward solving social issues and increasing the productivity of social projects by utilizing market principles, which has bolstered the trend of Social Impact Assessment for non-profit organizations mentioned earlier.

Furthermore, in the UK, initiatives regarding the standardization of such social value evaluation are being carried out. Social Value International, which was reorganized from the SROI Network in 2016, operates a database of social impact assessment indicators called the Global Value Exchange. Simultaneously, various guidelines are being developed to ensure compatibility between different evaluation methods.

Also in the UK, the Public Service (Social Value) Act came into effect in 2013. When public institutions purchase social services from the private sector, they are now obligated to consider the social value of the outcomes realized by the service as a criterion for selection, rather than deciding on a contractor based solely on work specifications and price.

At the same time, the point of how to utilize social impact assessment in the management of companies and non-profit organizations, rather than just evaluating it, became a major topic in the series of discussions. This is because if the point of how to use measured social value to lead to an improvement in the impact of the project is missing, it will not lead to a substantial expansion of social impact. These discussions provide the basis for the argument that the focus should shift from the previous "Social Impact Assessment" to "Social Impact Management."

In conjunction with these global movements in Social Impact Assessment, initiatives for Social Impact Assessment are also progressing in Japan. In 2012, SROI Network Japan was established and became a member organization of the UK SROI Network International (later renamed Social Value Japan).

In 2015, the "Social Impact Assessment Study Working Group" was established within the Cabinet Office's "Roundtable on Creating a Mutual Assistance Society." Discussions were held on the nature of evaluation for non-profit organizations in the Japanese context, and guidelines were presented. In 2016, based on these guidelines, case studies were conducted applying them to 60 non-profit organizations nationwide. Additionally, the "Social Impact Management Initiative" was launched as a voluntary organization through public-private collaboration. As of 2018, it forms a network in which more than 150 companies, NPOs, and administrative agencies participate.

What holds particularly important meaning in the spread of Social Impact Assessment in Japan is that the "Council for the Utilization of Dormant Deposits, etc." held by the Cabinet Office from 2017 to 2018 discussed utilizing dormant deposits to promote private-sector public interest activities. Within that, the implementation of Social Impact Assessment was positioned as the basis for fund allocation.

The dormant deposit fund is expected to be 4 to 5 billion yen in the first fiscal year (FY2019), reaching a maximum of about 50 billion yen per year. This amount is at most about 3% of the Japanese grant and donation market, which amounts to approximately 1.4 trillion yen per year, so its impact is limited. However, in the sense of preventing the scattering of the dormant deposit fund, which can be called quasi-public money, ensuring its effective utilization, and securing transparency of results, it seeks to mandate the evaluation of the social outcomes of the projects into which the fund is invested.

This positioning can be said to be a major change in Japanese society, where value evaluation of social activities had been deferred for the reasons mentioned above. However, there are non-profit organizations, mainly in regional areas, that question this. The premise that more funds and resources can be attracted by visualizing social impact as results and communicating them externally is specific to large metropolitan areas centered on Tokyo. In contrast, in environments where administrative subsidies and grants account for the majority of activity funds, there is a perception that an overemphasis on social outcomes in evaluation indicators will not lead to the maintenance and expansion of social welfare.

As the economy itself shrinks and the overall market for public projects contracts, there are concerns that Social Impact Assessment might only be used as an excuse to decide which projects to cut, or that indicators suitable for quantitative evaluation will be prioritized while qualitative evaluation recedes. Japanese society is no longer homogeneous, and in a situation where societies with different contexts coexist, the utilization of different tools is required.

Since the discussion on dormant deposits, "Social Impact Assessment" has been discussed particularly in connection with fund allocation, but its essential meaning is not limited to fund allocation. Furthermore, although there is some misunderstanding, methods involving monetary value conversion are only a small part of it; the original targets of evaluation are broad, and the methods cover both qualitative and quantitative aspects.

For social projects aimed at creating social value and solving social issues in the course of the project, value evaluation should be a prerequisite for the governance of the organization and the project itself, an important indicator for management decisions, and vital information for sharing "social value" with relevant stakeholders.

What does it mean that Social Impact Assessment for NPOs has become a major agenda in society? One thing is likely the birth of the recognition that NPOs are actors constituting a new society, as symbolized by the discussion of the "New Public" under the Democratic Party of Japan administration. They have a social responsibility to measure and communicate their results with a certain level of transparency as important members responsible for solving social issues, rather than simply complementing the administration.

NPOs are also positioned as a source of social innovation. They have a potential social role in creating innovations that embody social value that cannot be realized by the administration due to "government failure" or by business due to "market failure." To actually demonstrate such potential, NPOs, as entities forming equal partnerships with the administration and companies, are being questioned on their level of accountability, organizational foundation, execution power, and commitment to social impact.