Abstract
An ageing population has placed strains on health and social care systems. Innovative solutions have been sought to inject capacity and capability in order to deliver services to older people more efficiently and effectively. Over the last two decades, governments have actively encouraged third sector organisations to deliver public services on the assumption that they exhibit higher levels of innovation, efficiency and responsiveness. The evidence base, particularly for whether they provide better value for money, remains poor. We present the results of a systematic literature review on the costs and outcomes of services for older people delivered by third sector organisations. We combine this evidence with a framework for analysing the benefits and costs of third sector-led initiatives, and test this out empirically with a group of initiatives delivered for older people in an urban context. We find that our method may hold considerable promise for the evaluation of third sector initiatives.
Introduction
The world is ageing. In 2019, 9 per cent of the world’s population was aged 65 years or over, and this is expected to increase to 16 per cent in 2050 (UN, 2019), which is likely to put considerable strain on the capacity of health and social care systems to address such needs (Abdi et al, 2019; Morrow-Howell et al, 2020). To manage this ageing population, community-based initiatives led by organisations in the ‘third sector’ have been encouraged as alternative providers of social care, particularly to allow older people to remain living at home or within their communities as long as possible (Blas et al, 2008; Marmot, 2010; Alsaeed et al, 2016). The ‘tremendous diversity’ (Salamon and Sokolowski, 2016) of organisational forms and concepts under the ‘third sector’ umbrella includes ‘voluntary organizations, nonprofit organizations, nonprofit institutions (NPIs), nongovernmental organizations (NGOs), associations, civil society, social economy, solidarity organizations, cooperatives, mutuals, foundations, civil society and, more recently social enterprises’ (Enjolras et al, 2018: 2).
Over the last few decades, and particularly since the reforms represented by New Public Management (Dunleavy and Hood, 1994; Osborne, 2013), governments have actively shifted the delivery of public services, including health and social care services, onto the third sector (Donaldson et al, 2011; Roy et al, 2014; Caló et al, 2019) ostensibly on the premise that they exhibit higher levels of innovation, cost-efficiency and responsiveness (Buckingham, 2009; Hall et al, 2016). This has certainly been the case in the UK, where this research was undertaken. However, the evidence base on which these assumptions are based remains relatively poor. Despite a variety of instruments being developed to evaluate the outcomes of third sector organisations generally, to date, very little empirical evaluation research has taken place concerning the (economic) impact of third sector organisations delivering care and well-being services (Harlock and Metcalf, 2016; Dey and Gibbon, 2017; Liston-Heyes and Liu, 2021). We know that initiatives led by third sector organisations may have benefits over other types of organisations (whether public or for-profit) for elderly physical health, psychological health and well-being, and on health behaviours (Attree et al, 2011; O’Mara-Eves et al, 2013; Alsaeed et al, 2016), as they are locally based, locally owned and can respond more closely to community needs (Caló et al, 2018; 2019; Steiner et al, 2021). However, overall, the economic evidence base for their involvement in delivering care and well-being services, and the benefits they bring, remains relatively underdeveloped (Dickinson et al, 2012; Roy et al, 2014; Arvidson and Kara, 2017).
It is important that this evidence base is developed because funding is often awarded on a year-by-year basis to such organisations rather than being folded into mainstream health and social care budgets; in fact, they are rarely considered part of ‘formal’ health and social care systems at all (Roy et al, 2017). Funding in such a way can create unintended consequences, making third sector organisations precarious, fragile and subject to the whims of funders and politicians. Although funders often require some kind of evaluation, the form that such evidence should take is rarely specified (Buckingham, 2009; Hall et al, 2016). Mainstream approaches to measuring health impacts, such as through health technology assessment (HTA) frameworks (Banta, 2003), have seldom been used in the context of third sector organisations due to the time and cost commitments involved, particularly for those running the organisations (Caló et al, 2021). In particular, there is the challenge of balancing spend on the evaluation of initiatives with the relatively small numbers of beneficiaries that are usually associated with any local, community-led organisation. Third sector organisations, particularly those involved in delivering care and well-being services, are competing for resources with various forms of health ‘intervention’ that have (or are at least perceived to have) a better-quality and more robust evidence base supporting them. To be adopted within mainstream budgets (Shemilt et al, 2010), they would have to be considered alongside interventions that traditionally have an HTA to evaluate their clinical and cost-effectiveness compared with what is currently being provided (Taylor and Taylor, 2009).
In this study, we aim to move towards improving the evidence base relating to the economic contribution of third sector organisations to elderly health and well-being through presenting the results of a systematic review of initiatives being delivered by third sector organisations aimed at care and well-being for older people (aged 65+, based on the World Health Organization [WHO] definition). A systematic review summarises the results of available studies, providing evidence on what is known regarding the effectiveness of a specific intervention (Higgins and Green, 2011; Gough et al, 2012). Although systematic reviews are commonplace in public health, they have rarely been used to assess the effectiveness of third sector initiatives (but see Roy et al, 2014; Caló et al, 2018). We then attempt to utilise what is known about this ‘state of the art’ and combine it with a framework for economic evaluation, testing it with a range of initiatives delivered by third sector organisations within an urban setting in Glasgow, UK. The initiatives included in this study were funded by the Transformation Fund. The context of our study is Glasgow, where funding was made available by the government in 2011 from the Reshaping Care for Older People (RCOP) Change Fund, with Greater Glasgow and Clyde receiving £48.622 million. From this Change Fund, the Transformation Fund was created to aid the third sector in contributing to the RCOP agenda to build a more cohesive post-industrial city. The article is set out as follows. After setting out our methodology in detail, we present the synthesis of our findings, including the estimates of costs and benefits of each of the initiatives, as will be explained. We close by discussing the implications of our review both for policy and practice. First of all, however, we turn attention to our methods.
Methods
Two different methods were used to improve the evidence base relating to the economic contribution of third sector organisations to elderly health and well-being.
Systematic literature review
First, a systematic review was conducted to collate and synthesise all of the up-to-date empirical evidence relating to the impact of services delivered by third sector initiatives for older people. To ensure the validity of the chosen methods and the comprehensiveness of the study’s search strategy, the research team developed a protocol for the review in collaboration with an advisory group of local third sector representatives and advocates. The protocol can be found in Appendix 1. Following best practice, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used in the design of the research (Moher et al, 2009).
To be included, studies had to meet the criteria of being primary research (conducted through any type of method) evaluating and explaining the contribution of third sector organisations in terms of older people’s health and well-being. Cohort studies (one group pre- and post-evaluation) were also included, as their baseline results can mimic ‘usual care’.
Primary research that focused on third sector organisations providing traditional health services (dental or general practitioner [GP] services), such as National Health Service (NHS) social enterprise ‘spin-offs’, were excluded. To ensure added validity, and to be as comprehensive as possible, it was decided to consult with our advisory group on the key sectors in which the third sector is most active in order to ensure that we were being as comprehensive as possible in our search. These were identified as being (in no particular order of importance): community food, community transport, local community hubs, home care, long-term care, information and advice, arts and physical activity services at the local level, befriending models, and housing-based initiatives. These categories also helped us to organise our findings.
The following databases were searched: Sociological Abstracts, Web of Knowledge, Applied Social Sciences Index and Abstracts (ASSIA), International Bibliography of Social Sciences, Cochrane Collaboration, Campbell, PubMed, Ethos, OpenGrey, PLANEX and CINHAL. After piloting and adapting the search string, a full search strategy was conducted for studies. Specific journals related to the topics of the third sector, social enterprises and non-profits were also screened. All the materials identified in the research were evaluated using a two-stage screening process, with the inclusion criteria screened against titles and abstracts at the first stage. All the abstracts were double screened to decrease possible biases, and team discussion was organised when disagreement arose. Relevant studies were retrieved and their full text double screened. An excel database was created to record the reasons for exclusions at each stage. A form for data extraction was developed, adapted to analyse both quantitative and qualitative data. The majority of studies were excluded because they did not assess interventions provided by the third sector. Further information on the papers included in the review can be found in Appendix 2.
Transformation Fund initiatives evaluation
Second, initiatives funded by the Transformation Fund were grouped into the initiatives outlined under the systematic literature review. The Transformation Fund was created from funding provided by the government in 2011 to support third sector organisations to contribute to reshaping care for older people in Glasgow.
The initiatives submitted reports on a yearly basis, outlining their aims, objectives and what they achieved within that year. This included evidence on the outcomes for older people, for example, evidence on the reduction of social isolation for older people, and included the number of people that accessed the initiative. The costs and benefits of the initiatives were analysed using data collected from the reports and supplemented with the findings on costs and benefits from the literature review to understand the economic contribution of third sector initiatives (for more detail, see Table 1).
Outline of data type and source
| Data type | Source |
|---|---|
| Cost of the project | Amount of funding awarded to each initiative from the Transformation Fund |
| Service users | Numbers of people accessing the initiatives from the initiative reports |
| Reported impacts on well-being | Impacts reported in the reports by the initiatives submitted to the Transformation Fund and the literature review |
| Cost savings | Estimates of future resource savings reported by the initiatives and in the literature review, combined with local data on unit costs (for example, cost per day in hospital avoided) |
Using the results of the systematic review and the information shown in Table 1, a rudimentary framework for economic analysis, with outcomes and net costs plotted against each other, was used to assess the impact of certain groups of initiatives. This framework is shown in Figure 1. The aim of using this framework is to establish the net cost versus net outcomes (derived from primary data collection, combined with assumptions from the systematic literature review) in order to assess the balance of the resources allocated to the service with the benefits from that service and, thus, to provide an indication of whether such provision is worthwhile or not.
/img/Blank.svg)
Outcomes and net cost matrix
Citation: Voluntary Sector Review 15, 3; 10.1332/204080521X16826159230735
Results
Figure 2 shows the process and number of studies found at each stage. A total of 32 studies were included in the systematic review. Due to low numbers in seven of the categories, these are not reported on in this article. In addition, local community hubs and physical activities were grouped together for the analysis, as it became clear that local community hubs often provide physical activity classes for older people. Therefore, the focus of the results is on three categories: befriending activities (seven studies); local community hubs and physical activity (12 studies); and arts activities (five studies). Results are reported for each category in turn.
Results of the search and study selection process
Citation: Voluntary Sector Review 15, 3; 10.1332/204080521X16826159230735
Befriending initiatives
Befriending initiatives aim to support older people in maintaining independent lives within their own homes and to improve their general well-being. It is considered an ‘upstream’ community-orientated intervention. Befriending occurs on a one-to-one basis, either in person or over the phone at times agreed by the recipient.
A total of 37 full studies were reviewed; 30 were excluded and seven included. The focus of the befriending studies was predominantly around quality of life and mental health. With regard to studies reporting outcomes, an intervention that focused on providing social support for carers of those with Alzheimer’s disease found short-term positive benefits (that is, directly after the intervention was received) but no main effects for either depression or self-esteem (Pillemer and Suitor, 2002). This was confirmed by another study that assessed a community mentoring service, which showed that there was significant improvement in the short-form health survey (SF-12) mental health component in the short term (six months) but that this improvement was not maintained at 12 months, and no evidence of the service alleviating social isolation was found (Dickens et al, 2011). In addition, a randomised controlled trial (RCT) focusing on befriending for carers of people with dementia showed no effect on improving carers’ well-being or quality of life (Charlesworth et al, 2008).
With regard to studies reporting on both costs and outcomes, one study focused on the cost-utility of befriending carers with dementia and found that befriending was not especially cost-effective from a societal perspective. They calculated this by comparing the costs imposed on the statutory (NHS and social services), voluntary and household sectors, with the indirect costs representing the value of carers’ and family and friends’ informal care time (Wilson et al, 2009). Another study conducted an RCT for a visiting service for older widowed individuals in the Netherlands and found that the service was cost-effective in comparison with usual care, with a low cost per quality-adjusted life year (QALY) gained (Onrust et al, 2008). This study took a societal perspective, which included direct medical costs, patient costs, costs from not being able to perform domestic tasks and informal care costs. It was found that savings occurred in the use of healthcare services, which offset some of the additional cost of the intervention. Knapp, McDaid and Parsonage (2011) analysed the cost-effectiveness of a befriending initiative where the person is visited in their own home once per week or once every two weeks. They focused on the reduction of loneliness and the resulting impacts on depression, and found that the average cost for someone using a befriending service would be £80, with a saving of £35 for every £80 invested. If quality of life is considered to be improved due to a reduction in depressive symptoms, they highlighted a total savings per year of £300 per person compared with ‘usual care’.
From the review, the studies that reported on the costs of befriending showed that the average costs ranged from £141 to £1,168 per person. There were additional costs for all of the befriending interventions where costs were reported when compared to usual care. At the same time, the majority of the studies identified positive outcomes, at least in the short term, leading to potential savings in the use of healthcare services.
Local community hubs and physical activity initiatives
Local community hubs initiatives provide a variety of activities for older people in a group setting, aiming at reaching and building trust in people. They often promote physical activity to have a positive impact on the health of their beneficiaries.
Under this category, 25 studies were reviewed; 13 were excluded and 12 included. Of those included in the review, six looked at physical activity interventions: three looked at the physical outcomes from the exercise group (that is, static balance and mobility); three looked at reduction in falls; one looked at health-related quality of life (HRQoL) and quality of life separate to health; one looked at the effects on happiness and loneliness; and one looked at the consequences on NHS use. One study analysed initiatives that aimed to engage elderly people in sharing their knowledge and skills with volunteers and young people in the community. One other study looked at the impact of engaging elderly people in desired activities (for example, a sports club) alongside potentially stressful activities (for example, caring for disabled people). While one study focused on interventions aimed at engaging elderly people in activities to improve their cognitive vitality and memory (for example, memory training).
An RCT evaluating group-based exercise with a focus on improving balance in older people found that those in the intervention group improved more than the control group in their static balance indicators (Robitaille et al, 2005) and physical functions (Fishleder et al, 2019; Lyndsay-Smith et al, 2019). Reduction in falls was assessed in another RCT that again compared group exercise with no intervention, finding that there was an overall reduction of 40 per cent in falls (Barnett et al, 2003). Another study looked at the effect of strength and endurance training on modifying risk factors for falls (Buchner et al, 1997). This study found that 42 per cent of the exercise group reported a fall one year after randomisation, compared to 60 per cent of the control group. However, there was no change in the number of outpatient visits over time for the exercise group compared with an average increase in the control group. Robertson et al (2001) assessed the incremental costs and cost-effectiveness of implementing a home-based muscle-strengthening and balance-retraining programme delivered by a local third sector organisation. The programme reduced falls and injuries in older women in New Zealand, showing that 65 per cent of the control group reported a serious or moderate fall in Year 1, compared to 28 per cent of the exercise group. In Year 2, 26 per cent of the control group reported a serious or moderate fall, compared to 20 per cent in the exercise group. Another study focused on the effect of two physical activity modes – aerobic activity and stretching – on changes in the subjective well-being of old people (McAuley et al, 2000). Happiness was measured using the Memorial University of Newfoundland Scale of Happiness (MUNSH), and loneliness was measured using the UCLA Loneliness Scale. This showed that their physical activity programmes led to increased happiness and satisfaction with life, and decreased loneliness over the six-month intervention period. However, these improvements were not able to be sustained at the six-month follow-up.
An economic evaluation was conducted on an exercise programme that was provided in Sheffield in community centres and church halls (Munro et al, 1997). The health gains were measured in lives extended and life years saved by evaluating health events avoided for certain conditions. It was estimated that 77 deaths and 229 in-patient episodes per 10,000 participants per year could be avoided, and, thus, that the intervention was potentially worthwhile, giving health gains at low costs. It was calculated that for each avoided health event, there was a cost saving of £1,100.
Four studies that evaluated group activities for elderly people measured social connectedness, well-being, physical activity, cognitive vitality and quality of life (Kimberlee and Means, 2013; Potocnik and Sonnentag, 2013; Laforest et al, 2017; Millard, 2017). Laforest et al (2017) used a friendship scale to measure changes in isolation. This showed that, on average, at baseline, participants were isolated or had a low level of social support (14.53), which increased to being very or highly socially connected at follow-up (22.8). Office for National Statistics well-being indicators were used to measure changes in well-being. Kimberlee and Means (2013) showed an increase in all indicators of the index, and these were higher than the national average at follow-up. The International Physical Activity Questionnaire (IPAQ) was used to measure the increase in physical activity. This showed that there was a mean increase in the number of days of activity of 1.63 from baseline to follow-up. In addition, participants were also asked about any medication changes, and it was reported that 26 per cent (eight people) had stopped taking medication. Potocnik and Sonnentag (2013), meanwhile, looked at the impact of engaging in both desired activities (for example, a sports club) and potentially stressful activities (for example, caring for disabled people) on depression and quality of life over a two-year period. They found that there was an increased quality of life from regularly attending sports clubs, voluntary work and providing help to family and friends, and a decrease in depressive symptoms from volunteering. Similarly, Millard (2017) found that older adults participating in social activities with and without a physically active component reported a higher Health Related Quality of Life (HRQoL) measured using the 12-Item Short Form Survey version 2 (SF-12v2) and EQ-5D, and a higher quality of life separate to health measured by the ICEpop CAPability measure for Older people (ICECAP-O). Scores from the SF-12 and EQ-5D were not converted to utility scores. In terms of cognitive vitality, Laforest et al (2017) found that there was an increase in terms of memory strategies (2.52), while their programme also improved physical activity, whether reported in terms of frequency (5.64) or duration (225.63). The majority of the studies reported positive outcomes in terms of quality of life, physical functions and reduction in falls and loneliness.
Arts-based initiatives
Arts-based interventions aim to improve older people’s well-being and reduce social isolation. Arts-based initiatives incorporate a wide range of activities (painting, theatre and music making, singing, other visual arts making, and so on) (Bellazzecca et al, 2022). The activities are most usually undertaken in a group setting and include such activities as painting, creative writing, singing and arts and crafts. Under this theme, 20 studies were reviewed; 15 were excluded and five included. Of those included, four looked at the various effects of arts activities on the health and well-being of socially isolated people, and one looked at the impact of arts initiatives on the use of health services.
One study evaluated a creative activity intervention for socially isolated older people and found a variety of benefits, including a reduction in loneliness and falls risk, as well as increased well-being, from 18 participant interviews (Greaves and Farbus, 2006). On the SF-12 measure (Ware et al, 1996), this showed a statistically significant increase in both mental and physical components. However, at 12-month follow-up, the mental component change decreased and it was no longer significant, though the physical improvements had increased. This study also included the Geriatric Depression Scale, which showed a significant reduction in depressive symptoms that was maintained at 12-month follow-up. Positive results were also reported by Clift et al (2012), who assessed the various impacts 265 older people experienced through participating in a community singing programme in comparison to ‘usual activity’. The study also used SF-12, which showed a statistically significant increase in mental health on the self-reported Hospital Anxiety and Depression Scale (HADS) (see Zigmond and Snaith, 1983) and a significant reduction in anxiety scores. A cost analysis was also conducted, where the average costs of health and social care were calculated for both groups, and showed that over six months, both groups experienced an increase in cost compared to the baseline, though this was greater for the intervention group (Clift et al, 2012). Another study compared a choral intervention (consisting of weekly singing rehearsals for 30 weeks, as well as public performances several times during the intervention period) with usual activities, involving 166 people in total (Cohen et al, 2006). This study showed that visits to GPs and the use of over-the-counter medicine were lower in the intervention group but had increased in the control group. In addition, the intervention group reported a decrease in falls, while the control group reported an increase. Another study also analysed the benefits of participating in a community-based choral activity (Johnson et al, 2013), finding that there were significant correlations between the benefits developed during singing activities and three of the four quality-of-life domains (psychological, social relationships and environment).
Another study conducted in Finland that examined health service use compared three different types of intervention (writing and psychotherapy group, exercise group, and arts group) with control groups, using 235 older people suffering from loneliness (Pitkala et al, 2009). This study found that there was an improvement in general health in the intervention group compared to the control group, showing that they had used fewer health services. However, the outcomes were not separated out by groups, and so it was not possible to attribute the outcomes to the arts group. From the grey literature, COPE Scotland commissioned a social return on investment (SROI)1 study to be conducted on their Craft Café initiative, which offered a range of crafting opportunities for older people. This reported a return of £4.49 for every £1 spent. However, on further examination, this appeared to be an overestimate, as there was double counting of the benefits of relaxation. In addition, to capture the benefits from relaxation, the cost of a personal massage every week was used. When replaced by a yoga/meditation class and double counting was removed, this reduced the savings to £2.74 for every £1 spent. The majority of the studies identified a positive result, at least in the short term, as regards the health and well-being of the elderly people involved, which, as well as being important on its own right, shows the achievement of potential healthcare savings.
Putting the systematic review to work: economic evaluation phase
We examined details of initiatives that fell into each of the three categories that emerged from the systematic review, namely: (1) befriending initiatives; (2) local community hubs and physical activity initiatives; and (3) arts-based initiatives. Our assessment of the initiatives that fell into each category is discussed in turn.
Befriending initiatives
Four of the 25 initiatives funded by the Transformation Fund were assessed in the befriending category. This involved 111 service users in total, as Table 2 explains.
Assumptions and net costs with respect to befriending initiatives
| Assumption | Explanation | Sources | Net costs | Number of service users | Cost/saving per service user |
|---|---|---|---|---|---|
| NHS savings for mental health: for each £80 invested, there is a saving of £35 | This is the savings in the first year for a befriending service from a reduction in the need for treatment and support for mental health needs | Knapp et al (2011) | £57,505 | 111 | £518 |
| Savings of £300 per person | Building on the reduction in the use of mental health services, if quality of life is considered as a result of better mental health, then the savings of a befriending service would be £300 per person per year | Knapp et al (2011) | £29,200 | 111 | £263 |
| Return on investment for every £1 spent of £3.75 | Savings from reduction in mental health service spending and improvements to health from befriending services | King’s Fund and Local Government Association (2014) | –£234,375 | 111 | –£2,111 |
Both initiatives reported a reduction in social isolation, and through the literature, it is possible to relate this to a decrease in depression. We have assumed a reduction in the use of services for mental health (Knapp et al, 2011; King’s Fund and Local Government Association, 2014). The underpinning economic assumption, based on Knapp et al (2011) King’s Fund and Local Government Association (2014), is that for befriending initiatives, there is a saving to the NHS of £35 for each £80 in the first year of a befriending service due to a reduction in the need for treatment for mental health support. As a result of this reduction in the use of mental health services, quality-of-life improvement represents an additional £300 saving per person and a return on investment of £3.75 for each £1 spent, according to a report by the King’s Fund and Local Government Association (2014). Thus, the net cost ranges from a total saving of just over £234,300 (see A3 in Figure 1) to a total cost increase of over £57,500 (see C3 in Figure 1), depending on the assumptions used. This can be shown as ranging from a saving of just over £2,100 per service user to an incremental cost of up to £500 per service user.
Community hubs and physical activity
Analysing the projects financed by the Transformation Fund, 14 initiatives fell into the ‘community hubs and physical activity’ category. These involved 2,402 service users in total for a total cost of £505,599. All the initiatives reported a reduction in social isolation. Similarly to the previous category, we established NHS savings based upon the work of Knapp et al (2011) from a reduction in the need for treatment and support for mental health needs. In addition, evidence by Barnett et al (2003) and the report on the National Falls Programme (Scottish Government, 2014) considered that the rate of falls would decrease by 40 per cent, with a cost per fall of £1,720. There was also an assumption, based on Windle et al (2010), of a reduction in outpatient appointments but with cost savings offset by a slight increase in GP appointments.
As some community hubs do not provide physical activity sessions, they have been excluded from the analysis for falls reduction. Given the range of activities provided by community hubs, there could be a total potential saving of over £1.2 million (see A3 in Figure 1) to a net cost of up to £369,500 (see C3 in Figure 1). This equates to a range from a net saving of £1,676 per person per year up to an incremental cost of £180 per person, as shown in Table 3.
Literature-based assumptions and net costs with respect to local community hub and physical activities initiatives
| Assumption | Explanation | Sources | Net cost | Number of service users | Cost/saving per service user |
|---|---|---|---|---|---|
| NHS savings for mental health: for each £80 invested, there is a saving of £35 | This is the savings in the first year from a reduction in the need for treatment and support for mental health needs | Knapp et al (2011) | £397,509 | 2,402 | £165 |
| £300 saved per person per year | Building on the reduction in the use of mental health services, if quality of life is considered as a result of better mental health (taking evidence from The Partnership for Older People Projects (POPP) [4]), then the savings of a befriending service would be £300 per person per year | Knapp et al (2011) | –£215,001 | 2,402 | –£90 |
| 40% lower risk of a fall if involved in physical activity | Rate of falls in an intervention for providing physical activity is 40% lower than that in usual care Cost per fall = £1,720 1% of falls will result in a hip fracture Cost of hip fracture = £39,500 |
Barnett et al (2003) Scottish Government (2014) |
–£1,208,631 | 725 | –£1,676 |
| Reduction in outpatient appointments, saving £30.16 per person | For initiatives focused on emotional support and reducing social isolation, respondents reported a reduction in the use of outpatient appointments but an increase in GP appointments | Windle et al (2010) | £433,155 | 2,402 | £180 |
Arts-based activities
If we analyse the projects financed by the Transformation Fund and housing initiatives, seven local initiatives have been funded. These initiatives have involved a range between ten and 84 beneficiaries, for a total cost ranging between £7,332 and £48,457. More than 260 elderly people were involved, for a total cost of £157,792. The assumptions, derived from the literature and used for the calculations, relate to the reduction of symptoms for depression, alongside the costs of depression (McCrone et al, 2008), and the SROI explored in the COPE study adjusted for the overestimation detailed earlier. There was potential for total savings of just over £1 million (see A3 in Figure 1) (depending on the SROI sensitivity analysis) to a cost of an additional £99,287 (see C3 in Figure 1). This can be shown as ranging from a net saving of over £3,000 per service user to an incremental cost of £394 per service user, as shown in Table 4. Table 4 outlines the various assumptions drawn from the literature (see Greaves and Farbus, 2006; McCrone et al, 2008; Social Value Lab, 2011).
Literature-based assumptions and net costs with respect to arts-based activities
| Assumption | Explanation | Source | Net costs | Number of service users | Cost/saving per service user |
|---|---|---|---|---|---|
| 10% reduction in depression symptoms Cost of depression = £2,085 |
Geriatric Depression Scale was used in a community-based intervention with an emphasis on creative activity and social interaction: at baseline, 45% had mild clinical depression, which fell to 35% at six-month follow-up The average service costs in 2007 for people in treatment or where their condition was recognised, including contact with psychiatrists, psychologists, other doctors and community mental health nurses |
Greaves and Farbus (2006) McCrone et al (2008) |
£99,287 | 252 | £394 |
| SROI of £8.27 per £1 spent (maximum value) | Impact Arts commissioned a SROI to be conducted on their Craft Café initiative, which offers a range of crafting opportunities | Social Value Lab (2011) | –£1,702,247 | 252 | –£6,755 |
| SROI of £4.86 per £1 spent (minimum value) | Impact Arts commissioned a SROI to be conducted on their Craft Café initiative, which offers a range of crafting opportunities | Social Value Lab (2011) | –£1,000,353 | 252 | –£3,970 |
Discussion and conclusion
Systematic reviews are still not commonplace in fields of study or disciplines that regularly grapple with issues relating to the third sector. In this study, we have not only set out a systematic review but also then applied the findings to develop an evaluation framework. We tested the framework and found that after undertaking a retrospective analysis of initiatives aimed at older people in Glasgow, the majority of the initiatives that were supported either resulted in cost savings and better outcomes (see A3 of Figure 1) or increased costs combined with improved outcomes (see C3 of Figure 1). Although we have not compared third sector initiatives with other care and well-being services, this indicates that delivery by the third sector offers the potential to produce better outcomes for the allocated resource and can also target specific groups in society, in this case, older adults.
Even employing a relatively simplistic economic framework to analyse third sector initiatives has the potential to provide important evidence on the costs and benefits that can support evidence-based policymaking, and certainly takes the evaluation of third sector initiatives forward somewhat. Our findings provide evidence, from a small number of studies, that third sector initiatives could represent an effective provider of complementary public services, particularly in the case of befriending activities, local community hubs and physical activity and arts-based initiatives, confirming the literature that sees third sector organisations as an effective collaborator of the public sector (Mazzei et al, 2019; Steiner et al, 2021; Caló et al, 2023).
While the results appear positive and show potential for evaluating third sector initiatives using such an approach, there are quite obvious limitations to our study. First of all, the attribution of outcomes to only third sector initiatives was quite difficult to extract from some studies in the systematic review, and it was difficult to identify the comparator group of the initiatives. In addition, although the studies were analysing different contexts, contextual variables were rarely considered. We also faced a lack of reported outcomes from the initiatives for use in the economic evaluation phase of the research. While we derived assumptions from the literature, these were from an analysis of interventions that often substantially differed in design, intensity and contextual needs. Context is important in exploring and understanding the contribution of the third sector. Interestingly, due to their methodology, most of the studies included did not consider contexts in their explanation. If local databases collected primary data not only on outcomes but also on the contexts and mechanisms involved in delivering those outcomes, then this would enable some quite sophisticated analyses to be planned, enabling such questions as ‘What works?’, ‘For whom?’ and ‘In what circumstances?’ to be asked. This is the type of question that ‘realist evaluations’ (Pawson and Tilley, 1997; 2009) are designed to handle, which are becoming more commonplace in the design and evaluation of complex public health interventions (see Craig et al, 2008), including being combined with, or integrated with, systematic review methods (Pawson et al, 2005; Fletcher et al, 2016). Implementation of realist evaluation would also help to identify potential comparators for third sector interventions, as suggested in Caló et al (2019; 2020; 2021; 2023), enabling the development of more innovative third sector evaluation approaches (Baker et al, 2023) and supporting a better understanding and analysis of how third sector organisations can move towards addressing societal challenges and the relevant impacts of these challenges.
Third, only the funding awarded to the initiatives was used in place of the ‘true cost’ of the various initiatives. This may not show the full cost of these types of initiatives, as there is also the possibility that they were receiving funding from various different sources. In most of the aforementioned cases, the cost savings do not equate to actual cash savings. These are representations of resources saved that could have alternative uses.
In order to compete for funds aiming to address societal challenges, it is important to assess the extent to which the third sector can participate on a ‘level playing field’ with medical interventions, for which, it is perceived, the evidence base is more established (see Roy et al, 2013). If it can be established that the third sector can meet the complex needs of often vulnerable populations more effectively and/or more efficiently, then it is important that such evidence becomes part of local decision making. The tools presented herein (a systematic review and economic evaluation) are merely just the start of the process, while future studies should advance these tools by better exploring how context and mechanisms might be included, and how data on the outcomes of third sector services can be more routinely collected.
Note
SROI is a process for understanding, measuring and reporting the social, economic and environmental value created by an intervention, programme, policy or organisation (Banke-Thomas et al, 2015).
Funding
Funding was received from the Glasgow Council for the Voluntary Sector (GCVS) to undertake the research.
Conflict of interest
The authors declare that there is no conflict of interest.
References
Abdi, S. et al. (2019) Understanding the care and support needs of older people: a scoping review and categorisation using the WHO international classification of functioning, disability and health framework (ICF), BMC Geriatrics, 19(1): 195, doi: 10.1186/s12877-019-1189-9.
Alsaeed, D. et al. (2016) Older people’s priorities in health and social care research and practice: a public engagement workshop, Research Involvement and Engagement, 2: 2, doi: 10.1186/s40900-016-0016-0.
Arvidson, M. and Kara, H. (2017) Valuing third sector achievements in a service delivery context: evaluations and social value, in J. Rees and D. Mullins (eds) The Third Sector Delivering Public Services Developments, Innovations and Challenges, Bristol: Policy Press Scholarship Online, www.universitypressscholarship.com/view/10.1332/policypress/9781447322399.001.0001/upso-9781447322399-chapter-8 (Accessed 25.05.2022).
Attree, P. et al. (2011) The experience of community engagement for individuals: a rapid review of evidence, Health & Social Care in the Community, 19(3): 250–60, doi: 10.1111/j.1365-2524.2010.00976.x.
Baker, R.M. et al. (2023) Common health assets protocol: a mixed-methods, realist evaluation and economic appraisal of how community-led organisations (CLOs) impact on the health and well-being of people living in deprived areas, BMJ Open, 13(3): e069979, doi: 10.1136/bmjopen-2022–069979.
Banke-Thomas, A.O. et al. (2015) Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review, BMC Public Health, 15: 582, doi: 10.1186/s12889-015-1935-7.
Banta, D. (2003) The development of health technology assessment, Health Policy, 63(2): 121–32, doi: 10.1016/S0168-8510(02)00059-3.
Barnett, A. et al. (2003) Community-based group exercise improves balance and reduces falls in at-risk older people: a randomised controlled trial, Age and Ageing, 32(4): 407–14. doi: 10.1093/ageing/32.4.407
Bauer, A., Knapp, M., Wistow, G., Perkins, M., King, D. and Iemmi, V. (2017) Costs and economic consequences of a help-at-home scheme for older people in England, Health and Social Care in the Community, March, 25(2): 780–89, doi: 10.1111/hsc.12372.
Bellazzecca, E. et al. (2022) The health impacts of place-based creative programmes on older adults’ health: a critical realist review, Health & Place, 76: 102839, doi: 10.1016/j.healthplace.2022.102839.
Blas, E. et al. (2008) Addressing social determinants of health inequities: what can the state and civil society do?, The Lancet, 372(9650): 1684–89, doi: 10.1016/S0140-6736(08)61693-1.
Buchner, D.M., Cress, M.E., de Lateur, B.J., Esselman. P.C., Margherita, A.J., Price, R. and Wagner, E.H. (1997) The effect of strength and endurance training on gait, balance, fall risk, and health services use in community-living older adults, Journals of Gerontology Series A, July, 52(4): M218–24. doi: 10.1093/gerona/52a.4.m218. PMID: 9224433
Buckingham, H. (2009) Competition and contracts in the voluntary sector: exploring the implications for homelessness service providers in Southampton, Policy & Politics, 37(2): 235–54, doi: 10.1332/030557309X441045.
Caló, F. et al. (2018) Collaborator or competitor: assessing the evidence supporting the role of social enterprise in health and social care, Public Management Review, 20(12): 1790–814, doi: 10.1080/14719037.2017.1417467.
Caló, F., Roy, M.J., Donaldson, C., Teasdale, S. and Baglioni, S. (2019) Exploring the contribution of social enterprise to health and social care: a realist evaluation, Social Science and Medicine, 222: 154–61, doi: 10.1016/j.socscimed.2019.01.007.
Caló, F., Roy, M., Donaldson, C., Teasdale, S. and Baglioni, S. (2021) Evidencing the contribution of social enterprise to health and social care: approaches and considerations, Social Enterprise Journal, 17(1): 140–55.
Caló, F., Teasdale, S., Roy, M.J., Bellazzecca, E. and Mazzei, M. (2023) Exploring collaborative governance processes involving nonprofits, Nonprofit and Voluntary Sector Quarterly, 0(0), doi: 10.1177/08997640231155817.
Caló, F. et al. (2023) Exploring collaborative governance processes involving nonprofits, Nonprofit and Voluntary Sector Quarterly, doi: 10.1177/08997640231155817, (Accessed: 10.05.2023).
Charlesworth, G. et al. (2008) Befriending carers of people with dementia: randomised controlled trial, British Medical Journal, 336(7656): 1295–7, doi: 10.1136/bmj.39549.548831.AE.
Clift, S. et al. (2012) A Controlled Evaluation of the Health Benefits of a Participative Community Singing Programme for Older People (Silver Song Clubs), Folkestone: Sidney De Haan Research Centre for Arts and Health, Canterbury Christ Church University, https://www.singforyourlife.org.uk/sites/default/files_new/SSC%20FormativevEvalulation%20Summary.pdf.
Cohen, G.D. et al. (2006) The impact of professionally conducted cultural programs on the physical health, mental health, and social functioning of older adults, The Gerontologist, 46(6): 726–34. doi: 10.1093/geront/46.6.726
Craig, P. et al. (2008) Developing and Evaluating Complex Interventions: New Guidance, London: Medical Research Council, https://webarchive.nationalarchives.gov.uk/ukgwa/20170111123050/http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/, (Accessed: 20.12.2014).
Dey, C. and Gibbon, J. (2017) Moving on from scaling up: further progress in developing social impact measurement in the third sector, Social and Environmental Accountability Journal, 37(1): 66–72, doi: 10.1080/0969160X.2017.1285712.
Dickens, A.P., Richards, S.H., Hawton, A., Taylor, R.S., Greaves, C.J., Green, C., Edwards, R. and Campbell, J.L. (2011) An evaluation of the effectiveness of a community mentoring service for socially isolated older people: a controlled trial, BMC Public Health, Apr 8, 11: 218, doi: 10.1186/1471-2458-11-218. PMID: 21477273; PMCID: PMC3083357.
Dickinson, H. et al. (2012) The Role of Third Sector in Delivering Social Care. Scoping Review, London: NIHR School for Social Care Research.
Donaldson, C. et al. (2011) Social business, health and well-being, Social Business, 1(1): 17–35, doi: 10.1362/204440811X570545.
Dunleavy, P. and Hood, C. (1994) From old public administration to New Public Management, Public Money & Management, 14(3): 9–16, doi: 10.1080/09540969409387823.
Enjolras, B. et al. (2018) The Third Sector as a Renewable Resource for Europe: Concepts, Impacts, Challenges and Opportunities, Cham: Palgrave Macmillan.
Fishleder, S., Petrescu-Prahova, M., Harris, J.R., Leroux, B., Bennett, K., Helfrich, C.D., Kohn, M. and Hannon, P. (2019) Predictors of improvement in physical function in older adults in an evidence-based physical activity program (EnhanceFitness), Journal of Geriatric Physical Therapy, Oct/Dec, 42(4): 230–42, doi: 10.1519/JPT.0000000000000202. PMID: 29979352; PMCID: PMC6318072.
Fletcher, A. et al. (2016) Realist complex intervention science: applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions, Evaluation, 22(3): 286–303. doi: 10.1177/1356389016652743
Gough, D., Oliver, S. and Thomas, J. (eds) (2012) Introduction to Systematic Reviews, London: Sage Publications.
Greaves, C.J. and Farbus, L. (2006) Effects of creative and social activity on the health and well-being of socially isolated older people: outcomes from a multi-method observational study, The Journal of the Royal Society for the Promotion of Health, 126(3): 134–42. doi: 10.1177/1466424006064303
Hall, K., Miller, R. and Millar, R. (2016) Public, private or neither? Analysing the publicness of health care social enterprises, Public Management Review, 18(4): 539–57, doi: 10.1080/14719037.2015.1014398.
Harlock, J. and Metcalf, L. (2016) Measuring impact: prospects and challenges for third sector organisations, Voluntary Sector Review, 7(1): 101–8, doi: 10.1332/204080516X14534734765005.
Higgins, M., McKevitt, C. and Wolfe, C.D. (2005) Reading to stroke unit patients: perceived impact and potential of an innovative arts based therapy, Disability and Rehabilitation, Nov 30, 27(22): 1391–8, doi: 10.1080/09638280500164727. PMID: 16321921.
Higgins, J.P. and Green, S. (eds) (2011) Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011), London: Cochrane Collaboration, https://handbook-5-1.cochrane.org/whnjs.htm, (Accessed: 27.08.2012).
Johnson, J.K. et al. (2013) Quality of life (QOL) of older adult community choral singers in Finland, International Psychogeriatrics/IPA, 25(7): 1055–64, doi: 10.1017/S1041610213000422.
Kimberlee, R. and Means, R. (2013) Assessing the Impact of the LinkAge hub in Whitehall and St.George, Bristol: briefing report, https://uwe-repository.worktribe.com/output/932045, (Accessed: 14 Jul 2017).
King’s Fund and Local Government Association (2014) Making the case for public health interventions, www.kingsfund.org.uk/sites/default/files/media/making-case-public-health-interventions-sep-2014.pdf, (Accessed: 25.05.2022).
Knapp, M., McDaid, D. and Parsonage, M. (2011) Mental Health Promotion and Mental Illness Prevention: The Economic Case, London: Department of Health, www.pssru.ac.uk/, (Accessed: 10.07.2017).
Laforest, S., Lorthios-Guilledroit, A., Nour, K., Parisien, M., Fournier, M., Ellemberg, D., Guay, D., Desgagnés-Cyr, C.É. and Bier, N. (2017) Attitudes and lifestyle changes following Jog your Mind: results from a multi-factorial community-based program promoting cognitive vitality among seniors, Health Education Research, Apr 1, 32(2): 184–96, doi: 10.1093/her/cyx031. PMID: 28334988; PMCID: PMC5914362.
Lindsay-Smith, G., Eime, R., O’Sullivan, G., Harvey, J. and van Uffelen, J.G.Z. (2019) A mixed-methods case study exploring the impact of participation in community activity groups for older adults on physical activity, health and wellbeing, BMC Geriatrics, Sep 2, 19(1): 243, doi: 10.1186/s12877-019-1245-5. PMID: 31477054; PMCID: PMC6720859.
Liston-Heyes, C. and Liu, G. (2021) To measure or not to measure? An empirical investigation of social impact measurement in UK social enterprises, Public Management Review, 23(5): 687–709, doi: 10.1080/14719037.2020.1865435.
Marmot, M. (2010) Fair Society: Healthy Lives. The Marmot Review. Strategic Review of Health Inequalities in England Post-2010, London: University College London, https://www.instituteofhealthequity.org/resources-reports/fair-society-healthy-lives-the-marmot-review, (Accessed: 24.05.2016).
Mazzei, M., Teasdale, S., Caló, F. and Roy, M.J. (2020) Co-production and the third sector: conceptualising different approaches to service user involvement, Public Management Review, 22(9): 1265–1283. doi: 10.1080/14719037.2019.1630135.
McAuley, E. et al. (2000) Social relations, physical activity, and well-being in older adults, Preventive Medicine, 31(5): 608–17, doi: 10.1006/pmed.2000.0740.
McCrone, P.R. et al. (2008) Paying the Price: The Cost of Mental Health Care in England to 2026, London: King’s Fund, www.kingsfund.org.uk/sites/default/files/Paying-the-Price-the-cost-of-mental-health-care-England-2026-McCrone-Dhanasiri-Patel-Knapp-Lawton-Smith-Kings-Fund-May-2008_0.pdf Accessed [20.04.2021].
Millard, J. (2017) The health of older adults in community activities, Working with Older People, 21(2): 90–99, doi: 10.1108/WWOP-09-2016-0024.
Moher, D. et al. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement, Annals of Internal Medicine, 151(4): 264, doi: 10.7326/0003-4819-151-4-200908180-00135.
Morrow-Howell, N., Galucia, N. and Swinford, E. (2020) Recovering from the COVID-19 pandemic: a focus on older adults, Journal of Aging & Social Policy, 32(4–5): 526–35, doi: 10.1080/08959420.2020.1759758.
Munro, J. et al. (1997) Physical activity for the over-65s: could it be a cost-effective exercise for the NHS?, Journal of Public Health, 19(4): 397–402, doi: 10.1093/oxfordjournals.pubmed.a024667.
O’Mara-Eves, A. et al. (2013) Community Engagement to Reduce Inequalities in Health: A Systematic Review, Meta-analysis and Economic Analysis, Southampton: NIHR Journals Library (Public Health Research), www.ncbi.nlm.nih.gov/books/NBK262817/, (Accessed: 24.05.2016).
Onrust, S. et al. (2008) Cost-utility of a visiting service for older widowed individuals: randomised trial, BMC Health Services Research, 8: 128, doi: 10.1186/1472-6963-8-128.
Osborne, S.P. (2013) Voluntary Organizations and Innovation in Public Services, London and New York: Routledge (Routledge Studies in the Management of Voluntary and Non-profit Organizations).
Pawson, R. and Tilley, N. (1997) Realistic Evaluation, Thousand Oaks, CA: Sage.
Pawson, R. and Tilley, N. (2009) Realist evaluation, in H.U. Otto, A. Polutta and H. Ziegler (eds) Evidence-Based Practice: Modernising the Knowledge Base of Social Work, Opladen: Barbara Budrich Publishers, pp 151–80.
Pawson, R. et al. (2005) Realist review: a new method of systematic review designed for complex policy interventions, Journal of Health Services Research & Policy, 10(S1): 21–34.
Pillemer, K. and Suitor, J.J. (2002) Peer support for Alzheimer’s caregivers, Research on Ageing, March, 24(2): 171–192.
Pitkala, K.H. et al. (2009) Effects of psychosocial group rehabilitation on health, use of health care services, and mortality of older persons suffering from loneliness: a randomized, controlled trial, The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 64(7): 792–800, doi: 10.1093/gerona/glp011.
Potocnik, K. and Sonnentag, S. (2013) A longitudinal study of well-being in older workers and retirees: the role of engaging in different types of activities, Journal of Occupational and Organizational Psychology, 86(4): 497–521, doi: 10.1111/joop.12003.
Reed, J., Jones, D. and Irvine, J. (2005) Appreciating impact: evaluating small voluntary organizations in the United Kingdom, Voluntas: International Journal of Voluntary and Nonprofit Organizations, 16(2): 123–41, JSTOR, http://www.jstor.org/stable/27927959, (Accessed: 25.05.2023). doi: 10.1007/s11266-005-5694-4
Robertson, M. et al (2001) Economic evaluation of a community based exercise programme to prevent falls, Journal of Epidemiology and Community Health, 55(8): 600–6, doi: 10.1136/jech.55.8.600.
Robitaille, Y. et al. (2005) Moving forward in fall prevention: an intervention to improve balance among older adults in real-world settings, American Journal of Public Health, 95(11): 2049–56, doi: 10.2105/AJPH.2004.057612.
Roy, M.J. et al. (2013) Social enterprise: new pathways to health and well-being?, Journal of Public Health Policy, 34(1): 55–68, doi: 10.1057/jphp.2012.61.
Roy, M.J. et al. (2014) The potential of social enterprise to enhance health and well-being: a model and systematic review, Social Science & Medicine (1982), 123: 182–93, doi: 10.1016/j.socscimed.2014.07.031.
Roy, M.J., Baker, R. and Kerr, S. (2017) Conceptualising the public health role of actors operating outside of formal health systems: the case of social enterprise, Social Science & Medicine, 172: 144–52, doi: 10.1016/j.socscimed.2016.11.009.
Salamon, L.M. and Sokolowski, S.W. (2016) Beyond nonprofits: re-conceptualizing the third sector, VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 27(4): 1515–45, doi: 10.1007/s11266-016-9726-z.
Scottish Government (2014) The Prevention and Management of Falls in the Community: A Framework for Action for Scotland 2014/2015, Edinburgh: Scottish Government, www.nls.uk/scotgov/2014/9781784123345.pdf, (Accessed: 17.12.2019).
Shemilt, I. et al. (2010) Evidence-Based Decisions and Economics: Health Care, Social Welfare, Education and Criminal Justice, Oxford: Wiley-Blackwell.
Social Value Lab (2011) Craft Cafe - Creative solutions to isolation and loneliness. Social Return on Investment Evaluation, Accessible at: http://www.socialvaluelab.org.uk/wp-content/uploads/2013/05/CraftCafeSROI.pdf, (Accessed: 25.05.2023).
Steiner, A. et al. (2021) The role of governments and public policies in social innovation processes, Social Enterprise Journal, 17(2): 157–64. doi: 10.1108/SEJ-05-2021-138
Steiner, A., Caló, F. and Shucksmith, M. (2023) Rurality and social innovation processes and outcomes: A realist evaluation of rural social enterprise activities, Journal of Rural Studies, 99: 284–92.
Taylor, R. and Taylor, R. (2009) What is health technology assessment?, Hayward Medical Communications, NPR09(1114): 6.
United Nations, Department of Economic and Social Affairs, Population Division (2019) World Population Ageing 2019: Highlights, New York, NY: UNDESA.
Ware, J., Kosinski, M. and Keller, S.D. (1996) A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity, Medical Care, 34(3): 220–33, doi: 10.1097/00005650-199603000-00003.
Wikström, B.M. (2002) Social interaction associated with visual art discussions: a controlled intervention study, Aging & Mental Health, 6(1): 82–87, doi: 10.1080/13607860120101068.
Wilson, E. et al. (2009) Befriending carers of people with dementia: a cost utility analysis, International Journal of Geriatric Psychiatry, 24(6): 610–23, doi: 10.1002/gps.2164.
Windle, K. et al. (2010) National Evaluation of Partnerships for Older People Projects, London: Personal Social Services Research Unit, http://eprints.lincoln.ac.uk/7787/2/POPP_Executive_Summary.pdf, (Accessed: 17.12.2019).
Zigmond, A.S. and Snaith, R.P. (1983) The hospital anxiety and depression scale, Acta Psychiatrica Scandinavica, 67(6): 361–70, doi: 10.1111/j.1600-0447.1983.tb09716.x.
