Impact Framework
Canary exists to innovate and drive greater impact in the delivery of emergency relief funds in the workplace and beyond. So, having a thoughtful and deliberate approach to measuring impact is core to what we do every day and central to the design of our emergency relief fund platform, Grant Circle.
Understanding and tracking the impact of emergency relief grants is complex because of the variation in grantees’ circumstances pre- and post-grant, the host of external factors that play a role, and the practical limitations of data collection. It requires considering the potential benefits and limitations of modest but meaningful cash transfers, and how long we can realistically expect impact to persist. For example: the negative fallout arising from a car accident, for example, will differ for Individual A – who has insurance and is generally getting by, but can’t quite afford the entire deductible – as compared to Individual B – who is struggling to make ends meet on a daily basis, while juggling childcare and multiple jobs that require transportation via personal auto. When you layer in variation due to availability of resources within grantees’ social networks, different types of hardships, and timing – before versus after an event like the COVID-19 pandemic – the challenge is clear.
We use a multi-faceted approach to impact assessment to capture these different outcomes as much as we can. There are moments when even a relatively small amount of money can change the trajectory of someone’s life, and we want to identify those moments. On the other hand, even when that’s not the case, the grant can still be important and impactful. The impact measurement framework that we developed aims to assess immediate and longer-term impacts, asking applicants directly about the impacts they experienced while also capturing data on a handful of indicators across multiple points in time. We seek feedback from both approvals and declines, and gather data about applicants’ demographics and financial situation prior to grant application in order to contextualize impact findings.
Pilot
Collaborating with two partners, GreenPath Financial Wellness and the Red Tab Foundation, and with the generous support of Flourish and the Robin Hood Foundation, we disbursed over 60 grants during the pilot period from September 2019 – February 2020. We surveyed grantees seven days, three months, and six months after grant disbursement.(1)Grants were capped at $1,000, though over 80% of grants were for $400 or less. We received at least one post-grant survey related to 38 grants, 58% of those in the pilot.
Though small proof-of-concept programs generate small samples for analysis, what we found is consistent with other sources, including findings from The Workers Strength Fund – an emergency cash grant program focused on gig workers – and with data from a larger impact study we conducted with an emergency relief fund provider that will be released later this year.
What it adds is an examination of longer-term outcomes, with a focus on what we can glean when we have information from grantees at multiple post-grant junctures. This initial set of findings is a strong foundation which has informed our framework and ongoing measurement efforts.
Pilot Insights
Our quantitative data – as well as the qualitative interviews conducted with pilot participants – highlight the meaningful, immediate impacts of small-dollar cash grants, and suggest they may also contribute to lasting, positive effects.
Immediate Impacts
All grantees who responded to the seven-day survey report at least one positive by-product of the grant.(2)More than eight in ten respondents say the grant provided breathing room so they could figure out how to proceed. More than eight in ten respondents also feel supported by their employer in the immediate aftermath of grant receipt. Grants help employees avoid late fees, utility interruptions, and lateness or absence from work. This is particularly true for lower-income workers (those with annual household income under $60,000) who also report experiencing negative events prior to grant receipt at higher rates than their higher-income peers.
Longer-term Outcomes
All six-month surveys and about a third of three-month surveys were completed after the COVID-19 outbreak in the U.S., between March – August 2020. Given that, our findings about long-term stress and ability to pay bills are quite surprising.We observe a statistically significant decrease in high financial stress at all post-grant junctures when we compare rates of high financial stress at account creation (pre-grant) with that which is reported at each post-grant juncture (7 days, 3 months, and 6 months) by grantees who answered at least one post-grant survey.
We also observe a statistically significant increase in ability to pay most to all bills on time at three and six months post-grant, when we compare bill payment rates for the past year at account creation (pre-grant) with bill payment rates for the last 1-2 months reported by grantees who answered at least one post-grant survey (at 3 months and/or 6 months).From this small pilot dataset, we clearly cannot make claims about causation. The longer-term outcomes observed could be a function of response bias; individuals who chose to respond to the survey may be more likely to have jobs that were not affected by COVID-19 or may have been able to secure supplemental unemployment benefits, for example. They could be related to other relief efforts, such as moratoria on evictions, waiving of late fees, or postponement of debt payments.
The financial stress finding might also reflect how people process stress and when they feel it most acutely. For example, a specific hardship impacting an individual household may be experienced as more stressful than a global event affecting everyone. When we look at the survey data collected for grants made beginning in April 2020, however, we have early indications of another trend. For those grants, we see high rates of financial stress decline even more sharply at the seven-day mark, but they appear to rebound somewhat at three months. The sample size is still quite small, and it’s difficult to tease out all of the possible influences, but the impact of COVID is unmistakable, particularly as two-thirds of these grants are COVID-related. We look forward to examining these trends and questions more as we collect additional data.
Implications
Our conversations with grantees, coupled with their comments in surveys and during the grant management process, underscore the toll that hardships take on families when they do not have access to resources to manage them. Alleviating stress and anxiety, and preventing the negative domino effect that hardships can trigger, are worthwhile pursuits. That is further reinforced when we consider the positive impacts outlined above, achieved with an average grant size of just $438. However, the data also underscores that emergency relief grants are good for business.
In addition to feeling supported by their employer in the immediate aftermath of grant receipt, grantees who responded to surveys three and six months post-grant also say the program makes them feel better about working for their employer. Numerous sources report that satisfied employees take better care of customers and are less likely to leave their jobs. Given the high costs associated with dissatisfied customers and employee turnover, the potential return from these impacts is significant. For example, according to Gallup, the cost of replacing an individual employee can range (conservatively) from one-half to two times the employee’s annual salary. For a 100-person organization with an average salary of $50,000, turnover and replacement costs alone could range from $660,000 to $2.6 million per year. Impacts to customer relationships, morale, and reputation could drive additional financial and non-financial costs.
Similarly, financial stress at work is linked to absenteeism, lower productivity, and higher healthcare costs. Financial stress costs U.S. employers $250 billion annually, according to several sources, including MetLife and Mercer.(3) Another estimates that financial stress costs more than $1,900 per year per employee, totaling $1 million for midsized employers and $19 million for large employers.(4)
What’s Next
During our six-month pilot, we gleaned insights not just about impact and our measurement framework, but also about managing emergency relief fund programs. We applied these lessons to the design of our new Grant Circle platform which launched in April, with an evolved grant structure and slightly modified impact surveys. As we work with additional employers and Grant Circle program sponsors to support their communities through COVID-19 and beyond, we continue our commitment to impact measurement and look forward to sharing what we learn.
We are grateful for the support of Flourish and the Robin Hood Foundation, and partnership from GreenPath Financial Wellness and the Red Tab Foundation, without whom the pilot would not have been possible.
Want additional detail about the pilot or the insights we gleaned? Reach out to Aliza Gutman at aliza@workwithcanary.com; we’d love to discuss them further with you and share more detail. Want more information on Canary or the Grant Circle platform? Contact Kimberly Gartner at kimberly@workwithcanary.com.
1) We also surveyed declined applicants, but the declined sample was too small for analysis.
2) We received twenty-four responses to the seven-day survey.
3) According to a 2019 MetLife report: Financial stress can lead to distraction at work, absenteeism, and high turnover…For a company of 10,000, this can mean 1,922 hours and $28,830 in lost productivity every week. Overall, employers report $250 billion lost each year due to stressed employees at work.
According to a 2017 Mercer report: Employers could lose up to $250 billion in lost wages due to employees’ stress about their personal finances. Employees reported spending approximately 150 hours annually of their work time worrying about money.
