Estimating population-level wellbeing in real time

Qntfy
3 min readJun 17, 2020

Key results from recent work on the wellbeing and mental health of US healthcare providers during COVID-19.

We recently submitted a paper that demonstrates the feasibility of using social media data to measure the impact of specific events on the wellbeing of a population in real time. We focus on changes in the rates of anxiety, depression, and suicidality among healthcare providers (HCPs) and the general population in response to critical moments in the unfolding of the COVID-19 pandemic, the killing of George Floyd, and subsequent demonstrations for racial justice and police reform. We used machine learning techniques described in our previous work, to estimate the probability that a given user was experiencing anxiety, depression, or suicidality, based on their social media posts. (We report only aggregated statistics here and elsewhere.)

(We constructed a sample of HCPs (n=25,040) and a sample of users drawn from the general population across the country who were not healthcare providers (n=10,000). Below, we’ll refer to the latter as simply the Community sample.

We established a baseline for each group by calculating the average scores on all three mental health measures in the “pre-COVID” period prior to February 29, 2020 (up to two weeks prior to the declaration of a national emergency for COVID-19). The HCPs scored worse on every mental health measure during this pre-COVID period as compared to the community. In order to draw comparisons about the relative change within each group by these events, we examined normalized differences from that group’s baseline baseline in the figures below as z-scores.

Overall, we saw that every time period after COVID exhibits significantly increased rates for all mental health conditions we examined, and no group has yet returned to pre-COVID baseline levels for any of these measures. Moreover, the killing of George Floyd and subsequent unrest had a significant effect on mental health across all groups.

Drilling down into specific mental health conditions…

HCPs exhibit less change in their anxiety over time compared to Community (though HCPs are still at a higher base rate of anxiety).

HCPs show a larger change in suicide-related risk during Early Lockdown. This disappears in Mid Lockdown and gets closer to returning to baseline rates towards the end of May (note, again, that baseline rates for HCPs remain higher than for Community).

Longitudinal changes in depression do not differ between HCPs and Community (p>0.1).

Why does this matter?

Measurements of the mental health and wellbeing of large populations often become quickly outdated, given traditional techniques for data collection, analysis, and dissemination. For example, estimates of suicide rates within the United States are often delayed by two years. More up-to-date information about population-level mental health will allow for more appropriately tailored policies and interventions, while also providing more readily-accessible feedback for the evaluation of these interventions.

--

--