Abstract
This paper presents the findings of longitudinal research conducted in Ethiopia exploring the effects of COVID-19 school closures on children’s holistic learning, including their socio-emotional and academic learning. It draws on data from over 2,000 pupils captured in 2019 and 2021 to compare primary school children’s dropout and learning before and after school closures. The study adapts self-reporting scales used in similar contexts to measure grade 4–6 pupils’ social skills and numeracy. Findings highlight the risk of widening inequality regarding educational access and outcomes, related to pupils’ gender, age, wealth and location. They also highlight a decline in social skills following school closures and identify a positive and significant relationship between pupils’ social skills and numeracy over time. In conclusion, we recommend a need for education systems to promote children’s holistic learning, which is even more vital in the aftermath of the pandemic.
Key messages
In Ethiopia, COVID-19 school closures widened gender, age and wealth inequalities in education.
Longitudinal data show a decline in Ethiopian pupils’ social skills during the school closures.
There is a strong and positive relationship between pupils’ social skills and numeracy over time.
Introduction
School closures have affected the educational trajectories of many learners globally due to the COVID-19 pandemic. As a result, dropout from school and learning loss have become a pressing concern. This is especially true in education systems in low- and lower-middle-income countries that were struggling to achieve basic outcomes for all children even before the pandemic.
Ethiopia is an important example of already low levels of primary school completion and learning. As a result of the pandemic, an estimated 26 million learners were out of school for a minimum of five months, and often longer (Yorke et al, 2020; 2021a). During this time, efforts to support students’ learning remotely did not reach all students, especially those who are most disadvantaged. In addition, these children also missed out on important in-school support such as school feeding programmes, emotional support in safe and protective spaces, and opportunities for them to strengthen their capacities to build relationships and handle the heightened stresses of everyday life (Jenkins, 2020; UNESCO, 2020). Even after schools reopened, emerging evidence suggests that not all pupils have returned to continue their education (Favara et al, 2021). For many children in Ethiopia and other low-income countries, these school closures and subsequent dropout and learning loss may have lifelong consequences.
The impact of the COVID-19 school closures on students’ education has highlighted the need for a more holistic approach to assessing learning. A growing body of evidence has drawn attention to the importance of socio-emotional learning (SEL), especially in the context of the COVID-19 pandemic (Yorke et al, 2021b). More broadly, the literature has identified how SEL offers valuable benefits in terms of educational progress and attainment, wider life outcomes and the ability to navigate adverse situations and circumstances (Dercon and Krishnan, 2009; Brunello and Schlotter, 2011; INEE, 2016). In the context of growing inequalities and adversity, SEL therefore may be particularly relevant. However, the majority of evidence concerning the role and value of SEL has come from high-income contexts, while little is known about children’s SEL in low- and lower-middle-income countries, including how it interacts with the acquisition of academic skills such as literacy and numeracy.
This paper aims to address that gap using longitudinal data collected from over 2,000 primary school pupils in grades 4 to 6 in Ethiopia during the period 2019 to 2021, before and after the COVID school closures. Our study design allows us to track the educational trajectories of students during this critical period, identifying which children returned to school and which children dropped out. Importantly, we explore changes in students’ SEL, specifically their social skills, and academic outcomes over the period, and the relationship between these different aspects of learning.
The paper is structured as follows: the next section discusses the Ethiopian education context; we then review the relevant literature regarding children’s holistic learning focused on low- and lower-middle-income countries; the subsequent section outlines the methodology used and its limitations; then we set out the key results and discuss the findings before concluding the paper.
Ethiopian context
Since 2000, Ethiopia has made considerable progress to expand access to primary education. According to official government statistics, universal primary enrolment is close to being achieved nationally, although with considerable variations across regions and rural–urban locations (MoE, 2020a). Gender gaps have narrowed over time, although girls continue to face additional challenges in benefiting from education. Overall, learning levels are low (Woldetsadik, 2013; Iyer et al, 2020).
Following the confirmation of the first case of COVID-19 in Ethiopia in March 2020, the government closed all schools. To support learners during COVID-related school closures, the government put in place strategies to support students’ distance learning, which included educational programmes broadcast through radio for primary school pupils, and explicitly stated the need to provide additional support to disadvantaged students (MoE, 2020b). However, evidence suggests that the support provided did not reach all learners, especially those who are most marginalised due, for example, to the limited access to resources and infrastructure needed to continue learning, such as electricity and technology (Yorke et al, 2020).
From October 2020 onwards, schools started to reopen in a phased approach. As the school year runs from September to June, this meant that pupils missed out on around five months of schooling, as they were automatically promoted to the next grade. Challenges during this process included ensuring that all students returned to school, helping them to catch up on lost learning and catering for the diverse needs of different groups. In Ethiopia, as in many other countries around the world, there were fears that learners from poor families and facing other disadvantages would be at greater risk of dropping out and learning loss due to lower levels of household resources (such as radios) and parental support during school closures (Yorke et al, 2020; 2021a). Particular concerns have been raised with respect to girls’ educational opportunities in Ethiopia, given they have been disproportionately affected by the pandemic with evidence highlighting increased domestic responsibilities, greater risk of abuse and sexual exploitation, and increased likelihood of early marriage and teen pregnancy (Tiruneh, 2020; Ford et al, 2021).
Literature review
To date, much of the available literature has focused on the effects of school closures on students’ academic learning (Moscoviz and Evans, 2022; Patrinos et al, 2022). However, the pandemic has also highlighted the importance of assessing children’s holistic learning, including both academic and socio-emotional outcomes (Cardero, 2020; Partnership for Resilience, 2020; UNESCO, 2020).
SEL is generally understood as comprising the process of developing a wide set of holistic skills, traits and aptitudes that have value both in and of themselves, and also play a role in acquiring other key attributes and competencies. Various frameworks exist to conceptualise and operationalise SEL, sometimes using terms like ‘psychosocial skills’, ‘non-cognitive skills’ and ‘21st century learning’ to emphasise children’s particular outcomes (Duckworth and Yeager, 2015). Across these frameworks, however, there is consensus that SEL concerns learners’ social skills to communicate and collaborate with other people, their abilities to manage individual emotions and control behaviours, and several broader competencies to achieve goals, solve problems and make responsible decisions (Zins and Elias, 2007; Jones and Doolittle, 2017; CASEL, 2020).
Literature highlights numerous benefits of SEL for shaping learners’ lives and wider societal progress. Evidence from studies in both high- and low-income countries describes a positive and possibly bidirectional relationship between aspects of SEL and children’s early academic skills and subsequent progression (Blair and Razza, 2007; Brunello and Schlotter, 2011; von Suchodoletz et al, 2015; Wolf and McCoy, 2019). Beyond schooling, SEL appears to support coping skills for maintaining good mental health, lay foundations for building positive and healthy relationships, and even predict income and health many years later (Immordino-Yang and Damasio, 2007; Moffitt et al, 2011; Diamond, 2014; UNESCO, 2020). The importance of these benefits for holistic life outcomes has become even more pronounced in the context of the pandemic, given the increased challenges and inequalities facing many learners and their families.
To date, however, there has been relatively limited research on children’s holistic development and SEL in low- and lower-middle-income countries, particularly over time and in the context of major shocks like the pandemic. In part, this reflects some of the complexities of measuring SEL and key aspects like social skills, not least the need to situate them within the relevant cultural context, and to ensure comparability and equivalence when adapting tools from one setting for use in another (Jukes et al, 2018). Studies of SEL in low- and lower-middle-income countries have nevertheless examined children’s psychosocial development and well-being in contexts affected by disaster, conflict or migration, and evaluated the impact and success of different educational programmes and interventions to boost students’ holistic learning (Borisova et al, 2017; Abimpaye et al, 2019; Borzekowski et al, 2019; Forsberg et al, 2019; INEE, 2016; Tubbs Dolan, 2019). Longitudinal research by Wolf and McCoy (2019) on the bidirectional relationship between preschool children’s SEL and emerging academic skills in Ghana also revealed that while early literacy and numeracy predicted all aspects of SEL measured, the reverse was not necessarily true. Meanwhile, studies on learners’ social skills in low-income settings tend to focus on children with special educational needs, such as autism spectrum disorder (Nnamani et al, 2019; Major et al, 2021).
There is also some evidence available regarding children’s SEL and developmental trajectories in Ethiopia through the Young Lives programme. Since 2002, the programme has collected rich longitudinal data through a combination of quantitative and qualitative methods to investigate the association between SEL and household poverty, changes in agency and self-efficacy during adolescence, children’s well-being, and experiences following a parental death (Camfield and Tafere, 2009; Dercon and Krishnan, 2009; Himaz, 2009; Camfield, 2012; Espinoza Revollo and Ogando Portela, 2019). More recently, the programme has investigated adolescents’ experiences and welfare during school closures due to COVID-19, including with respect to their resilience and mental health, and the particular impact on girls (Favara et al, 2021; Ford et al, 2021; Porter et al, 2021; van der Gaag, 2021).
However, in low- and lower-middle-income countries worldwide, including in Ethiopia, there remains a clear gap in longitudinal evidence on how the SEL of primary school–aged children evolves over the course of their formal education. As yet, there have been no longitudinal analyses examining changes in both academic outcomes and SEL for primary school–aged children during the pandemic, either in Ethiopia or other low-income countries. This paper therefore addresses that research gap by drawing on quantitative data from 2019 and 2021, including data regarding children’s social skills, as the next section outlines.
Methodology
Research purpose and questions
We use longitudinal data from the same pupils in Ethiopia between 2019 and 2021 (before and after school closures) to understand pupil dropout and learning trajectories. In particular, we explore the interaction between academic outcomes and social skills as a proxy for SEL, taking account of wider child and other background factors. In particular, we seek to answer the following three research questions (RQs):
- 1.What factors are associated with Ethiopian children dropping out of primary school following the COVID-19 school closures?
- 2.How have Ethiopian children’s SEL and academic learning changed during the COVID-19 school closures?
- 3.What is the relationship between children’s SEL and numeracy over this period, once other factors are taken into account?
As described later, we draw on quantitative data collected by the Research on Improving Systems of Education (RISE) programme in Ethiopia in June 2019 (at the end of the 2018/19 academic year), and again in January 2021 (once schools had reopened). The RISE data tracked primary school students from the lower level of primary education prior to school closures, through the transition into the upper level of primary education once schools reopened.
Measuring social skills
We used established self-reporting scales to measure children’s social skills during both rounds of data collection. These involve participants rating themselves and their level of agreement with statements like ‘I make friends easily’ and ‘I like to share things with others’. Such scales provide an effective tool for capturing responses from large groups within narrow timeframes, offer access to feelings and attitudes that may be difficult to observe, and have been used by previous studies to examine children’s SEL in diverse settings (Haroz et al, 2017; Assessment Work Group, 2019; Forsberg et al, 2019).
In particular, we measured pupils’ social skills as an aspect of their wider SEL. At both time points, participants rated these skills using the same items from the Children’s Self Report Social Skills Scale, which had been previously used and validated in Ethiopia and other similar contexts (Danielson and Phelps, 2003). This enabled us to compare children’s responses before and after the school closures.
Prior to their use, the Children’s Self Report Social Skills Scale items were reviewed by Ethiopian experts to ensure that they were culturally sensitive and age-appropriate for the primary Ethiopian pupils. The specialists, for example, queried the inclusion of ‘I look at people when I talk to them’ as being unsuitable in the context given prevailing Ethiopian norms around children making eye contact with adults. Following this review process, we identified seven suitable items which were then finalised, translated carefully into seven languages and piloted with a smaller number of Ethiopian learners to check the psychometric properties of the scale before the main data collection. The items are set out in Appendix 1, Table 1A. For example, pupils rated themselves on statements like ‘Other people like me’ and ‘I help others when they need help’ using a five-point Likert scale with the following options: Strongly Disagree; Disagree; Undecided; Agree; and Strongly Agree (for information on using Likert scales, see Krosnick and Presser, 2010 and Leung, 2011).
The learners were guided in groups to complete the surveys, responding individually on their own with pencil and paper to keep their answers confidential. The guidance ensured that all pupils understood the statements and instructions, regardless of their reading abilities. For ethical reasons and common to other surveys of this kind in Ethiopia, we obtained the children’s voluntary informed assent at the start of the surveys and minimised the number of statements to reduce time out of lessons, and therefore any disruption to the pupils’ learning (Homan, 2001).
Learning assessments
The participating children also completed literacy and numeracy assessments in both 2019 and 2021, which included common items across the two rounds of data collection. The tests used established instruments adapted from Young Lives, which had been developed drawing on guidance from the Ethiopian Ministry of Education and the National Educational Assessment and Examinations Agency (Kim et al, 2021). Example tasks for literacy included reading exercises and answering basic comprehension questions.
Across regions in Ethiopia, a wide range of languages are spoken and used in the classroom. Given linguistic differences, direct comparability of pupils’ literacy and reading scores are problematic (Yorke and Ogando, 2018). For this reason, the study focuses on children’s numeracy in the analyses related to academic outcomes. The numeracy assessments comprised 25 multiple-choice items in 2019 and 30 items in 2021, with 15 common anchor items across the tests to enable comparison. These required children to perform basic tasks such as putting four numbers into ascending order, identifying shapes and solving problems using fractions and division.
Children’s performance on the tests was analysed using item response theory (IRT), which identifies the difficulty of specific tasks and places them on a common scale (Tiruneh et al, 2021). Specifically, we used a concurrent calibration approach and fitted a two-parameter logistic model to the item responses, which estimated them simultaneously on an interval scale using pooled data from both time points. Responses to the items that were unique to either 2019 or 2021 were treated as missing for the pupils that did not complete them. The anchor items thereby provided a link between the tests while the unique items increased the precision of the individual estimates. Learners’ scores were then transformed and standardised to have a mean of 500 and a standard deviation of 100. This approach enabled us to directly compare pupils’ performance in 2019 with their scores in 2021.
Covariates and background information
We further captured data regarding background characteristics of the children and their households at one or both time points. These variables were necessary to investigate factors associated with pupil dropout and changes in learning, and to control for factors that could affect the relationship between social skills and academic outcomes. Key background information about the children and their households were recorded before the pandemic in 2019, including their age, gender, family size and wealth, and whether or not their primary caregiver could read a basic sentence. We also captured data regarding their teachers’ qualifications and years of experience in order to understand their associations with different aspects of children’s learning. A narrower set of characteristics was then resurveyed after schools reopened in 2021, specifically the pupils’ age, class grade and the language spoken at home. In the longitudinal analyses, we used variables and covariates drawing on the 2021 responses wherever relevant, especially for factors like age and grade that were expected to have changed. For the other background characteristics, it was assumed that there would not have been significant changes over the two years, so we used data from the 2019 round of collection.
Sampling and descriptive statistics
Sampling for the research drew on the approach adopted by Young Lives in Ethiopia and built on by earlier rounds of RISE data collection (Hoddinott et al, 2019). Drawing on previous longitudinal data captured by RISE enabled us to achieve an illustrative picture of learning across Ethiopia and to track the same children over time to generate longitudinal evidence regarding their social skills and dropout. A total of 138 government schools were selected from six regions in proportions according to their population size and the balance of people living in rural and urban areas (Table 1). Within schools, pupils were randomly sampled. Changes in the sample are examined in further detail below with respect to the attrition analysis.
Data collection breakdown by region, round and pupil gender
| Region | Number of schools | 2019 data collection | 2021 data collection | ||||
|---|---|---|---|---|---|---|---|
| Female | Male | Total | Female | Male | Total | ||
| Addis Ababa | 20 | 236 | 239 | 475 | 215 | 219 | 434 |
| Amhara | 25 | 254 | 274 | 528 | 219 | 231 | 450 |
| Benishangul Gumuz | 14 | 161 | 195 | 356 | 101 | 128 | 229 |
| Oromia | 37 | 396 | 440 | 836 | 297 | 368 | 665 |
| SNNPRa | 22 | 252 | 224 | 476 | 196 | 179 | 375 |
| Somali | 20 | 163 | 150 | 313 | 141 | 122 | 263 |
| Total | 138 | 1,462 | 1,522 | 2,984 | 1,169 | 1,247 | 2,416 |
| Percentage | – | 49.0 | 51.0 | 100.0 | 48.4 | 51.6 | 100.0 |
Note: a Southern Nations, Nationalities and Peoples’ Region.
Table 2 sets out the descriptive statistics for the matched sample to provide a picture of the data and the types of children surveyed. In 2019, the children in the sample were enrolled in grade 4 and typically 11 years old. By 2021, these children were mainly in grade 6 with a mean age of 13 years, having transitioned from the lower to the upper level of Ethiopian primary education (MoE, 2020a). In 2021, 15% were over age for their grade. This is calculated as being two or more years older than the intended age, being 12–13 years old for pupils in grade 6. The vast majority of children, 91.1%, were ‘on track’ and had been promoted from grade 4 into grade 6 between 2019 and 2021, without the apparent need for them to repeat a year of schooling. Only just over a quarter of pupils’ caregivers could read a basic sentence and most of their teachers held a professional diploma or above. In some cases, data were missing for individual children, for example, regarding their household size or caregiver literacy. For those learners, we used multiple imputation to estimate their values on the missing variables, based on their other responses.
Descriptive statistics for Ethiopian sample
| Variable | Grade 6 |
|---|---|
| Child and household characteristics | |
| Female (%) | 48.4 |
| Average age (years) | 13.0 (1.5) |
| Over age (%) | 15.0 |
| ‘On track’ (%) | 91.1 |
| Rural (%) | 61.8 |
| Caregiver literacy (% literate) | 27.8 |
| Average household size | 5.9 (2.1) |
| Teacher characteristics | |
| Teacher holds certificate (%) | 10.8 |
| Teacher holds diploma (%) | 80.9 |
| Teacher holds degree (%) | 6.6 |
| Average teacher experience (years) | 4.8 (4.4) |
| Number of observations | 2,416 |
Note: Standard deviations shown in parentheses (where appropriate).
Analytical approach and attrition
We used a mix of bivariate analyses and multivariate regressions to answer the RQs in order to investigate children’s dropout and learning during the school closures. First, analysis for RQ1 entailed logistic regressions to explore the key factors that predict the likelihood of a child enrolled and surveyed in 2019 dropping out before the data collection in 2021. Second, we used standardised z-scores for pupils’ social skills in a panel data set combining 2019 and 2021 responses, as well as the IRT numeracy scores, to address RQ2 and ran t-tests to examine the significance of changes over time and differences by learners’ gender and location. This entailed examining the social skills scores for their psychometric properties at both time points, summarised in Appendix 2, creating standardised z-scores with means of 0 and standard deviations of 1 for each item, and then averaging the z-scores across the seven social skills items (Espinoza Revollo and Ogando Portela, 2019). (Histograms showing the distribution for both social skills and IRT numeracy scores are also set out in Appendix 2.) Finally, the analyses for RQ3 used a further panel data set and multivariate linear regressions to examine the relationship between pupils’ social skills and academic outcomes between 2019 and 2021. We further clustered standard errors by school to accommodate the effects of similarities between pupils according to the school they attend.
Before conducting these analyses, we investigated patterns of attrition between the data collection in 2019 and 2021. In total, 2,741 children participated in the 2019 data collection and completed the social skills assessment. Of these, 2,431 were still enrolled in 2021, of whom 2,416 participated in the second round of data collection. This reduction from 2,741 in 2019 to 2,416 in 2021 equates to an attrition rate of 11.9% of learners in 2019 who were missing from the 2021 data collection. The majority of these children had dropped out from school (see below in response to RQ1 for further details) but the difference between the attrition and dropout rates concerns learners who were enrolled in school but could not be physically located at the time of the 2021 data collection. Reasons given included both temporary absence for children who were ill or in hospital on the day of the surveys, to longer-term arrangements where learners had changed school or migrated to different areas with their families. Eight pupils also declined to take part in the study.
To examine patterns of attrition, we conducted analyses using logistic regressions to predict the likelihood of a child being unavailable for assessment in 2021, taking other variables into account. The results in Table 3 identify that older children were more likely to drop out between 2019 and 2021, but there is also lower attrition for learners from families in the wealthiest tercile. In light of these results and to minimise bias arising from non-random attrition, we calculated attrition weights which we used in the models. We followed the approach outlined by Fitzgerald et al (1998), which entailed running two probit models, one to estimate attrition as a function of the variables that we predicted to affect our outcome of interest, and a second that also included variables that we considered to only affect the likelihood that a child remains in school. We then used the predicted probabilities from both models to construct the attrition weights as the ratio of these predicted probabilities.
Logistic regression results showing the likelihood of attrition between 2019 and 2021
| Grade 4/6 | ||
|---|---|---|
| Gender (base group: female) | −0.21 (0.12) | |
| Age | 0.15*** (0.03) | |
| Caregiver literacy (base group: no/low literacy) | −0.01 (0.15) | |
| Family wealth (base group: first tercile/poorest) | Second tercile | −0.16 (0.15) |
| Third tercile | −0.41* (0.19) | |
| Household size | −0.02 (0.03) | |
| Location (base group: rural) | 0.08 (0.17) | |
| Region (base group: Addis Ababa) | Amhara | 0.26 (0.25) |
| Benishangul Gumuz | −0.63 (0.36) | |
| Oromia | 0.09 (0.25) | |
| SNNPR | 0.61* (0.25) | |
| Somali | 0.65* (0.27) | |
| Constant | −3.55*** (0.47) | |
| Observations | 2,741 | |
| Pseudo R-squared | .03 | |
Notes: * p < .05, ** p < .01, *** p < .001. We used the results in this model to inform the attrition weights used in the subsequent analyses.
Limitations
Notwithstanding these measures to minimise bias, the findings reported in the next section must be interpreted in light of several important limitations. First, given the research design, we cannot make claims regarding direct causality. As such, although we draw on longitudinal data to examine changes in children’s social skills, numeracy and school enrolment, we cannot unequivocally claim that any changes during the period of the COVID school closures necessarily result from disruption associated with the pandemic.
Second, the study only measured one aspect of learners’ SEL, their social skills, at both time points and future research could usefully examine more components of children’s SEL over time. Third, the findings must be understood subject to the limitations of using self-reporting measures to investigate pupils’ SEL. For example, more respondents agreed with the items than disagreed with them, which created a negative skew in the distributions. The ratings also represented how learners felt about their skills on the specific day, which could vary over time or differ from their actual competencies. There are further questions regarding if and how children’s perceptions and responses evolve as they grow up and mature. Indeed, recent OECD (2021) research discovered that 15-year-olds reported lower SEL than 10-year-olds, which could reflect short-term fluctuations in attitudes or older learners becoming more self-conscious, self-aware and therefore more self-critical as they progress into adolescence (Soto, 2016). However, the scales that we used have been validated in other related contexts, and we also validated them further using factor analyses, summarised in Appendix 2 (Sushama et al, 2000; Hussein, 2013; Yorke et al, 2021c).
Results
This section presents the findings of the study, taking each RQ in turn.
Dropout from primary school following school closures
The first RQ focuses on the factors associated with Ethiopian children dropping out of primary school during the COVID-related school closures. Overall, 2,741 pupils in the sample were enrolled in grade 4 during the data collection in 2019. Of those children, 2,431 were still enrolled in 2021 after the schools had reopened. The difference of 310 learners equates to a dropout rate of 11.3% of the total initial sample. To examine factors associated with dropout, we ran logistic regressions to identify variables that are associated with the probability of a child dropping out between 2019 and 2021. In the first model, we included learners’ gender, age, caregiver literacy, region, location (whether they lived in a rural or urban area) and their family’s size and wealth tercile. In the second model, we added a variable for pupils’ standardised IRT numeracy scores in 2019 to understand any relationship between learners’ dropout and their academic performance (Table 4).
Logistic regression results (odds ratios) showing the likelihood of dropout between 2019 and 2021
| Model 1: child, family and school characteristics | Model 2: Model 1 + 2019 IRT numeracy scores | ||
|---|---|---|---|
| Gender (base group: female) | 0.75* (0.09) | 0.76* (0.10) | |
| Age | 1.16*** (0.04) | 1.17*** (0.04) | |
| Caregiver literacy (base group: no/low literacy) | 1.00 (0.15) | 1.01 (0.16) | |
| Family wealth (base group: first tercile/poorest) | Second tercile | 0.84 (0.13) | 0.84 (0.13) |
| Third tercile | 0.69 (0.14) | 0.69 (0.14) | |
| Household size | 0.98 (0.03) | 0.98 (0.03) | |
| Location (base group: rural) | 1.08 (0.19) | 1.14 (0.20) | |
| Region (base group: Addis Ababa) | Amhara | 1.23 (0.32) | 1.16 (0.30) |
| Benishangul Gumuz | 0.59 (0.22) | 0.48 (0.18) | |
| Oromia | 1.17 (0.30) | 1.07 (0.27) | |
| SNNPR | 2.01** (0.52) | 1.74* (0.46) | |
| Somali | 1.71 (0.47) | 1.41 (0.41) | |
| 2019 IRT numeracy scores | – | 0.85* (0.06) | |
| Constant | 0.03*** (0.01) | 0.02*** (0.01) | |
| Observations | 2,741 | 2,741 | |
| Pseudo R-squared | .03 | .04 | |
Notes: * p < .05, ** p < .01, *** p < .001. Standard errors are shown in parentheses and based on the original log-odds scale. These models are also adjusted to include attrition weights, recognising that the attrition rate (11.9%) was slightly higher than the dropout rate (11.3%) as some enrolled learners could not be located on the day of the 2021 assessment.
The results reveal, first, that the learners’ gender predicts whether or not they dropped out during the closures. In both models, the significant odds ratios below 1 indicate that girls were significantly more likely to drop out than boys between the surveys in 2019 and 2021. Second, the significant odds ratios above 1 for pupils’ age shows that older learners dropped out with greater likelihood than younger children. Third, the significant odds ratio below 1 in the second model reveals that pupils achieving high numeracy scores in 2019 were less likely to drop out than their low-performing peers. In both models, family wealth is not significant, which potentially reflects its interaction with the other variables, such as household size, caregiver literacy or location, in an urban or rural area.
Taken together, these findings show that girls, older pupils and low performers in 2019 were significantly more likely than their peers to have dropped out of school by the time the surveys were conducted in 2021. This suggests that the pandemic has widened existing inequalities between learners, and that children who were already disadvantaged in 2019 may now face even greater obstacles to attend and complete school.
Changes in social skills and numeracy following school closures
To respond to RQ2, we next examined longitudinal changes in social skills and learning outcomes among pupils who were enrolled in school during both rounds of data collection. Such information is important for education systems in Ethiopia and elsewhere to understand changes in children’s learning during the pandemic and to inform strategies to support pupils to catch up as schools reopen.
Figure 1 shows the social skills z-scores for the panel of learners who were surveyed in both 2019 and 2021. The red line indicates the overall decline, while the other lines show the trends by groups disaggregated for learners’ location and gender. Tables 5 and 6 also set out the mean z-scores for different groups of pupils over time and a comparison of z-scores between groups at both time points respectively. In each case, the tables show the results of t-tests to explore the significance of the differences, using binary variables for the survey year and learners’ gender and location.
Mean z-scores and t-test results for differences in social skills between 2019 and 2021
| Group | 2019 social skills z-score | 2021 social skills z-score | Change | t-value |
|---|---|---|---|---|
| Overall | 0.16 | −0.17 | −0.33 | 16.83*** |
| Urban girls | 0.26 | 0.01 | −0.25 | 6.31*** |
| Rural girls | 0.15 | −0.26 | −0.41 | 10.48*** |
| Urban boys | 0.14 | −0.06 | −0.20 | 4.93*** |
| Rural boys | 0.13 | −0.26 | −0.39 | 10.87*** |
Note: * p < .05, ** p < .01, *** p < .001. n = 2,416.
Mean z-scores and t-test results for differences in social skills in 2019 and 2021
| Measure | Girls | Boys | t-value | Rural | Urban | t-value |
|---|---|---|---|---|---|---|
| 2019 social skills | 0.20 | 0.14 | 2.50* | 0.14 | 0.20 | −2.41* |
| 2021 social skills | −0.15 | −0.18 | 1.09 | −0.26 | −0.02 | −7.51*** |
| Change in social skills | −0.35 | −0.32 | −0.71 | −0.40 | −0.22 | −4.54*** |
Note: * p < .05, ** p < .01, *** p < .001. n = 2,416.
Changes in social skills between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Changes in social skills between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Changes in social skills between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Overall, most learners indicated similar levels of social skills in 2019, with the exception of urban girls whose responses were noticeably higher. However, all pupils showed a statistically significant decline in social skills over the period of the COVID-19 school closures. This means that fewer children in 2021 agreed with statements like ‘Other people like me’ and ‘I make friends easily’ than in 2019.
Table 6 also highlights significant differences between the groups of learners at one or both time points. Specifically, girls reported stronger social skills than boys in 2019 but this difference disappeared in 2021. By contrast, urban learners showed significantly higher social skills than rural children during both rounds of surveys. Further, although all learners reported reductions in social skills, the gradient of decline is steepest for the rural pupils, especially compared to the urban boys, which suggests widening disadvantage and the greatest losses in skills for children in rural locations.
The trends in learners’ numeracy during the school closures, however, look quite different. The red line in Figure 2 shows an overall small increase in numeracy scores while Table 7 compares mean IRT scores in 2019 and 2021, in both cases with performance further disaggregated by gender and location. At each time point, pupils in urban settings scored significantly higher than learners in rural areas. Boys also performed significantly better than girls in 2019 but this difference had disappeared by 2021. The chart further indicates that all groups of learners showed some gains in average numeracy, although these started from a low base and the extent of such improvements also varied depending on pupils’ gender and location. Specifically, urban and rural girls display similar numeracy gains during the COVID-19 school closures, whereas urban boys show the greatest progress and rural boys the smallest improvements.
Mean IRT scores and t-test results for differences in numeracy in 2019 and 2021
| Measure | Girls | Boys | t-value | Rural | Urban | t-value |
|---|---|---|---|---|---|---|
| 2019 numeracy | 507.65 | 521.99 | −3.29** | 489.18 | 559.29 | −16.44*** |
| 2021 numeracy | 522.05 | 529.72 | −1.73 | 497.86 | 574.16 | −17.70*** |
| Change in numeracy | 14.40 | 7.73 | 1.68 | 8.68 | 14.87 | −1.63 |
Note: * p < .05, ** p < .01, *** p < .001. n = 2,416.
Changes in IRT numeracy scores between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Changes in IRT numeracy scores between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Changes in IRT numeracy scores between 2019 and 2021
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
To better understand these gains and the difference between urban and rural learners, we also compared pupils’ changes in numeracy during the school closures with their progress during the 2018/19 academic year, before the disruption caused by the COVID-19 pandemic. Using grade 4 learners’ numeracy scores at the start and end of the 2018/19 academic year, we determined an average increase of 34 points (or 0.34 standard deviations) over the period. Assuming that pupils should have been able to achieve the same progress (of 34 points) between the end of grade 4 and start of grade 6, we predicted their performance on an assessment of similar difficulty in 2021 if the schools had not closed. However, between 2019 and 2021 numeracy scores only improved by an average of 12 points (0.12 standard deviations), suggesting a loss in learning during the COVID-19 school closures. Disaggregating the results further by location, we find that urban learners progressed by less than half (15 points compared to 37 points) and rural pupils by less than a third (10 points compared with 32), which suggests growing inequalities following the pandemic school closures.
To summarise, the findings from a descriptive analysis of the longitudinal data for Ethiopian pupils captured in 2019 and 2021 indicate that their social skills declined during the COVID-19 school closures, with rural learners experiencing the greatest losses. These could reflect the periods of isolation during the pandemic where learners were deprived of opportunities to interact with their peers and to refine important interpersonal skills. The difference between urban and rural areas could further reflect the reduced opportunities to meet with friends and peers outside family households for learners in more remote areas, or the limited resources to engage with distance learning, for example because of poor internet coverage. Over the same period, the pupils showed small gains in numeracy but these vary according to children’s gender and location, start from a low base and fall far short of the expected trajectories for learning before the pandemic closed Ethiopian schools.
Relationship between social skills and numeracy
The third RQ examines the bidirectional relationship between pupils’ social skills and academic outcomes, namely their social skills and numeracy, during the school closures and taking wider factors into account. This entailed running two sets of multivariate linear regressions, where children’s standardised scores for social skills and numeracy in 2021 comprised the respective dependent variables. Tables 8 and 9 present the regression results for social skills and numeracy, respectively. In each case, attrition weights were applied to the regressions as explained earlier and the standard errors were clustered by school.
Regressions for 2021 social skills z-scores
| (1) Basic model | (2) + Child variables | (3a) + Caregiver literacy | (3b) + Family variables | (4) + Teacher variables | (5) + School location | ||
|---|---|---|---|---|---|---|---|
| 2019 social skills | 0.10* (0.04) | 0.07 (0.04) | 0.07 (0.04) | 0.06 (0.04) | 0.07 (0.04) | 0.07* (0.03) | |
| 2021 numeracy | – | 0.23*** (0.03) | 0.23*** (0.03) | 0.21*** (0.02) | 0.21*** (0.02) | 0.18*** (0.02) | |
| Gender (base group: female) | – | −0.05 (0.03) | −0.05 (0.03) | −0.04 (0.03) | −0.05 (0.03) | −0.04 (0.03) | |
| Age | – | 0.00 (0.01) | 0.00 (0.01) | 0.01 (0.01) | 0.01 (0.01) | 0.00 (0.01) | |
| Class grade | – | −0.01 (0.04) | −0.01 (0.04) | −0.01 (0.04) | −0.01 (0.04) | −0.01 (0.04) | |
| Caregiver literacy (base group: no/low literacy) | – | – | 0.00 (0.04) | −0.07 (0.04) | −0.07 (0.04) | −0.05 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | – | – | – | 0.10 (0.06) | 0.09 (0.06) | 0.10 (0.06) |
| Third tercile | – | – | – | 0.21** (0.07) | 0.20** (0.07) | 0.24* (0.09) | |
| Household size | – | – | – | −0.02* (0.01) | −0.02* (0.01) | −0.01 (0.01) | |
| Teacher experience | – | – | – | – | 0.01 (0.01) | 0.01 (0.01) | |
| Teacher qualification (base group: no training) | Certificate | – | – | – | – | −0.17 (0.18) | −0.13 (0.17) |
| Diploma | – | – | – | – | −0.10 (0.13) | −0.08 (0.10) | |
| Degree | – | – | – | – | −0.09 (0.14) | −0.07 (0.13) | |
| Location (base group: rural) | – | – | – | – | – | −0.10 (0.09) | |
| Region (base group: Addis Ababa) | Amhara | – | – | – | – | – | 0.04 (0.07) |
| Benishangul Gumuz | – | – | – | – | – | −0.31 (0.16) | |
| Oromia | – | – | – | – | – | −0.04 (0.08) | |
| SNNPR | – | – | – | – | – | −0.17 (0.10) | |
| Somali | – | – | – | – | – | −0.06 (0.19) | |
| Constant | 0.02 (0.04) | 0.08 (0.29) | 0.08 (0.29) | 0.03 (0.28) | 0.13 (0.31) | 0.18 (0.29) | |
| Observations | 2,416 | 2,416 | 2,416 | 2,416 | 2,416 | 2,416 | |
| R-squared | .01 | .10 | .10 | .12 | .12 | .14 | |
Note: * p < .05, ** p < .01, *** p < .001. Standard errors are shown in parentheses.
Regressions for 2021 numeracy z-scores
| (1) Basic model | (2) + Child variables | (3a) + Caregiver literacy | (3b) + Family variables | (4) + Teacher variables | (5) + School location | ||
|---|---|---|---|---|---|---|---|
| 2019 numeracy | 0.72*** (0.02) | 0.68*** (0.02) | 0.68*** (0.02) | 0.66*** (0.02) | 0.67*** (0.02) | 0.59*** (0.02) | |
| 2021 social skills | – | 0.20*** (0.02) | 0.20*** (0.02) | 0.18*** (0.02) | 0.18*** (0.02) | 0.15*** (0.02) | |
| Gender (base group: female) | – | −0.02 (0.03) | −0.01 (0.03) | −0.01 (0.03) | −0.01 (0.03) | −0.01 (0.03) | |
| Age | – | 0.00 (0.01) | 0.01 (0.01) | 0.01 (0.01) | 0.01 (0.01) | 0.01 (0.01) | |
| Class grade | – | 0.13*** (0.03) | 0.12*** (0.03) | 0.13*** (0.03) | 0.12*** (0.03) | 0.13*** (0.03) | |
| Caregiver literacy (base group: no/low literacy) | – | – | 0.11* (0.05) | 0.04 (0.04) | 0.04 (0.04) | 0.01 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | – | – | – | 0.05 (0.04) | 0.05 (0.04) | 0.03 (0.04) |
| Third tercile | – | – | – | 0.20*** (0.05) | 0.19*** (0.05) | 0.11* (0.05) | |
| Household size | – | – | – | −0.02** (0.01) | −0.02** (0.01) | 0.00 (0.01) | |
| Teacher experience | – | – | – | – | 0.01 (0.01) | 0.01* (0.01) | |
| Teacher qualification (base group: no training) | Certificate | – | – | – | – | −0.25 (0.19) | 0.00 (0.17) |
| Diploma | – | – | – | – | −0.24 (0.18) | −0.09 (0.16) | |
| Degree | – | – | – | – | −0.29 (0.19) | −0.14 (0.17) | |
| Location (base group: rural) | – | – | – | – | – | 0.05 (0.05) | |
| Region (base group: Addis Ababa) | Amhara | – | – | – | – | – | −0.19* (0.10) |
| Benishangul Gumuz | – | – | – | – | – | −0.60*** (0.11) | |
| Oromia | – | – | – | – | – | −0.27** (0.08) | |
| SNNPR | – | – | – | – | – | −0.41*** (0.10) | |
| Somali | – | – | – | – | – | −0.59*** (0.10) | |
| Constant | 0.05 (0.03) | −0.75** (0.22) | −0.78** (0.22) | −0.80*** (0.22) | −0.54* (0.27) | −0.66* (0.26) | |
| Observations | 2,416 | 2,416 | 2,416 | 2,416 | 2,416 | 2,416 | |
| R-squared | .52 | .55 | .55 | .56 | .56 | .58 | |
Note: * p < .05, ** p < .01, *** p < .001. Standard errors are shown in parentheses.
In each case, we built the models step-by-step to include new controls and covariates, selecting the variables based on earlier RISE analyses and a review of the relevant literature in Ethiopia (Tiruneh et al, 2021). First, we considered the extent to which pupils’ social skills in 2019 predict their social skills in 2021, and their 2019 numeracy predicts their 2021 scores, before taking account of other factors (model 1, Tables 8 and 9). We then added individual learner factors, specifically their gender, age and class grade to accommodate the effects of any grade repetition, as well as their 2021 score on the other measure, 2021 numeracy as a predictor of 2021 social skills and vice versa (model 2). We next incorporated family variables, whether the pupils’ primary caregiver could read a basic sentence (model 3a) and the size and wealth of the household (model 3b). Finally, we included the learners’ teacher’s years of experience and highest qualification (model 4), and the school region and location, whether rural or urban (model 5).
To check the sensitivity of our results, we conducted further analyses set out in Appendix 3. Tables 3A and 3B show the results of regressions for pupils’ 2021 social skills and numeracy without using imputed data for the learners’ missing values. Tables 3C and 3D show the final regression models including a dummy variable for whether or not imputation was used to address data missingness. In each case, the variable is not statistically significant and supports the robustness of the results. Finally, Tables 3E and 3F set out the final models using both social skills and numeracy scores from 2019 to predict pupils’ social skills and numeracy in 2021 respectively. They highlight the relatively weak relationship between learners’ 2019 social skills and both their 2021 social skills and numeracy.
Overall, the results for the social skills regressions reveal that learners’ 2019 scores are significantly related to their 2021 responses but the coefficient is extremely small, suggesting that the relationship is relatively weak. This contrasts with the comparatively strong association between pupils’ 2021 social skills and their numeracy. Specifically, children’s 2021 numeracy significantly and positively predicts their 2021 social skills in all models (2–5), even when controlling for different child, household and school variables. Learners’ family wealth is also related to their social skills, with pupils from the most affluent backgrounds reporting significantly higher skills in their surveys.
The findings also shed further light on the trends presented in Figure 1 and the decline in children’s social skills during the school closures. Across the models, the coefficients for learners’ gender are not statistically significant, which is consistent with the t-test result in Table 6 that shows a lack of difference between girls’ and boys’ social skills in 2021. This contrasts with a significant gender difference in social skills in 2019 and reflects the slightly increased loss in social skills among girls during the period of the school closures. However, the coefficient for learners’ location is also insignificant, in contrast to the difference between rural and urban children indicated in Table 6. Several factors could account for this variation, such as the inclusion of attrition weights and variables for pupils’ region in the regressions. The interaction between learners’ location and household wealth is also likely to explain some of the difference, suggesting that children’s family resources and affluence account for at least some of the variation in social skills between pupils living in urban and rural areas.
The results for learners’ 2021 numeracy in Table 9 show, first, a strong, positive and statistically significant relationship between children’s numeracy performance in 2019 and their scores in 2021, even after controlling for individual, household and school variables. Second, both pupils’ 2021 social skills and their class grade significantly predict their 2021 numeracy, holding other factors constant. Third, although family variables like household size and caregiver literacy cease to be significant when taking region and location into account, learners taught by more experienced teachers scored significantly higher on numeracy, despite the coefficient being very small.
Together, these findings regarding children’s social skills and numeracy in 2021 offer valuable insights relating to their learning trajectories, especially in light of the COVID-19 pandemic. First, the relatively small association between pupils’ social skills in 2019 and 2021 suggests a potentially ruptured link in their SEL before and after the 2020 school closures. This is consistent with international concerns around the adverse effects of prolonged social distancing and isolation on children’s development as countries sought to contain the virus. For example, fewer Ethiopian respondents in 2021 than 2019 reported making friends easily, helping other people and sharing things, which may reflect the limited occasions for social interactions while schools were closed, and therefore the reduced opportunities for learners to practise their interpersonal skills.
By contrast, the positive association between pupils’ social skills and numeracy is much stronger, even after controlling for other related variables. Children who performed well on the 2021 numeracy tests also reported good social skills. Similarly, the relationship between learners’ numeracy scores in 2019 and 2021 is significant and large, showing close links between pupils’ early performance and their subsequent attainment. Comparing these associations, namely the relationship between children’s skills before and after the school closures, could suggest that the pandemic has had a greater and more adverse impact on learners’ social skills than on their numeracy.
In summary, the regression analyses highlight the limited relationship between Ethiopian children’s social skills in 2019 and 2021, but a strong and positive association between social skills and numeracy during the period between 2019 and 2021. This relationship remains significant even when taking factors such as pupils’ age, gender, family circumstances and location into account.
Discussion
This paper has drawn on longitudinal research regarding Ethiopian pupils’ social skills and numeracy outcomes assessed in both 2019 and 2021 to examine their learning trajectories in light of COVID-19. By using data captured before and after the school closures, we have explored the possible impact of the pandemic on children’s educational experiences, academic progress and SEL development. Although the study design precludes assertions regarding direct causality, the research nevertheless offers several valuable and useful insights.
First, the findings relating to RQ1 and patterns of dropout between 2019 and 2021 raise concerns regarding increasing inequalities among different groups of children. Specifically, the results reveal that girls, older pupils and poor performers were significantly more likely to fail to resume their education even after schools reopened. Numerous factors might account for their increased dropout: early marriage and pregnancy among female students; greater domestic responsibilities or expectations to contribute to the family purse; or fresh questions around the value and relevance of formal education, particularly from poor and illiterate parents of older learners and low achievers (Yorke et al, 2020; 2021a; Sabates et al, 2020; Tiruneh, 2020; Ford et al, 2021). Whatever the cause of an individual child dropping out, the unequal prevalence across different groups highlights the need to target those groups of children most at risk of not returning to school as they continue to contend with the effects of the pandemic. This suggests the need for increased support, investment and incentives for girls and learners in rural locations to continue and complete their basic education. It also highlights the possible need for alternative education pathways for those children from more disadvantaged backgrounds who have dropped out, perhaps through accelerated, parallel or informal learning initiatives like the nine-month Complementary Basic Education programme which has achieved success for some students in Ghana (Carter et al, 2020).
Second, the analyses for RQ2 identify a significant decline in pupils’ social skills during the period of the school closures. As shown in Figure 1 and Table 5, all groups of learners reported lower social skills in 2021 than in 2019, regardless of their gender and location. Furthermore, the regressions reveal a small association between children’s social skills across the rounds of data collection. This means that pupils who agreed with statements like ‘I make friends easily’ and ‘I feel confident talking to others’ in 2019 might not necessarily have expressed the same agreement in 2021. Such finding contrasts with the results of pre-pandemic research by Wolf and McCoy (2019) in Ghana who found that early SEL strongly and significantly predicted later SEL over successive time points.
Measurement issues may account for at least some of the difference. Although pupils responded to exactly the same seven items in 2019 and 2021, the results of psychometric tests summarised in Appendix 2 reveal better ‘goodness of fit’ indicators for the 2021 social skills measure and statistics, than for the 2019 data. Together with the negative skew of children’s 2019 responses and their tendency to agree with the item statements, the small association between learners’ reported social skills in 2019 and 2021 must be treated with some caution.
Overall, however, and taken together, the findings provide evidence of seemingly worsening social skills and ruptured learning trajectories in light of COVID-19. On the one hand, the decline in social skills could reflect the adverse consequences of prolonged periods of isolation, social distancing and school closures during the pandemic. On the other hand, the limited association between social skills in 2019 and 2021 could suggest a possible disruption to their SEL development, and underlines the importance of fostering social skills in addition to academic outcomes during major crises. In the event that schools close again for further protracted periods in the future, whether due to COVID-19 or otherwise, education ministries and schools themselves should consider mitigation measures to minimise the adverse impact on their learners’ SEL.
Between different pupils, the data regarding losses in social skills also highlight potentially widening learning inequalities. For example, Figure 1 shows the steepest declines and the greatest reductions in social skills among children in rural settings, which accords with evidence regarding lower levels of support for Ethiopian pupils’ learning during the school closures depending on their location (Yorke et al, 2020; 2021a). Similarly, Table 8 reveals significantly stronger social skills for students from the wealthiest families. This aligns with the findings of the OECD (2021) study, which identified a positive relationship between all aspects of learners’ SEL surveyed and their socio-economic status. Such differences could result from more affluent and middle-class parents making greater investments of time or resources in their children’s SEL, through home support, extracurricular activities or impressing on them the importance of wider learning for longer-term success. Conversely, poorer parents and caregivers may have less time to support their children’s SEL, whether during COVID or otherwise. In any case, these variations by wealth and location emphasise the need to promote SEL including social skills within schools to minimise learning inequalities and thereby mitigate the long-term effects of the pandemic.
Third, the trends in Figure 2 regarding pupils’ numeracy show more promising signs with small improvements in scores for all groups of children, notwithstanding the 2020 school closures. Again, there are clear performance gaps between urban and rural learners, and although boys scored higher than girls at both time points in line with wider RISE research in Ethiopia before the pandemic, these gender differences cease to be significant when other child, family and school factors are taken into account (Tiruneh et al, 2021).
A key finding from the analyses for RQ3 concerns the strong and positive relationship between pupils’ 2021 numeracy and social skills, which remains significant even after controlling for their 2019 results and other relevant variables. Numerous studies around the world have to date sought to examine and explain the links between children’s SEL and their academic learning. As with most other studies of this kind, the present research does not claim causality but rather highlights the positive and seemingly multiplicative interactions between Ethiopian pupils’ social skills and their academic outcomes. This aligns with existing literature, for example, the longitudinal study by Wolf and McCoy (2019), which suggested that higher attainment in skills like mathematics, reading and problem-solving could increase children’s self-esteem, self-efficacy and self-confidence, and thereby encourage more prosocial behaviours like sharing and cooperating with their peers. Pupils with stronger social skills may further be viewed more favourably by both teachers and classmates, and therefore better able to secure greater support for their learning (Liew, 2012). Similarly, a review by the Girls’ Education Challenge (2018: 17) emphasised the value of self-esteem, motivation and aspiration as a ‘softer middle layer of change’ to explain the connection between SEL and academic performance.
Between such literature and the current findings, the bidirectional association between social skills and academic learning like numeracy may therefore be best understood as akin to a virtuous circle, as set out in Figure 3. On one side, gains in mathematics, reading and other school subjects may nurture improvements in pupils’ self-confidence, self-efficacy and self-esteem, their sense of worth and belief in their own capacity, and consequently their social skills and disposition to engage in prosocial behaviours. On the other side, stronger social skills may enable children to forge effective relationships with their peers, teachers and even family members to secure support for their academic learning, whether during everyday life or times of crisis.
Emerging model for the relationship between social skills and academic outcomes
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Emerging model for the relationship between social skills and academic outcomes
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Emerging model for the relationship between social skills and academic outcomes
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Conclusion
Since schools closed in March 2020, education researchers, planners and policy makers have queried the impact of the COVID-19 pandemic on children’s learning, in both the short and the long term. Concerns were raised about the potential adverse consequences for students worldwide and the growing inequalities for learners contending with existing disadvantage (Ford et al, 2021; Kaffenberger, 2021; Porter et al, 2021). This paper contributes to understanding the effects of the pandemic on children’s educational trajectories by using longitudinal quantitative data from Ethiopian pupils before and after the school closures.
The findings advance current knowledge in three main respects: by highlighting the risk of widening inequalities regarding educational access and outcomes, whether relating to pupils’ gender, age, wealth or location; by quantifying the losses in participating Ethiopian children’s social skills during the school closures; and by identifying the positive and significant relationship between their social skills and numeracy over time. Although the findings must be understood in light of several limitations, including regarding the use of self-reported measures, the focus on social skills as an aspect of SEL and the inability to make causal assertions, they provide lessons on the potential effects of the pandemic for social change, for example, if girls and children from poor households are more likely to drop out of school following the pandemic. Similarly, the association between social skills and numeracy points at the interconnected and potentially interdependent nature of children’s learning, and the value of fostering all aspects of their holistic development.
The study therefore has implications for both future research and education planning in relation to children’s holistic learning and SEL, in Ethiopia and elsewhere. In particular, within schools, communities and households, the findings emphasise the need to promote SEL in conjunction with, rather than in isolation from, pupils’ academic outcomes. This implies that planners, teachers, headteachers and guardians need to work together to support the incorporation of SEL into children’s educational experiences. This could include establishing dedicated offices to develop approaches for mainstreaming SEL within education, including during crises and closures; designing and implementing specialised teacher training to share practical pedagogies and methods for promoting SEL; reviewing curricula, syllabi and related documents to ensure that they align with teaching approaches to foster SEL in schooling, which could include specifying standards and strategies for integrating SEL into academic subjects and the wider school culture; and committing additional resources to timetabled extracurricular activities that nurture SEL, such as clubs and safe spaces for girls. It could also support arguments for ‘self-contained classrooms’ during the lower primary years in Ethiopia and elsewhere, whereby one teacher guides and oversees children’s learning and progress across all domains of development, rather than rotating between classes to instruct different subjects (Melese and Tadege, 2019).
To conclude, this paper has highlighted the importance of children’s SEL, drawing on longitudinal data from Ethiopia captured before and after the 2020 school closures. Against the backdrop of COVID-19, students’ holistic learning and well-being have come into sharper focus, not least given their connection with academic progress, and their relevance for creating rounded, responsible and adaptable citizens who can handle the stresses and uncertainties of modern life.
Funding
The research described in this paper was funded by the LEGO Foundation and the UK Foreign, Commonwealth and Development Office, Australian Aid, and Bill and Melinda Gates Foundation through the Research for Improving Systems of Education (RISE) programme.
Acknowledgements
The authors would like to thank members of the RISE Ethiopia team at Addis Ababa University for feedback and advice during the study, including Dr Belay Hagos Hailu, Professor Amare Asegdom, Professor Tirussew Tefera and Professor Girma Lemma. Further thanks go to Ricardo Sabates, Dawit Tibebu Tiruneh, Janice Kim, Mesele Arraya and Chanie Ejigu. We are also grateful to Eve Hadshar, Celia Hsiao, Paul Frisoli, Garrett Jaeger and Margaret Laurie for their helpful input into the research, as well as members of the RISE Directorate for their feedback. Finally, we would like to thank the schools, teachers and pupils who participated in the study.
Data availability statement
The authors take responsibility for the integrity of the data and the accuracy of the analysis. The data set is not currently in the public domain.
Experimentation on humans and animals statement
The research was conducted in accordance with ethical approvals from Addis Ababa University and the University of Cambridge. These included requirements for informed consent in line with Ethiopian principles, and guidelines from the British Educational Research Association. The work did not involve any experimentation on humans or animals.
Conflict of interest
The authors declare that there is no conflict of interest.
References
Abimpaye, M., Dusabe, C., Nzabonimpa, J.P., Ashford, R. and Pisani, L. (2019) Improving parenting practices and development for young children in Rwanda: results of a randomized control trial, International Journal of Behavioral Development, 44(3): 205–15. doi: 10.1177/0165025419861173
Assessment Work Group (2019) Student Social and Emotional Competence Assessment: The Current State of the Field and a Vision for Its Future, Chicago: Collaborative for Academic, Social, and Emotional Learning (CASEL), https://casel.org/casel-gateway-student-sel-competence-assessment/.
Blair, C. and Razza, R.P. (2007) Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten, Child Development, 78(2): 647–63. doi: 10.1111/j.1467-8624.2007.01019.x
Borisova, I., Pisani, L., Dowd, A.J. and Lin, H.C. (2017) Effective interventions to strengthen early language and literacy skills in Low-income countries: comparison of a family-focused approach and a pre-primary programme in Ethiopia, Early Child Development and Care, 187(3/4): 655–71. doi: 10.1080/03004430.2016.1255607
Borzekowski, D.L.G., Singpurwalla, D., Mehrotra, D. and Howard, D. (2019) The impact of Galli Galli Sim Sim on Indian preschoolers, Journal of Applied Developmental Psychology, 64: art 101054, doi: 10.1016/j.appdev.2019.101054.
Brunello, G. and Schlotter, M. (2011) Non Cognitive Skills and Personality Traits: Labour Market Relevance and their Development in Education and Training Systems, Bonn: IZA.
Camfield, L. (2012) Resilience and Well-being among urban Ethiopian children: what role do social resources and competencies play?, Social Indicators Research, 107(3): 393–410. doi: 10.1007/s11205-011-9860-3
Camfield, L. and Tafere, Y. (2009) ‘No, living well does not mean being rich’: diverse understandings of Well-being among 11–13-year-old children in three Ethiopian communities, Journal of Children and Poverty, 15(2): 119–38. doi: 10.1080/10796120903310889
Cardero, D.L. (2020) Educating for Well-being: the need for systemic Socio-emotional learning and motion leadership, in A. Alfadala, J. Kirby, O. Zaki, A. Baghdady and D. Regester (eds) Education Disrupted, Education Reimagined: Thoughts and Responses from Education’s Frontline During the COVID-19 Pandemic and Beyond, Qatar: WISE (World Innovation Summit for Education), pp 80–2.
Carter, E., Rose, P., Sabates, R. and Akyeampong, K. (2020) Trapped in low performance? Tracking the learning trajectory of disadvantaged girls and boys in the Complementary Basic Education programme in Ghana, International Journal of Educational Research, 100: art 101541, doi: 10.1016/j.ijer.2020.101541.
CASEL (Collaborative for Academic, Social, and Emotional Learning) (2020) CASEL’s SEL Framework: what are the core competence areas and where are they promoted?, https://casel.org/wp-content/uploads/2020/12/CASEL-SEL-Framework-11.2020.pdf.
Danielson, C.K. and Phelps, C.R. (2003) The assessment of children’s social skills through self-report: a potential screening instrument for classroom use, Measurement and Evaluation in Counseling and Development, 35(4): 218–29. doi: 10.1080/07481756.2003.12069068
Dercon, S. and Krishnan, P. (2009) Poverty and the Socio-emotional competencies of children: evidence from the Young Lives sample in four developing countries, Children, Youth and Environments, 19(2): 138–63.
Diamond, A. (2014) Want to optimize executive functions and academic outcomes? Simple, just nourish the human spirit, Minnesota Symposia on Child Psychology, 37: 205–30.
Duckworth, A.L. and Yeager, D.S. (2015) Measurement matters: assessing personal qualities other than cognitive ability for educational purposes, Educational Researcher, 44(4): 237–51. doi: 10.3102/0013189X15584327
Espinoza Revollo, P. and Ogando Portela, M.J. (2019) Self-efficacy, Agency and Empowerment During Adolescence and Young Adulthood in Ethiopia, India, Peru and Vietnam, Working Paper 184, Oxford: Young Lives.
Favara, M., Freund, R., Porter, C., Sánchez, A. and Scott, D. (2021) Young lives, interrupted: Short‑term effects of the COVID-19 pandemic on adolescents in Low- and Middle-income countries, COVID Economics, 67, https://cepr.org/node/390717.
Fitzgerald, J., Gottschalk, P. and Moffitt, R. (1998) An analysis of the impact of sample attrition on the second generation of respondents in the Michigan Panel Study of Income Dynamics, Journal of Human Resources, 33(2): 300–44. doi: 10.2307/146434
Ford, K., Porter, C. and Pankhurst, A. (2021) A Lost Year of Learning for Girls in Ethiopia: Evidence from the Young Lives at Work COVID-19 Phone Survey, Policy Brief 44, Oxford: Young Lives.
Forsberg, J.T., Schultz, J.H., Lodi, C. and Tubbs Dolan, C. (2019) Development of the Student Learning in Emergency Checklist (26) (SLEC-26): A Measurement Tool of Promoters and Barriers for Learning Among Conflict-Affected Students in Palestine, technical working paper, Tromsø: University of Tromsø.
Girls’ Education Challenge (2018) Thematic review: Girls’ Self-esteem, https://girlseducationchallenge.org/media/hl2d344o/thematic-review-girls-self-esteem.pdf.
Haroz, E.E., Jordans, M., de Jong, J., Gross, A., Bass, J. and Tol, W. (2017) Measuring hope among children affected by armed conflict: Cross-cultural construct validity of the Children’s Hope Scale, Assessment, 24(4): 528–39. doi: 10.1177/1073191115612924
Himaz, R. (2009) The Impact of Parental Death on Schooling and Subjective Well-being: Evidence from Ethiopia Using Longitudinal Data, Working Paper 44, Oxford: Young Lives.
Hoddinott, J., Iyer, P., Sabates, R. and Woldehanna, T. (2019) Evaluating Large-scale education reforms in Ethiopia, RISE Working Paper 19/034. doi: 10.35489/BSG-RISE-WP_2019/034
Homan, R. (2001) The principle of assumed consent: the ethics of gatekeeping, Journal of Philosophy of Education, 35(3): 329–43. doi: 10.1111/1467-9752.00230
Huang, Y. and Bentler, P.M. (2015) Behaviour of asymptotically distribution free test statistics in covariance versus correlation structure analysis, Structural Equation Modelling: A Multidisciplinary Journal, 22(4): 489–503. doi: 10.1080/10705511.2014.954078
Hussein, M.H. (2013) The social and emotional skills of bullies, victims, and Bully–victims of Egyptian primary school children, International Journal of Psychology, 48(5): 910–21. doi: 10.1080/00207594.2012.702908
Immordino‐Yang, M.H. and Damasio, A. (2007) We feel, therefore we learn: the relevance of affective and social neuroscience to education, Mind, Brain, and Education, 1(1): 3–10.
INEE (Inter-agency Network for Education in Emergencies) (2016) INEE Background Paper on Psychosocial Support and Social and Emotional Learning for Children and Youth in Emergency Settings, New York: INEE, https://inee.org/resources/inee-background-paper-psychosocial-support-and-social-emotional-learning-children-youth.
Iyer, P., Rolleston, C., Rose, P. and Woldehanna, T. (2020) A rising tide of access: what consequences for equitable learning in Ethiopia?, Oxford Review of Education, 46(5): 601–18. doi: 10.1080/03054985.2020.1741343
Jenkins, R. (2020) How did education systems respond to the COVID-19 pandemic?, in A. Al-Fadala, J. Kirby, O. Zaki, A. Bagdady and D. Register (eds) Education Disrupted, Education Reimagined: Responses from Education’s Frontline During the COVID-19 Pandemic and Beyond, Qatar: WISE (World Innovation Summit for Education), pp 170–3.
Jones, S.M. and Doolittle, E.J. (2017) Social and emotional learning: introducing the issue, The Future of Children, 27(1): 3–11. doi: 10.1353/foc.2017.0000
Jukes, M.C.H., Gabrieli, P., Loti Mgonda, N., Nsolezi, F.S., Jeremiah, G., Tibenda, J.J. and Bub, K.L. (2018) ‘Respect is an investment’: community perceptions of social and emotional competencies in early childhood from Mtwara, Tanzania, Global Education Review, 5(2): 160–88.
Kaffenberger, M. (2021) Modelling the long-run learning impact of the COVID-19 shock: actions to (more than) mitigate loss, International Journal of Educational Development, 81: art 102326, doi: 10.1016/j.ijedudev.2020.102326.
Kim, J., Rose, P., Tiruneh, D., Sabates, R. and Woldehanna, T. (2021) Learning inequalities widen following COVID-19 school closures in Ethiopia, RISE blog, 4 May, https://riseprogramme.org/blog/learning-inequalities-widen-COVID-19-Ethiopia.
Krosnick, J. and Presser, S. (2010) Question and questionnaire design, in P.V. Marsden and J.D. Wright (eds) Handbook of Survey Research, 2nd edn, San Diego, CA: Elsevier, pp 263–313.
Leung, S.O. (2011) A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales, Journal of Social Science Research, 37(4): 412–21.
Liew, J. (2012) Effortful control, executive functions, and education: bringing self-regulatory and Social-emotional competencies to the table, Child Development Perspectives, 6(2): 105–11. doi: 10.1111/j.1750-8606.2011.00196.x
Major, S.O., Seabra-Santos, M.J., Martin, R.P. and Ventura, M.F. (2021) Preschoolers’ social skills and behavior problems: a Cross-cultural exploratory study of Angolan and Portuguese teachers’ perceptions, Current Psychology, ahead of print, doi: 10.1007/s12144-021-01375-6.
Matsunaga, M. (2010) How to Factor-analyze your data right: do’s, don’ts, and How-to’s, International Journal of Psychological Research, 3(1): 97–110. doi: 10.21500/20112084.854
Melese, S. and Tadege, A. (2019) The Ethiopian curriculum development and implementation Vis-à-Vis Schwab’s signs of crisis in the field of curriculum, Cogent Education, 6(1): art 1633147. doi: 10.1080/2331186X.2019.1633147
MoE (Ministry of Education) (2020a) Education Statistics Annual Abstract: September 2019–March 2020, Addis Ababa: Ministry of Education.
MoE (Ministry of Education) (2020b) Concept Note for Education Sector COVID-19 Preparedness and Response Plan, Addis Ababa: Ministry of Education.
Moffitt, T.E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R.J., Harrington, H., Houts, R., Poulton, R., Roberts, B.W., Ross, S. et al. (2011) A gradient of childhood self-control predicts health, wealth, and public safety, PNAS, 108(7): 2693–8. doi: 10.1073/pnas.1010076108
Moscoviz, L. and Evans, D.K. (2022) Learning Loss and Student Dropouts During the COVID-19 Pandemic: A Review of the Evidence Two Years After Schools Shut Down, CGD Working Paper 609, Washington, DC: Center for Global Development, https://www.cgdev.org/publication/learning-loss-and-student-dropouts-during-covid-19-pandemic-review-evidence-two-years.
Nnamani, A., Akabogu, J., Otu, M.S., Uloh-Bethels, A.C., Ukoha, E., Iyekekpolor, O., Omile, J.C., Obiezu, M.N., Dike, A.E., Ike, C.V. et al. (2019) Using rational-emotive language education to improve communication and social skills of adolescents with autism spectrum disorders in Nigeria, Medicine, 98(31): art e16550. doi: 10.1097/MD.0000000000016550
OECD (Organisation for Economic Co-operation and Development) (2021) Beyond Academic Learning: First Results from the Survey of Social and Emotional Skills, Paris: OECD Publishing.
Partnership for Resilience (2020) Rebuilding for a New Normal: A Study of the Impact of the COVID-19 Pandemic on Trauma-Responsive Schools and Key Recommendations for Communities, Lombard, IL: Partnership for Resilience.
Patrinos, H.A., Vegas, E. and Carter-Rau, R. (2022) An Analysis of COVID-19 Student Learning Loss, Policy Research Working Paper 10033, Washington, DC: World Bank, https://documents1.worldbank.org/curated/en/099720405042223104/pdf/IDU00f3f0ca808cde0497e0b88c01fa07f15bef0.pdf.
Porter, C., Favara, M., Hittmeyer, A., Scott, D., Sánchez Jiménez, A., Ellanki, R., Woldehanna, T., Duc, L.T., Craske, M.G. and Stein, A. (2021) Impact of the COVID-19 pandemic on anxiety and depression symptoms of young people in the Global South: evidence from a four-country cohort study, BMJ Open, 11: art e049653, https://bmjopen.bmj.com/content/11/4/e049653. doi: 10.1136/bmjopen-2021-049653
Sabates, R., Carter, E. and Stern, J. (2020) Using Educational Transitions to Estimate Learning Loss Due to COVID-19 School Closures: The Case of Complementary Basic Education in Ghana, Cambridge: REAL Centre, University of Cambridge.
Soto, C.J. (2016) The little six personality dimensions from early childhood to early adulthood: Mean-level age and gender differences in parents’ reports, Journal of Personality, 84(4): 409–22. doi: 10.1111/jopy.12168
Sushama, S., Sigafoos, J. and Carroll, A. (2000) Social skills assessment of Indian children with visual impairments, Journal of Visual Impairment and Blindness, 94(3): 172–6. doi: 10.1177/0145482X0009400305
Tiruneh, D.T. (2020) COVID-19 school closures may further widen the inequality gaps between the advantaged and the disadvantaged in Ethiopia, RISE blog, 21 April, https://riseprogramme.org/blog/COVID-19_ethiopia_school_closures.
Tiruneh, D., Hoddinott, J., Rolleston, C., Sabates, R. and Woldehanna, T. (2021) Understanding achievement in numeracy among primary school children in ethiopia: evidence from rise ethiopia study, RISE Working Paper 21/071. doi: 10.35489/BSG-RISE-WP_2021/071
Tubbs Dolan, C. (2019) Psychometric Analysis of the Pilot Holistic Assessment for Learning (HAL): Validity and Reliability, New York: Global TIES for Children, New York University.
UNESCO (United Nations Educational, Scientific and Cultural Organization) (2020) UNESCO COVID-19 Education response: education sector issue note 1.2 – April 2020: nurturing the social and emotional wellbeing of children and young people during crises, https://unesdoc.unesco.org/ark:/48223/pf0000373271.
van der Gaag, N. (2021) ‘Building Our Imagined Futures’: Supporting Resilience Among Young Women and Men in Ethiopia, Policy Brief 43, Oxford: Young Lives, https://www.younglives.org.uk/sites/www.younglives.org.uk/files/YL-PolicyBrief-43-Apr21.pdf.
Von Suchodoletz, A., Uka, F. and Larsen, R.A.A.A. (2015) Self-regulation across different contexts: findings in young Albanian children, Early Education and Development, 26(5/6): 829–46. doi: 10.1080/10409289.2015.1012189
Woldetsadik, G. (2013) National Learning Assessment in Ethiopia: Sharing Experiences and Lessons, Washington, DC: World Bank, https://olc.worldbank.org/sites/default/files/Session_6_Ethiopia_Woldetsadik.pdf.
Wolf, S. and McCoy, D.C. (2019) The role of executive function and Social‐emotional skills in the development of literacy and numeracy during preschool: a Cross‐lagged longitudinal study, Developmental Science, 22(4): art e12800. doi: 10.1111/desc.12800
Yorke, L. and Ogando, M.J.O. (2018) Psychosocial Scales in the Young Lives Round 4 Survey: Selection, Adaptation and Validation, Oxford: Young Lives, https://assets.publishing.service.gov.uk/media/5b9a89a240f0b6786cf0f4fa/YL-TN45.pdf.
Yorke, L., Rose, P., Hagos, B. and Woldehanna, T. (2020) The Effects of COVID-19 on Primary Education in Ethiopia: Perspectives of School Principals and Teachers, Research and Policy Paper 20/10, Cambridge: REAL Centre, University of Cambridge.
Yorke, L., Rose, P., Woldehanna, T. and Hagos, B. (2021a) Primary School-level responses to the COVID-19 pandemic in Ethiopia: evidence from phone surveys of school principals and teachers, Perspectives in Education, 39(1): 189–206.
Yorke, L., Rose, P., Bayley, S., Wole, D. and Ramchandani, P. (2021b) The importance of students’ socio-emotional learning, mental health and wellbeing in the time of COVID-19, RISE Insight Note 2021/025, 15 March, doi: 10.35489/BSG-RISE-RI_2021/025.
Yorke, L., Wole, D. and Rose, P. (2021c) An emerging strategy for the development of culturally relevant scales to capture aspects of students’ socio-emotional learning and social support for learning, RISE Insight Note 2021/031, 14 September, doi: 10.35489/BSG-RISE-RI_2021/031.
Zins, J.E. and Elias, M.J. (2007) Social and emotional learning: promoting the development of all students, Journal of Educational and Psychological Consultation, 17(2/3): 233–55. doi: 10.1080/10474410701413152
Appendices
Appendix 1: Social skills measure
Social skills scale items
| Social skills scale items | Strongly disagree | Disagree | Undecided | Agree | Strongly agree |
|---|---|---|---|---|---|
| 1. I feel confident talking to others. | |||||
| 2. Other people like me. | |||||
| 3. I like to share things with others. | |||||
| 4. I help others when they need help. | |||||
| 5. I make friends easily. | |||||
| 6. If I hurt someone, I say sorry. | |||||
| 7. I am polite towards others (e.g. greetings, saying thank you). |
Source: Children’s Self Report Social Skills Scale (CS4) (Danielson and Phelps, 2003).
Appendix 2: Distributions and psychometric properties
In addition to examining the distribution of the social skills scores, we conducted confirmatory factor analysis to test the hypothesised model and the extent to which it explained the pupils’ responses. Specifically, we used the asymptotically distribution free estimation method, which is preferable to maximum likelihood estimation where the sample size is large and the data violate assumptions of normality (Huang and Bentler, 2015). As shown in Figures 2A and 2B, the social skills responses were negatively skewed with more learners conveying their agreement with the scale items than disagreeing with them.
Distribution of z-scores for 2019 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of z-scores for 2019 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of z-scores for 2019 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of z-scores for 2021 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of z-scores for 2021 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of z-scores for 2021 social skills responses
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
The models were constructed based on the anticipated relationships, namely that each scale item was driven by the latent variable representing pupils’ social skills (Matsunaga, 2010). The models for each time point, 2019 and 2021, were then estimated and yielded standardised coefficients ranging from .64 to .85, indicating that every item contributed to the social skills scale. Table 2A shows the loadings, Cronbach’s alpha for reliability and various ‘goodness of fit’ statistics and thresholds for the different models, which indicate whether the scale shows unidimensionality with all items contributing to the social skills factor. These include the chi-squared test, comparative fit index (CFI), Tucker-Lewis index (TLI), the root mean square of error of approximation (RMSEA) and the standardised root mean square residual (SRMR). Multiple indicators of fit are important for such factor analyses because some are particularly sensitive to certain characteristics of the data, for example, the chi-squared test which often shows statistically significant results for large samples, such as that used in this study.
Psychometric properties for social skills scales
| Social skills | Threshold | 2019 data | 2021 data |
|---|---|---|---|
| Cronbach’s alpha | – | .729 | .881 |
| Coefficient loadings | – | .75–.85 | .64–.73 |
| Chi-squared | p>.05 | p<.05 | p<.05 |
| CFI | >.95 | .816 | .970 |
| TLI | >.90 | .724 | .955 |
| RMSEA | <.06 | .056 | .026 |
| SRMR | <.08 | .050 | .017 |
Source: Primary data 2019 and 2021.
Overall, the statistics indicate that the models for Ethiopian pupils’ social skills show a good level of fit for the 2021 data, but less so for the 2019 data. The 2019 figures for both CLI and TLI fall below the specified thresholds that indicate a good fit. The chi-squared test also shows lower fit for both models but this may be because such a large sample size was used. By contrast, RMSEA and SRMR show good fit for both the 2019 and 2021 data, being more robust with larger samples (Matsunaga, 2010).
Regarding learners’ numeracy, Figures 2C and 2D show the distribution of their IRT scores from the assessments conducted in 2019 and 2021 respectively.
Distribution of IRT scores for 2019 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of IRT scores for 2019 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of IRT scores for 2019 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of IRT scores for 2021 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of IRT scores for 2021 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Distribution of IRT scores for 2021 numeracy
Citation: Longitudinal and Life Course Studies 14, 2; 10.1332/175795921X16665759070534
Appendix 3: Sensitivity checks for social skills and numeracy regressions
Regressions for 2021 social skills z-scores without imputing missing values
| (1) Basic model | (2) + Child variables | (3a) + Caregiver literacy | (3b) + Family variables | (4) + Teacher variables | (5) + School location | ||
|---|---|---|---|---|---|---|---|
| 2019 social skills | 0.08 (0.04) | 0.05 (0.04) | 0.05 (0.04) | 0.05 (0.04) | 0.05 (0.04) | 0.04 (0.03) | |
| 2021 numeracy | – | 0.24*** (0.03) | 0.24*** (0.03) | 0.21*** (0.03) | 0.21*** (0.03) | 0.18*** (0.02) | |
| Gender (base group: female) | – | −0.03 (0.04) | −0.03 (0.04) | −0.03 (0.04) | −0.03 (0.04) | −0.02 (0.03) | |
| Age | – | 0.00 (0.01) | 0.00 (0.01) | 0.00 (0.01) | 0.01 (0.01) | 0.00 (0.01) | |
| Class Grade | – | −0.02 (0.04) | −0.02 (0.05) | −0.01 (0.04) | −0.02 (0.05) | −0.03 (0.04) | |
| Caregiver literacy (base group: no/low literacy) | – | – | 0.02 (0.04) | −0.04 (0.04) | −0.04 (0.04) | −0.03 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | – | – | – | 0.09 (0.07) | 0.09 (0.07) | 0.10 (0.07) |
| Third tercile | – | – | – | 0.19** (0.07) | 0.18* (0.07) | 0.25* (0.10) | |
| Household size | – | – | – | −0.02 (0.01) | −0.02 (0.01) | −0.01 (0.01) | |
| Teacher experience | – | – | – | – | 0.00 (0.01) | 0.00 (0.01) | |
| Teacher qualification (base group: no training) | Certificate | – | – | – | – | −0.19 (0.20) | −0.14 (0.18) |
| Diploma | – | – | – | – | −0.09 (0.13) | −0.07 (0.10) | |
| Degree | – | – | – | – | −0.08 (0.15) | −0.05 (0.14) | |
| Location (base group: rural) | – | – | – | – | – | −0.15 (0.10) | |
| Region (base group: Addis Ababa) | Amhara | – | – | – | – | – | 0.03 (0.07) |
| Benishangul Gumuz | – | – | – | – | – | −0.38* (0.16) | |
| Oromia | – | – | – | – | – | −0.10 (0.08) | |
| SNNPR | – | – | – | – | – | −0.24 (0.12) | |
| Somali | – | – | – | – | – | −0.19 (0.23) | |
| Constant | 0.02 (0.04) | 0.16 (0.31) | 0.15 (0.31) | 0.06 (0.30) | 0.16 (0.33) | 0.31 (0.33) | |
| Observations | 2,120 | 2,120 | 2,120 | 2,120 | 2,120 | 2,120 | |
| R-squared | .00 | .11 | .11 | .12 | .12 | .15 | |
Notes: * p < .05, ** p < .01, *** p < .001. Standard errors are shown in parentheses. The reduced number of observations compared with the total matched sample reflects missing values for certain pupil variables, for example, family wealth and location.
Regressions for 2021 numeracy z-scores without imputing missing values
| (1) Basic model | (2) + Child variables | (3a) + Caregiver literacy | (3b) + Family variables | (4) + Teacher variables | (5) + School location | ||
|---|---|---|---|---|---|---|---|
| 2019 numeracy | 0.72*** (0.02) | 0.68*** (0.02) | 0.67*** (0.02) | 0.65*** (0.02) | 0.66*** (0.02) | 0.58*** (0.02) | |
| 2021 social skills | – | 0.21*** (0.03) | 0.21*** (0.03) | 0.19*** (0.02) | 0.19*** (0.02) | 0.16*** (0.02) | |
| Gender (base group: female) | – | −0.01 (0.03) | −0.01 (0.03) | 0.00 (0.03) | −0.01 (0.03) | 0.00 (0.03) | |
| Age | – | 0.01 (0.01) | 0.01 (0.01) | 0.01 (0.01) | 0.01 (0.01) | 0.02 (0.01) | |
| Class grade | – | 0.13*** (0.03) | 0.13*** (0.03) | 0.14*** (0.03) | 0.12*** (0.03) | 0.15*** (0.03) | |
| Caregiver literacy (base group: no/low literacy) | – | – | 0.12* (0.05) | 0.04 (0.04) | 0.04 (0.04) | 0.02 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | – | – | – | 0.06 (0.04) | 0.06 (0.04) | 0.03 (0.04) |
| Third tercile | – | – | – | 0.22*** (0.05) | 0.21*** (0.05) | 0.09 (0.05) | |
| Household size | – | – | – | −0.02** (0.01) | −0.02** (0.01) | 0.00 (0.01) | |
| Teacher experience | – | – | – | – | 0.01 (0.01) | 0.01* (0.01) | |
| Teacher qualification (base group: no training) | Certificate | – | – | – | – | −0.19 (0.16) | 0.09 (0.15) |
| Diploma | – | – | – | – | −0.19 (0.15) | −0.06 (0.13) | |
| Degree | – | – | – | – | −0.24 (0.16) | −0.07 (0.14) | |
| Location (base group: rural) | – | – | – | – | – | 0.06 (0.06) | |
| Region (base group: Addis Ababa) | Amhara | – | – | – | – | – | −0.19 (0.11) |
| Benishangul Gumuz | – | – | – | – | – | −0.63*** (0.13) | |
| Oromia | – | – | – | – | – | −0.27** (0.09) | |
| SNNPR | – | – | – | – | – | −0.38** (0.12) | |
| Somali | – | – | – | – | – | −0.58*** (0.11) | |
| Constant | 0.06 (0.03) | −0.80** (0.24) | −0.85** (0.24) | −0.92*** (0.25) | −0.68* (0.27) | −0.84** (0.28) | |
| Observations | 2,157 | 2,157 | 2,157 | 2,157 | 2,157 | 2,157 | |
| R-squared | .51 | .54 | .54 | .55 | .55 | .58 | |
Notes: * p < .05, ** p < .01, *** p < .001. Standard errors are shown in parentheses. The reduced number of observations compared with the total matched sample reflects missing values for certain pupil variables, for example, family wealth and location.
Sensitivity check regression for 2021 social skills z-scores
| (5) Full model coefficients (standard errors) | ||
|---|---|---|
| 2019 social skills | 0.07* (0.03) | |
| 2021 numeracy | 0.18*** (0.02) | |
| Missing values imputed | 0.03 (0.07) | |
| Gender (base group: female) | −0.04 (0.03) | |
| Age | 0.00 (0.01) | |
| Class grade | −0.01 (0.04) | |
| Caregiver literacy (base group: no/low literacy) | −0.05 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | 0.10 (0.06) |
| Third tercile | 0.24* (0.09) | |
| Household size | −0.01 (0.01) | |
| Teacher experience | 0.01 (0.01) | |
| Teacher qualification (base group: no training) | Certificate | −0.13 (0.17) |
| Diploma | −0.08 (0.10) | |
| Degree | −0.08 (0.13) | |
| Location (base group: rural) | −0.10 (0.09) | |
| Region (base group: Addis Ababa) | Amhara | 0.04 (0.07) |
| Benishangul Gumuz | −0.31* (0.15) | |
| Oromia | −0.04 (0.08) | |
| SNNPR | −0.18 (0.10) | |
| Somali | −0.06 (0.19) | |
| Constant | 0.18 (0.29) | |
| Observations | 2,416 | |
| R-squared | .14 | |
Note: * p < .05, ** p < .01, *** p < .001.
Sensitivity check regression for 2021 numeracy z-scores
| (5) Full model coefficients (standard errors) | ||
|---|---|---|
| 2019 numeracy | 0.59*** (0.02) | |
| 2021 social skills | 0.16*** (0.02) | |
| Missing values imputed | −0.05 (0.05) | |
| Gender (base group: female) | −0.01 (0.03) | |
| Age | 0.01 (0.01) | |
| Class grade | 0.13*** (0.03) | |
| Caregiver literacy (base group: no/low literacy) | 0.01 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | 0.02 (0.04) |
| Third tercile | 0.10* (0.05) | |
| Household size | 0.00 (0.01) | |
| Teacher experience | 0.01* (0.01) | |
| Teacher qualification (base group: no training) | Certificate | 0.01 (0.17) |
| Diploma | −0.09 (0.16) | |
| Degree | −0.13 (0.17) | |
| Location (base group: rural) | 0.06 (0.05) | |
| Region (base group: Addis Ababa) | Amhara | −0.18 (0.10) |
| Benishangul Gumuz | −0.59*** (0.12) | |
| Oromia | −0.27** (0.08) | |
| SNNPR | −0.39*** (0.10) | |
| Somali | −0.57*** (0.10) | |
| Constant | −0.66 (0.26) | |
| Observations | 2,416 | |
| R-squared | .58 | |
Note: * p < .05, ** p < .01, *** p < .001.
Regression for 2021 social skills z-scores using 2019 predictors
| (5) Full model coefficients (standard errors) | ||
|---|---|---|
| 2019 social skills | 0.06* (0.03) | |
| 2019 numeracy | 0.12*** (0.02) | |
| Gender (base group: female) | −0.04 (0.03) | |
| Age | 0.01 (0.01) | |
| Class grade | 0.01 (0.04) | |
| Caregiver literacy (base group: no/low literacy) | −0.05 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | 0.11 (0.06) |
| Third tercile | 0.27** (0.09) | |
| Household size | −0.01 (0.01) | |
| Teacher experience | 0.01 (0.01) | |
| Teacher qualification (base group: no training) | Certificate | −0.13 (0.16) |
| Diploma | −0.10 (0.08) | |
| Degree | −0.10 (0.11) | |
| Location (base group: rural) | −0.09 (0.09) | |
| Region (base group: Addis Ababa) | Amhara | 0.01 (0.07) |
| Benishangul Gumuz | −0.42** (0.16) | |
| Oromia | −0.09 (0.08) | |
| SNNPR | −0.25* (0.11) | |
| Somali | −0.16 (0.19) | |
| Constant | 0.07 (0.30) | |
| Observations | 2,416 | |
| R-squared | .11 | |
Notes: * p < .05, ** p < .01, *** p < .001.
Regressions for 2021 numeracy z-scores using 2019 predictors
| (5) Full model coefficients (standard errors) | ||
|---|---|---|
| 2019 numeracy | 0.61*** (0.02) | |
| 2019 social skills | −0.02 (0.02) | |
| Gender (base group: female) | −0.01 (0.03) | |
| Age | 0.01 (0.01) | |
| Class grade | 0.13*** (0.03) | |
| Caregiver literacy (base group: no/low literacy) | 0.00 (0.04) | |
| Family wealth (base group: first tercile (poorest)) | Second tercile | 0.05 (0.04) |
| Third tercile | 0.15** (0.05) | |
| Household size | 0.00 (0.01) | |
| Teacher experience | 0.01* (0.01) | |
| Teacher qualification (base group: no training) | Certificate | −0.15 (0.16) |
| Diploma | −0.11 (0.15) | |
| Degree | −0.15 (0.15) | |
| Location (base group: rural) | 0.04 (0.05) | |
| Region (base group: Addis Ababa) | Amhara | −0.19* (0.10) |
| Benishangul Gumuz | −0.67*** (0.12) | |
| Oromia | −0.29** (0.09) | |
| SNNPR | −0.44*** (0.10) | |
| Somali | −0.62*** (0.11) | |
| Constant | −0.64* (0.27) | |
| Observations | 2,416 | |
| R-squared | .57 | |
Note: * p < .05, ** p < .01, *** p < .001.
