The landscape of longitudinal research today
This volume has presented examples of world class longitudinal research with policy and programme relevance. They join a growing number of researchers working with longitudinal data. In the last ten years, the number of publications citing use of longitudinal data has grown by 75%, shown in Figure 9.1. This growth presents an important resource for policy makers and practitioners towards meeting their sustainable development targets.
Growth of longitudinal research in peer-reviewed journal articles
Source: Author’s ownGrowth of longitudinal research in peer-reviewed journal articles
Source: Author’s ownGrowth of longitudinal research in peer-reviewed journal articles
Source: Author’s ownDespite the innovations presented, the potential relevance and impact, it is no doubt that longitudinal researchers today have experienced challenges in their dedication to this type of work.
To further unpack the landscape of longitudinal research, a systematic analysis of 122 longitudinal studies was conducted (Banati, 2019). A comprehensive search of the published literature was undertaken in Google Scholar, Pubmed and Scopus using the following search terms: birth cohort, longitudinal, child, life course and life stage. The study also drew from the largest open source database of longitudinal studies available – the Low and Middle Income Longitudinal Population Study Directory developed by the Institute for Fiscal Studies (2018). Inclusion criteria were (1) a minimum of two rounds of data collection; (2) first round conducted after 1970; (3) capturing information and responding to questions relevant for children. The studies are located in high-, middle- and low-income countries. Without a doubt, this was not an exhaustive process and may not have captured all available studies. Despite this, to our knowledge, we have identified and analysed the largest collection of longitudinal studies among children to date.
Challenges experienced by longitudinal research today
Although numerous studies have been published from ESRC-funded longitudinal studies, the review panel has found it difficult to trace specific evidence of the ‘instrumental’ or ‘direct’ policy impacts from these investments … If left unchecked, a lack of evidence is likely to undermine the case for future investments. (Economic and Social Research Council, 2017, p 6)
The next section focuses on this last challenge, identified as an absolute necessity to ensure the wealth of longitudinal evidence is more fully brought to bear on programme design, implementation and evaluation.
Bringing longitudinal research into the policy-practice space
Ensuring the relevance, timeliness and quality should be the top priorities of any new longitudinal research undertaking. However, research uptake strategies for such research are rarely considered beyond inclusion in the donor proposal, and infrequently translated into action plans. As demands to demonstrate ‘value for money’ increase, researchers are seeking ways to intensify the impact of their studies, and ways to measure it. In reflecting on their longitudinal research, Jones and Villar (2008, p 1) identify five dimensions of policy impact: ‘framing debates and policy agenda formulation; securing discursive commitments from key policy actors; bringing about procedural changes; policy reform; and behavioural change’. Their analysis highlights the roles of culture, politics and values in research uptake, and the challenges inherent in partnership models of knowledge translation. Acknowledging trust, co-creation and inclusion in the research process have been identified as key factors in the success of longitudinal impact evaluations such as the Transfer Project (2020, p 2). The authors note that ‘[e]ngaging government ministries in large-scale, highly public impact evaluations can also bring about an “evidence culture” with profound effects
A policy evaluation framework for longitudinal studies
Figure 9.1 proposes a research uptake framework for longitudinal research that rests on three elements of the ‘research dynamo’. The cornerstone of successful impact is quality research results that demonstrate strong internal validity and generate accurate policy and programme recommendations. Noting that over 80% of longitudinal studies surveyed used tablets for data collection, longitudinal research can exploit this use of technology to generate more real-time quality data sets, even extending to cell phone and GPS technology.
A wide and transparent partnership model is central to the dynamo. The majority of longitudinal research in developing contexts are anchored in northern academic institutions. Over one third of the studies surveyed received funding for their longitudinal study through an academic grant. A successful example of a wide partnership model is that utilized by the GAGE research programme, which has an active and expansive research consortium with members from leading research institutions from Africa, the Middle East and North Africa (MENA), South Asia, the UK and the US, and also globally renowned non-governmental organizations working on adolescence and gender.
Finally, the third piece of the dynamo brings continuous dialogue and participation to the research process. Dialogue at the governmental and agency level is perhaps most commonly seen among policy-engaged longitudinal studies. Participatory research models – involving the participation of communities and individuals – have been in existence a long time. The influential work of Robert Chambers using community-based participatory action research has raised the bar for researchers (Chambers, 1994). Engagement with communities and individuals in longitudinal research processes has not been
Recognizing that direct policy impact is uncommon, four proposed avenues of influence are described in Figure 9.2. Academic impact is identified as the contribution that research makes in ‘shifting understanding and advancing scientific, method, theory and application across and within disciplines’. It may also include methodological innovation. Instrumental impact describes the altering of programme or policy behaviours, including policy development, practice or service provision or directly shaping legislation. Conceptual impact is achieved through reframing debates, changing the understanding of the situation; or it might provide new or different ways of thinking about the policy problem or context being considered. Finally, capacity-building impact is achieved through technical and personal skill development, both among researchers and research users (Economic Social Research Council, 2020).
These avenues exploit a set of tools and instruments in support of reaching SDG impact, including programme adaptation and scale-up, policy agenda formulation or procedural change. An important part of policy process analysis that longitudinal studies could benefit from is a rigorous assessment of amplifier platforms at community, national, regional and global levels. These include governmental or intergovernmental events, media venues or traditional leaders’ groups.
The research dynamo for sustainable development impact
The research dynamo for sustainable development impact
The research dynamo for sustainable development impact
Donor perspectives of longitudinal research
In recent years, several large donors – the Economic Science Research Council (ESRC), Wellcome Trust and the Medical Research Council (MRC) – have undertaken reviews of their longitudinal investments. An independent process – undertaken by ESRC for example – can give rise to clarity in how funding should be prioritized going forward. A description of donor strategies for a selection of funders is described in Table 9.1. While not an exhaustive review, the table illustrates that donors seem to have a number of common elements of agenda or strategy in defining and sustaining their longitudinal investments.
Summary of selected foundations and agencies financing longitudinal research
Donor |
Financial investment |
Strategic focus |
|---|---|---|
Wellcome Trust |
Over the past ten years, more than £120 million has been invested in longitudinal population studies in the UK and low- and middle-income countries (Wellcome Trust, 2017). |
Data linkage, data sharing and discoverability; Co-ordination, networking between studies, standardization; Emerging technologies, cost efficiency; Capacity strengthening and research translation; Encourage data that is useful both locally and more widely; that answers several questions; that endures for currently unforeseen future uses (Wellcome Trust, 2017). |
Gates Foundation |
Approximately $76 million over the last 15 years. |
Global Development, Global Health and Global Growth and Opportunity Divisions and US Programs Division all exploit longitudinal data and research. Many investments focus on integrated data, or creation of data exchange platforms. There is also support for development of innovative methodologies for longitudinal science. However, an explicit data strategy is not readily available. |
U.K. Economic Social Research Council (ESRC) |
Over £20 million annually (Economic Social Research Council, 2020). |
2017 review identified the following priorities: Creation of an administrative data spine from which to sample a new cohort or refresh existing samples; A new birth cohort with accelerated longitudinal design; Continuation of understanding society and an additional ‘transition to adulthood’ sweep of the Millennium Cohort Study; Competitive bids to an innovation fund, including an option for the other cohorts to develop innovative bids for this resource; Continuation and further development of a longitudinal resource centre (currently CLOSER). |
US National Institutes of Health |
Over the last five years, over $500 million has been spent on longitudinal studies (NIH, 2020). |
NIH strategy for 2016–20 explicitly mentions longitudinal research. The strategy describes the Precision Medical Initiative, a cohort at the forefront, that takes advantage of emerging biomedical tools and technologies, such as availability of electronic health records, DNA sequencing, and exposure monitoring. |
£9.6m per year on 19 cohort studies (Medical Research Council 2014). |
2014 review identified the following priorities: Use standardized or validated sample collection, storage, tools and platforms for evolving technologies; Promote collaborations with centres of excellence such as the Farr Institute; Expand data linkage to the increasing number of routine health records and administrative data sets available in the UK; Increase discoverability and accessibility of studies (though data sharing and access to samples if possible); Adopt core common data standards, sharing knowledge and improving meta-data quality; Cost-effective methods such as digital technologies should be adopted; Effective models of two-way engagement between cohort study teams and policy makers. |
Given their significant potential to shape development policies and programmes, maximizing the returns to these studies is a worthwhile aim. Drawn from analysis of our survey of studies, as well as existing reviews (Economic Social Research Council, 2017, for example), four recommendations are made for improving the effectiveness of longitudinal studies.
Ensure data are open access: Democratizing data to enable the wider research and policy community to access and use it and facilitating better access to data will boost knowledge and is a public good in and of itself. In our analysis, only 37% of the 122 studies identified were available in the public domain. Access to data will allow timelier analysis and increase publication of the data. It will encourage triangulation of findings, and support improvements in quality. Very few longitudinal data resources are publicly available. Herbst (2002, p 43) notes that this challenge relates to the balance of priorities of data collectors, data subjects and data users. Employing his analysis, we see that while data collectors and analysts are interested
primarily in ensuring quality, generating useful findings and project sustainability, data subjects’ interests relate to protecting their own confidentiality, being informed of the use of the data and seeing the benefits of research. And finally, data users are interested mostly in setting research-based policies and informing programmes, as well as gaining credibility through their affiliation with the evidence. Young Lives provides a good example of data transparency, as all data sets are available in the public domain. To improve discoverability, an online portal repository that hosts all longitudinal studies can be developed, making an effort to capture those in low- and middle-income settings. Such an online portal can share technical lessons as well as examples of policy impact.
Foster data linkage: Linking data across sites or studies increases the value by increasing the number of variables, the size of the sample or the geographic scope of the analysis. Data sets that are linked can shed light in new areas, answering novel questions by expanding the potential of a singular data set. Linkage can overcome data-quality issues by allowing triangulation of outcomes, increasing completeness and improving ascertainment. Linkage of data sets can allow the evaluation of rare events. The use of data linkage in research studies has increased almost six-fold within the last two decades (Bohensky et al, 2010). Efforts to create interoperability with longitudinal data and other data sets (terms ‘longitudinal linkage’) is also becoming more common, with 63% of the studies in the analysis undertaking some form of linkage or harmonization activity. This was the most frequently mentioned methodological issue of UK-based longitudinal studies (Economic Social Research Council, 2017). The most common type of longitudinal linkage between data sets occurs between household data and facility-level data, such as patient health outcomes. The UK Administrative Data taskforce notes that linked data – longitudinal and administrative for example – could facilitate policy-relevant research, by expanding the number of data sets (and thereby variables) from which researchers can draw for analysis (UK Administrative Data Taskforce, 2012).
The Brazilian 100 million cohort provides a good example. The 100 million cohort project was set up in 2013 aiming to build a population-based cohort to be used to assess the effects of Brazil’s social programmes on health and other outcomes. A probabilistic linkage pipeline was used to link the cohort with different health databases (Pinto et al, 2016). Unique identifiers (NIS) are available for all individuals who are recorded in the CadastroUnico (CADU), a central register for all social programmes kept by the
Brazilian government. The cohort comprises all individuals who have received payments from Bolsa Familia (a conditional cash transfer programme) between 2007 and 2015, resulting in a total of 114 million records. Pita and colleagues (2017) report efforts to link this cohort to the Unified Health System (SUS), including hospitalizations (SIH), notifiable diseases (SINAN), mortality (SIM) and live births (SINAC) registers. The authors show 95% linkage is possible between CADU and Brazil’s Unified Health System (SUS) database using a combination of increasing sizes and manual review. The researchers note the importance of developing techniques in the absence of a ‘gold standard’. In the report by Pinto and colleagues (2016) estimating accuracy of linkage efforts, the conclusions note that, given the size of the cohort and number of records, manual reviews to improve accuracy are limited, and machine learning techniques are under exploration. Build interventions into the design of longitudinal data: Experimentalists are typically at odds with the observationists, yet many of the challenges and opportunities of longitudinal research are shared by both. The demand for rigorous determination of ‘what works’ drives donors and policy makers alike and has created an industry of longitudinal intervention research. Longitudinal intervention studies are repeated surveys that include an experimental intervention. Over the last decade, we have seen the rise of the Randomized Control Trial (RCT) – a type of longitudinal data collection effort – which in many ways is considered a gold standard for evaluating the effect of a given intervention (Rothman and Greenland, 1998). The main advantage of these surveys is that it is possible to study both the natural history of development and the impact of interventions in the same research project. At the same time, strictly defined RCTs are not flexible enough to embrace serendipity, and hence cannot take advantage of one of the key benefits of longitudinal research – capturing unintended or unplanned events and picking up problems not envisaged before the studies were planned. There are both advantages to and problems with longitudinal observational studies, randomized control trials and longitudinal-experimental designs (quasi-experiments) – and each brings its own value (Banati, 2017).
For example, the GAGE study has embedded a set of experimental and quasi-experimental impact evaluations within the panel data collection activities. This allows investigators to complement the observational study with causal questions on programme impact. Some questions regarding the ‘dynamism and diversity’ of adolescent transitions and trajectories, particularly those relating to gender
norms, are best understood through observational means. The presence of the embedded impact evaluations enables the study team to effectively disentangle the mediating roles that particular programme interventions play in gender socialization at different stages of adolescence. Noting that girls’ experiences differ during early and later adolescence, while most interventions group adolescent girls aged 10–19 together, GAGE uses the longitudinal design to answer the question: When is the best time to intervene in adolescence, using what types of change strategies, and in what contexts? Relatedly, their research addresses questions such as: Are current interventions too short and/or lack intensity? And which programmes must be delivered in early adolescence in order to see significant returns on investment? Creating cross-comparative (harmonized) studies: Some of the most powerful longitudinal studies exist as cross-comparative national studies, coordinated through large networks. Notably, the INDEPTH network and the Living Standards Measurement Surveys (LSMS) are examples of multi-country comparative initiatives that utilize similar tools to compare conditions and situations over time and across country contexts. INDEPTH, founded in 1998, is a network of longitudinal community field sites based on health and demographic surveillance systems to capitalize on the research and policy-informing capabilities organized by researchers from various field sites. Standardized data sets enable cross-site and cross-national research, increasing the power of each individual study. Increasingly, with global goals pushing the development community to scale up programmes, longitudinal research is also keenly interested to explore external validity, the validity of applying research findings across contexts. Here the questions that can be answered include: How generalizable are findings from a single research study to other contexts? And under what general conditions are the results confirmed?
Other benefits to cross-comparative studies include the establishment of benchmarks, the engagement of transboundary and increasingly global issues (such as internet use or migration). Common frameworks, survey instruments and methodologies also facilitate pooling, allowing studies to profit from larger sample sizes. The European Cohort Development Project (Chapter 7) describes an input-harmonized birth cohort survey that exploits many of the advantages described earlier. It comprises a common questionnaire, sampling and fieldwork procedures across EU member states which allow a direct comparison
The future of longitudinal research
A content analysis of SDG domains was undertaken in the 122 longitudinal studies we reviewed. Most studies were focused in the areas of health, poverty and inequality and education (SDGs 1, 3 and 4). Fewer studies focused on other SDGs such as clean energy; water, sanitation and hygiene; climate change and environment; or peaceful and inclusive societies (SDG 7, 8, 9, 12, 14, 15, 16). Among the survey respondents, when asked ‘what topical areas do you consider high priority for longitudinal research on children in the next 3–5 years?’ most researchers mentioned the terms ‘health’ and ‘development’.
‘many important issues, such as gender equality, health, sustainable consumption and production, or nutrition cut across different goals and targets. Similarly, the goals and targets are interdependent and must be pursued together since progress in one area often depends on progress in other areas’.
Extending current longitudinal research towards understanding and exploring the links between different development domains can serve as a useful basis for arguing for greater integration of service sectors (such as early childhood education; and water, sanitation and hygiene).
Two areas where longitudinal research could be launched in the future include:
Migration: The dynamism of migration makes longitudinal data a relevant tool for this topic. As migration steadily increases (UNICEF, 2016), an adequate evidence base on the drivers and possible determinants of migration has yet to be established. While most researchers might be concerned with attrition, adopting innovations that allow us to follow individuals over time, even when they move from one place to another, and interviewing them about the changes in their life, can help better describe the changes individuals experience when they migrate.
Climate and the environment: Environmental and sustainable aspects and their impacts on human well-being has not been the topic of many longitudinal studies. The interconnections between energy, food and water, as well as water and sanitation links to environmental health, human well-being and the quality of the economy make the longitudinal design worth exploring. Such a study evaluating the impacts of interventions that mitigate climate change on human well-being and natural systems is one example (UNRISD, 2015).
Conclusions
In what has been termed ‘the decade of action’, the United Nations Sustainable Development Agenda has much progress to make in order to realize its ambition. With less than 10% of developmental science research from regions accounting for over 90% of world’s population (Bornstein et al, 2012), the need to bridge research to solutions in low- and middle-income settings is even more acute. Longitudinal research has an influential role to play in delivering for those not yet counted within the current measurement framework. For policy and practice, new concepts and analyses are needed that place the individual within a life course of exposure, realities and opportunities. Current and conventional analysis both undervalues human futures and, in maintaining silos, undermines the fullness of human life. This volume has tried to open up and explore concepts and relationships between trajectories of human development and the sustainability agenda. It has exemplified these through a selection of high-impact longitudinal studies. Collectively, the studies have demonstrated the value of life-course examination – through analyses of transitions, trajectories and turning points – in answering questions of human development. In addition, the exploration of intergenerational transmission, inequalities, coherence, participation and sequencing, have enriched our understanding while simultaneously pointing to gaps in our current measurement efforts.
References
Banati P. (2017) In God we trust, all others must bring data: is better longitudinal research the best response to the new ‘post-truth’ order? UNICEF Connect Blog, 10 March. Available at https://blogs.unicef.org/evidence-for-action/is-longitudinal-research-the-best-response-to-the-post-truth-order/
Banati P. and Oyugi J. (2019) Longitudinal research for sustainable development: towards a policy driven agenda – research spotlight. Zeitschrift für Psychologie 227(2): 149–53.
Bohensky, M.A., Jolley, D., Sundararajan, V., Evans, S., Pilcher, D.V., Scott, I. and Brand, C.A. (2010) Data linkage: a powerful research tool with potential problems. BMC Health Services Research 10(1): 1–7.
Bornstein, M., Britto, P., Nonoyama-Tarumi, Y., Ota, Y., Petrovic, O. and Putnick, D. (2012) Child development in developing countries: introduction and methods. Child Development 83(1): 16–31.
Chambers R. (1994) The origins and practice of participatory rural appraisal. World Development 22(7): 954–69.
Economic and Social Research Council (2017) Longitudinal studies strategic review. Available at https://esrc.ukri.org/files/news-events-and-publications/publications/longitudinal-studies-strategic-review-2017/
Economic and Social Research Council (2020) UK’s ESRC impact statement. Available at https://esrc.ukri.org/about-us/50-years-of-esrc/50-achievements/longitudinal-studies/
Herbst K. (2002) Wider accessibility to longitudinal data sets: a framework for discussing the issues. Paper presented at Workshop on Leveraging Longitudinal Data in Developing Countries hosted by the National Research Council.
Institute for Fiscal Studies (2018) Low- and middle-income longitudinal population study directory. Available at www.ifs.org.uk/tools_and_resources/
Jones N. and Villar E. (2008) Situating children in international development policy: challenges involved in successful evidence-informed policy influencing. Evidence and Policy: A Journal of Research, Debate and Practice 21: 31–51.
Medical Research Council (MRC) (2014) Maximising the value of UK population cohorts: MRC strategic review of the largest UK population cohort studies.
National Institutes of Health (NIH) (2020) National Institutes of Health Research Portfolio Online Reporting Tools (RePORT). Available at https://report.nih.gov/
Pinto C., Dantas R., Sena S., Reis S., Fiaccone R., Amorim L. et al (2016) Accuracy of probabilistic linkage: the Brazilian 100 million cohort. Available at www.atyimolab.ufba.br/BHI2017_Accuracy_vFinal.pdf
Pita R., Mendonça E., Reis S., Barreto M. and Denaxas S. (2017) A machine learning trainable model to assess the accuracy of probabilistic record linkage. In L. Bellatreche and S. Chakravarthy, International Conference on Big Data Analytics and Knowledge Discovery (pp 214–27). Regensburg, Germany: Springer.
Robeco SAM (2019) SDG Impact Framework. Available at www.robecosam.com/en/insights/2019/robecosam-impact-framework-bridging-impact-investing-to-mainstream-investors.html
Rothman K.J. and Greenland S. (1998) Modern Epidemiology. Philadelphia, PA: Lippincott-Raven.
SDSN (2015) Sustainable development solutions network: indicators and monitoring framework for the SDGs. Available at https://sustainabledevelopment.un.org/content/documents/2013150612-FINAL-SDSN-Indicator-Report1.pdf
Transfer Project (2020) Beyond internal validity: towards a broader understanding of credibility in development policy research. World Development 127: 104802.
UK Administrative Data Taskforce (2012) The UK administrative data research network: improving access for research and policy. Available at https://esrc.ukri.org/files/news-events-and-publications/publications/themed-publications/improving-access-for-research-and-policy/
UNICEF (2014) UNICEF’s Office of Research – Innocenti 2014 Results Report. Available at www.unicef-irc.org/publications/774-2014-results-report.html
UNICEF (2016) Uprooted: the growing crisis for refugee and migrant children. Available at www.unicef.org/publications/index_92710.html
UNICEF (2017) UNICEF Innocenti Research Uptake Strategy (unpublished).
UNICEF (2020) Global Longitudinal Research Initiative (GLORI). Available at www.unicef-irc.org/research/longitudinal-and-lifecourse-research-for-children/
UNRISD (2015) 100 key questions for the post-2015 development agenda. Available at www.unrisd.org/80256B3C005BCCF9/(httpAuxPages)/AC45AB8A6F43AFA2C1257E0E006A9993/$file/100%20questions.pdf
Wellcome Trust (2017) Longitudinal population studies strategy. Available at https://wellcome.org/sites/default/files/longitudinal-population-studies-strategy_0.pdf
