Introduction: Why Panel Data Matters for Economic History

Historical economic research has undergone a profound methodological transformation over the past three decades. The adoption of panel data methods—techniques that analyze data tracking the same units (countries, regions, firms, or individuals) across multiple time periods—has allowed scholars to move decisively beyond simple cross-sectional comparisons or time-series narratives. By combining the spatial breadth of cross-sectional data with the temporal depth of time-series data, panel methods enable economic historians to control for unobserved heterogeneity, identify causal relationships, and trace dynamic processes that would otherwise remain hidden. This article explores how panel data methods are applied in historical economic research, their advantages and challenges, and their growing indispensability for understanding long-run economic change.

What Is Panel Data? A Primer

Panel data, also known as longitudinal data, consists of observations on the same entities—such as countries, states, industries, or households—over two or more time periods. For example, a dataset tracking annual GDP, population, and institutional quality across 50 European countries from 1800 to 2000 constitutes a panel. This structure captures both variation across entities (the cross-section) and variation within entities over time (the time-series). In historical work, panels often span decades or centuries, making them uniquely suited to study slow-moving processes like institutional change, demographic transitions, or technological diffusion.

Panel data can be balanced (all entities observed in all periods) or unbalanced (missing observations for some periods). In historical research, unbalanced panels are the norm due to incomplete records, wars, or shifts in political boundaries. The key feature distinguishing panel data from pooled cross-sections is the ability to follow the same units repeatedly, enabling researchers to control for time-invariant unobservable factors that would otherwise bias estimates.

Common panel data models include:

  • Fixed-effects (FE) models: Control for all entity-specific, time-invariant characteristics (e.g., geography, culture, initial institutions) by focusing on within-unit variation over time. This is the workhorse model in historical panel analysis.
  • Random-effects (RE) models: Assume that entity-specific effects are uncorrelated with explanatory variables, allowing both within- and between-unit variation to be used. The Hausman test helps decide between FE and RE; in historical settings, FE is almost always preferred because unobserved heterogeneity is likely correlated with regressors.
  • First-difference (FD) estimation: Removes time-invariant effects by subtracting the previous period’s observation from the current period. Particularly useful when serial correlation is present or when the panel has a short time dimension.
  • Dynamic panel models (e.g., Arellano-Bond GMM): Include lagged dependent variables to model persistence—common in studies of growth, conflict, or institutional stickiness. These require careful instrumenting to avoid Nickell bias in short panels.

These methods are extensively described in econometric textbooks such as Wooldridge’s Econometric Analysis of Cross Section and Panel Data (2020). A good introduction can be found at the American Economic Association’s econometrics resources page.

Historical Applications: From the Industrial Revolution to the Great Depression

Panel data methods have been used to tackle a wide range of historical economic questions. Below are several illustrative examples that demonstrate the versatility of these approaches across different time periods and units of analysis.

Long-Run Effects of Institutions

A seminal contribution is the work of Acemoglu, Johnson, and Robinson (2001, 2002), who used cross-country panel data to examine the impact of colonial institutions on long-run economic development. By exploiting variation in the timing of colonization and the type of institutions established (extractive vs. inclusive), they showed that early institutional environments have persistent effects on GDP per capita. Their use of instrumental variables within a panel framework helped address endogeneity concerns—specifically, they instrumented for institutions using settler mortality rates. Subsequent studies have extended this analysis using decadal panels covering the 19th and 20th centuries, confirming that institutional quality matters for economic growth over the very long run. A later refinement by Dell (2010) used a within-country panel of Peruvian districts to show that the mita—a forced labor system imposed by Spanish colonizers—reduced contemporary consumption and stunted growth, even after controlling for district fixed effects.

Technological Change and Productivity

Panel data is particularly well-suited for studying the diffusion of technology, where adoption decisions are made over time and across regions. For example, economic historians have constructed firm-level panels for 19th-century textile mills to examine how adoption of the steam engine affected output per worker. By including fixed effects for each mill, researchers can control for unobserved management quality or location-specific advantages, isolating the causal effect of the new technology. A well-known study by Atack, Bateman, and Margo (2008) used a panel of U.S. counties from 1850 to 1880 to show that railroad expansion significantly increased agricultural productivity and land values. Their panel allowed them to include county fixed effects (holding constant soil quality, initial market access) and year fixed effects (controlling for national business cycles), thus identifying the railroad’s impact from within-county variation over time.

Welfare Effects of Economic Shocks

The Great Depression (1929–1941) has been re-examined using state-level panel data from the United States. Researchers have analyzed how New Deal spending influenced employment, bank recovery, and social unrest. By controlling for state fixed effects and including year fixed effects, they can separate the impact of federal policies from underlying regional trends. A classic example is Fishback, Horrace, and Kantor (2005), who used annual panel data across states to estimate the employment multiplier of public works and relief spending, finding that each dollar of New Deal spending raised incomes by about 60 cents. More recent work by Hausman (2016) employed a panel of U.S. cities to show that New Deal spending lowered unemployment but also crowded out private investment—a dynamic that only a panel approach could capture by tracking the same cities before, during, and after the program.

Regional Disparities and Convergence

Panel unit-root tests and convergence regressions have been applied to historical data on income per capita across European regions from the 19th century onward. These methods help determine whether poorer regions are catching up to richer ones (beta-convergence) and whether the dispersion of incomes is shrinking (sigma-convergence). For instance, a study using a panel of 71 French and German regions from 1840 to 1910 found strong evidence of convergence within each country, driven largely by trade liberalization and factor mobility. The panel structure allowed the researchers to control for country-specific shocks (e.g., the Franco-Prussian War) while examining within-region growth patterns. Another innovative study by Crafts and Wolf (2014) used a panel of European NUTS-2 regions from 1850 to 1910 to show that convergence accelerated after the adoption of the gold standard, which reduced transaction costs and integrated capital markets.

Human Capital and Demographic Change

Panel data has also enriched our understanding of the demographic transition. By constructing historical panels of birth rates, death rates, and literacy across hundreds of districts in Sweden or the United States, researchers have examined the causal role of mass education in reducing fertility. Fixed-effects models that control for permanent district characteristics (such as religious composition or climate) reveal that the introduction of compulsory schooling laws significantly accelerated the decline in birth rates. A notable example is the work of Becker, Cinnirella, and Woessmann (2010), who used a panel of Prussian counties from 1849 to 1914 to demonstrate that education reduced fertility by altering the quality-quantity tradeoff—parents invested more in fewer children. The panel allowed them to control for county-level characteristics that could jointly determine education and fertility decisions, such as local labor demand or cultural norms.

Conflict and Political Economy

Panel methods have increasingly been used to study historical conflicts. For example, Besley and Reynal-Querol (2014) constructed a panel of African ethnic groups over the 20th century to examine how precolonial institutions shaped the incidence of civil war. By including group fixed effects, they could control for time-invariant factors like geography or language, isolating the effect of changes in political centralization. Similarly, Dincecco and Prado (2012) used a panel of European states from 1600 to 1900 to show that fiscal centralization—measured by the share of tax revenue collected by the central government—reduced the frequency of interstate wars. Their panel allowed them to include country and century fixed effects, reducing omitted variable bias from slow-moving cultural or geographic factors.

Advantages of Panel Data Methods in Historical Research

The benefits of using panel data in economic history are numerous and often critical for credible causal inference. These advantages explain why panel methods have become the default framework for quantitative historical analysis.

  • Control for unobserved heterogeneity: Historical studies are plagued by omitted variables such as culture, geography, or persistent institutional quality. Fixed-effects models sweep away all time-invariant confounders, even those we cannot measure, providing a powerful defense against spurious correlations.
  • Uncover dynamic relationships: Panel data allows researchers to model lagged effects, adjustment processes, and path dependence. For example, the effect of a policy change in one decade may only fully materialize decades later, which cross-sectional data cannot capture. Dynamic panel models explicitly estimate the speed of adjustment to equilibrium.
  • Increase sample size and power: Combining time periods multiplies the number of observations, leading to more precise estimates and the ability to detect small but economically meaningful effects. This is particularly valuable when the cross-sectional dimension is limited by data availability.
  • Handle measurement error through multiple observations: When historical records contain random errors but are unbiased, panel estimators average out some of the noise. In some cases, using two time periods can even allow correction for classical measurement error through instrumental variables.
  • Compare across space and time simultaneously: Panel datasets allow researchers to test whether a relationship holds not only across regions but also within regions as historical conditions change. This within-unit variation is often the only source of identification when units differ in many unmeasured ways.

For a detailed survey of these advantages in historical contexts, see this article in the Journal of Economic History.

Challenges and How Historians Overcome Them

Despite their power, panel methods present unique difficulties when applied to historical data. Fortunately, researchers have developed strategies to mitigate many of these issues, often combining econometric techniques with deep historical knowledge.

Data Availability and Quality

Historical data is often sparse, irregularly collected, and subject to changing definitions. For example, GDP estimates for the 19th century rely on proxies such as tax receipts, urban population, or industrial output. Missing observations are common, leading to unbalanced panels. Solutions include:

  • Imputation techniques: Multiple imputation or interpolation can fill gaps when data is missing at random. For example, if a country’s GDP is missing for a few years due to war, interpolation from neighboring years may be acceptable.
  • Using alternative proxies: Night lights (from satellite imagery backcasted), property records, or price indices have been used as substitutes for missing income measures in pre-industrial contexts.
  • Restricting the sample to periods with reliable data: Many panel studies focus on the post-1820 era when national accounts became more standardized, or they limit analysis to periods without major territorial changes.

Model Specification and Heterogeneity

Choosing between fixed and random effects depends on whether unobserved heterogeneity is correlated with the regressors. In historical settings, this correlation is almost certain—for instance, regions with a more favorable initial endowment may also adopt better policies. Therefore, fixed-effects models are generally preferred. Yet even within fixed effects, researchers must decide whether to include time effects (to control for common shocks such as wars or global economic cycles). Hausman tests and the inclusion of time trends are standard practice. Additionally, when the panel has a long time dimension, researchers must consider whether the effect of interest varies over time—for example, the impact of railroads on land values may have been larger in the early 19th century than later. Interaction terms with time dummies can test for such heterogeneity.

Potential Biases

Several forms of bias are particularly acute in historical panel data:

  • Attrition bias: In long panels, some units may drop out (e.g., due to state dissolution or annexation). If attrition is correlated with the outcome, estimates become inconsistent. Researchers test for this using attrition tests (e.g., comparing characteristics of stayers and leavers) or weighting methods like inverse probability weighting.
  • Measurement error: Classical measurement error in a time-varying regressor leads to attenuation bias. Instrumental variables (IV) methods, such as using a third time period to instrument for the mismeasured variable, can help. Alternatively, if the error is non-classical (e.g., correlated with the true value), more sophisticated approaches like using multiple noisy measures are needed.
  • Serial correlation: Errors in panel data are often correlated over time. Robust standard errors clustered at the entity level (e.g., country) are now standard, and some studies use panel-corrected standard errors or autoregressive specifications. Failure to account for serial correlation can severely understate standard errors.

Endogeneity and Time-Varying Confounders

Fixed effects remove time-invariant confounders, but time-varying confounders remain a threat. For example, a policy change that coincides with a recession may appear harmful even if it is beneficial. Difference-in-differences (DiD) designs with staggered adoption are one common solution, often estimated via two-way fixed effects (unit and time effects). However, recent research has shown that two-way fixed effects can be biased when treatment effects are heterogeneous across units or over time. More recently, economists have developed dynamic DiD methods (Callaway & Sant’Anna, 2021; Goodman-Bacon, 2021) that are valid with heterogeneous treatment effects—a crucial advance for historical studies where policies unfold at different rates in different places. For instance, a study of the introduction of public schooling across U.S. states from 1830 to 1900 could use these new methods to estimate the average treatment effect on the treated, even though some states adopted earlier than others and the effect of schooling may have changed over time.

Emerging Methods: Dynamic Panels and Spatial Panels

The frontier of panel data methods in economic history increasingly involves dynamic models and spatial dependence. Dynamic panels—including lagged dependent variables or autoregressive parameters—allow researchers to model persistence and adjustment speeds. The Arellano-Bond estimator (1991) and its extensions (Blundell-Bond, 1998) are now used to study topics like the persistence of regional inequality, where current income depends on past income. However, historical panels often have a long time dimension (T large) and a moderate N, meaning the Nickell bias (which plagues dynamic FE when T is small) is less severe. Researchers must still be careful with instrument proliferation.

Spatial panel models account for spillovers across units—for example, the spread of innovation from one region to neighboring regions. Ignoring spatial dependence can lead to omitted variable bias because outcomes in one unit are correlated with outcomes in nearby units through unobserved shocks. Spatial lag and spatial error models, integrated with fixed effects, are becoming more common in historical analyses of technology adoption, conflict contagion, and trade networks. A recent example is the study by Juhász (2018), who used a panel of French departments during the Napoleonic Wars to show that temporary protectionism (the Continental System) led to persistent industrialization in protected regions, with spatial spillovers to neighboring regions through market integration.

Conclusion: Expanding the Possibilities of Historical Inquiry

Panel data methods have become indispensable tools for economic historians. By leveraging repeated observations of the same units over time, researchers can control for unobservable confounders, model dynamic processes, and obtain more reliable estimates of causal effects. The historical record is full of natural experiments—institutional changes, wars, technological breakthroughs—that can be analyzed credibly with panel techniques. As more historical data becomes digitized and harmonized—projects like Clio-Infra, the Medieval Cartography project, and the Global Historical Databases initiative are expanding coverage—the potential for panel data studies will only grow.

However, historians must remain vigilant about data quality, model assumptions, and the historical context that might generate biases. Panel methods are not a panacea; they require careful attention to the structure of the data, the definition of units, and the historical appropriateness of identifying assumptions. When used carefully, panel data methods illuminate patterns that would otherwise remain buried in archival silences. They allow us to ask not only what happened in history, but why it happened—a question at the very heart of economic history.