Cliometric research – the application of quantitative methods to economic history – has revolutionized our understanding of long-run economic development, institutional change, and the roots of modern prosperity. By stitching together centuries of price series, census records, trade ledgers, and tax rolls, cliometricians test hypotheses that qualitative historians can only debate. Yet beneath the elegant regressions and counterfactual simulations lies a persistent and often underappreciated challenge: the twin problems of data scarcity and data quality. Without reliable historical data, even the most sophisticated econometric models produce misleading inferences. This article examines the sources, consequences, and mitigation strategies for these data challenges, drawing on insights from decades of cliometric practice and recent advances in computational methods.

The Nature of Data Scarcity in Historical Economic Research

Data scarcity in cliometrics is not merely a matter of small sample sizes. It reflects the fundamental incompleteness of historical records: governments did not always collect the statistics we now need; wars, fires, and bureaucratic decay destroyed archives; and recording conventions were uneven across regions and eras. For early modern Europe, for example, systematic national income accounts do not exist before the nineteenth century. Researchers must piece together proxies from tithe records, trade statistics, and urban tax registers. In pre-colonial Africa or pre-contact Americas, written records are often absent altogether, forcing reliance on archaeological proxies, colonial censuses, or oral traditions, each with its own limitations.

Sources of Scarcity

  • Physical loss: The Library of Alexandria is only the most famous example. Thousands of parish registers, mercantile ledgers, and state papers have perished through fire, flood, neglect, or deliberate destruction (e.g., the 1943 bombing of Naples destroyed centuries of medieval trade data).
  • Selective survival: Records that survive often favor the literate, the wealthy, the urban, and the male. Rural peasants, women, and indigenous populations appear only intermittently – if at all – creating systematic gaps that can bias economic analyses of welfare or inequality.
  • Changes in administrative boundaries: A municipality that merged, split, or changed name across centuries makes longitudinal panels nearly impossible without painstaking geo-referencing and harmonization.
  • Low frequency of measurement: Most pre-modern statistics were collected at irregular intervals – sometimes every decade, sometimes once a century. Cross-sectional data may be missing for entire generations.

These scarcities force cliometricians to adopt what might be called a “pragmatic epistemology”: working with the best available evidence while openly acknowledging its lacunae. The resulting datasets are often unbalanced panels or cross-sections with many missing cells, which complicates the use of standard econometric techniques.

Consequences of Scarcity for Economic Models

When variables are missing, researchers may resort to proxy variables that only weakly capture the concept of interest. For instance, using the number of railway stations as a proxy for market integration ignores road and river transport. Alternatively, they might drop observations with missing data, shrinking sample size and potentially introducing selection bias. If missingness correlates with unobserved characteristics (e.g., poorer regions kept worse records), the resulting estimates are inconsistent. Time-series analyses become especially fragile: a single missing data point in an autoregressive model can break the dynamic structure.

Perhaps more insidious is the tendency to rely on strong assumptions about data-generating processes. Researchers often assume that missing data are “missing at random” conditionally on observables, a claim rarely defensible in historical settings. The upshot is that many cliometric findings come with wide confidence intervals and a high sensitivity to specification – a fact that non-technical historians sometimes mistake for methodological weakness rather than honest uncertainty.

The Quality Dimension: Error, Bias, and Inconsistency

Even when historical data survive, their quality is far from guaranteed. Data quality encompasses accuracy, consistency, completeness, and freedom from systematic bias. Historical sources were created for administrative, fiscal, or legal purposes, not for modern scientific analysis. As a result, they carry the imprints of their creators’ motives and limitations.

Common Forms of Poor Data Quality

  • Transcription errors: Handwritten ledgers are prone to misreading; one study found that clerks in nineteenth-century British factories miscounted production by 5–10%. When digitized via optical character recognition (OCR), historical fonts introduce further errors – e.g., “1642” becomes “164Z”.
  • Changing definitions: The category “urban” might have shifted from 2,000 inhabitants in 1800 to 10,000 in 1900. “Wheat prices” could exclude transportation costs in one archive but include them in another. Such inconsistencies can generate phantom structural breaks.
  • Rounding and heaping: Age heaping – the tendency to report ages ending in 0 or 5 – is notorious in census data from the nineteenth century, biasing demographic estimates. Prices were often rounded to the nearest shilling or peseta, suppressing variance.
  • Selection bias: Courts recorded crimes, but not unreported offenses. Tax assessments omitted the poorest households. Newspapers covered dramatic events, not everyday trade. Using such sources without correction yields a distorted picture of economic activity.
  • Measurement units: A “bushel” varied by commodity and region; “pounds sterling” changed in silver content over centuries. Without careful metrological conversion, price series become non-comparable.

The cumulative effect of these quality issues is measurement error, which biases regression coefficients toward zero (classical errors-in-variables) or in unpredictable directions (non-classical error). For example, per capita income estimates based on largely urban salary data will overstate national income if the rural sector is underrepresented. Mis-measured growth rates can produce spurious “poverty traps” or “take-offs”.

Bias in Historical Record-Keeping

A particularly vexing quality issue is systematic bias introduced by the political, social, or economic context of record creation. Colonial administrators, for instance, often reported inflated tax revenues to please their superiors. Land registration in many societies excluded women or communal ownership, making it impossible to reconstruct true asset distribution. Nineteenth-century factory inspectors focused on large mills, ignoring small workshops. These biases are not random; they are correlated with the very economic phenomena cliometricians wish to study, such as inequality, state capacity, and industrialization.

Failure to account for such bias can lead to published empirical findings that contradict later archival discoveries. The classic example is the “Great Divergence” debate: early cliometric estimates of pre-1800 Chinese GDP were based on European-standard price data, but later work using Ming dynasty tax rolls revealed much higher levels of agricultural productivity, challenging the narrative of a stagnant Asian economy. Data quality, in this sense, is not only a technical issue but a historiographical one.

Strategies for Mitigating Scarcity and Quality Problems

Cliometricians have developed a rich toolkit to address these challenges. The choice of strategy depends on the nature of the missingness or error and the research question. Below we survey the most important approaches, moving from simple to sophisticated.

Data Imputation and Multiple Imputation

When data points are missing but the pattern is plausibly random conditional on observables, imputation can fill the gaps. Single imputation (e.g., replacing missing values with the mean or last observation) is simple but artificially reduces variance. Modern cliometrics increasingly uses multiple imputation (MI), which creates several plausible completed datasets, runs analyses on each, and combines estimates with Rubin’s rules. MI is well-suited to historical panels where missingness is monotonic (e.g., a country’s GDP not recorded before 1820). However, MI assumes that the variables predicting missingness are included in the imputation model – a strong requirement when unobserved historical shocks cause data loss.

Cross-Verification and Triangulation

Before any statistical correction, researchers should cross-verify data against independent sources. For example, a price series from a merchant ledger can be checked against municipal market registers, newspaper reports, and even ship manifests. Triangulation reveals transcription errors, exposes local biases, and sometimes uncovers entirely new data points. The Cliometric Society maintains databases that encourage such multi-source validation, but the process remains labor-intensive and domain-specific.

Robust Statistical Models

Some econometric techniques are inherently robust to data imperfections. Instrumental variables (IV) can correct for measurement error in a key regressor, provided a valid instrument exists. Fixed effects absorb time-invariant unobservables (e.g., persistent data collection biases in a given village). Bootstrap standard errors and Bayesian estimation allow researchers to incorporate uncertainty about missingness into confidence intervals. For instance, Bayesian structural time series models can handle irregularly spaced historical observations by specifying prior distributions over missing periods.

Machine Learning and Digitization Advances

Recent years have seen a surge in computational approaches to historical data. Optical character recognition (OCR) has become more accurate for historical fonts, and handwriting recognition (HTR) can now transcribe cursive manuscripts with error rates below 10%, enabling mass digitization of census returns and parish registers. Natural language processing (NLP) extracts structured data from unstructured historical texts (e.g., price quotes from newspapers). Record linkage algorithms – based on machine learning – can match individuals across censuses, creating longitudinal panels from repeated cross-sections. These tools reduce both scarcity (by unlocking previously inaccessible archives) and quality (by standardizing heterogeneous sources). A notable project is Historical Data, which combines HTR, NLP, and geocoding to build global datasets from colonial trade records.

Sensitivity Analysis and Replication

Given the fragility of historical data, sensitivity analysis should be mandatory. Researchers can test how results change when missing values are imputed with different algorithms, when outliers are trimmed, or when the sample is limited to high-quality subsets. Reporting multiple specifications – a “multi-model” approach – allows readers to gauge the robustness of conclusions. The replication movement in cliometrics has also improved quality: many journals now require data and code deposits, enabling other researchers to uncover errors or test alternative assumptions. The Economic History Association maintains a repository that facilitates such transparency.

Case Study: Reconstructing GDP Growth in Early Modern England

To see these challenges and strategies in action, consider the reconstruction of English GDP from 1600 to 1800. Early cliometric work by W. A. Cole and Phyllis Deane drew on scattered customs records, agricultural output estimates, and wage data, but faced severe scarcity: no systematic national accounts existed before 1855. Later researchers like Stephen Broadberry and colleagues (2015) tackled data quality by:

  • Digitizing thousands of parish register entries for agricultural output (using HTR for handwriting).
  • Cross-verifying industrial production data from Quaker business records (high quality) with excise tax returns (systematic but possibly evaded).
  • Using multiple imputation to fill gaps in years where London port records were destroyed in the Great Fire of 1666.
  • Applying Bayesian dynamic factor models to smooth erratic price series.

The resulting dataset reveals that English per capita GDP growth was steady but modest (~0.3% per year) across the seventeenth century, not the stagnation previously assumed. Without careful management of scarcity and quality, the earlier “stagnation” narrative would have persisted – a testament to the importance of these methodological corrections.

Future Directions

Looking ahead, three developments promise to ease data scarcity and quality burdens in cliometrics. First, large-scale digitization initiatives – such as the Transkribus platform for historical manuscripts – are making rare sources machine-readable at unprecedented scale. Second, probabilistic record linkage and entity resolution algorithms will allow researchers to merge datasets across regions and centuries, creating richer panels. Third, causal inference methods (e.g., difference-in-differences with synthetic controls) are being adapted to settings with missing pre-treatment data, offering more robust tests of historical hypotheses.

Yet technology alone cannot substitute for domain knowledge. Understanding why a particular archive was created – who funded it, what political pressures existed, which groups were excluded – remains essential for evaluating data quality. The best cliometric research combines computational muscle with historical deep reading, treating data scarcity and quality not as nuisances but as objects of inquiry in their own right.

Conclusion

Data scarcity and data quality are not peripheral nuisances in cliometric research; they are central to the validity of every claim about economic history. Scarcity forces researchers to work with incomplete, non-random samples; quality issues introduce errors and biases that can overturn conclusions. By adopting rigorous imputation techniques, cross-verifying sources, exploiting robust statistical methods, and leveraging new digitization tools, cliometricians can mitigate these problems – but never eliminate them. The future of the field lies in transparent reporting, replication, and a willingness to quantify the uncertainty that historical distance imposes. Only by facing these challenges directly can cliometrics fulfill its promise of providing a quantitative foundation for our understanding of the past.