How to Rescue a Population Genomics Project with Missing Data, Low Coverage, or Unbalanced Groups

For Research Use Only. Not for use in diagnostic procedures or clinical decision-making.

Most population genomics projects do not fail catastrophically. They degrade gradually — a batch of samples with lower coverage than expected, a metadata file with missing population labels, a VCF where 15% of variants are absent in half the samples. The question is rarely "is this dataset usable?" but "what can I still do with it, and what needs to be fixed?"

This guide is for researchers who have already generated data and discovered problems. It covers the most common rescue scenarios: diagnosing missing data patterns, handling low-coverage samples, fixing unbalanced group sizes, detecting and correcting batch effects, and using genotype likelihood-based methods to salvage low-depth data. It also defines the point at which re-sequencing is the better option than continued rescue.

Figure 1: Common population genomics data quality problems and their primary rescue strategies — most issues can be addressed without re-sequencing.

What Rescue Looks Like

Data rescue is not about salvaging a failed project — it is about systematically identifying what analyses are still valid, which samples or variants need to be excluded, and what analytical adjustments compensate for remaining weaknesses. The goal is to maximize the biological insight you can extract from the data you have, while honestly documenting the limitations so reviewers can assess the evidence appropriately.

The rescue process follows four steps:

Diagnose. Quantify the problem: which samples and variants are affected, at what magnitude, and with what pattern. A missing data problem that is random across samples and variants is manageable. A missing data problem that is correlated with population group, phenotype, or sequencing batch is a confound that must be addressed before any downstream analysis.
Contain. Remove or down-weight the most severely affected data points — samples with extreme missingness, variants with near-zero call rates, genomic regions with systematic coverage gaps — to prevent them from distorting downstream results.
Adapt. Switch to analytical methods that are robust to the remaining data limitations. For low coverage, this means genotype likelihoods instead of hard genotype calls. For unbalanced groups, this means down-sampling or using methods with explicit population size priors.
Validate. Confirm that your rescue strategy produced reliable results by checking internal consistency — do PCA and ADMIXTURE agree? Does F_ST between known-divergent populations exceed F_ST between known-similar populations? — and by comparing rescued results to published expectations.

Before attempting rescue on a dataset you did not generate yourself, review our guide to preparing FASTQ, BAM, or VCF files for analysis to ensure you have all the necessary files and metadata.

Diagnosing Missing Data Patterns

Missing data is the most common data quality problem in population genomics. The key diagnostic question is not "what percentage of data is missing?" but "is the missingness random or structured?"

Table 1: Missing Data Diagnosis

Pattern	What It Looks Like	Most Likely Cause	Rescue Strategy
Random missingness	Missing genotypes distributed evenly across samples and variants	Typical stochastic variation in sequencing and variant calling	Standard missingness filters — exclude variants >10% missing, samples >10% missing
Sample-correlated missingness	Specific samples have consistently high missingness across all variants	Low-quality DNA, low library concentration, or degraded samples	Exclude affected samples if >10% missing; investigate DNA quality for retained samples
Variant-correlated missingness	Specific variants missing across most or all samples	Low-complexity regions, paralogous sequences, or reference assembly gaps	Exclude affected variants; these are not biologically informative
Population-correlated missingness	Missingness rate differs systematically between population groups	Reference bias — reads from diverged populations map less efficiently to the reference	Use a reference-free approach or genotype likelihoods; consider a population-specific reference
Batch-correlated missingness	Missingness clusters by sequencing batch or library preparation date	Batch effects in library preparation or sequencing	Include batch as a covariate; test whether excluding one batch changes results

The most dangerous pattern is population-correlated missingness, because it can create artificial genetic differentiation between populations. If Population A has 5% missingness and Population B has 20% missingness at the same variant set, the variants that pass a uniform missingness filter will be biased toward Population A's allele frequency spectrum, producing inflated F_ST estimates and distorted PCA projections.

For a systematic approach to evaluating data quality before analysis, review the per-sample and per-variant QC metrics described in our population genomics report deliverables guide.

Side-by-side comparison of random missingness pattern versus population-correlated missingness pattern in genotype matrices Figure 2: Missing data pattern comparison — random missingness (left) is manageable with standard filters; population-correlated missingness (right) is a dangerous confound that produces artificial F_ST inflation between populations.

When Coverage Falls Short

Low sequencing coverage — below 4–5× for population genomics — reduces confidence in individual genotype calls. But low coverage does not mean no analysis. It means switching from genotype calling to genotype likelihood-based methods.

The critical distinction is between hard genotype calls and genotype likelihoods:

Hard genotype calls (0/0, 0/1, 1/1) are binary decisions made at each site for each sample. At low coverage, these decisions carry high uncertainty — a heterozygous site covered by 2 reads may be called homozygous if both reads happen to sample the same allele. Hard calls at low depth produce downward-biased heterozygosity estimates, inflated false positive rates in GWAS, and unreliable individual-level metrics.
Genotype likelihoods (GLs) express the probability of the observed sequencing data given each possible genotype, without making a hard call. Tools like ANGSD, bcftools mpileup with the -G flag, and GATK's --genotyping-mode GENOTYPE_GIVEN_ALLELES propagate this uncertainty into downstream analyses, producing more accurate allele frequency estimates, diversity statistics, and population structure at low coverage.

When to use GL-based methods:

Coverage below 5× across most samples — use ANGSD for PCA (PCAngsd), admixture (NGSadmix), F_ST estimation, and neutrality tests.
Coverage between 5–10× — GL-based and hard-call methods often agree for common variants; GL methods provide better rare variant sensitivity.
Coverage above 10× — hard-call methods with standard QC are generally reliable; GL methods add marginal benefit at increased computational cost.

For a detailed comparison of low-coverage WGS (with ANGSD and GL-based approaches) versus reduced-representation alternatives, see our low-coverage WGS vs ddRAD comparison.

Fixing Group Imbalance

Unbalanced sample sizes between population groups — 200 samples from Population A, 12 from Population B — is a study design problem that data rescue cannot fully solve, but can mitigate.

For population structure analyses (PCA, ADMIXTURE, phylogenetic trees): Group imbalance is generally manageable. PCA is relatively robust to unbalanced groups — a small, genetically distinct population will form its own cluster regardless of the size difference. ADMIXTURE can handle unbalanced groups, but cross-validation error may be dominated by the larger group, making K selection less reliable. Down-sampling the larger group to match the smaller group for K-selection analysis can help, but report both the full-dataset and balanced-subset results.

For F_ST and differentiation analyses: Group imbalance inflates the variance of F_ST estimates for small populations but does not systematically bias the point estimate. Report confidence intervals alongside point estimates for small population comparisons.

For GWAS and association analyses: Group imbalance is a serious problem. GWAS statistical power is driven by the smaller group in a case-control comparison — a GWAS with 200 cases and 12 controls effectively has the power of a 12-sample study. Imbalanced group sizes also inflate type I error in standard association tests. Use SAIGE or REGENIE for binary traits with unbalanced case-control ratios, and always report effective sample size alongside nominal sample size.

For diversity and selection statistics within small populations: With fewer than 10–15 samples per population, within-population diversity statistics (π, θ_W, H_E) have wide confidence intervals, and selection scans (Tajima's D, iHS) lose power. Limit analyses of small populations to between-population comparisons (F_ST, D-statistics) and population structure, and avoid reporting within-population selection statistics unless the small group is genuinely the population of interest and reduced power is acceptable with appropriate caveats.

Batch Effects and Confounding

Batch effects — systematic differences between samples driven by technical rather than biological variation — are pervasive in population genomics. They arise from differences in DNA extraction protocol, library preparation kit, sequencing instrument, flow cell, lane, and even the date of processing. The most insidious batch effects are those that are confounded with the biological variable of interest: all samples from Population A processed in Batch 1, all samples from Population B processed in Batch 2.

Detection:

Color PCA plots by batch identifier. If samples cluster by batch rather than by population, batch effects dominate the biological signal.
Run a per-variant association test between genotype and batch. A Manhattan plot of batch-association p-values should be flat. Peaks indicate genomic regions where batch systematically affects genotype calls.
Compare F_ST between batches within the same population to F_ST between populations. Within-population batch F_ST should be close to zero.

Mitigation:

If batch and population are partially confounded (some Population A samples in each batch, but unbalanced), include batch as a covariate in association models and report batch effects transparently.
If batch and population are fully confounded (all Population A in Batch 1, all Population B in Batch 2), you cannot statistically separate batch from biology. The most honest approach is to report this confound explicitly and acknowledge that population differences could be partially or entirely technical.
For fully confounded designs, re-sequencing a subset of samples from each population in a new batch provides a bridge that enables batch effect estimation. For projects requiring complete re-analysis after rescue, joint variant calling across all samples — combining rescued and newly sequenced data in a single pipeline — is the most reliable way to eliminate batch effects at the variant detection stage. When re-sequencing is unavoidable, whole genome resequencing at ≥10× coverage provides the cleanest analytical foundation.

For a full treatment of QC metrics that matter at cohort scale, see our cohort-scale QC guide.

Genotype Likelihood Rescue Strategies

When coverage is low (below 5×) but re-sequencing is not feasible, genotype likelihood-based methods offer the most powerful rescue path. These methods do not call genotypes — they work directly with the per-site likelihoods of each possible genotype, propagating read-level uncertainty through to population-level inference.

Table 2: Genotype Likelihood-Based Rescue Tools

Analysis	Hard-Call Tool	GL-Based Alternative	Key Advantage of GL Approach
PCA	PLINK, EIGENSOFT	PCAngsd	Accounts for genotype uncertainty; works at coverage as low as 0.5×
Admixture	ADMIXTURE	NGSadmix	Estimates ancestry proportions without hard genotype calls
F_ST	VCFtools, PLINK	ANGSD (F_ST module)	Uses sample allele frequency likelihoods; unbiased at low depth
Nucleotide diversity	VCFtools, pixy	ANGSD (diversity module)	Estimates θ_π and θ_W directly from GLs
Selection scans	selscan, rehh	ANGSD (thetas, Tajima's D)	Site frequency spectrum from GLs; avoids downward bias from low-depth hard calls
GWAS	PLINK, BOLT-LMM	REGENIE (with dosage data)	Handles imputed dosage data from low-coverage imputation

The typical rescue workflow for a low-coverage dataset is: (1) generate genotype likelihoods in BEAGLE format using ANGSD or bcftools, (2) run PCAngsd and NGSadmix for population structure, (3) estimate F_ST and diversity statistics in ANGSD, and (4) if GWAS is the goal, impute to a reference panel using GLIMPSE or Beagle and run association tests on imputed dosages.

Pipeline workflow showing genotype likelihood-based rescue path from raw BAM files through ANGSD to PCAngsd, NGSadmix, and FST analysis Figure 3: Low-coverage rescue workflow using genotype likelihoods — from raw BAM files through ANGSD genotype likelihood estimation to publication-ready population genomics results without hard genotype calls.

Deciding When to Re-Sequence

Data rescue has limits. Some problems cannot be fixed analytically and require generating new data. Knowing when to stop rescuing and start re-sequencing saves time, money, and credibility.

Re-sequencing is the better option when:

More than 25% of samples have coverage below 2×, and joint variant calling across all samples produces a VCF where most variants are called in fewer than half the samples. GL-based methods can compensate for low coverage but cannot create information that was never sequenced.
Batch and biological variable of interest are fully confounded, and bridge samples cannot be arranged. No statistical method can disentangle what is confounded by design.
DNA quality for a subset of samples was so poor (DIN below 4, extensive degradation) that the resulting data shows systematic coverage gaps in gene-rich regions, and those regions are central to the research question.
The reference genome has been substantially upgraded (scaffold-level to chromosome-level), and re-alignment and re-calling against the new reference are expected to recover thousands of variants in previously unmappable regions.

Rescue is the better option when:

Missingness is random or sample-correlated and below 15%, and excluding the worst samples leaves sufficient sample sizes in all groups.
Coverage is moderate (3–8×) and GL-based methods can recover accurate population-level statistics even if individual genotypes are uncertain.
Group imbalance is manageable (the smaller group has at least 10–15 samples) and analyses are limited to population structure and between-population comparisons.
Batch effects are detected and partially confounded but bridge samples exist or batch can be included as a covariate.
Budget or sample availability makes re-sequencing impossible, and a well-documented rescue analysis — with all limitations transparently reported — is the only path to publication.

Frequently Asked Questions

Can population genomics analysis still be performed with low coverage data?

Yes. Coverage as low as 1–2× supports population structure analysis (PCA, ADMIXTURE) and between-population comparisons (F_ST) when genotype likelihood-based methods are used. Within-population diversity statistics and individual-level analyses lose accuracy below 4–5×. The key is switching from hard genotype calls to genotype likelihoods — the analytical framework, not the coverage depth, determines what is recoverable.

At what missing data rate should I exclude a sample or variant?

Standard thresholds are 10% missingness for samples and variants in most population genomics analyses. Variants with more than 10% missing data across samples are excluded; samples with more than 10% missing data across variants are flagged for investigation. These thresholds are guidelines, not rules — a sample with 12% missingness that is otherwise high-quality and belongs to a small but critical population group may be worth retaining with documented justification. The threshold should be set based on the missingness pattern (random vs. structured), not applied mechanically.

How many samples per population are enough for rescue?

For population structure (PCA, ADMIXTURE), 5–10 samples per population is generally sufficient to detect major clusters, though differentiation between closely related populations becomes less reliable with small sample sizes. For within-population diversity statistics, 10–15 samples is the minimum for reasonably precise estimates. For GWAS and selection scans, effective sample size — the smaller group in a comparison — drives power, and rescue strategies should focus on between-population comparisons rather than within-population inference when samples are limited.

What is the difference between missing data and low coverage?

Missing data means a genotype was not called at a specific site in a specific sample — typically because coverage at that site was zero or below the variant caller's confidence threshold. Low coverage means the average sequencing depth across the genome is low, which increases the rate of missing data but does not guarantee it at every site. A sample with 2× average coverage may still have confident genotype calls at sites where both alleles were sampled; a sample with 30× coverage may still have missing genotypes at sites where mapping quality is low due to repetitive sequence. Low coverage is a cause; missing data is a consequence.

Can I combine rescued data with newly sequenced samples?

Yes, but only if the new samples are processed through the same bioinformatics pipeline from the same starting point. If the rescued data is in VCF format and the new samples are in FASTQ format, both should be re-processed together — alignment and joint variant calling across old and new samples — rather than calling variants in new samples and merging VCFs. Joint processing ensures consistent variant calling, reduces batch effects, and produces a single, unified dataset with uniform QC metrics.

References:

Korneliussen TS, Albrechtsen A, Nielsen R. ANGSD: analysis of next generation sequencing data. BMC Bioinformatics. 2014;15:356. doi:10.1186/s12859-014-0356-4
Meisner J, Albrechtsen A. Inferring population structure and admixture proportions in low-depth NGS data. Genetics. 2018;210(2):719-731. doi:10.1534/genetics.118.301336
Lou RN, Therkildsen NO. Batch effects in population genomic studies with low-coverage whole genome sequencing data: causes, detection and mitigation. Molecular Ecology Resources. 2022;22(5):1678-1692. doi:10.1111/1755-0998.13559
Li JH, Mazur CA, Berisa T, Pickrell JK. Low-pass sequencing increases the power of GWAS and decreases measurement error of polygenic risk scores compared to genotyping arrays. Genome Research. 2021;31(4):529-537. doi:10.1101/gr.266429.120
Rubinacci S, Ribeiro DM, Hofmeister RJ, Delaneau O. Efficient phasing and imputation of low-coverage sequencing data using large reference panels. Nature Genetics. 2021;55(1):168-179. doi:10.1038/s41588-023-01315-1

For Research Use Only. Not for use in diagnostic procedures or clinical decision-making.

* Designed for biological research and industrial applications, not intended for individual clinical or medical purposes.