Prepare sample metadata • popgenVCF

Metadata

Metadata are optional. Without a metadata file, popgenVCF uses VCF sample names as public labels and runs analyses that do not require groups or geography.

When metadata are supplied, the file must contain exactly one row for every VCF sample. Do not omit a row merely because some annotations are unavailable.

Recommended file structure

Use a headered, tab-separated UTF-8 text file. Comma-separated and whitespace-separated files are accepted, but tabs are the safest choice when optional fields are empty or labels contain spaces.

The parser detects tabs, commas, or whitespace from the first line. Header names are converted to lowercase and non-alphanumeric runs become underscores. Headerless files support only the first column as sample and the second as population; use a header for aliases, coordinates, and other annotations.

Configure the file as follows:

input:
  metadata: /data/metadata.tsv
  metadata_header: auto
  geographic_columns: [latitude, longitude]

metadata_header accepts auto, yes/true, or no/false. Prefer auto or yes for a headered file. The default geographic-column order is latitude first and longitude second.

Column reference

Column	Requirement and meaning	Missing values
`sample`	Required immutable key; must match one VCF sample name exactly and uniquely	Never allowed
`alias`	Optional public label used in tables, figures, trees, and reports	Falls back to `sample`
`population`	Group used by diversity, FST, DAPC, AMOVA, bootstrap, and spatial modules	Allowed for annotation-only use, but any missing value disables population-dependent modules
`latitude`	North-south coordinate in signed decimal degrees	Sample is excluded from spatial calculations if either coordinate is missing
`longitude`	East-west coordinate in signed decimal degrees	Sample is excluded from spatial calculations if either coordinate is missing
`individual`	Optional biological-individual grouping	Allowed
`family`	Optional family or pedigree grouping	Allowed; repeated values are expected
`replicate`	Optional technical or biological replicate grouping	Allowed; repeated values are expected
`display_order`	Optional positive integer for stable identity-table ordering	Allowed; non-missing values must be unique
`sex`	Optional annotation (e.g. `male`/`female`)	Allowed; see below for its PCA-colouring role
Any other column	Preserved as descriptive metadata	Allowed

Any repeated-value annotation column – sex is a common example – also drives an extra PCA panel automatically: once a column has enough samples per distinct value (analyses.pca_metadata_color_min_group, default 3) and not too many distinct values (analyses.pca_metadata_color_max_levels, default 12), the report gains a panel recolouring the same PCA fit by that column (figures/07b_PCA_PC1_PC2_by_<column>). population, coordinates, and identifier-like columns are excluded automatically since they are already handled elsewhere. See the interpreting-results article for a real example coloured by sex.

Use the exact names above in new files. The reader also recognizes common sample-key headers (sample_id, id, individual, individual_id) and pop as a population header, but explicit sample and population names avoid ambiguity when an individual grouping column is also present.

Using a differently named column

If a metadata column already holds population-equivalent information under a name the built-in detection does not recognize (a pathotype, lineage, or clade column, for example), point input.population_column (and, if needed, input.sample_column) at it instead of renaming the column in the file itself:

input:
  sample_column: null
  population_column: pathotype

An explicit sample_column/population_column always wins over the built-in auto-detection, even if a column already named sample or population is also present. The name is matched case-insensitively after the same normalization applied to every header (non-alphanumeric runs become underscores), so Pathotype, pathotype, and patho-type all match. Both require a headered metadata file (input.metadata_header not no).

Complete example

sample  alias   population  latitude    longitude   location    individual  family  replicate   display_order
VCF_001 Birch_01    North   45.4215 -75.6972    Ottawa  Tree_01 Family_A    NA  1
VCF_002 NA  North   NA  NA  Ottawa  Tree_02 Family_A    NA  2
VCF_003 Birch_03    South   -33.8688    151.2093    Sydney  Tree_03 Family_B    Batch_2 3
VCF_004 Birch_04    South   48.8566 2.3522  Paris   Tree_04 Family_B    Batch_2 4

Here VCF_002 remains available to non-spatial analyses, uses VCF_002 as its public label, and is omitted only from calculations requiring coordinates.

Sample aliases

The metadata sample column remains the immutable identifier used to match the VCF and GDS sample IDs exactly.

An optional alias column may provide a clearer public name for each sample:

sample  alias   population
VH01903_AAGMVNWM5_S12_L001  Plant_01    Ontario
VH01903_AAGMVNWM5_S18_L001  Plant_02    Quebec

Column names are normalized to lowercase internally, so Alias, alias, and equivalent capitalization are accepted.

Alias rules

sample is still mandatory and must match the VCF sample name exactly.
alias is optional.
Blank or missing aliases fall back to the original VCF sample ID.
Non-missing aliases must be unique.
The final public names must also be globally unique. For example, an alias cannot equal another sample’s unaliased VCF ID.

Output behavior

The public name is used in sample-level QC tables, PCA coordinates and labels, IBS/MDS matrices, neighbour-joining trees, diversity tables, DAPC coordinates and membership matrices, figures, and downstream report artifacts.

The original VCF identity is retained as vcf_sample in identity-sensitive tables and provenance. The VCF/GDS itself is never renamed, so exact genotype-to-metadata linkage remains auditable.

Geographic coordinates

Coordinates are interpreted as geographic latitude and longitude in signed decimal degrees. Pairwise distances are calculated with the Haversine formula and reported in kilometres.

Use the following conventions:

latitude is positive north and negative south, from -90 to 90;
longitude is positive east and negative west, from -180 to 180;
WGS84/EPSG:4326 coordinates are recommended;
use a period as the decimal separator, with no degree symbol;
do not use degrees-minutes-seconds, N/S/E/W suffixes, UTM, or projected easting/northing values.

Examples:

Location	Latitude	Longitude
Ottawa, Canada	`45.4215`	`-75.6972`
Sydney, Australia	`-33.8688`	`151.2093`

Do not use 0 as a missing marker: zero is a valid coordinate. Values that cannot be converted to numbers become missing; out-of-range values are not a substitute for a missing marker and must be corrected before analysis.

A sample is spatially usable only when both coordinates are finite. Spatial analysis excludes incomplete coordinate rows and requires at least four retained samples with complete pairs. It also requires complete population annotations because spatial modules are population-dependent in this workflow.

Missing metadata values

Use the literal NA or an empty tab/CSV field for an unavailable optional value. Keep the row and its sample key. Explicit NA is clearer and is the recommended form for reproducible files.

Do not use ., N/A, unknown, -999, or 0 as generic missing markers; they may be interpreted as real labels or coordinates.

Situation	Behavior
Missing `sample`	Invalid; every VCF sample must have one metadata row
Missing `alias`	Public label falls back to the immutable VCF sample ID
Missing `population`	Population-dependent and spatial modules are unavailable for the run
One or both coordinates missing	That sample is excluded only from spatial calculations
Missing optional descriptive/grouping field	Preserved as missing; unrelated modules can still run
Entire metadata row omitted	Invalid because VCF and metadata sample sets must match exactly

Population annotations are deliberately all-or-nothing for population-level analysis: one missing value makes the population capability unavailable. If population is not known for every sample, either complete the annotation or run in sample-metadata/VCF-only mode rather than inventing a population label. Population-comparison modules such as FST, AMOVA, and chromosome-specific FST also require at least two distinct populations among the samples retained after genotype QC. If only one population remains, those modules are skipped during preflight and analysis_capabilities.tsv records the population count and the reason.

Metadata missingness is separate from missing genotypes in the VCF. Genotype calls such as ./. contribute to sample and variant missingness and are controlled by qc.max_sample_missing and qc.max_variant_missing; those QC thresholds do not fill or remove missing metadata annotations.

Preflight checklist

Before starting a long run, confirm that:

the file has a header and consistent tab delimiters;
sample values match the VCF header exactly and case-sensitively;
every VCF sample occurs once, with no extra or duplicate metadata rows;
aliases are unique and do not collide with another public sample name;
population labels are complete and consistently spelled when population analyses are intended;
coordinates use signed decimal degrees in latitude-longitude order;
missing optional values use NA, not sentinel numbers or ad hoc strings;
coordinate and sample exclusions are scientifically documented.

After startup, inspect 02_sample_metadata_match.tsv, 01_sample_QC.tsv, and analysis_capabilities.tsv before interpreting results. These files show the identity mapping, genotype-missingness exclusions, and modules enabled by the retained metadata.