Bioinformatics File Formats: FASTQ, SAM/BAM, VCF, GTF/GFF & BED Explained

Recognize the files omics data arrives in, know what each one stores, read its columns — and avoid the 0- vs 1-based coordinate trap that silently shifts every result

A practical reference to the core bioinformatics file formats a data scientist gets handed: FASTQ (raw reads), SAM/BAM (alignments), VCF (variants), GTF/GFF (gene annotation), and BED (genomic intervals). For each one, learn what it stores, what its key columns mean, and how to read it — plus the single most important gotcha: BED is 0-based and half-open while GTF/GFF, SAM, and VCF are 1-based, an off-by-one that ruins coordinates if you miss it.

TipKey takeaways

• Real omics data arrives as files with cryptic extensions — .fastq.gz, .bam, .vcf, .gtf, .bed. Each sits at a known stage of the pipeline: FASTQ (raw reads) → SAM/BAM (alignments) → counts; plus VCF (variants), GTF/GFF (gene annotation), and BED (intervals).
• A FASTQ record is 4 lines: ID, sequence, +, and a per-base Phred quality string (one quality character per base — they must be the same length).
• SAM is human-readable tab-delimited alignment text; BAM is the same data binary-compressed — never cat a BAM, view it with samtools view.
• VCF stores variants (8 fixed columns: CHROM POS ID REF ALT QUAL FILTER INFO); GTF/GFF stores gene models (9 columns ending in a free-text attributes field); BED stores plain genomic intervals.
• The trap that bites everyone: BED is 0-based, half-open; GTF/GFF, SAM, and VCF are 1-based. Mix them up and every coordinate is off by one — silently.

Introduction

You have been handed a folder of sequencing data and asked to “just load it.” Inside are files ending in .fastq.gz, .bam, .vcf, .gtf, .bed — and none of them open cleanly in a spreadsheet. Before any analysis, you need to know what each file holds, where it sits in the pipeline, and how to read its columns. That recognition is the whole job of this lesson.

These formats are not arbitrary. They map onto the stages data passes through: the sequencer emits raw reads (FASTQ), an aligner places those reads on the genome (SAM/BAM), and downstream tools turn the alignments into counts of reads per gene, or into variants (VCF). Two more formats describe the genome itself rather than your sample: GTF/GFF says where the genes are, and BED marks arbitrary intervals. Learn to recognize the five and you can sit down in front of any omics project and know what you are looking at.

Where each format sits in the pipeline

The core sequencing workflow is a short chain — reads, then alignments, then either expression counts or variant calls — with the annotation formats feeding in from the side to tell the tools where the genes are:

flowchart TD
  FQ["FASTQ<br/>raw sequencing reads"]
  BAM["SAM / BAM<br/>reads aligned to the genome"]
  COUNTS["Count matrix<br/>reads per gene"]
  VCF["VCF<br/>genetic variants"]
  ANNO["GTF / GFF · BED<br/>gene &amp; interval annotation"]
  FQ -->|align| BAM
  BAM -->|quantify| COUNTS
  BAM -->|call variants| VCF
  ANNO -. "where the genes are" .-> COUNTS
  ANNO -. "regions of interest" .-> VCF

Now take the five formats one at a time.

FASTQ — raw sequencing reads

A FASTQ file is what comes off the sequencer: millions of short reads, each one a string of bases plus a quality score for every base. It is plain text, almost always gzip-compressed (.fastq.gz). Every read is exactly four lines:

Line	Starts with	Content
1	@	Read identifier (instrument, run, coordinates)
2	—	The sequence — the bases A/C/G/T/N
3	+	Separator (optionally repeats the line-1 ID)
4	—	Phred quality — one ASCII character per base

It really is just four lines of text. Read one record with base R and the structure is obvious:

record <- c(
  "@SEQ_ID_001",                  # 1. identifier (starts with @)
  "GATTTGGGGTTCAAAGCAGTATCGATCA", # 2. the read sequence
  "+",                            # 3. separator
  "IIIIIIIIIIIIIII9999IIIG+++!!"  # 4. one quality character per base
)
record

[1] "@SEQ_ID_001"                  "GATTTGGGGTTCAAAGCAGTATCGATCA"
[3] "+"                            "IIIIIIIIIIIIIII9999IIIG+++!!"

# The sequence and the quality string must be the SAME length:
c(seq = nchar(record[2]), qual = nchar(record[4]))

 seq qual 
  28   28 

Twenty-eight bases, twenty-eight quality characters — they line up one-to-one. That equality is the sanity check: if line 2 and line 4 differ in length, the record is corrupt.

Phred quality, decoded

Line 4 looks like gibberish, but each character encodes a Phred quality score = the ASCII code of the character minus 33 (the “Phred+33” encoding). A higher score means the base-caller was more confident. Decode it in one line:

# First six bases vs the worst base in our read:
utf8ToInt(substr("IIIIIIIIIIIIIII9999IIIG+++!!", 1, 6)) - 33   # the 'I' run

[1] 40 40 40 40 40 40

utf8ToInt("!") - 33                                            # the '!' character

[1] 0

I decodes to 40 (a 1-in-10,000 chance of error — excellent); ! decodes to 0 (the lowest possible — the caller had no confidence). So this read starts strong and degrades to junk at the 3′ end, the classic quality drop-off you trim before alignment. The real tools parse FASTQ for you:

# Conceptual — the real parser (not run here):
library(ShortRead)
fq <- readFastq("reads.fastq.gz")
sread(fq)     # the sequences
quality(fq)   # the Phred qualities

SAM/BAM — aligned reads

Once an aligner (BWA, Bowtie2, STAR) has placed each read on the genome, the result is a SAM file: tab-delimited text with one row per aligned read. BAM is the exact same information binary-compressed — smaller and indexable, but not human-readable. You convert between them with samtools; downstream tools almost always want the BAM.

Each alignment row has 11 mandatory columns. The ones you actually read:

Column	Name	Meaning
1	QNAME	Read name (matches the FASTQ ID)
2	FLAG	Bitwise flags (paired? mapped? reverse strand? duplicate?)
3	RNAME	Reference name the read mapped to (e.g. chr1)
4	POS	Leftmost mapping position — 1-based
5	MAPQ	Mapping quality (Phred-scaled confidence in the position)
6	CIGAR	Compact match/insert/delete string (e.g. 28M, 5S23M)
10	SEQ	The read sequence
11	QUAL	Per-base Phred quality (as in FASTQ)

(The skipped columns 7–9 — RNEXT, PNEXT, TLEN — give the mate read’s reference, position, and template length for paired-end reads; you rarely read them by hand.)

POS is 1-based — the first base of a chromosome is position 1, not 0 (remember this; it is half of the coordinate trap below). To read a BAM in R you use Rsamtools, not readLines:

# Conceptual — BAM is binary; use the parser, never cat/readLines:
library(Rsamtools)
bam <- scanBam("aligned.bam")[[1]]
bam$pos    # 1-based leftmost positions
bam$cigar  # the CIGAR strings

VCF — variants

A VCF (Variant Call Format) file lists positions where your sample’s genome differs from the reference — SNPs, insertions, deletions. After a ##-prefixed header block, it is tab-delimited with 8 fixed columns, then per-sample genotype columns:

Column	Name	Meaning
1	CHROM	Chromosome
2	POS	Position of the variant — 1-based
3	ID	Variant ID (e.g. an rsID, or . if none)
4	REF	Reference allele (the base in the genome)
5	ALT	Alternate allele (what your sample has)
6	QUAL	Phred-scaled confidence in the call
7	FILTER	PASS, or why the call was filtered
8	INFO	Semicolon-separated annotations (depth, allele frequency, …)

Two more columns usually follow: FORMAT (what each genotype field means, e.g. GT:DP) and one column per sample holding that sample’s genotype (0/1 = heterozygous, 1/1 = homozygous alternate). Like SAM, POS is 1-based. Read one with VariantAnnotation:

# Conceptual — the real parser (not run here):
library(VariantAnnotation)
vcf <- readVcf("variants.vcf.gz", "hg38")
rowRanges(vcf)   # CHROM/POS as a GRanges
ref(vcf); alt(vcf)

GTF/GFF — gene annotation

GTF (Gene Transfer Format) and GFF (General Feature Format) describe where genes are on the genome — the map an aligner or a counting tool uses to decide which gene a read belongs to. Both are tab-delimited with 9 columns:

Column	Name	Meaning
1	seqname	Chromosome / contig
2	source	Who produced the annotation (e.g. ensembl)
3	feature	gene, transcript, exon, CDS, …
4	start	Feature start — 1-based
5	end	Feature end — inclusive
6	score	Score, or .
7	strand	+ or -
8	frame	Reading frame for coding features, or . (GFF3 calls this column phase)
9	attribute	Free-text key/value pairs (gene_id, gene_name, …)

The whole gene name and ID live in that last attribute column. GTF separates the pairs with key "value";; GFF3 uses key=value;. Split a GTF attribute string with base R:

attr <- 'gene_id "ENSG00000012048"; gene_name "BRCA1"; gene_biotype "protein_coding";'
fields <- strsplit(attr, ";\\s*")[[1]]
fields[fields != ""]

[1] "gene_id \"ENSG00000012048\""     "gene_name \"BRCA1\""            
[3] "gene_biotype \"protein_coding\""

Each pair becomes one element — that is how a parser pulls gene_name "BRCA1" out of the line.

Note

GTF vs GFF — what’s the difference? Same 9 columns; only the attribute syntax in column 9 differs. GTF uses gene_id "X"; transcript_id "Y"; (space, quotes, semicolons). GFF3 uses ID=gene:X;Name=BRCA1; (= signs, no quotes) and adds a Parent= key to nest exons under transcripts under genes. Counting tools (featureCounts, HTSeq) typically expect GTF.

The real loader builds a transcript database rather than parsing text yourself:

# Conceptual — the real parser (not run here):
library(rtracklayer)
anno <- import("annotation.gtf")            # a GRanges of features
library(txdbmaker)
txdb <- makeTxDbFromGFF("annotation.gtf")   # a queryable transcript database

BED — genomic intervals

A BED file is the simplest of all: plain genomic intervals, one per line, used for peaks, regions of interest, primer targets — anything you mark on the genome. Only the first three columns are required:

Column	Name	Required?	Meaning
1	chrom	yes	Chromosome
2	chromStart	yes	Start — 0-based
3	chromEnd	yes	End — exclusive (half-open)
4	name	optional	Feature name
5	score	optional	Score (0–1000)
6	strand	optional	+ or -

Notice columns 2–3: BED uses a 0-based, half-open coordinate system — the opposite convention to every format above. That difference is the single most common silent bug in genomics, so it gets its own section.

# Conceptual — the real parser (not run here):
library(rtracklayer)
peaks <- import("peaks.bed")   # a GRanges; rtracklayer converts BED's
                               # 0-based coordinates to R's 1-based GRanges for you

The coordinate-system trap (read this twice)

Here is the gotcha that ruins more analyses than any parsing error:

BED is 0-based and half-open. GTF/GFF, SAM, and VCF are 1-based and inclusive.

In a 1-based system (GTF/GFF, SAM, VCF), the first base of a chromosome is position 1, and the end coordinate is included in the feature. In a 0-based, half-open system (BED), counting starts at 0, and the end coordinate is not included — it points just past the last base. The upshot: the same physical stretch of DNA is written with different numbers depending on the format.

Take the first five bases of a chromosome. In GTF/GFF it is start = 1, end = 5. In BED it is chromStart = 0, chromEnd = 5. Both describe the identical 5-bp feature — and the length arithmetic is different in each system:

# Same 5-bp feature, written in two coordinate systems:
gtf_len <- 5 - 1 + 1   # GTF/GFF (1-based, inclusive): end - start + 1
bed_len <- 5 - 0       # BED     (0-based, half-open): end - start
c(GTF = gtf_len, BED = bed_len)

GTF BED 
  5   5 

Both give 5 — but only because you used the right formula for each. Subtract BED-style on GTF coordinates (or forget the + 1) and every length and overlap is off by one. The practical rules:

• Converting GTF/VCF/SAM start → BED start: subtract 1. The end stays the same.
• Converting BED start → 1-based start: add 1. The end stays the same.
• In R, let the tools do it: rtracklayer::import() and GenomicRanges use 1-based coordinates internally and convert BED for you on import — so you reason in one system and don’t hand-edit numbers.

Format	Coordinate system	First base is	End coordinate
FASTQ	(no coordinates)	—	—
SAM/BAM (POS)	1-based	1	—
VCF (POS)	1-based	1	inclusive
GTF/GFF (start/end)	1-based	1	inclusive
BED (chromStart/chromEnd)	0-based, half-open	0	exclusive

When in doubt, write a known feature’s coordinates both ways and check the length is what you expect — the two-line calculation above is faster than debugging a shifted result downstream.

How this connects to the rest of the pillar

These files are the raw material; the analysis series turn them into something you can model. The aligned reads in a BAM are quantified against a GTF into the RNA-seq count matrix — genes × samples — which, together with its sample and gene tables, lives in a SummarizedExperiment container. Recognize the formats here, and those lessons are about analysis, not about fighting the file.

Common issues

Off-by-one between BED and everything else. A region looks shifted by one base, or an overlap that should hit comes back empty. The cause is almost always mixing 0-based BED coordinates with 1-based GTF/VCF/SAM coordinates. Convert explicitly (BED start + 1 → 1-based start) or, better, import both with rtracklayer/GenomicRanges and let R hold everything 1-based.

“Why won’t my GTF/BED load?” These are tab-separated, not space-separated — a file edited in a word processor or split on spaces will mangle. They may also be gzip-compressed (.gtf.gz), and GTF/GFF carry #-prefixed header lines plus a free-text column 9 that naïve CSV readers choke on. Read them with a format-aware loader (rtracklayer::import) rather than read.csv.

Trying to read a BAM as text. cat, readLines, or head on a .bam prints binary garbage — BAM is compressed. View it with samtools view aligned.bam (or read it with Rsamtools::scanBam). The text twin, SAM, is readable, but it is large; you normally keep BAM and decompress on demand.

Frequently asked questions

NoteWhat is a FASTQ file?

A FASTQ file holds raw sequencing reads. Each read is four lines: an @ identifier, the DNA sequence, a + separator, and a per-base Phred quality string (one character per base, ASCII code minus 33 = the quality score). It is plain text, usually gzip-compressed as .fastq.gz, and it is what comes straight off the sequencer before alignment.

NoteWhat is the difference between SAM and BAM?

They store the same aligned-read data. SAM is human-readable tab-delimited text; BAM is that text binary-compressed (and indexable). BAM is smaller and is what downstream tools consume; you convert between them with samtools. Never cat a BAM — view it with samtools view.

NoteWhat is a VCF file?

A VCF (Variant Call Format) file lists positions where a sample differs from the reference genome — SNPs and small insertions/deletions. After a ## header it has 8 fixed columns (CHROM POS ID REF ALT QUAL FILTER INFO), then a FORMAT column and one genotype column per sample (0/1 heterozygous, 1/1 homozygous alternate). POS is 1-based.

NoteGTF vs GFF — what’s the difference?

Both describe gene annotation with the same 9 columns; only column 9 (the attributes) differs in syntax. GTF uses gene_id "X"; gene_name "Y"; (quotes, semicolons); GFF3 uses ID=X;Name=Y; (= signs, no quotes) and a Parent= key to nest exons under transcripts under genes. Counting tools such as featureCounts typically expect GTF.

NoteIs BED 0-based or 1-based?

BED is 0-based and half-open: the first base is position 0 and the end coordinate is not included. This is the opposite of GTF/GFF, SAM, and VCF, which are all 1-based and inclusive. So the first five bases of a chromosome are 0–5 in BED but 1–5 in GTF — the same feature, different numbers. Always check which system a file uses before doing coordinate math.

Test your understanding

ImportantYour turn: convert coordinates between BED and GTF

A gene feature in a GTF file spans start = 100, end = 200 (1-based, inclusive). (1) How long is it, in base pairs? (2) What are its chromStart and chromEnd in a BED file (0-based, half-open)? (3) Confirm the BED length matches.

TipHint

For a 1-based inclusive feature the length is end - start + 1. To go to BED, subtract 1 from the start and leave the end unchanged; a BED length is chromEnd - chromStart.

TipSolution

gtf_start <- 100; gtf_end <- 200

# (1) GTF length (inclusive):
gtf_len <- gtf_end - gtf_start + 1     # 101 bp

# (2) Same feature in BED (0-based start, end unchanged):
bed_start <- gtf_start - 1             # 99
bed_end   <- gtf_end                   # 200

# (3) BED length (half-open) must match:
bed_len <- bed_end - bed_start         # 101 bp

c(gtf_len = gtf_len, bed_start = bed_start, bed_end = bed_end, bed_len = bed_len)
#> gtf_len bed_start bed_end bed_len
#>     101        99     200     101

The feature is 101 bp. In BED it becomes chromStart = 99, chromEnd = 200 — the start dropped by one, the end stayed the same, and the length still works out to 101. Get the start - 1 and forget nothing else.

NoteQuick check: which file would you open to see a sample’s genetic variants?

A. A FASTQ file. B. A BAM file. C. A VCF file.

TipShow answer

C. A VCF lists the positions where the sample differs from the reference (SNPs and indels). FASTQ holds raw reads, and BAM holds the aligned reads — variants are called from a BAM and written to a VCF.

Conclusion

Five formats cover almost everything a data scientist gets handed at the start of an omics project. FASTQ is raw reads (four lines each, sequence plus Phred quality); SAM/BAM is those reads aligned to the genome (text vs binary-compressed); VCF is the variants called from them; GTF/GFF is the gene map the tools count against; and BED is plain genomic intervals. Read their key columns, reach for the right parser instead of read.csv, and above all keep the coordinate systems straight — BED is 0-based and half-open, everything else is 1-based. With the files demystified, the next step is turning aligned reads into a count matrix and holding it safely in a SummarizedExperiment.

Related lessons

• The RNA-Seq Count Matrix & QC — turns aligned reads (BAM) plus a GTF into the genes × samples matrix. · The SummarizedExperiment Container — the object that holds that matrix with its sample and gene tables aligned. · Biology for Data Scientists — what a read and a count actually mean. · Bioinformatics — the pillar.
• Where this fits: file formats → count matrix & QC → SummarizedExperiment → differential expression — recognize the files, then analyze what’s inside them.

Was this page helpful?

Prove you can do it. Master the whole Bioinformatics Foundations in R series — track your path, build projects, and earn a certificate.

Start free →

Go Pro — unlimited Prova on your own data and a verifiable certificate that proves the skill.

from $15/mo billed yearly

Go Pro →

✓ You're Pro — keep going. The runtime is the judge.

Get new R & Python lessons by email

Practical, reproducible, no spam. Unsubscribe anytime.

Double opt-in. We never share your email.

Share this pageXLinkedInRedditHN

Note

This lesson is reproducible: every number on this page was produced by the code shown — copy any block and run it to reproduce them. The runtime is the judge.