Records · Preservation · Access AboutHow we workContact
The Working Archivist

Preservation

Formats that survive, and how fixity proves they did

Preservation has two quiet enemies: the format nobody can open anymore, and the file that rots a few bits at a time while looking perfectly healthy. Here is how to defend against each, using tools you already have.

Preservation has two quiet enemies. The first is the format nobody can open anymore. The second is the file that looks fine in a listing but has rotted a few bits at a time on the drive. Both failures are invisible until the day you actually need the record, which is the worst possible day to discover them. This guide covers how to defend against each, using tools most people already have.

Format choice is a preservation decision

When you save a document, you are betting that software will still read that format in ten or twenty years. Some bets are safe and some are reckless, and the difference comes down to a few properties.

A format is a good preservation bet when its specification is published openly, when many independent programs can read and write it, and when it does not depend on a single vendor staying in business or a license staying valid. Plain text is the extreme case: a UTF-8 text file will open in anything, forever, because the encoding is documented and universal. A proprietary format from a discontinued program sits at the other end, where the only thing that reads it is a copy of software you may no longer be able to install.

The practical move is to prefer open, well-documented formats for anything you intend to keep. A rough working set that most preservation programs converge on:

  • Text and documents: plain text (UTF-8), PDF/A for page-faithful documents, and open formats such as ODF where structure matters.
  • Images: TIFF (uncompressed or lossless) for masters, PNG for lossless graphics, and JPEG only when the original already is one.
  • Audio: WAV (PCM) or FLAC for lossless audio.
  • Video: this is the hard case; FFV1 in a Matroska container has become a common lossless preservation choice, with the caveat that video preservation is genuinely difficult and worth dedicated reading.
  • Tabular data: CSV alongside a documented schema, not a spreadsheet binary.

You will not always control the format you receive. That is fine. The rule is about what you migrate to for long-term keeping, not about refusing material that arrives in an awkward shape. When you do convert, keep the original too, because a migration can lose something you did not know was there.

The one-line version For anything you mean to keep, prefer formats that are openly specified and readable by many programs. Keep the original alongside any preservation copy you create.

Fixity: proving a file has not changed

Fixity is the property of a digital object staying bit-for-bit identical over time. You verify it with a checksum, also called a hash: a short string calculated from the full contents of a file. Change a single bit and the hash changes completely. Store the hash when you first take custody of a file, recompute it later, and compare. If the two match, the bytes are intact. If they differ, something changed, and you know to investigate before that copy becomes your only copy.

This matters because storage fails quietly. Drives develop bad sectors, bits flip, a botched copy truncates a file, and none of that announces itself. A file can sit in a folder for years looking perfectly healthy in every directory listing while its contents decay. Fixity checking is how you catch that while you still have a clean copy to restore from.

Which hash to use

For preservation and integrity checking, use SHA-256. It is widely supported, fast enough, and free of the known weaknesses that make older algorithms a poor choice. MD5 and SHA-1 still appear in older systems and are adequate for catching accidental corruption, but both are broken against deliberate tampering, so do not rely on them where authenticity is in question. When in doubt, SHA-256 is the safe default.

A routine you can actually run

You do not need specialized preservation software to start. The command-line tools built into macOS, Linux, and Windows can generate and verify checksums for a whole directory. The pattern is always the same three steps.

First, when you take custody of a set of files, generate a manifest that lists every file and its SHA-256 hash. On macOS or Linux:

find . -type f -exec shasum -a 256 {} \; > manifest-sha256.txt

Second, store that manifest somewhere safe, ideally in more than one place and separate from the files it describes. The manifest is now your record of what "intact" means for this collection.

Third, on a schedule, verify the files against the manifest:

shasum -a 256 -c manifest-sha256.txt

Every file that still matches reports OK. Any file that changed, or went missing, is flagged. On Windows, certutil -hashfile filename SHA256 produces a hash for a single file, and PowerShell's Get-FileHash does the same across many. The BagIt specification, widely used in archives, formalizes exactly this manifest-and-verify pattern, and free tools such as Bagger and the bagit command-line libraries implement it if you want structure around the routine.

Cadence Generate hashes at the moment you take custody. Verify on a regular schedule and after every move or copy. A check you run once and never repeat tells you almost nothing about decay over time.

Fixity is necessary but not sufficient

A checksum tells you whether a file changed. It does nothing to bring back a file that is already gone, and it cannot fix corruption on its own. That is why fixity lives inside a larger habit: keep more than one copy, on more than one kind of storage, in more than one location, and check all of them. The often-cited rule of thumb is three copies, on two different media, with one kept offsite. Fixity is what lets that rule work, because it tells you which copy is still good when the copies start to disagree.

When a check fails, the response is simple: do not panic and do not overwrite. Identify a copy whose hash still matches the manifest, restore from it, verify the restored file, and record what happened. The failure itself is useful information about that drive or that storage location, and it is worth logging so you can spot a pattern before it becomes a pattern of losses.

Where to go deeper

Two references worth knowing

The Library of Congress maintains public Sustainability of Digital Formats resources that assess specific formats for preservation suitability, which is the most useful reference when you have to defend a format decision. And the National Digital Stewardship Alliance publishes the Levels of Digital Preservation, a tiered model that lets you place your current practice and see the next concrete step up. Both are free, and both are written for practitioners rather than researchers.