HedgeGreat workflows start with great apps

Redoing tape archival without losing time

by Alex
Redoing tape archival without losing time

Halfway through an archive, the drive pings its cleaning light. Or the source volume drops off the desk. Or macOS decides that 2am is a fine moment to install something.

The copy stops, and because the copy stops, verification never runs. The files that made it to tape are almost certainly fine. "Almost certainly" is doing a lot of work in that sentence, and it isn't the standard anyone archives to. Canister 26.3 for Mac closes that gap with Verify Duplicates.

The Chain

Tape does one thing at a time. Unlike a hard drive or an SSD, it can't read and write at once, so the write is sequential and verification has to be a separate pass: Canister reads your data back off the tape and checks it against the source. If the transfer never gets to the verification stage, you have files on tape but no evidence about any of them.

They are most likely not corrupt. You just can't prove it.

Second Hand News

Until now, the answer was to redo the transfer, and rely on Duplicate Detection doing its job: anything matching in name, size, and modification date is already on tape, so it isn't written twice. Efficient, yet also a problem. Canister marks those files as transferred and moves on, so they never get read back, never get checksummed, and never reach the Transfer Log.

That log is the paper trail. In DIT and post workflows it travels with the job to prove the data landed, and chain of custody lives or dies by it. A file sitting safely on tape with no entry in the log is, to anyone auditing the job, not archived at all.

Which leaves one grim option: format a perfectly good tape and write the whole thing again, purely to produce a document. Nobody enjoys wiping a tape to get checksums. It is, by a wide margin, the thing you have asked us to fix most often.

Don't Stop

In Canister 26.3, simply turn on Verify Duplicates in its Settings, then point Canister at the same source and the same tape. When Duplicate Detection finds an item that is already there, Canister doesn't wave it through: it reads the item back off the tape and checksums it against the source, with the same xxHash verification every other transfer gets. Canister writes and verifies anything not yet on tape, as usual.

One transfer, one log, and a complete set of checksums covering every file, whether it arrived on the first attempt or the second. Paperwork intact, tape untouched.

Little Lies

If a checksum doesn't line up, you hear about it. The transfer completes with warnings, and the Transfer Log gains a Duplicate Items Verified section listing what was checked and what disagreed.

A tape that quietly differs from your source is the worst kind of archive: one you trust. Better to know on the day, while the source is still online and the fix is cheap.

Go Your Own Way

Verify Duplicates ships switched off, deliberately. Reading data back from tape costs time, and most transfers are fresh material with nothing to recheck. Switch it on when it will earn its keep: after an interrupted archive, after a drive misbehaves, or any time you need a fresh set of checksums for a tape you already have. If every item turns out to be a duplicate, Canister reverifies the lot rather than telling you there's nothing to do. Pro tip: hook up Canister to Connect, and get a push notification on your phone when a transfer runs into a snag.

Everywhere

Canister for Windows has had Verified Duplicates since 25.2, so whichever platform your workflow relies on, your tapes and your paperwork behave the same. Canister for Mac 26.3 is available now. Update in-app, or download a fresh copy from the Releases page.

Redoing tape archival without losing time