Deduplication & integrity

With deduplicate enabled, identical bytes on the same (disk, visibility) are stored once and shared by reference, as long as the shared file is still on the disk — and attach() and a same-disk copy() share an original the same way. A reference count on the stored file keeps sharing safe: a physical original is removed — by a delete, a cross-disk move or a replace — only when no other row, soft-deleted rows included, still points at the same file (disk + path), so a soft delete followed by a restore stays lossless even while the file is shared:

// Identical bytes on the same (disk, visibility) are stored once.
$a = MediaLibrary::for($userA)->add($path)->toBucket('docs');   // the source path is only read
$b = MediaLibrary::for($userB)->add($path)->toBucket('docs'); // same bytes -> shared storage

// Deleting $a keeps the file alive while $b still references it.
MediaLibrary::delete($a); // the original stays until the last reference is gone

$b->verifyIntegrity();    // bool — false on drift or a missing file

A content checksum (sha256 by default) backs dedup and optional read-time integrity verification — set verify_checksum_on_read to re-hash the original on stream/download and throw ChecksumMismatch on drift. Each file is verified with the algorithm its checksum was taken with, so changing media.checksum_algorithm never reports older media as drifted. Variants are never shared: each media generates its own. A still-shared original is never overwritten either — replacing one sharer writes its new bytes to a path of its own — and directories are only tidied away once nothing another row points at is left in them.

Dedup scope — who may share a file

With the default scope, global, identical bytes on one disk with one visibility are stored once for every owner. That is the cheapest layout, and the wrong one for a multi-tenant application: tenant B’s upload of a logo tenant A uploaded first becomes a row pointing at tenant A’s file — under tenant A’s path when your path generator puts the tenant first. Inside the package that is safe, but anything that deletes a tenant’s files or path prefix outside the package — an offboarding script, a bucket lifecycle rule — removes a file another tenant still uses, and tenant A’s usage pays for tenant B’s file. Narrow it with deduplicate_scope:

// config/media.php
'deduplicate' => true,
'deduplicate_scope' => 'owner',   // or 'path_prefix', or your own DeduplicationScope class

Or name your own DeduplicationScope, resolved from the container:

use Illuminate\Database\Eloquent\Builder;
use RoundlyConsulting\MediaLibrary\Contracts\DeduplicationScope;
use RoundlyConsulting\MediaLibrary\Models\Media;

// config/media.php: 'deduplicate_scope' => App\Media\TenantDeduplicationScope::class,
final class TenantDeduplicationScope implements DeduplicationScope
{
    /** Narrow the rows with identical bytes; null = this media stores its own copy. */
    public function constrain(Builder $candidates, Media $media): ?Builder
    {
        $tenant = $media->custom_properties['tenant_id'] ?? null;

        return $tenant === null ? null : $candidates->where('custom_properties->tenant_id', $tenant);
    }
}

• $candidates are the non-deleted rows on the same disk with the same visibility and checksum; $media is the row being stored (unsaved on an add) with its owner, bucket, draft token and custom properties set. The scope applies to adds and replaces — attach() and a same-disk copy() share the file you hand them on purpose, whatever the scope.
• Re-checked under the lock: under any scope other than global the scope runs again on the chosen row while it is locked, so a row moved to another owner or path in between is not shared — the new media stores its own copy.
• Concurrent uploads of the same bytes in one scope, before either has committed, cannot see each other: each stores its own copy. Nothing is lost or shared across scopes.
• Changing the scope affects new adds and replaces only; rows already sharing a file keep sharing it.
• php artisan about shows the scope next to dedup. The next major release changes the default to owner — set 'deduplicate_scope' => 'global' explicitly to keep today’s behaviour after upgrading.

Measuring stored bytes

Every row knows its own bytes — the original’s size plus the size each variant recorded when it was written ($media->storedSize()). Summing that over rows over-counts once originals are shared, so MediaLibrary::usage() counts what the disks physically hold:

use RoundlyConsulting\MediaLibrary\Facades\MediaLibrary;
use RoundlyConsulting\MediaLibrary\Models\Media;

$usage = MediaLibrary::usage();                     // every row, soft-deleted included

$usage->originalBytes;            // each stored original once, per distinct (disk, path)
$usage->variantBytes;             // every recorded variant size (variants are per-row files)
$usage->totalBytes();             // originalBytes + variantBytes — what the disks hold
$usage->referencedOriginalBytes;  // originals counted once per row (the sum of size)
$usage->savedBytes();             // referencedOriginalBytes - originalBytes — what sharing saves

// Scope it with any media query: an owner, a bucket, a column of your own.
MediaLibrary::usage(Media::query()->forModel($team)->withTrashed());
MediaLibrary::usage(Media::query()->inBucket('invoices'));

• The identity is (disk, path), the same one the shared-original reference count uses, so each stored file is counted once.
• Scoped per query: a file two scopes share counts once in each scope’s usage and once in the unscoped total. With a dedup scope that matches how you measure (owner and Media::query()->forModel($team)), no file counts in two owners’ usage, so the per-owner figures add up to the total.
• Soft-deleted rows keep their files until they are force-deleted, so the default query includes them; pass a query without withTrashed() to leave them out.
• A variant recorded before sizes were counts 0 until media:backfill-variant-sizes measures it. MeasureStorageUsageAction is the same measurement for dependency injection; under MediaLibrary::fake() it still reads the stored rows.

Concurrent writes and rollbacks

A row that starts sharing a file is invisible to other connections until it commits, so the reference count alone could miss it. The package closes that gap with row locks and after-commit deletes:

• Every way a row starts sharing an original — a deduplicated add or replace, attach(), a same-disk copy() — saves it under a row lock (SELECT … FOR UPDATE, inside a transaction of its own — a savepoint in yours) on a row that already stores the file. A concurrent delete or move of that row waits for the new referrer to commit, then sees it. Locking one referrer is enough: while it still stores the file, no other row’s check removes it.
• Should the row being shared be gone — or hold other bytes — once the lock is held, a deduplicating add or replace stores its own copy instead, while attach() and copy() throw ModelNotFoundException.
• A replace locks its own row before it decides whether to overwrite its original in place, so it never overwrites bytes a new referrer is about to share.
• A force delete removes its files once the outermost transaction commits (at once when none is open): a rollback keeps them, and the reference count then sees a row another connection made share the original meanwhile. A move, a variant move, a draft bind and a replace release their old files the same way.
• On a real database engine two such flows can deadlock, each holding a row the other wants. The database aborts one with a deadlock error, which the caller can retry.