Suprnova's storage facade gives you a single, named-disk API over local
filesystems, in-memory backends, and the major object stores (S3, Azure Blob,
Google Cloud Storage). Under the hood it is built on
opendal - but the consumer surface is shaped to
match Laravel's Storage::disk(...) calls, so PHP muscle memory translates
straight across.
use ;
# async
Registering disks
Every disk is registered once at boot via Storage::register_* and looked up
by name through Storage::disk(name). There is no "default backend" the
others degrade into - each driver is a peer.
| Constructor | Backend | Feature |
|---|---|---|
Storage::register_fs(name, root) |
Local filesystem | filesystem |
Storage::register_memory(name) |
In-process memory (tests) | filesystem |
Storage::register_s3(name, cfg) |
Amazon S3 or S3-compatible (e.g. RustFS, MinIO, R2, B2) | filesystem |
Storage::register_azblob(name, cfg) |
Azure Blob Storage | filesystem-azure |
Storage::register_gcs(name, cfg) |
Google Cloud Storage | filesystem-gcs |
Storage::register_read_through(name, cfg) |
Read-through composite | filesystem |
filesystem is on by default; the Azure and GCS features are not. Turn one
on in your Cargo.toml:
[dependencies]
suprnova = { git = "https://github.com/eas4ai/suprnova.git", tag = "v4.0.3", features = ["filesystem-gcs"] }
Without the feature, register_azblob / register_gcs and their config
structs do not exist - you get a compile error naming the missing item, not
a runtime failure.
Every constructor has a _with variant that hands you the suprnova::opendal::Operator
just before it lands in the registry so you can install retry/timeout/logging
layers around it:
use Duration;
use ;
use Storage;
register_fs_with?;
The cloud constructors (register_s3, register_azblob, register_gcs)
apply a RetryLayer (3 attempts) by default since transient throttling /
5xx errors are routine on object stores. Use the _with variants when you
need full control.
The full set of opendal layers wired in by Suprnova is RetryLayer,
TimeoutLayer, LoggingLayer, TracingLayer (bridges to OTel via
tracing-opentelemetry when the framework's otel feature is on), and
PrometheusClientLayer (exports histograms and counters into a
prometheus_client::registry::Registry you own). Layer order matters -
the outermost layer wraps everything inside it - and the idiomatic stack
is RetryLayer → TimeoutLayer → LoggingLayer so a timed-out attempt
still logs and a retry covers transport failures.
Re-registering the same name replaces the previous operator and emits a
warn! log - disks are meant to be registered once at boot, and an
accidental duplicate could swap a production disk for a memory one. The
replacement still happens; the warning just makes the swap audible.
Why Suprnova diverges
Laravel's config/filesystems.php lists every disk driver and you pick one
at runtime; nothing is compiled out. Suprnova gates Azure and GCS behind
features because in Rust the choice has a dependency cost, and this one has
a security dimension: both opendal service crates pull rsa, which carries
RUSTSEC-2023-0071 (the
Marvin timing attack) with no fixed release upstream. Making them opt-in
means an app that stores files locally or on S3 never carries that crate.
S3 is deliberately not gated - its signer never depended on rsa, so
gating it would break the most-used cloud backend and remove nothing.
Atomic local writes
On a local disk, every operation that publishes bytes at a path publishes them
in one step. disk.write(...), disk.writer(...), and disk.copy(...) all
land in <root>/.suprnova-atomic/ first, are flushed and synced there, and are
then renamed onto the target; disk.rename(...) is already a single step. A
concurrent reader therefore sees either the previous object or the finished new
one and never a partial length, and a process that dies mid-write leaves the
target untouched rather than truncated at the live path.
append is the one in-place operation, because staging an append would mean
copying the whole object first. That holds for the append that creates the
object as much as for every append after it, so two writers appending to the
same new object both land. Being in place is also what an append costs you: one
that fails or is aborted leaves the object behind, empty or short, exactly as an
append onto an existing object always has.
A conditional write is published with link(2) rather than a rename, which
keeps it a real exclusive create rather than a check followed by an overwrite:
// Exactly one of any number of racing callers gets Ok here. Every other one
// gets an `ErrorKind::ConditionNotMatch` error and writes nothing.
disk.write_with.if_not_exists.await?;
That publish needs a filesystem with hard links. On FAT, exFAT, and some network
filesystems link(2) is unsupported, and a conditional write fails there rather
than silently degrading into a check followed by an overwrite - which would hand
you an exclusivity guarantee that does not hold. Every other operation is
unaffected.
Publishing by rename replaces the object's inode. A rewrite therefore does not preserve the previous file's mode, owner, or hard links, and a reader holding an open descriptor keeps reading the old content instead of seeing the new bytes. That is the usual trade for atomic publishing, but it is a change if you were relying on either.
A path that reaches the disk through a symlink the guard cannot resolve - a dangling one, whose target does not exist - is refused rather than treated as a free name to create. Creating through such a link would create the link's target, anywhere on the host, so the guard cannot tell a harmless dangling link apart from an escape and refuses both.
The .suprnova-atomic name is reserved at the root of every local disk. Any
path whose first component is that name is refused with a permission error, and
so is any path that resolves into the directory through a symlink, so you
cannot read another writer's staging file, write into the directory, or delete
it. The entry is filtered out of files, directories, all_files, and
all_directories, so it never shows up as an object. The name is exported as
suprnova::ATOMIC_STAGING_DIR because backup and sync tooling needs it:
exclude the directory the way you would exclude a lock directory. It holds
in-flight temp files plus whatever a process that died mid-publish left behind,
and nothing sweeps those, so a host in a crash loop will grow it until someone
empties it - which is safe to do while nothing is writing.
Path-traversal guard
Local filesystem disks have a PathGuardLayer applied before any user-supplied
layers. A request like disk.write("../escaped.txt", ..) is rejected before
it reaches the OS - no .. component or absolute prefix can escape the disk
root. Object stores and the in-memory backend do not get the guard (a key
like ../foo is just an ordinary key character on those backends).
After rejecting .. and absolute components, the guard canonicalizes the
local disk root and the requested on-disk target. Existing targets resolve
every symlink component; for a path that does not exist yet, the guard walks
up to and canonicalizes the nearest existing ancestor. The operation is
rejected if that resolved path lies outside the canonical root, so an in-root
symlink observed during validation cannot redirect a read, write, list, copy,
or rename outside the disk.
A check followed by a syscall by pathname is a race. Someone who can rename
entries inside the root could swap a checked directory for a symlink after the
check and redirect the syscall. On Unix the local disk does not act by
pathname. Every operation resolves its path one directory at a time, holds each
directory open, and names the next component relative to it, opening each
directory and the final entry with O_NOFOLLOW. Symlinks inside the root are
still followed, but by that walk: a link whose target leaves the root is
refused, and a symlink swapped in after the walk looked at a component fails
the operation. A write is published with renameat or linkat relative to its
held parent directory, so a directory swapped mid-upload cannot carry the
upload out of the root. On other platforms the guard is a check followed by the
operation; use OS-level isolation or a dedicated filesystem there when other
principals can change the storage tree concurrently.
Streaming writers, listers, and copiers perform the resolved-path check once, immediately before their first backend I/O, so each chunk or item does not block on filesystem canonicalization. On Unix the directory-relative walk still runs at publish time, which is what keeps a long upload confined. Copier and writer aborts always forward cleanup to their backends, even before activation or when validation can no longer complete.
The Laravel-shape disk surface
Storage::disk(name) returns a suprnova::opendal::Operator directly so you
can use its full streaming surface (writer, reader, presign_read, list,
stat, ...). On top of that, the DiskExt trait - blanket-implemented on
Operator and re-exported as suprnova::DiskExt - adds every Laravel
convenience method you'd reach for through Storage::disk('local')->....
Bring it into scope with use suprnova::DiskExt;.
Existence checks
disk.exists.await?; // raw opendal
disk.missing.await?; // negation
disk.file_exists.await?; // file only (not a directory)
disk.file_missing.await?;
disk.directory_exists.await?;
disk.directory_missing.await?;
Reading and writing
| Laravel name | Rust-native equivalent | Note |
|---|---|---|
get(path) |
read(path) |
get returns Vec<u8>; read returns opendal's Buffer. |
put(path, contents) |
write(path, contents) |
Both accept any Into<Bytes>. |
json::<T>(path) |
- | Reads + deserializes via serde_json. |
put_json(path, &value) |
- | Pretty-prints via serde_json. |
prepend(path, data) |
- | Joins with \n. Use prepend_with_separator for a custom join. |
append(path, data) |
- | Joins with \n. Use append_with_separator for a custom join. |
prepend and append create the file if it does not yet exist, so they are
safe as the first write to a log file.
Metadata
let bytes = disk.size.await?; // u64
let when = disk.last_modified.await?; // Option<DateTime<Utc>>
let mime = disk.mime_type.await?; // Option<String>
let digest = disk.checksum.await?;
mime_type first asks the backend - S3, Azure, and GCS pass the stored
Content-Type through. If the backend does not have one, it sniffs the first
16 KiB via the infer crate. Ok(None) is reserved for unrecognised binary
blobs.
checksum supports Md5, Sha1, and Sha256 via ChecksumAlgorithm.
MD5 and SHA-1 are included for parity with Laravel and object-store ETags;
choose SHA-256 for any new integrity check.
Listing
let files = disk.files.await?; // top-level files
let all = disk.all_files.await?; // recursive
let dirs = disk.directories.await?;
let all = disk.all_directories.await?;
All four return sorted Vec<String> so callers can rely on stable ordering
across backends. Directories are filtered out of files, and vice versa.
Directory paths are returned without a trailing slash ("docs/sub") to
match Laravel's Storage::directories() output - opendal's underlying
list reports "docs/sub/" but we strip the slash for parity.
Mutating directories and files
| Laravel name | opendal native |
|---|---|
make_directory(path) |
create_dir(path) |
ensure_directory_exists(path) |
create_dir(path) unless the directory exists |
delete_directory(p) |
delete_with(p).recursive(true) |
move_to(from, to) |
rename(from, to) |
make_directory and ensure_directory_exists take a directory path with or
without its trailing slash, as Laravel does. opendal's create_dir accepts a
directory only when the path ends in /, so the two add the slash for you. An
empty path, the root, reaches create_dir unchanged. ensure_directory_exists
checks for the directory first and succeeds when it is already there, so you
can call it before every write into the directory:
disk.ensure_directory_exists.await?;
disk.put.await?;
move_to falls back to copy + delete if the backend doesn't support
rename, and to read + write + delete if it doesn't support copy either -
so it works against the in-memory driver used in tests as well as against
production backends.
Pre-signed URLs
let read_url = disk.temporary_url.await?;
let upload = disk.temporary_upload_url.await?;
// upload.url, upload.method ("PUT"), upload.headers
temporary_url returns the URL as a String. temporary_upload_url returns
a TemporaryUploadUrl with the URL, the HTTP method, and the headers the
signature covers. Send the upload with that method and every one of those
headers: some backends refuse an upload without them. Azure Blob requires
x-ms-blob-type, and an S3 bucket set up for server-side encryption requires
its encryption headers. The value serializes to
{ "url": .., "method": .., "headers": { .. } }, so a handler can return it to
the browser as JSON, the shape Laravel's temporaryUploadUrl returns.
Both are backed by Operator::presign_read and presign_write, so they error
with an Unsupported message on backends that do not implement presigning
(the in-memory and local-filesystem drivers fall in this bucket; S3, Azure
Blob, and GCS support it).
Upload URLs from your own callback
A disk that does not presign can still hand out upload URLs. Give it a
callback with Storage::build_temporary_upload_urls_using, for example one
that points at a route of your application that takes the upload. The
callback receives the path and the lifetime and returns the
TemporaryUploadUrl:
use BTreeMap;
use Duration;
use ;
# async
The callback comes first on every disk: a read-through disk and a disk that
presigns by itself, such as S3, answer from the callback too. Setting a
callback again replaces the one before, and the disk keeps its public URL.
Take the disk with Storage::disk after you set the callback: a handle you
took before keeps answering as it did. Registering the disk again drops the
callback.
Storage::provides_temporary_upload_urls(disk) tells you whether
temporary_upload_url can answer on a disk: it is true when the disk has a
callback or its backend presigns writes. A read-through disk answers for its
primary.
Why Suprnova diverges
Laravel's local and read-through disks ask the callback first, while its S3 disk keeps its own presigning and ignores the callback. Suprnova applies one rule to every disk, so the callback you set is the one that answers.
Public URLs
A pre-signed URL expires. For a file that is meant to be public, give the
disk a public base URL and ask Storage::url for the link.
use Storage;
register_fs?;
set_public_url?;
let url = url?;
assert_eq!;
Storage::set_public_url(disk, base_url) takes the name of a registered
disk and the place where the root of that disk is served from. The base URL
is an absolute URL or a path of your own host, such as /storage. A path
of your own host gets the public root in front, so behind a proxy that
serves the application under /billing the link is
/billing/storage/avatars/7.png; an absolute base is left as it is.
Storage::url(disk, path) joins the base URL and the path and returns a
String. It does not ask the disk, so it returns a URL for a file that
does not exist.
Storage::url takes the name of the disk, not the Operator that
Storage::disk returns, because the operator carries no name.
Storage::url writes each path segment so that it stays one segment. Every
character that a URL gives a meaning to becomes %XX, so c++ notes.pdf
is written c%2B%2B%20notes.pdf.
Both functions return an error in these cases:
set_public_urlis called for a disk that is not registered.- The base URL has a scheme other than
httporhttps, no host, a user or a password, a query or a fragment, a backslash, a.or..segment, or a space or control character. A path base must begin with one/, not two. The error does not repeat the URL, because a URL can carry a token. urlis called for a disk with no public base URL. A private disk hands out no link that can be guessed. Usetemporary_urlfor those files.- The path is empty or has a
.or..segment.
Storage::url doesn't serve the files. A local disk needs a route, such as
one that returns Storage::response, or a web server in front of its
directory. A bucket needs to be public or behind a CDN.
Serve a file from a disk
Storage::download(disk, path, name) and Storage::response(disk, path, name) return a file on a disk as an HTTP response. They match Laravel's
Storage::disk($disk)->download($path, $name) and
Storage::disk($disk)->response($path, $name).
use ;
// The browser saves the file.
pub async
// The browser shows the file.
pub async
Both read the file through the disk, so they work on every driver. A local
disk keeps its path-traversal guard: a path such as
../secret is refused before anything is read. That makes these the forms
to use when a request names the file. S3, Azure Blob, GCS, and in-memory
disks have no local path, and they're read the same way.
The response follows the same rules as HttpResponse::download and
HttpResponse::file:
downloadsendsContent-Disposition: attachmentandresponsesendsinline. The filename isname, or the last segment ofpathwhennameisNone. Non-ASCII names are written per RFC 6266, as described in Filenames andContent-Disposition.- The
Content-Typecomes from the extension ofpath, or isapplication/octet-streamwhen the extension is unknown. A content type stored with the object is not used, because whoever uploaded the object chose it. - A file of 1 MiB or less is read whole. A larger file is streamed from the
disk in 64 KiB chunks. Both carry a
Content-Length.
You receive Last-Modified when the disk reports a modification time, and
Accept-Ranges: bytes. Your GET range requests receive 206 with the
selected bytes and Content-Range, or 416 with Content-Range: bytes */<size> when the range is unsatisfiable. You read only the selected range
through the disk, so its path guard still applies.
You use Storage::download_with(disk, path, name, headers, disposition) or
Storage::response_with(disk, path, name, headers, disposition) to pass
extra headers and select ContentDisposition::Inline or
ContentDisposition::Attachment. Your Content-Type and Cache-Control
are kept. You cannot override the body's length or range metadata.
The errors map to these status codes:
| Case | Status |
|---|---|
Nothing at path, or path is a directory |
404 |
The disk refuses the path, such as .. on a local disk |
403 |
| The disk isn't registered, or the read fails | 500 |
The 404 and 403 messages don't name the path.
Cross-disk streaming copy
copy_between_disks(src, src_path, dest, dest_path) streams the source
object into the destination in 64 KiB chunks, regardless of the backend
pair. Source and destination can be backed by any opendal driver - local
filesystem to S3, S3 to Azure Blob, in-memory to GCS, and so on.
use copy_between_disks;
register_fs?;
register_memory;
let bytes = copy_between_disks.await?;
You also call Storage::copy_to_disk(source, source_path, destination, destination_path).await? or Storage::move_to_disk with those same arguments.
You pass a string name or an owned or borrowed Operator for either disk,
through DiskReference. You receive the transferred byte count as u64.
You keep the source after a copy. You delete it only after a move finishes
copying and closing the destination writer. You receive an error for a missing
disk, a failed read or write, or a failed source deletion. You keep both copies
when deletion fails. You cannot copy or move a file onto itself on the same
operator.
let archive = disk?;
copy_to_disk.await?;
move_to_disk.await?;
You abort the destination writer before a copy error propagates. The backend cleans up its staged writes. On a backend that writes in place, you can still see partial bytes; abort cannot restore the previous contents.
Read-through disks
A read-through disk pairs a fast primary with a slower fallback and moves objects from the second to the first as they are read. Point the primary at the store you are migrating to and the fallback at the one you are migrating from, and the working set crosses over under real traffic - no maintenance window, no bulk copy of objects nobody asks for.
use ;
register_s3?;
register_s3?;
register_read_through?;
let assets = disk?;
// Reads `logo.png` from `legacy-store` and writes it to `new-store` on the way
// out. Every later read is served by `new-store`.
let bytes = assets.read.await?;
Storage::disk("assets") returns an ordinary Operator, so every method on
it and every DiskExt convenience works unchanged.
Which disk answers which operation
| Operation | Disk |
|---|---|
read |
Primary if it holds the object, otherwise the fallback - and, unless copy is false, the fallback hit is promoted |
exists, size, last_modified, mime_type, stat |
Primary if it holds the object, otherwise the fallback |
write, make_directory |
Primary only |
files, directories, list |
Primary only - fallback entries are invisible to a listing |
delete |
Both, fallback first |
copy, rename / move_to |
Primary if it holds the source, otherwise streamed across from the fallback; a rename also deletes the fallback source |
temporary_url |
Primary if it holds the object, otherwise the fallback |
temporary_upload_url |
Primary only - an upload has to land where writes land; a callback from Storage::build_temporary_upload_urls_using answers ahead of the primary |
Listing is primary-only by design. A union listing would have to reconcile
paging and ordering across two backends, and it would report objects that a
later listing no longer returns once they are promoted. Use
Storage::disk("legacy-store") directly when you need to enumerate what is
left on the fallback.
Delete removes the object from both disks. If it only removed the primary copy, the next read would promote the fallback copy straight back. The consequence is that a read-through disk over a read-only fallback cannot delete: the fallback delete fails and the error reaches you.
When a promotion fails
By default a promotion failure is logged at warn and swallowed. You still
receive the bytes you asked for; the disk simply degrades to reading the
fallback every time until the primary is writable again. Set
throw_on_promotion_failure: true when a silent loss of promotion would hide
a fault you need to see - a migration you are trying to finish, for instance:
register_read_through?;
Registration rejects a configuration that cannot work: an empty primary or
fallback, a pair that names the same disk twice, a disk that names itself,
or a name that is not registered. Each returns a FrameworkError naming the
problem, and no disk is registered.
Reading without promoting
Set copy: false to serve fallback hits without writing them through:
register_read_through?;
The disk then reads like a transparent overlay: the primary answers what it holds, the fallback answers everything else, and nothing moves between them. Use it when the primary is a small cache you do not want a one-off read to fill, or when the fallback is authoritative and the primary only ever holds objects you put there deliberately.
The flag governs read-time promotion and nothing else. Writes, deletes,
metadata, listings, and the destinations of copy and rename all behave
exactly as they do with promotion on - so a copy: false disk still lands a
copied or moved object on the primary. Because nothing is written back, a
read with copy: false fetches only the range you asked for rather than the
whole object.
Copying and moving across the fallback
copy and rename resolve the source against the primary first. When only
the fallback holds it, the object is streamed across in 64 KiB chunks and the
destination lands on the primary:
let assets = disk?;
// `logo.png` lives only on `legacy-store`. The copy streams it across and
// writes `branding/logo.png` to `new-store`; the legacy object stays put.
assets.copy.await?;
// A move does the same and then deletes the legacy source.
assets.rename.await?;
A move deletes the fallback source on both paths - whether the primary held the source or not. Without that, the next read would promote the fallback copy back and undo the move.
The two paths differ in when they delete it, and the difference is what a failed move leaves behind:
- The primary held the source. The fallback copy goes first, before the rename. While the primary holds the path, the fallback's copy is unreachable through this disk, so removing it first changes nothing you can observe - and if the delete fails, nothing has moved yet. Retry the move. If instead the delete succeeded and the rename then failed, the fallback holds nothing for that path, the destination is unwritten, and the primary still holds the source - so a retry takes this same path and renames again. The failure costs the cold copy and nothing else.
- Only the fallback held it. The delete can only come after the destination is in place, so a move that fails on the delete leaves the destination written and the source still on the fallback. Retry the move; the source is now on the primary, so the retry takes the first path.
Either way a failed move is safe to retry, and the destination you end up with is the object the move started from.
Conditions travel with the operation on the streaming path too. if_not_exists
becomes a conditional write, so a guarded copy or move still refuses an
existing destination rather than clobbering it, and a copy that names a source
version gets that version out of the fallback. A copy's if_match is the one
exception: it is a condition the backend applies inside its own copy, which is
the call this path cannot make, so it is refused with an Unsupported error
naming the condition rather than quietly ignored.
That makes conditions the one place where which disk holds the source shows
through. A local directory advertises copy and rename but neither of their
conditional forms, so copy_with(a, b).if_not_exists(true) succeeds when only
the fallback holds a (it becomes a conditional write) and is refused with
Unsupported when the primary holds it. Check the condition you need against
the primary driver rather than assuming it holds for every object on the disk.
A move the primary would refuse is refused before anything is deleted. A
primary with no rename at all, a guarded move onto a primary with no
conditional rename, and a guarded move onto a destination that already exists
all fail with the fallback source still in place - a move that never happens
must not cost you the cold copy.
If the stream fails partway, the writer is aborted and a destination the
transfer created is deleted before the error reaches you, so a failed transfer
is not observable as a truncated object. A destination that was already there
is left alone - a failed copy must not be the thing that destroys an object it
never wrote. A local-filesystem primary honors that too, because it stages the
transfer under .suprnova-atomic/ and only renames on success; aborting the
writer removes the staged file, so a failed transfer leaves neither a partial
destination nor a leftover temp file.
Versioned and conditional reads
A read that carries a version or an If-Match, If-None-Match,
If-Modified-Since, or If-Unmodified-Since condition is passed on with that
condition intact, so the answer means what you asked it to mean. Such a read
is served but never promoted: writing an old version or a validator-matched
body to the primary would publish it as the live object, and every later
plain read would get it.
Which disk answers one is decided the usual way. The first probe is an ordinary existence check, so a read-through disk delegates a versioned or conditional read to the primary whenever the primary holds the path at all; it reaches the fallback only when the primary does not.
The primary also decides which of these a read-through disk accepts at all, because the primary's reader is opened first. A versioned read against a read-through disk whose primary is a local directory is rejected before it reaches the fallback, since a local directory has no versions.
Why Suprnova diverges
Laravel builds a read-through disk from a config/filesystems.php entry whose
primary and fallback keys accept either a disk name or an inline driver
config. Suprnova takes disk names only, because disks here are registered by
typed constructors rather than described by arrays - register the inner disk
first, then name it.
Laravel's promotion re-checks the primary after reading the fallback, which makes a concurrent writer win. Suprnova keeps that check and publishes the promotion atomically, which Laravel does not. On a local-filesystem primary the bytes are staged at a temporary sibling and renamed into place; writing them straight to the target would leave a growing, half-written file visible for the length of the write, and a read-through disk routes readers by exactly that existence check. On a primary without a rename - in-memory, S3, Azure Blob, GCS - a write is already a single indivisible publish, so the promotion writes the target directly, conditional on the object not already existing so two concurrent readers do not both promote.
That condition is the part a staged promotion cannot have: the staging path is unique, so a no-clobber condition on it would be vacuous, and the target is published by a rename that overwrites. A read-through disk on a local-filesystem primary therefore trades it away - a write that lands on the primary in the moment between the promotion's last existence check and its rename is overwritten by the promoted copy. On a primary without a rename the condition holds and no such window exists.
The staging object is a real entry on the primary while it lasts, so a listing
taken mid-promotion can show a .suprnova-promote-<id>.tmp sibling. A read
that completes, fails, or gives up tries to remove its own sibling, and logs a
warning if that delete fails rather than failing the read. Nothing sweeps a
sibling left by a failed delete, a process that crashed, or a read future
cancelled mid-promotion: those have to be removed by hand.
A promoting read streams the cold object from the fallback into the primary,
then answers from the primary. Nothing holds the whole object in memory, but
the first read of a cold object waits for the whole transfer, even when it
asks for a small range. A read that is not promoted fetches only the range it
asked for: a read with copy: false, a versioned or conditional read, and the
rest of a read whose promotion failed. A chunked read does not retry a failed
promotion for every chunk. A fallback that answers a range with more bytes than
the range holds, the way a server that ignores Range does, fails the read
after one chunk instead of being read to the end. On S3, Azure Blob, and GCS
the service refuses such a response before reading its body, for an
open-ended range as well.
Laravel hands back the fallback's own stream when copy is false and
buffers through php://temp when it is true. Suprnova narrows the fallback
fetch to the requested range when nothing is promoted, and streams the
promotion instead of buffering it.
A delete or a move of a path that overlaps a promotion of the same path is not undone by it, on one node or across several. Within one process, the promotion publishes only if no delete or move of its path ran after it started fetching; otherwise it discards its staged copy. Across processes nothing is shared, so the storage decides: a delete and a move remove the fallback copy before the primary copy, and a promotion checks the fallback again just before it publishes and once more after. When the first check finds the object gone or changed, the promotion discards its unpublished bytes. When the second one does, the promotion withdraws the copy it published. A cancelled read still finishes that check.
Each check compares the fallback object's ETag or version, or its
modification time and length. A length alone never counts: a same-size
overwrite keeps it. A fallback that reports none of the others - the
in-memory disk, and so Storage::fake() - is checked by content instead. The
promotion hashes the bytes it streams, and each check reads the object again
and compares hashes, so a promotion from such a fallback reads the object up
to three times. A modification time is only as fine as the fallback records
it, so a same-size overwrite within one tick of a local disk's clock reads as
unchanged.
A withdrawal never deletes a writer's object. It removes only the exact copy the promotion wrote, and that needs a primary that names each write by version and can delete one version, such as an S3 bucket with versioning enabled. On any other primary - a local directory, memory, or an unversioned bucket - the promotion keeps its copy and logs a warning, because OpenDAL has no delete that checks what the path holds. Three limits remain. On those primaries, a delete on another node that completes in the moment between the promotion's last check and its publish can leave the promoted copy on the primary. Between the publish and the check after it, a reader on another node can see the object for a moment. A process that stops between the two leaves its copy on the primary.
A versioned or conditional read reaches the fallback even when the primary cannot express the version or condition. The primary's refusal applies only when the primary holds the object.
Laravel's cross-fallback copy and move also buffer the source through
php://temp. Suprnova streams it in 64 KiB chunks instead, because the
fallback is where the large, rarely-touched objects live, and deletes a
half-written destination before returning the error. Two more differences
follow from OpenDAL. Deleting a path that is not there counts as success, so a
move clears the fallback source without first checking that it exists. And
OpenDAL carries conditions on copy and rename that Flysystem has no
equivalent for, so Suprnova has to decide what each one means when the source
is only on the fallback: if_not_exists and a copy's source version are
honored, and a copy's if_match is refused rather than dropped.
Before it deletes anything, a move checks that the primary accepts the destination: a local primary's path guard refuses a destination outside its root or inside its staging directory, and a refused move leaves both disks as they were.
Laravel deletes the fallback source after the move on both paths. Suprnova deletes it first when the primary holds the source, because the two orders differ under a retry: the source is unreachable through the disk either way, but deleting last means a move that lost its delete to a transient fault comes back as a move whose source is now fallback-only, and streams the fallback's stale copy over the destination the first attempt already wrote correctly.
Registry hygiene
let removed = forget; // bool: was any named disk present?
forget;
purge; // drop one named disk
purge_all; // drop every disk
purge_all; // drop every disk
set; // store a ready-made Operator
let names = disks; // Vec<String>, sorted
You pass one name, an array, a vector, or a slice to forget through
DiskNames. You receive true when at least one disk was removed, and false
for missing names or an empty list. You use purge(name) for Laravel's
named purge and purge_all() for every disk; Rust does not overload
functions by argument count, so the two forms carry two names. You store a
ready-made operator with set; you
replace the previous disk of that name and clear its old public URL. You keep
any operator handle you already obtained usable after removal or replacement.
These mirror Laravel's FilesystemManager::forgetDisk / purge and are
useful for configuration reloads and admin dashboards. They are not
test-only: production code occasionally needs to drop and re-register a
disk at runtime (e.g. after a secrets rotation).
Testing
Storage::fake() returns a guard that:
- Acquires a process-global mutex so concurrent
#[tokio::test]cases do not race on the shared registry, and - Resets the registry on construction and on drop, leaving the suite in a clean state for whichever test runs next.
A "default" memory disk is pre-registered for convenience.
use DiskAssertExt;
use ;
async
The five assertion helpers - assert_exists, assert_contents,
assert_missing, assert_count, assert_directory_empty - are exposed via
the DiskAssertExt trait, gated on #[cfg(any(test, feature = "testing"))]
so production code cannot reach for them.
Parity quick reference
Laravel Storage::disk(...)->... |
Suprnova |
|---|---|
exists($path) |
disk.exists(path) |
missing($path) |
disk.missing(path) |
fileExists($path) / fileMissing |
disk.file_exists(path) / file_missing(path) |
directoryExists($p) / directoryMissing |
disk.directory_exists(p) / directory_missing(p) |
get($path) |
disk.get(path) (Vec<u8>) |
json($path) |
disk.json::<T>(path) |
put($path, $contents) |
disk.put(path, bytes) |
prepend($path, $data) |
disk.prepend(path, data) |
append($path, $data) |
disk.append(path, data) |
size($path) |
disk.size(path) |
lastModified($path) |
disk.last_modified(path) |
mimeType($path) |
disk.mime_type(path) |
checksum($path, ['checksum_algo' => 'sha256']) |
disk.checksum(path, ChecksumAlgorithm::Sha256) |
files($dir, $recursive) |
disk.files(dir, recursive) |
allFiles($dir) |
disk.all_files(dir) |
directories($dir, $recursive) |
disk.directories(dir, recursive) |
allDirectories($dir) |
disk.all_directories(dir) |
makeDirectory($path) |
disk.make_directory(path) |
File::ensureDirectoryExists($path) |
disk.ensure_directory_exists(path) |
deleteDirectory($path) |
disk.delete_directory(path) |
move($from, $to) |
disk.move_to(from, to) (or opendal-native rename) |
copy($from, $to) |
disk.copy(from, to) (opendal-native) |
delete($path) |
disk.delete(path) (opendal-native) |
temporaryUrl($path, $expiry) |
disk.temporary_url(path, expire) (or opendal-native presign_read) |
temporaryUploadUrl($path, $expiry) |
disk.temporary_upload_url(path, expire) (or opendal-native presign_write) |
buildTemporaryUploadUrlsUsing($cb) |
Storage::build_temporary_upload_urls_using(disk, callback) |
providesTemporaryUploadUrls() |
Storage::provides_temporary_upload_urls(disk) |
download($path, $name) |
Storage::download(disk, path, name) |
response($path, $name) |
Storage::response(disk, path, name) |
Storage::fake() |
Storage::fake() |
Storage::disk()->assertExists() |
disk.assert_exists(path).await |
FilesystemManager::forgetDisk($n) |
Storage::forget(name) |
FilesystemManager::purge() |
Storage::purge(name) and Storage::purge_all() |
Configuration
You register disks in two ways. In code, bootstrap() calls
Storage::register_* and you address a disk by name at the call site
(Storage::disk("public")). From the environment, the server registers one
S3 disk named s3 when it boots, beside the queue, the rate limiter and the
mail transport.
An S3 disk from the environment
You configure the s3 disk with S3_BUCKET or its Laravel alias
AWS_BUCKET. The server registers it under suprnova::ENV_S3_DISK at boot.
When neither bucket is set, you get no environment disk. A blank value
counts as unset. Each S3_ value wins over its AWS alias.
| Variable | Meaning |
|---|---|
S3_BUCKET, AWS_BUCKET |
The bucket. Without either there is no disk. |
S3_REGION |
The region. The driver needs one. When S3_REGION is not set, AWS_DEFAULT_REGION is read, then AWS_REGION. A service that is not AWS takes any name, such as us-east-1 or auto. |
S3_ENDPOINT, AWS_ENDPOINT |
The endpoint of a service that is not AWS: MinIO, RustFS, R2, B2. |
S3_ACCESS_KEY, S3_SECRET_KEY |
The keys. You use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY when the corresponding S3 name is unset. You set both effective keys or neither. |
S3_ROOT |
A prefix inside the bucket. Every path is under it. |
S3_PUBLIC_URL, AWS_URL |
The public base URL of the disk, as Storage::set_public_url takes it. See Public URLs. |
S3_USE_PATH_STYLE_ENDPOINT, AWS_USE_PATH_STYLE_ENDPOINT |
You set true for endpoint/bucket paths or false for bucket hostnames. With neither set, you keep path-style URLs. An invalid value fails boot. |
With no keys set, the driver uses the default credential chain of AWS. That
chain reads AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, the profile and
the role of the instance.
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY=<access key>
S3_SECRET_KEY=<secret key>
S3_BUCKET=local
S3_REGION=us-east-1
S3_PUBLIC_URL=http://localhost:9000/local
use ;
let disk = disk?;
disk.put.await?;
The server does not boot when the variables describe no usable disk. The
cases are no region, one key without the other, a driver that refuses the
configuration, and an S3_PUBLIC_URL that is not a valid base URL. No error
repeats the value of a variable.
If your bootstrap already registered a disk named s3, that disk stays and
the variables are not read. Your registration is a decision, and a fault in
variables that describe nothing in use must not stop the boot.
To register the environment's disk under a name of your own, call
S3Config::from_env(). It returns Ok(None) when neither S3_BUCKET nor AWS_BUCKET is set.
use ;
if let Some = from_env?
The disk exists after the server has booted its drivers. A booted callback
runs before that, and the console binary boots no driver of the environment.
If a callback or a console command needs the disk, call
suprnova::filesystem::bootstrap_from_env()? at the end of your own
bootstrap, after you register your disks and set the default disk: it
checks the default disk too, so a call before the disk is registered
fails.
Each driver is a peer. Azure Blob and GCS disks have no environment variables: register them in code.
The default disk
FILESYSTEM_DISK names the application's default disk, and
Storage::default_disk() returns it. A name set in code with
Storage::set_default_disk wins over the variable, the way a disk the
bootstrap registers wins over the one the environment describes:
use Storage;
// bootstrap.rs
register_fs?;
set_default_disk; // or FILESYSTEM_DISK=uploads
// anywhere after boot
default_disk?.write.await?;
The boot checks the name once the bootstrap and the s3 disk of the
environment are in place: a default disk that names no registered disk
stops the server with an error that names the disk and FILESYSTEM_DISK,
rather than failing the first upload. With neither setting present, you use
the registered local disk. When you have not registered local,
Storage::default_disk() returns an error naming local and
FILESYSTEM_DISK. A call that names its disk, Storage::disk("s3"), is
not affected.
Why Suprnova diverges
Laravel ships a local disk and defaults FILESYSTEM_DISK to it. Suprnova
registers no local disk on its own, so you register local in your
bootstrap to use the implicit default. You reach it through
Storage::default_disk() rather than Storage::put.
See Configuration for the wider rule on where the framework reads from the environment.
Next
- Configuration - what the framework reads from
.env - Requests - file uploads land on a disk via
UploadedFile::store_as - Responses - streaming bytes back out of a disk
- Cache - the other named-driver registry, same shape
- Testing - the wider fake-everything testing surface
