s3fs – FUSE file system for Amazon S3

Computers & Mobile Technology

As the title suggests, s3fs is a FUSE-based file system for Amazon S3. What this means is that you can mount an S3 bucket and use it just like a standard disk—a crazy-huge distributed disk that you can mount from any machine.

s3fs supports mode (e.g., chmod), mtime (e.g, touch) and uid/gid (chown). s3fs stores the values in x-amz-meta custom meta headers and uses x-amz-copy-source to efficiently change them.

If enabled via “use_cache” option, s3fs automatically maintains a local cache of files in the folder specified by use_cache. Whenever s3fs needs to read or write a file on s3 it first downloads the entire file locally to the folder specified by use_cache and operates on it. When fuse release() is called, s3fs will re-upload the file to s3 if it has been changed. s3fs uses md5 checksums to minimize downloads from s3.

The folder specified by use_cache is just a local cache. It can be deleted at any time. s3fs re-builds it on demand.

While this will technically function as a cvsroot or mysql store, it’s probably a bad idea to use it this way, as writes mean copying the whole file across the network, making that use particularly error prone and inefficient.

More interesting is using this as a simple way to manage web files that you are delivering over S3 via HTTP. Most content management systems can be configured with an alternate path for file uploads, and you could basically cloud-enable all of your asset storage without rewriting a line of code.

This could also serve as a network drive for less frequently updated data, such as your desktop’s mp3 or photo folders, or a backup drive for nightly rsyncs.

s3fs – FUSE-based file system backed by Amazon S3

Discuss this article with the rest of the community on our Discord server!
Tagged

ADVERTISEMENT

Maker Faire Bay Area 2023 - Mare Island, CA

Escape to an island of imagination + innovation as Maker Faire Bay Area returns for its 15th iteration!

Buy Tickets today! SAVE 15% and lock-in your preferred date(s).

FEEDBACK