dotCloud now supports FUSE!
This has been a long-requested feature. Well, not FUSE support itself—some of you might wonder what it even is! But since the early days of our private beta, users have been asking for some kind of shared filesystem support. Lots of apps need a shared folder to store, for example, content uploaded by their visitors (the most common case); or content generated by one service instance, but potentially served or consumed by another one.
Well its finally here! Let’s see how FUSE helps with that.FUSE lets you mount any kind of filesystem
FUSE is Filesystem in Userspace. It is a kernel interface allowing unprivileged users (a.k.a. non-root users) to run any filesystem in userspace, i.e. in a regular UNIX process (as opposed to within a kernel module that requires escalated privileges to load and run). The list of filesystems supported by FUSE is quite impressive, and if your favourite one isn’t in there, you can implement your own.
Sharing files between services with S3FS
The most popular use-case will probably be S3FS, which allows you to mount a S3 bucket and access it as if it were a local filesystem. The same S3 bucket can be mounted multiple times across different service instances; which means that when you create a file in one service, this file will end up on S3 (without having to go through AWS APIs), and will eventually show up in other services as well. Since we thought that many of you would be interested by this use-case, we wrote a (hopefully) convenient S3FS tutorial to explain how to do that on dotCloud.
You will need you own AWS account, and your data will be stored in your S3 buckets. Remember that data transfer is free between S3 and EC2 (where dotCloud’s default cluster is located). Therefore, you will only have to pay for storage fees; and even if you need to store a few GB of data, that will only amount to pennies a month.Encrypted storage of sensitive data with encfs
Sometimes, for example compliance reasons, you must make sure that some data is not stored in clear. FUSE can help, thanks to encfs! When using encfs, each file you write is encrypted on-the-fly (and decrypted when you read it back). The data is never stored in clear. Of course, you will need to provide the secret crypto key each time you redeploy, restart, or even scale the service; that’s the price to pay for the increased security.
If you’re paranoid, you could even add a watchdog within your dotCloud service, to track suspicious activity. If any unexpected process appears within your service, boom, you terminate the FUSE encfs process, and the data becomes unavailable for malicious prying eyes — until you inspect the service and unlock it again by providing the crypto key.High-performance file access with GlusterFS
Some of you might not like the “eventual consistency” features of S3; or you might want the low-latency, higher-throughput, of a true distributed filesystem. Well then, have a look at GlusterFS: it provides exactly this, and has a FUSE interface! If you already have an existing GlusterFS storage somewhere, you can now use it from within your dotCloud services. You could also spin up your own GlusterFS storage cluster, or use dotCloud for that purpose. Recipes to do exactly that might come up in the future!
...And much, much more!
Use GMailfs to store files in a GMail account. Mount a remote home directory (on a server to which you have SSH access) with SSHFS. Mount a Hadoop HDFS and access it as if it were a local filesystem. Etc.
If you have interesting or original ideas of cool use-cases for FUSE on dotCloud, let us know!
