[tahoe-dev] Feasibility of using Tahoe-LAFS on home server

[Jason Harrigan]

Hi,

I'm investigating the feasibility of using Tahoe-LAFS on my home server.
I've got a Proliant N40L with 8 GB of ram and a dual-core 1.5 GHz turion
processor. It's got space for about 4 - 6 hard drives in it, right now I've
got 3.

I'm booting from a usb stick with ESXi on it. The plan is to have redundant
storage to hold all files - things like important documents and media.
Ideally a virtual machine will take care of the redundant storage, and then
share it back transparently back to ESXi so all the other virtual machine
servers can access it. Specifically, one server will stream the media to
devices around the house. I'd also like to put a router VM on this box, and
maybe other kinds of servers like mail.

I'm looking at Tahoe-LAFS because it does a good job at redundancy and
protecting the data. An important feature is that I can use all my
nonuniformly sized external drives, and the extra disk space on my desktop
to provide additional redundant space. I can also set up some disk space at
work, etc, to provide the geographical diversity that's needed.

I'm thinking that I'd set up my system such that I could lose one or two
disks in my home server box, but still retrieve all the data from the
server box only. So it would be very low latency (hopefully I'd never need
to get a share from outside of the box). All the other space from external
drives and the network would just be added redundancy.

I don't want to use raid because it's not cheap (ESXi requires hardware
raid) and uniform disk sizes are needed. There's no geographical diversity.

I'd prefer not to use zfs because I wouldn't feel comfortable having
everything on one machine. I'd have to figure out how to make backups.
Also, it's not as ad-hoc such that new storage can be easily added or
removed, and certainly not from the network.

My main question is about speed. Since my entire system would essentially
be contained on one box, I'd have multiple processes running (one for each
disk) as well as the gateway all simultaneously.
- Does the encryption/decryption on the turion processor bottleneck the
system?
- Even though all needed shares are within one box, are there latencies
with querying the network?
- Will streaming media from the filesystem be feasible?
- Even if the performance is good enough, can other virtual machines simply
use it as a transparent file system?  I'd probably want to share it back to
ESXi as a nas or drive so it is available to all the other virtual machines.
- Do I have to worry about making the gateway highly redundant? If the
drive the gateway is on is lost, is access to the entire storage array
lost? Can there be multiple gateways connected to the same storage array?
I'd probably have the gateway running on an ssd for example, and I don't
want that to be a potential point of failure for the entire system.

Thanks, I really appreciate the good work being done on this file system.
Btw, I'm not an expert in file systems or servers, so some of my
terminology or planning might be wrong.

Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20121224/8d092ec5/attachment.html>