Completed, over about 3 days last week, upgrading the last of the 400GB drives in the file server. These new ones are Toshiba 2TB drives (which I paved the way for by buying a 2TB hot spare drive last time I upgraded).
Everything went fairly smoothly, though I messed up one command that dropped the redundancy at one point. No actual data loss, and I had two complete current backups at the time.
If I were doing this from scratch today, I’d use an AMD motherboard (because it’s much easier to find consumer-price AMD motherboards that will support ECC memory) and a 5-in-3 hot-swap cage (fits in the opening where 3 5.25″ drives are supposed to go, holds 5 3.5″ drives in hot-swap trays). That would mean a much cheaper and smaller case, plus I wouldn’t need an additional disk controller card (most such motherboards have 6 SATA ports).
And I’d put FreeNAS software one it. That’s FreeBSD-based rather than Solaris, but still supports ZFS. ZFS is absolutely wonderful for this sort of use. It supports many enterprise-level features that you won’t get in any other cheap approach to building a home fileserver.
I’ll stick with mirrored pairs rather than parity, though. I can upgrade in place very easily with this setup. Five drives is perfect for two pairs of data disks plus a hot spare. The hot spare is useful in emergencies, but also is vital to upgrading the disks in a mirror without reducing the redundancy. I’ve upgraded the current server in 5 steps from 800GB usable space to the current 4TB. A file-server built to this outline (just 4 data drives) would support 8TB of usable space today, far more than I need.
And I’ll boot it off USB thumb drives inside the case rather than from a real disk (the current server has a mirrored pair of 2.5″ disk drives for the system disk; that’s an expense and use of controller slots that’s not really necessary).
I’ve speced out parts at various stores a couple of times; I can build an empty FreeNAS file server of this sort for $300 to $500 depending on details (largely how much memory; the current server ran fine in 2GB, runs fine in 4GB, but FreeNAS documentation suggests it’s memory-hungy; but I suspect that’s for deduplication, which I don’t need).