Thread: Guard against data loss on a home fileserver or seedbox. Not RAID, a better sollution

Note: This isn't a technical tutorial, but more of a guide. I assume you have a fairly good understanding of *NIX, and like to tinker with your systems. If you absolutely want my help I can be reached for more details. This is for hardware you have access to. The title mentions a seedbox, but I'm talking about my home seedbox. Continue...

I knew I had to buy some big disks soon, and I was looking for a solution to data loss. I considered a RAID mirror, but I decided on an old school approach. What it basically boils down to is you have to give up a real time mirror (RAID 1) and settle on a scheduled backup. What you get in return is better control and flexibility over the backup process. I may be goofy, but I've often found myself in a position where I just deleted something, and immediately realized I shouldn't have. I like the idea of having a day or so of buffer time to revert changes. Of course, this means that a days worth of changes are always at risk, but when we're talking about large non-changing media files, the risk is low. Also, there's more flexibility in non-RAID arrays. I don't have to drive match, and I can just replace a failed drive with whatever is most economical at the time. If you're liking were I'm going with this, I'm going to describe my set up, which I'm pretty happy with.

- First, keep the OS and APPs away from the data. I have a lone separate disk for the OS.

- Acquire some hard disks. Best to buy in pairs and as large as you can get them. Also, you can get yourself a very cheap controller card. You don't need RAID ability, so that will save you some money. I just bought a Promise SATA TX4, very cheap and it'll do everything I need.

- Next, use LVM to create two volume groups. One labeled "primary" and the other labeled "backup". Of course, you put one of each disk on each of the volume groups. Note, use the whole disk for each volume group, and use the whole volume group for each logical volume. IMO, there's no point in breaking the disks up into separate partitions, or making your logical volumes smaller than the full size of the volume group. Of course, if you know what you need, use your own discretion.

- Once you've built up your logical volumes, create two mount points for them. Something like "/mnt/primary" and "/mnt/backup". Make appropriate changes to fstab, and get them mounted.

- Now use primary for all your data needs, and schedule a copy operation to backup your primary volume to your backup volume. Edit: I just wanted to make a note that rsync works amazing for doing the backup copy operation.

- If primary fails? Just mount your backup volume to your primary mount point, and your system shouldn't be able to tell the difference, while you fix your primary volume. If a disk fails, just replace it with whatever makes sense.



- If you're running low on space? Just go out and buy another pair of drives, add them to each volume group, and extend the logical volumes.

This setup suits me better than a RAID mirror. I thought I'd offer this up as an option to anyone who might be thinking of setting up a mirror.

Good idea, it is also easier to setup and more flexible. The problem with some of us though is that we are too greedy for space, I always end up using all my space as primary :P

Originally Posted by rawr

Good idea, it is also easier to setup and more flexible. The problem with some of us though is that we are too greedy for space, I always end up using all my space as primary :P

Yeah, I feel your pain. I just bought a total of 4TBs of disk, but after install it turned into effectively 1.8TB (mirrored and formatted). :'( Anyway, better that type of pain, than the pain of losing a media collection that you worked hard to acquire. In my opinion anyway.

Wow, sounds like an interesting alternative. Can this be implemented easier on 1 OS over another? example it works better on a linux box but not so well on a windows box..

Originally Posted by dog2

Wow, sounds like an interesting alternative. Can this be implemented easier on 1 OS over another? example it works better on a linux box but not so well on a windows box..

Well, any server type application will work better over a *nix type system. Windows was never intended for such a job contrary to what microsoft tells you. If you want to do something similar to this with windows, you're going to have to research what pieces of software are required. There must be a way of doing something similar. Maybe, someone here has some experience with doing something similar will post a comment.

I have a couple of questions for you. I hope you don't mind.

I guess by now you've run your first backup and are adding to it? The first time you hit "return," did you see a lot of decrease in throughput as your drives filled up? What kind of speeds were you seeing?

Do you use rsync or one of the evaluative programs (which one?) for incremental backups?

Thanks in advance!

P.S. I ran across this interesting article on linux and rsync. The author is using a couple of recycled old pc's and doing something similar, running incremental and full backups with cron, and he explains it pretty well, though it's on a much smaller scale and not about media. He's even using it to backup windows machines. And if I'm not mistaken, I believe he's doing it on a student budget.