Backup Solution
Saturday, July 21st, 2007
I have been taking a lot of pictures over here in Europe and I have also thought about how awful it would be if I had a disk failure and lost all of them, so I decided to come up with a backup solution to make sure this never happened to me. I looked around for an online solution that would run in the background and continually sync my files to web space, but I couldn’t find anything I wanted for the price I wanted (read: free). Also, I have like 48GB of web space on mattfaus.com that I’m not using, so I figured I should utilize it.
So, I used Christopher Evans’ blogpost Backing Up Your Files: A Comprehensive Guide for CG Artists and Users Alike to come up with this command:
rar a -u -msjpg;mp3;zip;rar;xsn;mpg;mpeg -rv10 -v524288k BackupC.rar @CDriveBackupList.lst
a = add files that don’t exist in the archive to the archive
-u = update files in the archive that need updating
-m[list] = copy files with these extensions without compression into the archive (HUGE speed boost)
-rv10 = create 10 recovery volumes - each of the recovery volumes can replace one of the regular volumes
-vXk = Create split volumes at 512MB each
BackupC.rar = name of archive (split comes out like BackupC.part01.rar, BackupC.part02.rar, etc.)
@CDriveBackupList.lst = the list of files and folders to archive (one per line, no quotes necessary)
Rar or WinRar is a great tool that I install on all my machines. It works on loads of compression formats and adds great commands to the right-click context menu in Explorer.
I have 4 physical disks in my machine, 2 of which are RAIDed (striped), for a logical total of 3 disks. The RAID disks are where I run my OS and the other two disks are my storage.
C = 138GB = Operating System
F = 232GB = Storage2
G = 232GB = SATA Storage
Total: 602GB
So, I backup F onto G and G onto F, and C (which is much smaller) onto both F and G. As soon as I get back to the land of fast Internet connections I will upload to my web space. I stored the command string in a .cmd file, of course, but I don’t plan on running it as a scheduled task - I think I’ll just make an outlook reminder to get me to do it on my own. I should also probably run disk defragmenter at that time.
Recovery volumes
I learned about this after I started using USENET to download files and it is a great concept. It provides parity on the files so that if any of the individual archives are corrupted, you can replace it with one of the recovery volumes. I haven’t ever actually had to use it, but I think it’s a good idea to have it. I have loads of disk space so I do 10-20% parity. G backup resulted in 40 archives, so I made 6 recovery volumes on top of this. The recovery volumes are the same size as any of the individual archives.
Running Times
G = 18.4GB = 20 mins archive + 44 mins parity = 1 hour 4 minutes
F = 10.8 GB = 30 mins archive + 37 mins parity = 1 hour 7 minutes
That SATA drive really shows it’s true colors here.
Also, if you’re marvelling at my speeds, I do have a pretty beefy machine:
AMD Athlon 64 X2 Dual Core 5200+
2.61 GHz per core = 5.22 GHz theoretical
4GB RAM
Microsoft Windows Server 2003
Enterprise x64 Edition
Service Pack 2, v.2825
I have been taking a lot of pictures over here in Europe and I have also thought about how awful it would be if I had a disk failure and lost all of them, so I decided to come up with a backup solution to make sure this never happened to me. I looked around for an online solution that would run in the background and continually sync my files to web space, but I couldn’t find anything I wanted for the price I wanted (read: free). Also, I have like 48GB of web space on mattfaus.com that I’m not using, so I figured I should utilize it.
So, I used Christopher Evans’ blogpost Backing Up Your Files: A Comprehensive Guide for CG Artists and Users Alike to come up with this command:
rar a -u -msjpg;mp3;zip;rar;xsn;mpg;mpeg -rv10 -v524288k BackupC.rar @CDriveBackupList.lst
a = add files that don’t exist in the archive to the archive
-u = update files in the archive that need updating
-m[list] = copy files with these extensions without compression into the archive (HUGE speed boost)
-rv10 = create 10 recovery volumes - each of the recovery volumes can replace one of the regular volumes
-vXk = Create split volumes at 512MB each
BackupC.rar = name of archive (split comes out like BackupC.part01.rar, BackupC.part02.rar, etc.)
@CDriveBackupList.lst = the list of files and folders to archive (one per line, no quotes necessary)
Rar or WinRar is a great tool that I install on all my machines. It works on loads of compression formats and adds great commands to the right-click context menu in Explorer.
I have 4 physical disks in my machine, 2 of which are RAIDed (striped), for a logical total of 3 disks. The RAID disks are where I run my OS and the other two disks are my storage.
C = 138GB = Operating System
F = 232GB = Storage2
G = 232GB = SATA Storage
Total: 602GB
So, I backup F onto G and G onto F, and C (which is much smaller) onto both F and G. As soon as I get back to the land of fast Internet connections I will upload to my web space. I stored the command string in a .cmd file, of course, but I don’t plan on running it as a scheduled task - I think I’ll just make an outlook reminder to get me to do it on my own. I should also probably run disk defragmenter at that time.
Recovery volumes
I learned about this after I started using USENET to download files and it is a great concept. It provides parity on the files so that if any of the individual archives are corrupted, you can replace it with one of the recovery volumes. I haven’t ever actually had to use it, but I think it’s a good idea to have it. I have loads of disk space so I do 10-20% parity. G backup resulted in 40 archives, so I made 6 recovery volumes on top of this. The recovery volumes are the same size as any of the individual archives.
Running Times
G = 18.4GB = 20 mins archive + 44 mins parity = 1 hour 4 minutes
F = 10.8 GB = 30 mins archive + 37 mins parity = 1 hour 7 minutes
That SATA drive really shows it’s true colors here.
Also, if you’re marvelling at my speeds, I do have a pretty beefy machine:
AMD Athlon 64 X2 Dual Core 5200+
2.61 GHz per core = 5.22 GHz theoretical
4GB RAM
Microsoft Windows Server 2003
Enterprise x64 Edition
Service Pack 2, v.2825
