Switching to RAID on existing drives?

[d4005]

Hi,

I've read up on RAID at the wikipedia page, so I'll say first what I learned and what I believe is true.

I've decide that RAID 1 is for me. I want it for the backup/redundancy feature and understand that I don't get the full drive space usage that I'd get with RAID 0, but I'm mostly after the benefit of data safety. If I've understood correctly, I do still get the double data rate benefit when it comes to reads (just like RAID 0), although writes maybe slightly delayed. I also understand that when it comes to the BIOS and OS, the controller just presents the two RAID 1'ed drives as one single drive. I also think that should a drive fail, I can just replace it and the controller will be busy (for hours or days) getting the new drive up to full mirrored equality.

So now to my questions. I want to apply the RAID 1 to existing installed drives that aren't empty and I'm not sure if it's possible, or if it's easy, or what I would need to buy.

My machine is currently configured like this:

- Asus P5B Deluxe WiFi motherboard (http://usa.asus.com/products4.aspx?l1=3&l2=11&l3=307&model=1179&modelmenu=1)
- Three sata 2.0 drives

Sata drive 1 = 400 GB (logical drives C,D,E) boot drive and programs
Sata drive 2 = 500 GB (logical drive G) 300+GB of data
Sata drive 3 = 500 GB (logical drive H but currently totally empty)

What I'd like to do is combine drives 2&3 in a RAID 1 config. Somehow I'd like to "tell the controller (however that might work)" that drive 2 is the primary drive in this RAID 1 partnership, and that drive 3 is the secondary drive and should be sync'ed up so that it mirrors drive 2, just as if it would if I'd added a new drive to replace one that failed.

I'll also add another 400GB drive (drive 4) which I'll similarly RAID 1 up with drive 1, and tell the raid controller to mirror from drive 1 to drive 4 until they're equal.

I've looked around at RAID controllers and none of them go into enough detail (nor does wikipedia) on whether or not what I want to do is possible. I might find that you can only setup RAID pairings on drives that start out blank (or are made blank by the controller). Nobody explains this part of it :confused:

Also, when looking at RAID controllers, I've seen that some are "4 channel" controllers. It's my guess that these are probably no good for me. I'm guessing (totally) that these are probably for RAID 5 or some other > 2 drive RAID config. I'm thinking that I probably need 2 x 2-channel raid controllers. If there's a 2x2 single controller, or I have to buy two 2-channels controllers, I'm not sure.

I hope the question isn't too long. I wanted to explain that I already read up on it but just couldn't find anything that was practical info, it's all theoretical info.

[Paul Komski]

I also understand that when it comes to the BIOS and OS, the controller just presents the two RAID 1'ed drives as one single drive
A Hardware RAID's BIOS (for any Hardware RAID array) will present just one volume to a DOS-based OS and arrange for it to be the chosen boot device. Linux and the NT-based OSes go deeper and dig out additional information for themselves. This means that the latter OSes must have the appropriate drivers for the array or they will see all hard drives and probably break the array into the bargain.

There is no consistent way that all RAID controllers work so the documentation to read carefully is the motherboard manual or the manual for a PCI card when one of them has the RAID chip on it.

It is "normal" to be able to add two drives (one with data and one blank) to a RAID-1 array and then build the mirror using one as the basis. Some must do this during the POST - others can do this in the background after booting to the OS. If in doubt backup the data before mirroring or experiment with two drives before and after creating the mirror and also by deliberately breaking the mirror by booting to the array with one drive disconnected.

The reason this is a difficult area is, as already alluded to, because no two bits of hardware work the same and so only generalisations are possible. Once you get familiar with your own hardware the whole things becomes much more simple to understand and use.

Four channel controllers would allow up to 4 drives to be added to an array configured on that chip. They could be configured as two 2-drive arrays or one 3 or 4-drive array. It wouldnt be mandatory to use all 4 drives and you should also be able to use drives on the controller but not included in any array.

[d4005]

Thanks Paul, you've pretty much answered all my concerns. I understand that it's not guaranteed that any specific controller will work the way I want it, but I now at least know it's possible and that I can seek out the ones that will work that way :D :D

[Paul Komski]

I forgot one thing and that is that if you installed WinXP onto the drive when it was not on a RAID, you should ensure that you do install the RAID drivers BEFORE you add it to a RAID array. This would be equivalent to using F6 at the start of a clean installation. If you dont have the drivers on board when the drive reappears in an array it won't be able to see itself properly.

It should certainly be possible since what you are doing is essentially replacing a "bad drive" in a mirrored array. The only difference being how the "source" drive got into the array in the first place.

[d4005]

I forgot one thing and that is that if you installed WinXP onto the drive when it was not on a RAID, you should ensure that you do install the RAID drivers BEFORE you add it to a RAID array.[/B]Good tip, I have XP already installed on it so this is totally relevant.

btw, knowing what my requirements are, do you have any recommendations on which RAID controller to go for? I guess a 2x2 (4 channel) controller will be better than two 2 channel controllers, then again considering that I might want to copy gigabytes of data from one RAID PAIR to another RAID pair, perhaps separate controllers would be better. Maybe a single controller would then be a bottleneck

[Paul Komski]

do you have any recommendations on which RAID controller to go for?Not really. It looks as if you can mix one SATA one eSATA and two PATAs on the Jmicro chip - but I didn't read everything in the manual - so that could be the most versatile. The Southbridge chip can configure 6 internal SATAs so that might be the most stable. Its always best to match the drives included in an array as closely as possible. I don't think it makes any difference if you have two arrays on one controller or on two - the quality and matching of the drives themselves being of greater importance.

[d4005]

Not really. It looks as if you can mix one SATA one eSATA and two PATAs on the Jmicro chip - but I didn't read everything in the manual - so that could be the most versatile. The Southbridge chip can configure 6 internal SATAs so that might be the most stable. Its always best to match the drives included in an array as closely as possible. I don't think it makes any difference if you have two arrays on one controller or on two - the quality and matching of the drives themselves being of greater importance.Thanks again Paul :) I'll post back with the story of how it went :)

[Variable]

I mostly work on server based systems but since you talk about backups and recovery as important, I would like to point out a few things. If something took out your machine (like a lightening strike) or your single controller card failed you would lose all data. An external drive should be used for scheduled backups and then unplugged and put in a different location than your machine.

RAID 1 and 5 in general, are about keeping a system online in case of a drive failure. It is not for disaster recovery.
Whether you choose two mirrored sets or RAID 5 you should have an external drive for recovery. If you have two controllers you might want to look at RAID 10. You would stripe two drives on one controller and create a mirrored set on the second controller. That gives you the speed of striping and the redundancy of a mirrored set. In the BIOS or of your controller card you can set which is the primary set. By default it will be the first container you create or should be. Either way, you can set the primary or boot container manually.

Again though, RAID1, 5, 10, etc are not backup solutions, it is about keeping a system online during a drive failure. So the question becomes what you plan to do with the machine and which RAID gives you the best performance and redundancy. Overall, RAID 10 is a good choice.

[d4005]

since you talk about backups and recovery as important, I would like to point out a few things ~~ An external drive should be used for scheduled backups and then unplugged and put in a different location than your machineHi Variable, Great point. I never went into enough detail. I do have an e-sata hot-swap bay into which I plugin external backup drives. When I was mentioning backup/recovery in the RAID context, I was referring really to the mirror feature and that if a drive dies, then I have that "backup" of the mirror and I don't have to go through the process of restoring from a backup. It's a good point to raise though, upgrading to a RAID environment shouldn't make me complacent enough to stop doing backups.

RAID 1 and 5 in general, are about keeping a system online in case of a drive failure. It is not for disaster recovery.
Whether you choose two mirrored sets or RAID 5 you should have an external drive for recovery. If you have two controllers you might want to look at RAID 10. You would stripe two drives on one controller and create a mirrored set on the second controller. That gives you the speed of striping and the redundancy of a mirrored set. ~ Overall, RAID 10 is a good choice.I read up on striping and sure the increased speed is nice, but I'm more impressed by RAID 1 mirroring only. OK I don't get the 2x speed increase when it comes to writes, and I lose 50% of the storage space of the drives involved, but I at least get the 2x (almost) speed increase when it comes to reading data (I assume).

Also, striping just multiplies the DTMF of the drives so that you're far more likely to get a failure, and one failure = both effectively fail. Hang on a second though, as I'm writing this I think I'm seeing the benefits of RAID 5 or 10 come to the surface. If I understand correctly, my total storage (un-RAID'ed) with 2x400 and 2x500 is 1.8TB. With them paired up in RAID 1, I'm looking at 900GB and no write-speed increase. With them in a RAID 5 or 10 config, I'd have two of them striped (RAID 0 style) and they'd then also be mirrored to the other two drives. The result being I have a twice-as-fast RAID 0 pairing with the full mirroring safety. I see it now. Increased speed, but also increased risk of failure, but mirrored anyway for uptime. And just like in my twin RAID 1 config, any single drive failure can be replaced because of the mirroring. Not something you can do in a RAID 0, but in the RAID 5 or 10 you can. The only downside is that 400GB drive bringing my stripeability of the other 500GB drive down to 400GB also. Not sure why I went for that now :( I wish I'd just got 3x500GB :(

[Variable]

Do you mean MTBF? Mean Time Between Failure? That is based on the drive. So two identical drives have the same MTBF.
Striping is not "far more likely to get a failure" than anything else. A dead drive is a dead drive. A striped set is considered one logical drive. If you mean a striped with two drives has a greater chance of failure because their are two drives then I would agree. But by itself striping does not increase drive failures. Striping alone has no redundancy. It is for performance.

RAID 10 is about getting the speed of striping with the redundancy of mirrored set. I use it for DB servers or any application that needs high I/O. Rebuild rate is faster than RAID5.

RAID 5 is striping with Parity, with 4 disks it might be worth looking at. RAID 10 is faster but RAID 5 will give you more space. A RAID 5 array with your drives would give you 1.2TB.

The question of what RAID level to use comes down to what you want to do with the machine. RAID can increase I/O and allow for uptime during a drive failure. That is all it really is.

[d4005]

Striping is not "far more likely to get a failure" than anything else. If you mean a striped with two drives has a greater chance of failure because their are two drives then I would agree. But by itself striping does not increase drive failures. Striping alone has no redundancy. It is for performance.Sorry, that is what I meant. The increased chance of failure.

RAID 10 is about getting the speed of striping with the redundancy of mirrored set. I use it for DB servers or any application that needs high I/O. Rebuild rate is faster than RAID5.:D :D That's what I figured out half way through my last post :D :D

RAID 10 it is then. Now I just need to ditch my current 400GB boot drive and replace it with a 500GB otherwise it'll drag down the total size I can achieve (with RAID lowering the capacity to smallest drive in the array). I want to have 4 x 500GB in RAID 10 as my final config.

Does anyone know of a good backup program that can backup my 400GB drive (with a C boot, and D and E logical drives). I want to back it up to a currently empty 500GB drive and then boot from that drive. I'm hoping I can do this in one operation, rather than backup from 400 and restore to 500.

[Variable]

If Sylvander is around I think he regularly re images his system :)

I have used Ghost with good success. Acronis True Image is good. The more I look at Acronis the more I like it.

[Paul Komski]

The more I look at Acronis the more I like it.Be very wary of Acronis and RAID. It runs from a Linux kernel and has broken any arrays I have tried to access with it on my ASUS A8NE board, where it doesn't see the RAID volumes but does see all four individual SATA drives, which are configured into two RAID-1 Mirrors. It's a particular PITA to have to rebuild them since it has to be done before booting the OS on this system. With striping it would be even more horrendous.

Ghost should be fine but I fell out of favour with it after it was taken over by Symantec from PQ's Drive Image. BiNG is extremely versatile and excellent value and is particularly adept at writing and recovering from images on USB2 HDDs and DVDs.

I agree with all the comments about mirroring and the prevention of down time and storing important data on removable media in a safe place.

If you are starting over then you might consider creating an image of your current boot volume, create an empty RAID array (of your choice) and then copy the image file directly to the array. This would avoid any rebuilding of an array starting from a source drive. The likelihood is that you could only do this anyway with a RAID-1 mirror since the striping inherent in RAID-10 is likely to disallow this and have to be created from scratch.

[Variable]

Hmm, good to know. I have never used Acronis but heard people here mentioning it and I looked at their site and read their user guide. If it has problems with RAID that is bad. I have never seen a problem writing an image to a RAID 10 array but I have only done it on SCSI based RAID controllers.

[Paul Komski]

I have seen arrays broken in two ways by software. Firstly when using Acronis and secondly when installing an NT-based OS WITHOUT installing the RAID drivers using F6 during the beginning of the install. I'm not saying it will happen with every BIOS but any OS that polls the BIOS for itself (after selection of the boot device by the BIOS) must be prone to this if it then differentiates between the hard drives AND writes a single byte to an individual hard drive. It's about a year since I experimented with Acronis and, from memory, it writes to the MBR as soon as it is installed - so the mere fact of installing it and rebooting was enough turn on the hazard lights for a broken array. At least with a mirror it would still boot but with striping it would have meant downtime for sure. I just ran on one drive for a couple of days until I had a slot to rebuild.

PS
Since that time I have realsised that if I have a suitable system image of an array on a large enough removable medium it can be quicker to restore the whole backup to a brand new array using BiNG than to rebuild the mirror from scratch. This is only OK of course if one's data and system partitions are segregated and one has planned for the eventuality.

[d4005]

I've been considering my options. Raid 10 (1+0) does sound like it would be perfect, with a striped pair and that pair mirrored. Considering my current config though, 1x400 and 2x500 drives, that would mean to do it sensibly I'd need to ditch the 400 otherwise it would meand I'd waste 100GB on each of the other drives. I'd have to buy two new 500GB drives.

Having read around a bit more, it seems like the main advantage of striping (the speed increase in writes) isn't nearly as useful as it sounds. Only if you're hammering the drive with very large files (where the striped chunks come into play) do you start to see this 2x speed increase. With regular small files it's hard to notice an improvement (so people have told me). When I consider that RAID 1 already offers the increase in speed when it comes to reads (because it can read stripedly from the two drives in the mirror pair), then it seems that RAID 10 (for me at least) becomes an increased cost (one drive more) for little gain.

So I'm going to opt for RAID 1. This will mean pairing my two 500gb drives together, and buying one new 400GB drive to pair with my existing boot drive. Total 900GB but only had to buy one new 400GB drive :)

This is not only the least cost route, it's also the least effort - add one new drive, add the controller and power it up and go. The RAID 10 effort would have involved some kind of ghosting/backup/restore to a new 500GB drive.