RAID 0 for Video Editing... Good enough?

[imashination]

A RAID 0 isn't much more risky than a regular harddrive, you wont be losing data monthly or even yearly. The only extra risk is that you have 2 harddrives which might fail instead of 1, thats it.

[rocarpen]

Hmm.. why not get two 74GB Raptors instead of four 36GB? Seems like it'd be faster (the 74GB line apparently got a lot of features missing in the 36GB), cheaper, and less prone to failure (two drives versus four).

[mustique]

Are 2 74GB Raptors really faster than 4 X 36 GB ones? If that's true, than I definately would go for it.

[lots]

I doubt that the 74GB Raptor is twice as fast as the 36. Thus getting two 74s in RAID0 vs 4 36s in RAID0 would probably not be faster.. Then again I havnt looked at benchmarks of this type in a while, since HD speed is of little conciquence to what I do.

[rocarpen]

Hmm... good question re: 4x36 versus 2x74. I suppose it's a question of how well performance scales with the RAID0 setup. If I were you, I'd post this exact question over at the Storage Review forums: www.storagereview.com (http://forums.storagereview.net/index.php?showforum=2).

[lots]

I do agree that the 74GB raptor is faster than the 36GB, its just probably not twice as fast.

[maX_Andrews]

If you have four drives then you don't need raptors. A four-drive set of good 7200RPM SATA drives will give you much more bang for the buck and transfers of over 200MB/s, almost enough to work with uncompressed 1080 HD footage. And the more space you have the better, especially with 32-bit uncompressed footage. Drives slow down as they fill up, so the more space you have, the longer your read/write speed can be maintained.

I'd get four of these: http://www.newegg.com/Product/Product.asp?Item=N82E16822144425
The diamondmax drives are rated for 24/7 use in servers and as such are highly reliable. SO for $240 you get a 320GB RAID 0 set with over 200MB/s read and write speeds.
You could also go with eight 7200RPM 40-gig drives and get transfers of over 500MB/s for the same price as four raptor drives.

[nessus]

I would say use a RAID0 for editting and temp footage storage only when you are editting them, at the same time have a huge HD like 400GB or so to act as your real storage. As most of the video editing software out there these days are using "non-destructive" mechanism, the actual saved "project" file is small. So this is how i would do it:
store your footage at your regular media (could be dvd's, or single reliable HD's). when you start editting, copy the footage you need onto the RAID0 drives in order to increase the performance. and save the "project" files on your regualr HD. since the actual "project" files are relavtively small, it wouldnt effect the profermance much even if its not on the RAID0 HDs.
and once you are done with all the editting and output them to RAID0 and copy over to a more reliable media, or simply just output to that media directly.

although i have never done video editting this way (i dont have a 400GB HD yet :P), this is how i would set up my system if i have the all the equipments.

hope this helps!

[mustique]

maX_Andrews thanks for your help.

nessus, I was thinking to use the RAID 0 setup exactly the way you described it. Thx.

I've read reviews on RAID controllers here and there. But does anybody have
real-world experiences from a video editor POV with SATA-Raid controllers?

Can I plug a PCI-EXP compatible RAID controller into one of the PCI-EXP slots
of an nForce SLI Mainboard?

What kinds of experiences do you have with software RAID as found on the nForce4 boards?

[Framed]

I have been using two SATA 160GB barracuda's in RAID0 for two years as main system drive on my main workstation. Never had a problem. I had it setup this way just to test what the fuzz of RAID was and got stuck with it. Only had to reinstall windows once and it had nothing to do with the RAID array. I worked with Premiere with this RAID and the difference from one disk vs two-disk array was huge. The RAID Controller is the (on-board) FastTrack 378.

IMO, the number one reason people experience harddrive failures is they underestimate the importance of adequate cooling of the drives. With poor cooling (specially in raid, when they often are stacked next to eachother) a drive will happilly tug along not showing any problem and then suddenly go byebye a year later.

[Tarrbot]

imashination states:A RAID 0 isn't much more risky than a regular harddrive, you wont be losing data monthly or even yearly. The only extra risk is that you have 2 harddrives which might fail instead of 1, thats it.
This is one of the great myths of RAID 0. The potential for disaster with RAID 0 is much greater than a simple ratio of disks.

PC Guide (http://www.pcguide.com/ref/hdd/perf/raid/concepts/relRel-c.html) has a nice explanation for it. I used to have a link to IBM that had this much more in depth but I seem to have lost it.

[imashination]

imashination states:
This is one of the great myths of RAID 0. The potential for disaster with RAID 0 is much greater than a simple ratio of disks.

PC Guide (http://www.pcguide.com/ref/hdd/perf/raid/concepts/relRel-c.html) has a nice explanation for it. I used to have a link to IBM that had this much more in depth but I seem to have lost it.

That article says exactly what I stated, although it does make some very fundamental maths mistakes:

"The implications of this are clear. If you create a RAID array with four drives, each of which has an MTBF figure of 500,000 hours, the MTBF of the array is only 125,000 hours!"

If you have two drives, you have double the chance of a drive failing, if you have 4 drives, you have 4 times as much chance of the drives failing. But the chances of a drive failing are still very slim, and they will still last for years, not months. The article's mistake is assuming that the estimated time til failiure is linear, it isnt. The drives dont last between 0 and 500,000 hours, the failure rate increases at the upper end of the curve, a typical lifespan might be 450,000 to 550,000, with 2% failing before 450,000.

Thus having a 10 drive raid wouldnt give 500,000/10 = 50,000 hours, it would give something in the region of 400,000 hours. Obviously I dont have the manufacturer's numbers so cannot give an accurate result.

Basically the article has the right idea and makes a valid point, but it has been grossly oversimplified to the point that it is misleading. If you have 4 drives in a system, they are 4 times more likely to have one fail, but that is not the same as reducing the average run time by a factor of 4, because the using the mean is meaningless. Take the mode and you will have something more useful.

[Tarrbot]

By that logic, if I have a 1 in 15,000,000 chance of winning the lottery and buy 2 tickets, I will have a 1 in 7,500,000 change in winning which just isn't the case.

You would have a 2 in 15,000,000 chance of winning and nothing more.

I think your math is a bit off.

[maX_Andrews]

It's all wrong.

You have to multiply the probability of each drive failing. Let's say 2% of drives fail before 500,000 hours, which is about 3.5 years of continuous operation. Let's say you have a four-drive raid set. The probability of each single drive failing is 2%, so multiply 2x2x2x2=16% chance of the system failing before 3.5 years of use. For a two-year period, let's say 1.3% of drives have failed. 1.3x1.3x1.3x1.3=about 3% chance of failure in two years, i'll take those odds.
You can't "rate" it in hours because it just doesn't make sense. You can only look at the probaility of failure of the system by evaluating each drive. There is a large possibility that your system will work for six years with no issues. It's not any more prone to failure than a normal single drive, the risk is that of one part fails the whole thing must be wiped.

[jbo]

maX-
why would you have to multiply? you'd add. by your logic, if each drive was 50% likely to fail within a given time, with 4 drives, it would be 6,250,000% likely to fail. hmmmmm... something's not right there.

imashination is right on this one. i don't know what the rest of you guys are smoking.

[MadMax]

Honestly I'd go with Serial Attached SCSI and forget this SATA crap for video editing.

[imashination]

By that logic, if I have a 1 in 15,000,000 chance of winning the lottery and buy 2 tickets, I will have a 1 in 7,500,000 change in winning which just isn't the case.

You would have a 2 in 15,000,000 chance of winning and nothing more.


2 in 15,000,000 *IS* 1 in 7,500,000 ;-) If the lottery number combinations give 15 million possible answers, you could buy 15 million tickets and have a 15,000,000/15,000,000 chance of winning; ie 1/1, you are guarenteed to win.

I should warn you I used to work in a betting shop and my father manages quite a few, I know a bit about statistics :) I don't claim to always get them right but I can spot a glaring error.

Max: Sorry thats also wildly wrong, you simply cannot multiply numbers in that way:
"2x2x2x2=16% chance of the system failing"
So what if I have 3 more drives, that gives 2x2x2x2x2x2x2=128% chance of failing. You can't honestly tell me I have a 100% chance of losing a drive in 3.5 years just because I have 7 of them are you? You can rate an estimated life of a system in hours, of course you can, what else would you rate it in?

[Tarrbot]

Technically, when dealing with odds, 2 in 15,000,000 is not the same as 1 in 7,500,000 because when you buy 3 lottery tickets in your way of calculating you would again halve the odds, and another ticket halves the odds once again.

In reality, you still only have x amount of lottery tickets in 15,000,000 chance of winning not x amount halving continuously since that would mean after 24 lottery tickets you would have a less than 1:1 odd of winning the lottery when instead it should be 24:15,000,000.

No, your odds making is quite a bit off.

Would Storage Review (http://www.storagereview.com/guide2000/ref/hdd/perf/raid/concepts/relRel.html) have the exact same 'article' then?

I thought not...

[jbo]

Technically, when dealing with odds, 2 in 15,000,000 is not the same as 1 in 7,500,000 because when you buy 3 lottery tickets in your way of calculating you would again halve the odds, and another ticket halves the odds once again.

In reality, you still only have x amount of lottery tickets in 15,000,000 chance of winning not x amount halving continuously since that would mean after 24 lottery tickets you would have a less than 1:1 odd of winning the lottery when instead it should be 24:15,000,000.

No, your odds making is quite a bit off.



no, you're the one that's a bit off. no one said anything about it halving each time... that only happens the first time when your number of tickets goes from 1 to 2. you are twisting what was said into something else. obviously they wouldn't double again with 3 tickets.

[Tarrbot]

I'm sorry. Why would it double the first time then? What pattern are you using to determine the halving of the first time?

You aren't. There is no mathematical concept of using the halving concept on the 'first ticket' when making odds.

If you are calculating odds of something, it has a mathematical construct behind it. I've shown two valid sources of the identical article explaining the concept behind drive failure. Storage Review and PC Guide.

Give me a mathematical explanation why that is incorrect, or just stop blathering bullshit.

[jbo]

I'm sorry. Why would it double the first time then? What pattern are you using to determine the halving of the first time?

You aren't. There is no mathematical concept of using the halving concept on the 'first ticket' when making odds.

If you are calculating odds of something, it has a mathematical construct behind it. I've shown two valid sources of the identical article explaining the concept behind drive failure. Storage Review and PC Guide.

Give me a mathematical explanation why that is incorrect, or just stop blathering bullshit.

it doubles because THE NUMBER OF TICKETS YOU HAVE DOUBLES. that only happens the first time when you go from 1 ticket to 2 tickets. think about it before you start with the insults.

as for the article... it doesn't really have anything to do with this particular argument, but it IS wrong for exactly the reasons that imashination stated earlier. hard drive life expectancy cannot be measured linearly... it follows a curve where drive failure becomes increasinly probably the older the drive gets.

[Tarrbot]

So at four tickets it halves again? Is that what you are saying? So instead of my original 24 tickets = a 1:1 odds it would be more in the tune of 48 tickets?

I'm sorry, that just doesn't make any mathematical sense.

Still, the point remains that I have given you two sources showing that drives fail in a higher rate in RAID 0 than was specifically mentioned.

Have another (http://www.redbooks.ibm.com/redbooks/pdfs/sg242510.pdf) (PDF file from IBM, page 75 where it states:
RAID 0: Mean Time To Failure = Mean Time to Failure of a Single Disk / Number of Disks in the Array.

If you think PC Guide didn't have their numbers right, maybe Storage Review and IBM can convince you.

I tried being nice about it earlier, but you're the one who originally insinuated that everyone else besides imashination was smoking something when the fact remains that he was incorrect in his original assessment.

[jbo]

So at four tickets it halves again? Is that what you are saying

yes.

So instead of my original 24 tickets = a 1:1 odds it would be more in the tune of 48 tickets?

i just don't understand what you are trying to say here.

I'm sorry, that just doesn't make any mathematical sense.
i assure you it does.

Still, the point remains that I have given you two sources showing that drives fail in a higher rate in RAID 0 than was specifically mentioned.

Have another (http://www.redbooks.ibm.com/redbooks/pdfs/sg242510.pdf) (PDF file from IBM, page 75 where it states:


If you think PC Guide didn't have their numbers right, maybe Storage Review and IBM can convince you.

I tried being nice about it earlier, but you're the one who originally insinuated that everyone else besides imashination was smoking something when the fact remains that he was incorrect in his original assessment.

your sources are all incorrect. they all make the same exact mistake. they are probably all copied from the same source that was incorrect to begin with. but really this has nothing to do with the argument at hand. i disagree with all those articles, but they have nothing to do with the lottery argument we are having.

[Tarrbot]

Fine, I'll concede the loss of the lottery argument because I really don't want to talk lottery when dealing with drives and subsystems and chances are I'm smoking that elusive crack that I can't seem to find around here.

You can disagree with the article all you want but I have shown you three direct and credible sources that say otherwise.

Is it so hard to believe that a RAID array can lose one drive prior to MTBF lifespan being spent?

Tell me why these articles are incorrect? Why do you not believe them? Is it some government conspiracy or something?

Technically, RAID 0 is no worse than RAID 5 or 50 or 01 when determining this. You have an array of x amount of drives. The fact that RAID 0 is bad in that it gives the false impression of redundancy when in fact it doesn't have redundancy at all like its brethren. The numbers don't lie. Ask any SysAdmin who manages RAID arrays about the drives that die on them.

Sure, there are many people out there that have good luck with RAID 0 arrays. A lot of them do. There are many more people out there who have had problems and likely blame it on 'cheap' crappy RAID adapters and move on when it's likely they were just bitten by the numbers game. <-- but enough anecdotal evidence.

You can argue all you want about these numbers, but until you explain why the numbers aren't valid, you're spouting bullshit.

[jbo]

Fine, I'll concede the loss of the lottery argument because I really don't want to talk lottery when dealing with drives and subsystems and chances are I'm smoking that elusive crack that I can't seem to find around here.

You can disagree with the article all you want but I have shown you three direct and credible sources that say otherwise.

Is it so hard to believe that a RAID array can lose one drive prior to MTBF lifespan being spent?

no it is not hard to believe. it is certainly possible, but the articles assume that it is AS likely as the drives dying after a normal lifespan.

Tell me why these articles are incorrect? Why do you not believe them? Is it some government conspiracy or something?


it's simple. imashination explained it perfectly in his first reply to you. the article assumes that a drive is just as likely to die in a week as it is in a year. this is obviously not the case. it does not take into account the fact that drives become more likely to fail the older they get.

if you have two drives that are supposed to last 5 years, chances are that they will both last near 5 years. the articles assume that it's more than likely that one drive will die within 2.5 years. this is just flat out wrong. i've owned probably 20 hard drives in my life and not one has ever died within 2.5 years. according to the odds they give, i must be the luckiest person on the entire planet.

[Framed]

So at four tickets it halves again? Is that what you are saying? So instead of my original 24 tickets = a 1:1 odds it would be more in the tune of 48 tickets?

I'm sorry, that just doesn't make any mathematical sense.

.

It halves everytime you double the amount of tickets from the amount of tickets you had before.

1 ticket = 1 in a million that you'll hit it.
2 tickets = 1 in 500 000
4 tickets = 1 in 250 000
(...)
1.000.000 tickets = 1 in 1

[maX_Andrews]

today will go down in history as the day all threads were derailed...

[imashination]

Aahaha, this is all highly entertaining :bounce: Being told you're wrong by people who lack basic maths you should have a firm grasp of by age 15. hold on, I'll go ask my sister to explain for you....

"RAID 0: Mean Time To Failure = Mean Time to Failure of a Single Disk / Number of Disks in the Array."

That equation is correct, but it is still meaningless and misleading, because the lifespans of the drives are still not linear. Lets try a simple example to see if we can get you to understand:

Say we have a mouse, the mouse will last for 1 million clicks, then the buttons will fail, theres a range of +/1 1000 clicks, but the 1 million click lifespan is pretty accurate. So it therefore has an average life of 1 million clicks. I buy 100 mice for my studio.... by your calculations:

1,000,000 clicks / 100 mice = 10,000 clicks before one fails

Can you see the problem yet? My mice which I know will last for 1 million clicks have suddenly given me an average failure time of just 10,000 clicks before one dies; even though I know this wont happen.

The harddrives are exactly the same, they will have an expected life, where most will last, they do not have a linear ramp of life expectancy, otherwise you would have a huge backlash as a quarter of the harddrives on the planet suddenly dont last past the first year.

If you don't understand how this works then please for everyone else's sake, stop replying. So far this thread has been bombarded with guesses, redneck maths and quotes about things you don't have the faintest understanding of.

[Carina]

If you have a 2% risk of one drive failing, and you multiply that risk for each drive you have (this was the reasoning in one of the earlier posts), then say you have 4 drives, and you do 2% * 2% * 2% * 2%, which is 0.02 * 0.02 * 0.02 * 0.02, that doesn't give you 16% i'm afraid:)

Anyhoo.. that reasoning never really worked anyway. What you're really doing there is calculating the risk that all the drives will fail.

[parallax]

On topic:

I have been running Raptors in RAID-0 since december 2003. Works like charm, and is fast enough for 2 streams of uncompressed SD, and i'm not even running decklink.

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.