Advice for storage and backup for animation?

#1

I’m looking for advice on what other C4D animators are using for external storage and backup of large amounts of data?

More specifially, I have been using OWC HDD raid 1s (mirrored) for years for external storage, with the idea that the redundant drive would protect my data in case one drive went down. Last week, however, my latest 10 TB raid 1 died (I believe it was the controller that went) so I had to take the drive to the shop for data recovery. Happily they got the data off okay, but the people at the shop–and advice online–maintains that “redundancy isn’t backup,” which I’m beginning to believe after this.

I produce C4D animation/video, which, of course, generates huge files and TBs of video and rendered tifs over the life of a project, which can last several years, so I need very large amounts of storage–larger than most SSDs I can find–as well as something with a relatively long lifespan, but also massive backup needs, since I don’t want to lose years of work!

My initial thoughts now are, if a mirrored raid isn’t really a backup, why bother doing it, since it really just slows things down and doesn’t truly “back up” your data? For my latest project, I will probably need at least 5 TB of storage. Also, since I will ideally be rendering animation frames to the external drive, speed is a factor, since writing to the drive on each frame factors into the time per frame. As for backup, I imagine the safest is probably offsite cloud backup, but will that be a problem where I generate files several GBs at a time? Would a “local” wireless backup drive be better for me?

My motherboard is an Aorus x399 Pro with 8 x USB 3.1 Gen 1 ports, 1 x USB type C 3.1 Gen 2, and 1 x USB 3.1 Gen 2 Type A.

Anyway, looking for any advice on this!

Thanks!

#2

With your modest data requirements you should easily be able to get away with 2x USB RAID boxes. This is what the data recovery people meant by redundancy isn’t backup. You need another back up. One of the boxes you work from and the other is the true back up. The second box should ideally be a 5 drive RAID5 enclosure but could also be a second RAID1 enclosure.

We have a SAN which has our archive on it which is backed up to LTO which we take home. We have RAID5 enclosures attached to our machines which contain our current project files and we’ll copy these files across the office. We drag files off these RAIDs onto our internal SSDs while working. At the end of a project we archive to the SAN and wipe the internal SSDs and RAID5s.

I’m sure someone could pick holes in what we do but in my experience the cost can run away with you very quickly if you try and cover every eventuality so we try and be pragmatic.

#3

Thanks, Infograph! This is helpful. I agree about being pragmatic–it seems you can go nuts if you let yourself. Just checking, but I assume that your internal working SSDs are dedicated for working files, and not the same SSD where your OS and applications reside?

#4

Yes we have separate OS and Media SSDs in our machines. We also have FireCuda SSHD in the machines that hold textures and general stuff. These hybrid drives work extremely well as they swap the most used files onto the SSD part of the drive so you get a good balance of storage and performance.

If your main OS SSD is big enough you can get away with using it as your work drive. The performance impact is negligible. I’ve cut 4k promos on my macbook pro internal SSD and it didn’t feel any different to editing with an external drive.

#5

Data and backups all in the same location are not desirable because of the possibility of disasters such as theft and fire etc.

Cloud backup is definitely desirable, or at the very least, ensure you make a duplicate of your data that is stored off-site.

I have set up Synology NAS with Cloud Sync (a Synology App), that automatically syncs the NAS with Backblaze Cloud Storage. I can recommend Backblaze B2 Cloud Storage as the most cost-effective based on research, but do your own of course.

#6

Thanks Infograph, I think the second internal SSD for “temporary” working files makes a lot of sense in terms of speed, and it would also mean I could back up onto cheaper (and slower) HDD.

Drew, thanks for the recommendation for a NAS/Backblaze solution, I’ll look into it!