Backup those files…. but how

We all push around some pixels and we all have to store that amount of files somewhere.
To be honest there is no 100% solution that works for everyone, but over the years I think I found one that might benefit a lot of people out there and that’s easier than you might think…… and a lot cheaper than other solutions 😀

Video or Photo, were did those files go.
Let’s first discuss proper storing of the files.
You probably know the problem about not being able to find a file or a project, and you also don’t remember which year it was……… well that can be a real problem, especially when your customer is breathing down your neck. However it shouldn’t be a problem.

DAM / Digital Asset Managment
For some people storing everything on a drive by year, month, day makes sense. They probably have a great memory of when something happened. For other people they store everything by name, and again others just plug in drives until they are full. DAM you say ?

In reality it’s a difficult word for something vital and simple. It’s keeping track of your files and finding them back. You can do this in many different ways but the easiest is to use a special program for it. The two choices I find best are Aperture 3 and Lightroom, myself I’m an Aperture user.

Within Aperture and Lightroom you can give any file (or a whole bunch of files) keywords, they are stored within the database or the file (depending on settings and program) and this way you can always find a file back very quickly. However don’t trust the programs too much. For example in Aperture you can choose to just copy everything in the Aperture database, big NO !!!! If for some reason Aperture crashes or screws up your database you are lost and left with a lot of files were you can’t make any sense off. What you can do, and what I advise, is to work with referenced masters. They can be stored on your harddrive in any order or form you like and when Aperture or Lightroom is not usable for some reason you can still find them and use your library.

Backing up everything.
This can be expensive if you follow the commercial route, or in other words if you believe everything can go wrong and want to be sure. Well sorry to say, you can never be 100% sure, so why spend a lot of money on materials that bring you into a load of pain when they crash. I’ve worked with several raid based NAS solutions and trust me when the device crashes it’s a lot of work to get your files back from the drives, when one drives crashes it’s often an easy task to get things back. So what did I do ?

Keep it simple.
Why spend more money than needed ?
Let’s say you need 8TB of storage (which is on the low side for people like me, but on the high side for most), that would mean you need 4x a 2TB drive to store everything. If you want this in a NAS you are automatically pointed towards the more expensive devices. And than you have 1.

Why not keep it simple ?
There are several companies out there that sell so called storage towers, look for example at Addonics (the brand I also use), they have several towers including some for the really big needs. Our example person would need one tower with 4 drive slots. Add to this tower a port multiplier and add the same card to your PC or Mac and you have one tower with 4 drives and one e-sata cable running from that tower to your PC/Mac. All drives are seen as different drives and the speed (as long as you use one drive at a time) is blazing fast. Now we have 4 drives, but no backup I hear you think……..

The easy rule of harddrives.
The easy rule of harddrives is that they ALWAYS brake, however it’s never known when, but it’s always at a time you don’t want it. So let’s make sure that this rule will not bring us into trouble. Remember the one tower ?
To make sure we have a good backup add an exact same tower, this is your mirror. There are several programs on both the Mac and PC that let you sync between several harddrives. Personally I sync every day at the end of the day and I can this way never loose a file because I also use something else…….

For the big needs….. the file server
Remember those from the old days…. the file server.
This was the mystical big machine that made a lot of noise and was locked up in his case (his own personal room) with airconditioning and you needed a keycard to get in… (ok sometimes I overact :D)
However the files server can still be used. In todays market with 1GB network speeds as the standard and fiber and other options overing even more speed, there is no reason why not to use something with a network adaptor. Same rules applies however to the server. Keep it simple. Don’t chain drives to make it one volume, when one goes they all go. Make sure you have an external tower as mirror…. Why external ? simple when the machine fails it could in some cases fry your drives and with an external tower you could opt to power that down after every sync.

The advantages of a good file server are plenty.
1. costs can be really low compared to NAS solutions that are sold to photographers.
2. there is no real limit to drives, expect when you run windows where there is a limited amount of drives letters (but also that can be solved)

3. It can do many things more than just store the files of your photography/video, think about movies, music etc.
4. It can double as a security unit with webcams connected.
5. Take over the desktop to control the machine from any location, so even when abroad you can reach your files.
and much much more.

Also copying to a new drive is much easier, just plug it in and copy the data the fastest way possible, remember with a NAS the only way to copy data is back to another NAS or to a PC/MAC and than back, a lot of work when you have a large archive.

For you ?
As mentioned at the start it really all boils down to what YOU need.
Some people only use one harddrive at a time and love the Sharkoon Quickport solution which is great by the way, but if you need more space (and in the future we all will) it’s wise to invest in a future proof solution and although one can chain several NAS boxes to each other it’s still a closed system that is not so easy to upgrade or rescue (and try to backup 10TB over the netwerk with most NAS units, you will not like it), in the end it will cost you a lot of money and you will have a lot of boxes. In the past our data stream was always smaller than the biggest harddrives, or in other words by the time you ran out of space there was a bigger harddrive. lately however that has stopped and until they release a 5TB harddrive a lot of photographers and video guys will have the need for many harddrives, and for me the safest way to work is with a mirror, and the best way to have a lot of drives available for the cheapest price is the file server.

I hope this small article gives you something to think about, and I would love to hear your thoughts and your solutions, so press the comment button and join the conversation.

3 replies
  1. Kay
    Kay says:

    While I do agree with your keep-it-simple strategy, I would not exclude RAID from the mix. In your scenario, if your single drive dies before you sync, you still lose the day’s work.

    I would like to comment on RAID-5 and RAID-1. Both these are business continuity tools. They’re really there so that, when a drive fails, we can tell users to save their work and go take an extended smoke break while we rebuild the RAID. It’s not meant to allow us to keep working on a degraded RAID. On RAID-5, this is particularly important, because if a second drive fail (as often happens if the drives are from the same batch), you lose everything.

    The other consideration is that drives have gotten a lot bigger, but not (much) faster. Rebuilding a RAID-5 with three 2TB drives takes a *very* long time. Incidentally, Drobo, which is so popular with photographers, suffers the same problem – they dress it up in fancy language. If a drive dies, or if you yank out a drive and put a bigger one in, you have no redunancy until it’s finished rebuilding.

    For continuity, with really big drives, I like RAID-1, preferrably with three drives in a set. That way, if one drive fails, you don’t lose redundancy. If two drives fail, you still have your complete data set on one drive, and it can be easily recovered.

    Of course, this only deals with the redundancy issue. You still need to back up your data.

    –K

  2. Frank Doorhof
    Frank Doorhof says:

    @Kay,
    I agree a raid solution is safe.
    However I store the files temporary also on a hdd in our mediafileserver so that when I sync it can be deleted, before I sync it’s still there.

    The worked on files are stored on my local computer (the machine I work on) and when they are copied to the tower they are immediately synced.

    For me this workd, and I don’t want to give up any harddrive space for a Raid solution 😀
    But it varies of course per situation, there is no one size fits all in backup I’m afraid.

    Keep the suggestions coming, a lot of people struggle with this issue.

Comments are closed.