Be Excellent To Each Other

As I do more and more work, I get more and more annoyed... but that's another story.

The result of me doing more work is that I'm having to back up more and more files. I currently use external hard drives and keep everything off my main PC and backed up.

I believe (but I'm not sure) that NAS storage could help here so, without going to Google, could someone please help and explain what it is and then I'll be able to see if it's suitable.

NAS stands for "network-attached storage". Basically it's a box that holds X number of hard drives and runs a little OS that allows it to connect to your network and serve files up.

Ah cool, thank you. So, I could potentially have this little box attached to my router, which is in turn attached to my PC and have it running so that my files can be stored on it? The little OS, does that back stuff up automatically or would I need to tell my PC to send files to it? I'm guessing that there's software that would do this for me.

Also, the little box... would it need a monitor/keyboard/mouse permanently attached to it or could I access it via my current PC?

You attach it to the router, and it will make itself available to everything on your network. It will not back things up automatically - it can't see the files on your PC. It doesn't need any peripherals, you drive it from your PC.

Some NAS boxes can do more stuff, like run web servers, torrent clients or make video available to your TV.

Thanks Grim... That's all very useful!

I'm guessing that I'll have to buy the box and hard drive(s) for it separately?

With any modern NAS you basically just connect your shiny new box of tricks to your router using a standard Ethernet cable and turn it on, it'll then grab an IP address from your router and present itself on your LAN, and you can then access it by typing in the default URL into any web browser.

Any decent NAS will essentially offer a full 'Windows style' experience in a web browser, everything is GUI driven, so can set up your shares and permissions and all that stuff in a web browser. Most also support external drives so you can also hang external drives out of the NAS (they generally have a couple of USB ports), and then access them on the network rather than directly from your PC.

I recently bought a Synology NAS which had a lot of the spadework already done, it has 2x4TB drives and it was pre-configured with a single 4TB volume using RAID-1, which means that the two drives are mirrored, so if one drive in the NAS fails, I don't lose all my data. All I had to do was set up my shares, users, and permissions.

Modern NAS boxes are generally fairly capable from a hardware perspective (featuring dual/quad core CPUs, and 512MB to 1GB of RAM), so can run all sorts of other swizzy stuff if you want, such as media servers, do hardware transcoding and whatnot.

In some regards it would be more useful to think of them as 'mini computers' or 'mini servers', rather than a dumb box with a couple of hard drives in them.

As for backing stuff up to the NAS, you can either use the built in Windows backup, a free bit of software such as VEEAM, or most NAS manufacturers offer their own backup solution along with the NAS, which can often be installed in the form of an app.

This is the fella I got earlier in the year, and I've been very impressed with it - https://www.ebuyer.com/830170-synology- ... 18j-8tb-iw

That's really useful... Thank you.

Can that one be upgraded with more drives or has it only got space for two?

Nope it's only a two-bay NAS and it comes full populated, so to upgrade the storage I'd have to yank the drives and replace them, which is so much of a faff it becomes not worth it. (Nearly all NAS boxes run on Linux, so the filesystem will not be even remotely recognised by a Windows machine. The NAS itself presents stuff on the network that Windows will recognise. So if I wanted to keep my data when replacing the drives, I'd have to copy everything off the NAS to somewhere, replace the drives, and then copy everything back.)

So in the case of a pre-populated NAS, pick the one that has the capacity you need now, and for the next 3-5 years or so. (Depending on how long you want it to last for before upgrading. I actually got ten years out of our last NAS because I bought a 2TB model when that was really rather over the top for the time.)

You can buy an empty NAS bay which is the box and the 'guts' of the NAS, but no drives, which you have to purchase separately. These can have anything from 2 -10 bays, so you could maybe fill half of the drive bays early on, and then add more drives later on to extend the capacity, but you are getting into some fairly serious cash at that point.

A decent recommendation IMO is to pick a pre-populated NAS from a major manufacturer such as Synology, and size it right to give you room to grow over at least three years.

The perfect NAS is one you do all the initial set up on, get it how you want it, and then you never need to think about it again, because it just sits there and does it thing without fuss or failure.

Also note that if you're talking about moving hundreds of gigabytes of data around, you really want to be on a Gigabit Ethernet connection, so that's a a Gigabit NIC in your PC, a Gigabit LAN port on your router, and a Gigabit NIC on the NAS.

I also think you'd be best off with something off the shelf (despite having self-built my last new NASs). A mid-range Synology model, maybe 4-6 bays, would likely be a good fit for you. Probably not a two-bay. Do you know what sort of storage size you need?

You also need to think about RAID levels, which is how the data is stored across the disks: https://uk.pcmag.com/storage-devices/79 ... -explained You probably want RAID-5 for your NAS.

You also need to remember that -- and I cannot stress this enough -- none of this is a backup. A home NAS gives you a cheap and convenient way to build a big hunk o' discs. With RAID, you can get some protection against failing discs. But you are not protected against power surges, theft, fire, data corruption, accidental deletion, ransomware, and other mishaps. As this data is now literally your livelihood, you need a better disaster management plan than "I will store the only copy I have on this NAS and that's all."

A lot of the older NAS's will not handle things that quickly - I have the much older version of the one you've posted but even on yours it has



Yes you can stream media from it / store stuff on it / stream backups to it etc etc but dont expect that quick a response and they are generally 'slow' to move lots of data around (well slow in comparison to other things)

I just checked back and mines is this one

https://www.amazon.co.uk/Synology-DS411 ... B0049MPQGS

Bought in 2012 - I've had one disk go bad but it had RAID and I just replaced it with a similar drive and off it went
Media streaming and backup (locally to disks and cloud backups) is all we use it for but its been doing that pretty much daily for the last 7 years.

No, Synology's OS allows you to replace drives in place and thus expand the storage array: https://www.synology.com/en-uk/knowledg ... place_disk

Well that's the limit of Gigabit Ethernet, and mine can saturate that no problem, I've seen sustained reads and writes of just over 110MB/s when copying to and from it.

TBH I'd expect any modern NAS with decent drives and Gigabit Ethernet to be able to manage that. Obviously things will be slower with lots of small files, but that's the same with any file system.

FYI the Synology at least has cloud backup options where you can set a time and let it upload stuff to a cloud solution - I have that set on mine for pictures because there were not great options for this when i was looking before and I had my photo's syncing between my desktop , a folder on the NAS and then backed up online via the NAS overnight.

Nowadays my pictures sync on my phone to a cloud backup solution without having to go through the middleman of moving them to the NAS first.

That's pretty nifty, and I did not know about that.

Cheers Doc, something to bear in mind for the future

Thanks everyone for the input. I do appreciate your thoughts. They're a bit more expensive than I thought they'd be but pretty essential by the way things are going.

@Doc you are right about backing things up, I need to get better with this and perhaps go out of my way more to build up my backups. My mom has a fire proof safe which I should really take advantage of at the very least.

Investigate leasing options as it’s tax deductible and will protect your cash flow

Yes I've seen a scary number of people who think that anything they put on an external hard drive or a NAS immediately becomes a 'backup' just because it's not on their PC or laptop.

If it's the only copy of the data, then it's not a backup!

As DocG also notes, RAID arrays are great for protecting against failed drives, but that's it, a million other things can still wipe it out. (I'm super paranoid so I keep an encrypted hard drive at a different property of all the family critical data, on top of all the local backups, to protect against the 'house burning down' scenario.)

Right, we're going about this wrong. We're brainstorming solutions but we don't know the problem.

What data are you dealing with? I would guess, but could be wrong:
1) some amount of admin stuff: bookkeeping, emails, invoices, project briefs
2) raw footage
3) finalised output, ready to send to client
4) anything else?

What is the lifecycle of the data? I'm guessing
1) in-flight; anything until the client signs off
2) archived; after that point
Does raw footage get deleted or archived forever?

What are the consequences of losing each type of data at each stage? The one of most interest here is raw footage in the archive stage, I think. Specifically: can it be deleted? And I imagine losing anything during the in-flight stage is a disaster.

How big, per project, are these data types? How many projects are you booking per year? We need a capacity plan.

Not as good as many think. Buy four drives of the same model and vintage. Run them through the same workload until one fails. Replace the failure and do a RAID rebuild; this puts the remaining three disks through a punishing workload for several days or up to a week depending on various factors like data size. Data is thin but what there is suggest you have maybe 10-15% odds you'll kill a second disk before the array rebuilds. This is why serious operators like hot spare disks.

Oh god yeah, we’d often have situations where a drive would need to be replaced, then another, then sometimes another, all in the space of a few days. And this is in arrays with 100 disks, perhaps.

Not now, though. I no longer have any dignity

I’ve just gaslighted you into believing that.

Pretty much all of this for my business but I guess I'd be storing ISO's and ROMs there too.. Oh, and family pictures.

Again, spot on. At the moment I keep all raw footage forever and I'd like to keep on doing that for as long as possible. Obviously, some completed projects can get archived and never touched again but it would be nice to keep them just in case.

A couple of months ago for instance, I had someone come to me and ask if I still had the videos we did about 18 months ago. It's nice to say 'yes' when this happens.


You are right. Anything that I'm working on that gets lost would be a disaster. If I lose completed projects after they're sent to the client and signed off then like wise, that would be pretty bad. Plenty of times things have been signed off before they come back with one more change. So let's presume losing any kind of data would be a disaster.


As you can imagine, the size of the files depends on the job. For example, the job I'm currently working on is pretty lean at a mere 10gb so far. It'll probably be 12gb when it's done. The largest job so far was around 260gb but let's say around 50gb per job. On average I think we'll look at 50 jobs per year.... I think?!?

Right, then, I think you want a three-pronged approach.

Projects in flight continue to live on whatever computer you are using right now. That's the fastest storage you have. (If you don't have fast storage in there -- ie SSDs -- consider the productivity gain from adding some.)

This computer is regularly backed up to a NAS you keep in your house. For this I'd suggest:
* a four-bay Synology unit -- these run £375-£500 depending on model. I'm not clear on the differences between models.
* 4x Western Digital RED NAS drives. 3 TB drives are around £90, 4 TB drives are around £110 (larger sizes are also available) You want at least the 3 TB ones.

In a RAID-5 array, with 3 TB drives, you'd have 9 TB of total space. That's about three years of projects at your guesstimate. You might want to pad that by using larger discs, however.

In addition to backups of in-flight stuff, the NAS is also your primary archive storage point. So once a project is done and delivered, you move it off from the computer (freeing up your fast storage space) and onto the NAS (where you can still get at it at any time.)

Then you have a third tier, which is off-site backup for the NAS. For this I'd suggest a cloud service, like Amazon Glacier, Google Cloud Storage Nearline, Synology's own C2 (I've heard good things about this), Backblaze's B2, or similar. These are storage services that are optimised for backups that you read from very infrequently. They tend to cost around $0.01 per GB per month. So a full backup of 9 TB would cost you $90/month. Configuring of this is quite easy; you can click buttons in the Synology UI and tell it to copy snapshots up to the cloud regularly.

There's some fine print to be aware of here and further research would be wise before you commit. For example, you also pay to copy stuff in. With at least one of these options -- Glacier -- you pay to copy stuff out, and it can become very expensive (like, hundreds of dollars to do a full restore if you needed it urgently.) On the upside, Glacier is one of the cheaper services (it's around $0.007 per GB per month.)

Forgot to mention: how fast is your internet? It matters for the cloud backup. And potentially it could really hurt if you need to restore a backup in a hurry. Some backup vendors will post you a hard disc if you ask them.

Thanks for all this Doc. I appreciate you taking the time and there's definitely some good options there, and more importantly they all make sense.

My internet by the way is 104Mbps download and 6Mbps upload with Virgin.

Translation: fucking hell this is going to be way more expensive than I first thought

Well, yes. This as well... but, you get what you pay for I guess!

yes but

That's not really sufficient upload speed to be moving any significant amount of data to the cloud, unless you're happy to kick things off overnight and just leave it going. (You'd be talking about ten hours to upload a single 50GB project, which would saturate your upload for the duration and impact on other users on the same internet connection.)

DocG's outline plan is solid, but as you can see, the costs start racking up once you commit to a proper belt and braces approach for securing large amounts of data.

Thing is, if this is Vision's business, he needs to treat it like a business and expect these things to cost business rates, not home rates.

There’s a few points where you can choose to flex the cost if you can accept more risk. I imagine most of that cost is coming from the decision to fully back up all raw forage forever. If you must cut corners, I’d start there: either don’t fully back it up (eg keep it on the NAS but not in the cloud, accepting the risk of losing it) or only keep it for a fixed time window of a year or two.

Another option as an alternative to cloud backup is keeping files on an external HDD at a relative’s house. Needs a lot more discipline from you to regularly refresh it, though. As opposed to a cloud solution which is set-and-forget.

I suspect Virgin will give you more speed if you ask for it.

Another option is to compress the video once it gets to a certain age.

And don't forget to get the VAT back!

Factor the cost of a 1TB hard drive into every job and give the client the raw footage. Then it's on them if they want a re-edit.

That's a pretty good idea, after all, the project is completed when the finished product is handed over.

I can see it's a 'nice thing to do' from the customer's perspective (shoulder the costs and hassle of storing the complete project, raw footage and all, forever), but adding on £50 as an optional extra to the cost per project for a portable drive containing all project material, to be handed over at completion, is a neat way of addressing it.

Depends on the vat scheme you are on
I suspect he will be on a flat rate scheme, which means he can't claim the vat back for purchases under £2k

Convenient timing. 8TB Buffalo NAS for a reasonable sum.

https://www.ebuyer.com/630704-buffalo-l ... ent=630704

Two bay though, so it's really a 4TB unless you're brave.

I've heard enough bad things about Buffalo to give them a swerve, they're certainly cheaper than the likes of Synology but I was put off.



It depends if the NAS is a second copy of everything though, in which case all you're losing by utilising the full capacity is protection against a drive failing. We've already established that a NAS by itself, even with two mirrored drives, isn't a backup.

Depends on how much hassle it's going to be to get the stuff from your primary backup if you lose a drive. 4TB is a spicy meatball.

Just for reference I have a 4-bay D-Link ShareCenter with 16TB, and it's very nice. Discontinued now though, so you can't have one.

And it's practically full

This.


My self-built six-disk 24 TB FreeNAS machine is pretty nice, y'know. Although I have two redundant disks, so only 16 TB usable space.

My *huge* 2TB media drive is approaching capacity. I'm tempted to overkill it with something huge. But the bank manager disagrees.

Actually I do have a NAS. It's a Netgear STORA MS2000 2-bay from 2011. I can't remember why I stopped using it.

cs,b

Speed, I suspect.

Nah, I don't do drugs

This is a nice idea but I wouldn't want the customer to have all the raw footage in case they take it all to a competitor and get them to edit it... or worse still, have a go themselves!

But also making it easier for the customer to use someone else for video editing that project in the future.

o/