«608 »
  • Post
  • Reply
evil_bunnY
Apr 2, 2003



Astro7x posted:

What would be a good way to back up a 16TB server offsite for a small business? My boss is starting to get concerned about liability of losing data in case of a catastrophic event, and even though we are backed up internally, if the building were to burn down we'd be kind of screwed. This isn't stuff that is archived, I'm talking about 16TB of data we need access on almost a daily basis.
Where's the data now? (desktops, SAN, NAS?)

Astro7x
Aug 4, 2004
Thinks It's All Real

DrDork posted:

Is this data that is continuously updated and time critical? Eg, banking data or something where you literally need backups running continuously. Or is this more like live/production data that you want backed up regularly, but you could stand to wait to batch process it at the end of the day/week, and if the place burned down it wouldn't hurt too much to lose a day or two of data?

Also, why would you need to access it on a daily basis if you're already backing it up internally--you've already got your working set, your backup set, and now you're looking at a third copy; I can't for the life of me think of why you'd actually need to read from said third copy unless both your first and second set died.

Additionally, how much new/changed data are you producing daily? I understand that 16TB is a lot of data to push over the net, but there's a world of difference between "16TB total collected over years with only a few gigs new/changed a day" and "16TB almost all of which rotates constantly."

Time critical in the sense that if the building burned down, we couldn't afford to spend over a week downloading all our data from an internet backup and would like a service that could send us drives or tape or something.

I don't need access to the third copy on a daily basis, or probably ever hopefully. I meant that we would be backing up data that probably gets accessed on a semi-daily basis but is usually created once and then just sits there not being modified. It's large video files. It could wait to be backed up at night after we leave for the day, it doesn't need to be backed up in real time as we are creating files.

The amount of new data that we produce on a daily basis is small (I should actually calculate this today now that you mention it). It's that when project starts there is a ton of data. Footage comes in, so we load the camera original footage, then it gets converted to a video editor friendly format. So for example... new project came in yesterday and the project folder was at 500GB of new data, I'd say that's average for when a project to start. Over the course of the next month we probably create an additional 200GB related to that project if I had to guess. It's all video files, media... just huge stuff.

Our internal backup is within the Raid. But the 'what if" scenario of the building burning down is what my boss is worried about which is not covered, which we would be liable of losing client project files.

evil_bunnY posted:

Where's the data now? (desktops, SAN, NAS?)

It's in a single 16TB SAN Raid, hooked up to a dedicated computer just for it. About 8 other computers access it.

evil_bunnY
Apr 2, 2003



Astro7x posted:

It's in a single 16TB SAN Raid, hooked up to a dedicated computer just for it. About 8 other computers access it.
What model? What's the setup?

What kind of hookup can you get at the office?

DigitalMocking
Jun 8, 2010

Wine is constant proof that God loves us and loves to see us happy.
Benjamin Franklin

Telex posted:

But a Roku can't play anything from your NAS, so that's a shitty suggestion.

Get the WDTV Live, or a Boxee. Skip the Roku because it's honestly a pile of shit compared to the WD, especially for the price.

It can't directly, but I run a Plex Media server anyway, and the Roku Plex app works flawlessly.

I've been really happy with it.

Astro7x
Aug 4, 2004
Thinks It's All Real

evil_bunnY posted:

What model? What's the setup?

What kind of hookup can you get at the office?

We use a Graintestore Raid from Small Tree (I can't find the model number, as I don't see it on their site anymore) that connects to a MacPro which we call the server computer, then we all connect to that machine via Ethernet San in some special way that Smalltree recommends for optimal speeds.

Astro7x fucked around with this message at 19:52 on Sep 12, 2012

Nystral
Feb 6, 2002

Every man likes a pretty girl with him at a skeleton dance.

Lipstick Apathy

For raidz under nas4free or freenas or whatever is it just format drives as zfs, add them to a pool them, then build RAID? Or am I missing something like a checksum drive or the like?

DEAD MAN'S SHOE
Nov 23, 2003

We will become evil and the stars will come alive


No, just stick them in, create a ZFS featuring all drives with Raid-Z and you're good to go. It's really user friendly.

titaniumone
Jun 10, 2001



Telex posted:

I didn't do XBMC, but I tried doing that sort of setup with a Plex server and FreeNAS running off an ESXi to stream to an Apple TV.

Kinda sucked when two people were doing it at the same time and a download was un-rar'ing with SABnzbd or SickBeard was doing its searches, I think mostly because I only had 8GB of ram in the machine.

It's tempting to try and re-do it with like a quad-core setup and a lot of ram but honestly the effort needed to make that all work seems to be stupid compared to just getting one of the set-top boxes to handle it all for cheap.

I'd blame ESXi, your disk controller, or your disks themselves.

My system is FreeBSD 9 (upgraded from 8, to 8.2, to 9 over the years). Using an LSI 1068E PCI-express 4x SAS card to connect 8x WD Black 2TB drives. 8GB of ram.

According to bonnie++, my RAID-Z1 pool gets about 500MB/s read and 400MB/s write, sequential.

I have a 4-port gigabit network card with link aggregation. I can simultaneously:
  • copy to my desktop at 100MB/s
  • copy to my roommate's desktop at 100MB/s
  • unrar downloads in SABnzbd
  • stream HD to XBMC on HTPC
  • stream HD to XBMC on roommate's AppleTV
without getting any slowdowns.

Nystral
Feb 6, 2002

Every man likes a pretty girl with him at a skeleton dance.

Lipstick Apathy

DEAD MAN'S SHOE posted:

No, just stick them in, create a ZFS featuring all drives with Raid-Z and you're good to go. It's really user friendly.

Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM.

titaniumone
Jun 10, 2001



Nystral posted:

Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM.

Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc.
code:
vfs.zfs.prefetch_disable=0
vfs.zfs.arc_min="1024M"
vfs.zfs.arc_max="4G"
vm.kmem_size_max="16G"
vm.kmem_size="12G"
No noticeable performance difference.

Nystral
Feb 6, 2002

Every man likes a pretty girl with him at a skeleton dance.

Lipstick Apathy

titaniumone posted:

Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc.
code:
vfs.zfs.prefetch_disable=0
vfs.zfs.arc_min="1024M"
vfs.zfs.arc_max="4G"
vm.kmem_size_max="16G"
vm.kmem_size="12G"
No noticeable performance difference.

Where do I learn more about this wizardry?

Longinus00
Dec 29, 2005
Ur-Quan

titaniumone posted:

Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc.
code:
vfs.zfs.prefetch_disable=0
vfs.zfs.arc_min="1024M"
vfs.zfs.arc_max="4G"
vm.kmem_size_max="16G"
vm.kmem_size="12G"
No noticeable performance difference.

To be fair this sounds more like a filesystem/kernel bug of some kind than an actual performance thing. I mean, disc cache exhaustion panicing the kernel?

titaniumone
Jun 10, 2001



Longinus00 posted:

To be fair this sounds more like a filesystem/kernel bug of some kind than an actual performance thing. I mean, disc cache exhaustion panicing the kernel?

Without arc_max set, ZFS will sometimes chew up all kernel memory, especially with bittorrent running.

edit: my point was that if you constrain arc_max too much it can limit performance but the settings I used have kept the system stable without any performance loss

titaniumone fucked around with this message at 01:05 on Sep 13, 2012

Longinus00
Dec 29, 2005
Ur-Quan

titaniumone posted:

Without arc_max set, ZFS will sometimes chew up all kernel memory, especially with bittorrent running.

As it should, it is a disc cache after all. The bug would be that ARC isn't releasing any memory (and causing a kernel panic of all things!) under memory pressure. I suspect this is freebsd specific issue though as I remember there were all sorts of issues with running UFS and ZFS at the same time before (I think it was something along the lines of UFS being unable to cache anything because the freebsd kernel couldn't differentiate between memory used by ARC and other kernel memory).

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

In new fucked up ZFS storage confusion, my N40L with NAS4Free that randomly picks "fast" and "slow" files has gotten even more confusing. I found a slow 9GB file that would transfer at 10MB/s, and started it transferring to a desktop, then I simultaneously started another transfer of the same file to a laptop. 10MB/s again. So the NAS is happy pushing 2x 10MB/s transfers of the same file, but won't do 20MB/s to a single client for this file, and will happily let other files zip by at 100MB/s.

What the hell is going on here?

evil_bunnY
Apr 2, 2003



Nystral posted:

Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM.
Others have said that but max it out if you can. ZFS *loves* ram.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

I can't wait for the 2TB WD Reds to become available, no one knows anything.

Is there a significant difference between the WD Blacks and RE4s, beyond TLER? I've a bunch of old 500GB RE2s still trucking with over 34000 hours, so I'm leaning towards the RE4s. But if the Blacks the same hardware, I'd rather save some money.

Civil
Apr 21, 2003

Do you see this? This means "Have a nice day".

You can buy them now, but Amazon's price isn't that great.

http://www.amazon.com/Western-Digit...e/dp/B008JJLZ7G

Since we're talking network storage anyway, I'm not sure why anyone would be going for super high-performance drives. My 2 year old WD green drives nearly hit the limits of gigabit ethernet.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

I'm in Belgium, pretty close to the German border. None of the shops in any vicinity in either country have any stock or delivery dates.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Combat Pretzel posted:

Is there a significant difference between the WD Blacks and RE4s, beyond TLER? I've a bunch of old 500GB RE2s still trucking with over 34000 hours, so I'm leaning towards the RE4s. But if the Blacks the same hardware, I'd rather save some money.
Sans TLER, my understanding of WD's lineup is that the Blacks and RE4's are pretty damned similar. The RE4's may have an edge in terms of reliability, but it's probably not by enough of a margin to justify a 30% price increase over the Blacks. Part of that price is also the "enterprise tax" where you're paying for drives that have gone through validation for this and that, and which are made for and marketed to companies with big 'ol tech budgets who aren't going to squirm over $50/drive.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

It's a bit less than 30% here. The Black is 167€ and the RE4 is 188€. I would have wanted to wait it out for the Reds, because they're just 133€ and still rated for 24/7. But they're not coming closer and my RAID1 is pretty much screaming "DO FUCKING SOMETHING!" at me.

r u ready to WALK
Sep 29, 2001



Keep in mind that the extra money for the RE4 also buys you higher performance compared to the Reds.
If the server is hidden away in the basement where you can't hear it the RE4 is an excellent drive, but if you keep it in your livingroom the Reds are great because they're nearly dead silent.
They also generate roughly half the heat, which adds up when you have lots of drives in a small room.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

For those of you looking to pick up some HDD's, NewEgg this weekend has the 2TB Samsung F4 for $99. I know that drive has been pretty popular with NAS-builders in the past, and it's a pretty good price these days.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams


DrDork posted:

For those of you looking to pick up some HDD's, NewEgg this weekend has the 2TB Samsung F4 for $99. I know that drive has been pretty popular with NAS-builders in the past, and it's a pretty good price these days.

Oh sonnuva...

WeaselWeaz
Apr 11, 2004

Life, Liberty and the pursuit of Biscuits and Gravy.

What is a good NAS solution that I can couple with cloud storage? I'm currently debating CrashPlan+, Netgear ReadyNAS Ultra, or both. My priorities, in order, are 1) Backup photos and financial data; 2) Backup music and videos; 3) Media server for music and videos (we have a separate HTPC that currently streams videos from my desktop). I'm concerned with the always on functionality of CrashPlan slowing down my wife's laptop, but it appears that some NAS servers can backup directly to CrashPlan (or other cloud services) instead. Also, if I have cloud storage I do not need mirroring so I can either have more space or wait to get a second HDD.

TaTaToothy
Jun 5, 2004
Hey Botth!

Astro7x posted:

I started this discussion in the Video Editing thread since I work for a small post house, but I guess it is more appropriate for this thread.

What would be a good way to back up a 16TB server offsite for a small business? My boss is starting to get concerned about liability of losing data in case of a catastrophic event, and even though we are backed up internally, if the building were to burn down we'd be kind of screwed. This isn't stuff that is archived, I'm talking about 16TB of data we need access on almost a daily basis.

I'm assuming sites like Crashplan and Carbonite are too expensive for that size, or too slow to transfer so much data over the net. Of the few I knew of and looked into, I was surprised to see that Dropbox Teams was the most affordable. But yeah, was hoping SHSC had something proved you knew of to recommend.

If it is a Windows server you might want to look at Dell AppAssure. You would need storage at both ends but it will do a good job with the backup replication.

Astro7x
Aug 4, 2004
Thinks It's All Real

TaTaToothy posted:

If it is a Windows server you might want to look at Dell AppAssure. You would need storage at both ends but it will do a good job with the backup replication.

Unfortunately no, we use Final Cut Pro so we are Mac based.

I threw some of these prices in front of my boss to just give him a general idea on how much it would cost to back up the entire server, and he though it would be much cheaper than what I showed him. Sounds like we might just end up backing up just the essentials instead (like project files, since assets could be redownloaded or recreated from those project files)

Devian666
Aug 20, 2008

Take some advice Chris.



Fun Shoe

Astro7x posted:

Unfortunately no, we use Final Cut Pro so we are Mac based.

I threw some of these prices in front of my boss to just give him a general idea on how much it would cost to back up the entire server, and he though it would be much cheaper than what I showed him. Sounds like we might just end up backing up just the essentials instead (like project files, since assets could be redownloaded or recreated from those project files)

Backing up project files makes a lot more sense. Backing up large amounts of raw footage or temporary video files is a lot more difficult. The offsite storage for my office is in the gigabytes rather than terabytes.

We do have a duplicate of our server on site which is in a fire rated room. If the server dies we could switch over to that server. If there is a fire and the fire rated room holds up then I would take the server off site to copy the data before the corrosion from smoke sets in (typically you have about a week before components start failing).

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Jesus Christ, the WD Black/RE4 must have platters made of depleted uranium. Compared to my other drives, they take long to spin up, and free wheel god knows how long when powering down. My Seagate finally gave the salute, so I took it apart. It also has four platters, yet seems way more nimble in spin up and down.

evil_bunnY
Apr 2, 2003



Combat Pretzel posted:

Jesus Christ, the WD Black/RE4 must have platters made of depleted uranium. Compared to my other drives, they take long to spin up, and free wheel god knows how long when powering down. My Seagate finally gave the salute, so I took it apart. It also has four platters, yet seems way more nimble in spin up and down.
The motors are probably weaker to save energy and heat. It also means they're easier to freewheel when the magnets aren't powered. Neat

Nam Taf
Jun 25, 2005

I am Fat Man, hear me roar!



So both as a general FYI and some questions, I figured I'd retell my story of replacing my raid-z1 hard drive that was starting to get faulty. For record, the zpool is 4x3TB drives in an N36L microserver.

I had a drive (Hitachi 5k3000) that'd decided to get a significant amount of reallocated sectors (31/100 left, with 5 being the threshold) and pending sectors. This was causing the zpool to hang when accessing it in certain ways if it hit that bad data. If I dropped it out and ran degraded, it was fine.

So I got a new hard drive (WD green 3TB, I know, but it's by far the cheapest at $40 less than a WD Red and they didn't have Red drives in stock) as a stop-gap til the 5k3000 is RMAed, at which point I'll use the WD as a cold spare.

In an effort to take the load off the other 3 healthy drives, I decided to take the approach of running ddrescue across the broken drive on to the new drive, as the vast majority of the drive was still fine. This process got about 2 or 2.2 of the 3TB done in the 6 hours overnight as I slept, and then started to grind right down and take another 24 hours after that to complete (I woke up this morning to it completed).

I could've just resilvered the array, but that runs the risk of throwing an error on a second drive. Not necessarily fatal in my situation but I'd rather put the strain of reading 3TB of data on the drive that I'm about to replace, then resilver only the broken parts. I also wanted to see how zfs would handle a dd'ed drive replacement as opposed to letting it resilver on to a clean slate.

ddrescue went fine and there was 151kB of data that was lost, as expected. I plugged back in the 3 good hitachis and the WD, fired the server back up and imported the array again. It's displaying and reading perfectly, showing a healthy status, so it seems to have accepted the WD drive perfectly in place - I think partially by luck because the WD drive is /dev/ada3 which is what the broken hitachi was too. I'm now scrubbing to resilver the lost 151kB of data.

I guess my one question from this is does anyone expect this to pose a problem? Everything seems to be working fine and maybe I just fluked it with the device names working out, but I just want to confirm whether anyone can think of a reason why this might go to shit on me?

Oh, and lastly, the dd process was relatively gentle on the drive. As this is a WD with its infamous head parking fun, it will load cycle if I read with 8 second pauses. Even still, ddrescue only incurred 52 load cycles over the entire process of 3TB:
code:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       10
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       31
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       10
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       7
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       52

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

ZFS uses EFI to wrap the disk and stores and UUID. You could have attached it via USB and it would have worked. ZFS is pretty cool, too bad Oracle won't let it leave its home turf and keep that farce of BTRFS going.

A Windows IFS driver would also be pretty rad. Storage Spaces is limited as fuck and probably won't see a big improvement outside the Windows release cycle.

caberham
Mar 18, 2009

by Smythe


Grimey Drawer

So I'm just wandering around the computer mall today to build my own freenas server, anyone got any tips for itx enclosures? Sure they look small but I'm missing the slot loading feature from the set top nas and N40L.

Is a h77 chip set and i3 good enough for Usenet, VPN, plex media repository? Or am I good enough with an atom. Or just buy a N40L and load free nas. I also have a synology 2 bay at home and plan to add sabnzd to it.

My dream machine is a 10 bay atx goon treasure chest. What CPU do I need?

text editor
Jan 8, 2007



caberham posted:

So I'm just wandering around the computer mall today to build my own freenas server, anyone got any tips for itx enclosures? Sure they look small but I'm missing the slot loading feature from the set top nas and N40L.

Is a h77 chip set and i3 good enough for Usenet, VPN, plex media repository? Or am I good enough with an atom. Or just buy a N40L and load free nas. I also have a synology 2 bay at home and plan to add sabnzd to it.

My dream machine is a 10 bay atx goon treasure chest. What CPU do I need?

Usenet un-rarring and par2 checks would be pretty slow on the atom, plus the Atom locks you into 2-4GB RAM. Might as well get at least an i3 and load it with cheap RAM.

Cervix-A-Lot
Sep 29, 2006
Cheeeeesy

No idea where to post this but I need some help.

Boss at works wants a file server for the company. He wants raid 10 at least 2tb of space and able to auth users from a Windows 2008 domain controller. I don't know shit about Raid and file servers so I need help. We currently have some cheap Seagate NAS device but it's already failed on us twice.

Any suggestions would be great.

FISHMANPET
Mar 3, 2007

Sweet 'N Sour
Can't
Melt
Steel Beams


So, 4 3TB diks in just about any Dell server. Pretty much any 2U server should be able to hold 6 drives (2 mirrored system drives, 4 data drives). Install Windows Server, join the domain, ???, profit.

E: Holy shit this is a pain in the ass to build on Dell's website, but it looks like an R520 can do it for around $5k, though I fucked up the RAID math in my head and you'd be fine with 2 TB or even 1 TB drives.

FISHMANPET fucked around with this message at 03:09 on Sep 21, 2012

evol262
Nov 30, 2010
#!/usr/bin/perl

Easy Mac posted:

No idea where to post this but I need some help.

Boss at works wants a file server for the company. He wants raid 10 at least 2tb of space and able to auth users from a Windows 2008 domain controller. I don't know shit about Raid and file servers so I need help. We currently have some cheap Seagate NAS device but it's already failed on us twice.

Any suggestions would be great.

The previous answer is the correct one. Get the cheapest (enterprise, prosumer, or something that's not a ReadyNAS or something) server you can get and put drives in. It'll outperform a Drobo, QNAP, or Synology, with a lot more features, and you can do whatever you want with it. Plus it'll probably cost less.

Longinus00
Dec 29, 2005
Ur-Quan

evol262 posted:

The previous answer is the correct one. Get the cheapest (enterprise, prosumer, or something that's not a ReadyNAS or something) server you can get and put drives in. It'll outperform a Drobo, QNAP, or Synology, with a lot more features, and you can do whatever you want with it. Plus it'll probably cost less.

Assuming the person who posted the question (aka Mr. "I don't know shit about Raid and file servers") is going to have to maintain this maybe the extra money spent on a dedicated device with support will be worth it.

evil_bunnY
Apr 2, 2003



Yes you need support, no you don't want a drobo-like device. Just get a dell big enough for the amount of drives you need and get on with your life.

caberham
Mar 18, 2009

by Smythe


Grimey Drawer

text editor posted:

Usenet un-rarring and par2 checks would be pretty slow on the atom, plus the Atom locks you into 2-4GB RAM. Might as well get at least an i3 and load it with cheap RAM.

Thanks I'm looking at the HP proliant m110 series http://h10010.www1.hp.com/wwpc/us/e...5942.html?dnr=1

Load it with 32gb ram and stuff it with 4tb disks. And add freenas. Costs a pretty penny but still cheaper than the ready made nas. That should do the trick right?

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »