|
Astro7x posted:What would be a good way to back up a 16TB server offsite for a small business? My boss is starting to get concerned about liability of losing data in case of a catastrophic event, and even though we are backed up internally, if the building were to burn down we'd be kind of screwed. This isn't stuff that is archived, I'm talking about 16TB of data we need access on almost a daily basis.
|
![]() |
|
DrDork posted:Is this data that is continuously updated and time critical? Eg, banking data or something where you literally need backups running continuously. Or is this more like live/production data that you want backed up regularly, but you could stand to wait to batch process it at the end of the day/week, and if the place burned down it wouldn't hurt too much to lose a day or two of data? Time critical in the sense that if the building burned down, we couldn't afford to spend over a week downloading all our data from an internet backup and would like a service that could send us drives or tape or something. I don't need access to the third copy on a daily basis, or probably ever hopefully. I meant that we would be backing up data that probably gets accessed on a semi-daily basis but is usually created once and then just sits there not being modified. It's large video files. It could wait to be backed up at night after we leave for the day, it doesn't need to be backed up in real time as we are creating files. The amount of new data that we produce on a daily basis is small (I should actually calculate this today now that you mention it). It's that when project starts there is a ton of data. Footage comes in, so we load the camera original footage, then it gets converted to a video editor friendly format. So for example... new project came in yesterday and the project folder was at 500GB of new data, I'd say that's average for when a project to start. Over the course of the next month we probably create an additional 200GB related to that project if I had to guess. It's all video files, media... just huge stuff. Our internal backup is within the Raid. But the 'what if" scenario of the building burning down is what my boss is worried about which is not covered, which we would be liable of losing client project files. evil_bunnY posted:Where's the data now? (desktops, SAN, NAS?) It's in a single 16TB SAN Raid, hooked up to a dedicated computer just for it. About 8 other computers access it.
|
![]() |
|
Astro7x posted:It's in a single 16TB SAN Raid, hooked up to a dedicated computer just for it. About 8 other computers access it. What kind of hookup can you get at the office?
|
![]() |
|
Telex posted:But a Roku can't play anything from your NAS, so that's a shitty suggestion. It can't directly, but I run a Plex Media server anyway, and the Roku Plex app works flawlessly. I've been really happy with it.
|
![]() |
|
evil_bunnY posted:What model? What's the setup? We use a Graintestore Raid from Small Tree (I can't find the model number, as I don't see it on their site anymore) that connects to a MacPro which we call the server computer, then we all connect to that machine via Ethernet San in some special way that Smalltree recommends for optimal speeds. Astro7x fucked around with this message at 19:52 on Sep 12, 2012 |
![]() |
|
For raidz under nas4free or freenas or whatever is it just format drives as zfs, add them to a pool them, then build RAID? Or am I missing something like a checksum drive or the like?
|
![]() |
|
No, just stick them in, create a ZFS featuring all drives with Raid-Z and you're good to go. It's really user friendly.
|
![]() |
|
Telex posted:I didn't do XBMC, but I tried doing that sort of setup with a Plex server and FreeNAS running off an ESXi to stream to an Apple TV. I'd blame ESXi, your disk controller, or your disks themselves. My system is FreeBSD 9 (upgraded from 8, to 8.2, to 9 over the years). Using an LSI 1068E PCI-express 4x SAS card to connect 8x WD Black 2TB drives. 8GB of ram. According to bonnie++, my RAID-Z1 pool gets about 500MB/s read and 400MB/s write, sequential. I have a 4-port gigabit network card with link aggregation. I can simultaneously:
|
![]() |
|
DEAD MAN'S SHOE posted:No, just stick them in, create a ZFS featuring all drives with Raid-Z and you're good to go. It's really user friendly. Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM.
|
![]() |
|
Nystral posted:Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM. Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc. code:
|
![]() |
|
titaniumone posted:Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc. Where do I learn more about this wizardry?
|
![]() |
|
titaniumone posted:Get 16. Even in FreeBSD 9.0, ZFS still loves to eat a shitload of memory. My system was kernel panicing every 30 days or so until I limited the arc. To be fair this sounds more like a filesystem/kernel bug of some kind than an actual performance thing. I mean, disc cache exhaustion panicing the kernel?
|
![]() |
|
Longinus00 posted:To be fair this sounds more like a filesystem/kernel bug of some kind than an actual performance thing. I mean, disc cache exhaustion panicing the kernel? Without arc_max set, ZFS will sometimes chew up all kernel memory, especially with bittorrent running. edit: my point was that if you constrain arc_max too much it can limit performance but the settings I used have kept the system stable without any performance loss titaniumone fucked around with this message at 01:05 on Sep 13, 2012 |
![]() |
|
titaniumone posted:Without arc_max set, ZFS will sometimes chew up all kernel memory, especially with bittorrent running. As it should, it is a disc cache after all. The bug would be that ARC isn't releasing any memory (and causing a kernel panic of all things!) under memory pressure. I suspect this is freebsd specific issue though as I remember there were all sorts of issues with running UFS and ZFS at the same time before (I think it was something along the lines of UFS being unable to cache anything because the freebsd kernel couldn't differentiate between memory used by ARC and other kernel memory).
|
![]() |
|
In new fucked up ZFS storage confusion, my N40L with NAS4Free that randomly picks "fast" and "slow" files has gotten even more confusing. I found a slow 9GB file that would transfer at 10MB/s, and started it transferring to a desktop, then I simultaneously started another transfer of the same file to a laptop. 10MB/s again. So the NAS is happy pushing 2x 10MB/s transfers of the same file, but won't do 20MB/s to a single client for this file, and will happily let other files zip by at 100MB/s. What the hell is going on here?
|
![]() |
|
Nystral posted:Is the 1GB of RAM per TB in a RAIDZ advice still hold true? I'm looking to build a new system and debating 8 or 16GB of RAM.
|
![]() |
|
I can't wait for the 2TB WD Reds to become available, no one knows anything. Is there a significant difference between the WD Blacks and RE4s, beyond TLER? I've a bunch of old 500GB RE2s still trucking with over 34000 hours, so I'm leaning towards the RE4s. But if the Blacks the same hardware, I'd rather save some money.
|
![]() |
|
You can buy them now, but Amazon's price isn't that great. http://www.amazon.com/Western-Digit...e/dp/B008JJLZ7G Since we're talking network storage anyway, I'm not sure why anyone would be going for super high-performance drives. My 2 year old WD green drives nearly hit the limits of gigabit ethernet.
|
![]() |
|
I'm in Belgium, pretty close to the German border. None of the shops in any vicinity in either country have any stock or delivery dates.
|
![]() |
|
Combat Pretzel posted:Is there a significant difference between the WD Blacks and RE4s, beyond TLER? I've a bunch of old 500GB RE2s still trucking with over 34000 hours, so I'm leaning towards the RE4s. But if the Blacks the same hardware, I'd rather save some money.
|
![]() |
|
It's a bit less than 30% here. The Black is 167€ and the RE4 is 188€. I would have wanted to wait it out for the Reds, because they're just 133€ and still rated for 24/7. But they're not coming closer and my RAID1 is pretty much screaming "DO FUCKING SOMETHING!" at me.
|
![]() |
|
Keep in mind that the extra money for the RE4 also buys you higher performance compared to the Reds. If the server is hidden away in the basement where you can't hear it the RE4 is an excellent drive, but if you keep it in your livingroom the Reds are great because they're nearly dead silent. They also generate roughly half the heat, which adds up when you have lots of drives in a small room.
|
![]() |
|
For those of you looking to pick up some HDD's, NewEgg this weekend has the 2TB Samsung F4 for $99. I know that drive has been pretty popular with NAS-builders in the past, and it's a pretty good price these days.
|
![]() |
|
DrDork posted:For those of you looking to pick up some HDD's, NewEgg this weekend has the 2TB Samsung F4 for $99. I know that drive has been pretty popular with NAS-builders in the past, and it's a pretty good price these days. Oh sonnuva...
|
![]() |
|
What is a good NAS solution that I can couple with cloud storage? I'm currently debating CrashPlan+, Netgear ReadyNAS Ultra, or both. My priorities, in order, are 1) Backup photos and financial data; 2) Backup music and videos; 3) Media server for music and videos (we have a separate HTPC that currently streams videos from my desktop). I'm concerned with the always on functionality of CrashPlan slowing down my wife's laptop, but it appears that some NAS servers can backup directly to CrashPlan (or other cloud services) instead. Also, if I have cloud storage I do not need mirroring so I can either have more space or wait to get a second HDD.
|
![]() |
|
Astro7x posted:I started this discussion in the Video Editing thread since I work for a small post house, but I guess it is more appropriate for this thread. If it is a Windows server you might want to look at Dell AppAssure. You would need storage at both ends but it will do a good job with the backup replication.
|
![]() |
|
TaTaToothy posted:If it is a Windows server you might want to look at Dell AppAssure. You would need storage at both ends but it will do a good job with the backup replication. Unfortunately no, we use Final Cut Pro so we are Mac based. I threw some of these prices in front of my boss to just give him a general idea on how much it would cost to back up the entire server, and he though it would be much cheaper than what I showed him. Sounds like we might just end up backing up just the essentials instead (like project files, since assets could be redownloaded or recreated from those project files)
|
![]() |
|
Astro7x posted:Unfortunately no, we use Final Cut Pro so we are Mac based. Backing up project files makes a lot more sense. Backing up large amounts of raw footage or temporary video files is a lot more difficult. The offsite storage for my office is in the gigabytes rather than terabytes. We do have a duplicate of our server on site which is in a fire rated room. If the server dies we could switch over to that server. If there is a fire and the fire rated room holds up then I would take the server off site to copy the data before the corrosion from smoke sets in (typically you have about a week before components start failing).
|
![]() |
|
Jesus Christ, the WD Black/RE4 must have platters made of depleted uranium. Compared to my other drives, they take long to spin up, and free wheel god knows how long when powering down. ![]()
|
![]() |
|
Combat Pretzel posted:Jesus Christ, the WD Black/RE4 must have platters made of depleted uranium. Compared to my other drives, they take long to spin up, and free wheel god knows how long when powering down.
|
![]() |
|
So both as a general FYI and some questions, I figured I'd retell my story of replacing my raid-z1 hard drive that was starting to get faulty. For record, the zpool is 4x3TB drives in an N36L microserver. I had a drive (Hitachi 5k3000) that'd decided to get a significant amount of reallocated sectors (31/100 left, with 5 being the threshold) and pending sectors. This was causing the zpool to hang when accessing it in certain ways if it hit that bad data. If I dropped it out and ran degraded, it was fine. So I got a new hard drive (WD green 3TB, I know, but it's by far the cheapest at $40 less than a WD Red and they didn't have Red drives in stock) as a stop-gap til the 5k3000 is RMAed, at which point I'll use the WD as a cold spare. In an effort to take the load off the other 3 healthy drives, I decided to take the approach of running ddrescue across the broken drive on to the new drive, as the vast majority of the drive was still fine. This process got about 2 or 2.2 of the 3TB done in the 6 hours overnight as I slept, and then started to grind right down and take another 24 hours after that to complete (I woke up this morning to it completed). I could've just resilvered the array, but that runs the risk of throwing an error on a second drive. Not necessarily fatal in my situation but I'd rather put the strain of reading 3TB of data on the drive that I'm about to replace, then resilver only the broken parts. I also wanted to see how zfs would handle a dd'ed drive replacement as opposed to letting it resilver on to a clean slate. ddrescue went fine and there was 151kB of data that was lost, as expected. I plugged back in the 3 good hitachis and the WD, fired the server back up and imported the array again. It's displaying and reading perfectly, showing a healthy status, so it seems to have accepted the WD drive perfectly in place - I think partially by luck because the WD drive is /dev/ada3 which is what the broken hitachi was too. I'm now scrubbing to resilver the lost 151kB of data. I guess my one question from this is does anyone expect this to pose a problem? Everything seems to be working fine and maybe I just fluked it with the device names working out, but I just want to confirm whether anyone can think of a reason why this might go to shit on me? Oh, and lastly, the dd process was relatively gentle on the drive. As this is a WD with its infamous head parking fun, it will load cycle if I read with 8 second pauses. Even still, ddrescue only incurred 52 load cycles over the entire process of 3TB: code:
|
![]() |
|
ZFS uses EFI to wrap the disk and stores and UUID. You could have attached it via USB and it would have worked. ZFS is pretty cool, too bad Oracle won't let it leave its home turf and keep that farce of BTRFS going. A Windows IFS driver would also be pretty rad. Storage Spaces is limited as fuck and probably won't see a big improvement outside the Windows release cycle.
|
![]() |
|
So I'm just wandering around the computer mall today to build my own freenas server, anyone got any tips for itx enclosures? Sure they look small but I'm missing the slot loading feature from the set top nas and N40L. Is a h77 chip set and i3 good enough for Usenet, VPN, plex media repository? Or am I good enough with an atom. Or just buy a N40L and load free nas. I also have a synology 2 bay at home and plan to add sabnzd to it. My dream machine is a 10 bay atx goon treasure chest. What CPU do I need?
|
![]() |
|
caberham posted:So I'm just wandering around the computer mall today to build my own freenas server, anyone got any tips for itx enclosures? Sure they look small but I'm missing the slot loading feature from the set top nas and N40L. Usenet un-rarring and par2 checks would be pretty slow on the atom, plus the Atom locks you into 2-4GB RAM. Might as well get at least an i3 and load it with cheap RAM.
|
![]() |
|
No idea where to post this but I need some help. Boss at works wants a file server for the company. He wants raid 10 at least 2tb of space and able to auth users from a Windows 2008 domain controller. I don't know shit about Raid and file servers so I need help. We currently have some cheap Seagate NAS device but it's already failed on us twice. Any suggestions would be great.
|
![]() |
|
So, 4 3TB diks in just about any Dell server. Pretty much any 2U server should be able to hold 6 drives (2 mirrored system drives, 4 data drives). Install Windows Server, join the domain, ???, profit. E: Holy shit this is a pain in the ass to build on Dell's website, but it looks like an R520 can do it for around $5k, though I fucked up the RAID math in my head and you'd be fine with 2 TB or even 1 TB drives. FISHMANPET fucked around with this message at 03:09 on Sep 21, 2012 |
![]() |
|
Easy Mac posted:No idea where to post this but I need some help. The previous answer is the correct one. Get the cheapest (enterprise, prosumer, or something that's not a ReadyNAS or something) server you can get and put drives in. It'll outperform a Drobo, QNAP, or Synology, with a lot more features, and you can do whatever you want with it. Plus it'll probably cost less.
|
![]() |
|
evol262 posted:The previous answer is the correct one. Get the cheapest (enterprise, prosumer, or something that's not a ReadyNAS or something) server you can get and put drives in. It'll outperform a Drobo, QNAP, or Synology, with a lot more features, and you can do whatever you want with it. Plus it'll probably cost less. Assuming the person who posted the question (aka Mr. "I don't know shit about Raid and file servers") is going to have to maintain this maybe the extra money spent on a dedicated device with support will be worth it.
|
![]() |
|
Yes you need support, no you don't want a drobo-like device. Just get a dell big enough for the amount of drives you need and get on with your life.
|
![]() |
|
text editor posted:Usenet un-rarring and par2 checks would be pretty slow on the atom, plus the Atom locks you into 2-4GB RAM. Might as well get at least an i3 and load it with cheap RAM. Thanks I'm looking at the HP proliant m110 series http://h10010.www1.hp.com/wwpc/us/e...5942.html?dnr=1 Load it with 32gb ram and stuff it with 4tb disks. And add freenas. Costs a pretty penny but still cheaper than the ready made nas. That should do the trick right?
|
![]() |