«608 »
  • Post
  • Reply
30 TO 50 FERAL HOG
Mar 2, 2005





Ramrod XTreme

Synology really is the best. The web interface and general ease of setup is well worth the price. I've been using a roll your own solution for a while and it's given me nothing but headaches: rebuilds after restarting, slow everything since most of the processing power goes to the array, slow read/write access.

It's going to take me a while to get all of the drives that I want (Fry's only had 3 WD Red 3TB and one failed within minutes), but when I'm done this should last me basically forever.

Bob Morales
Aug 18, 2006

I love the succulent taste of cop boots

BackBlaze did a good write up on hard drive failures -

http://blog.backblaze.com/2013/11/1...sk-drives-last/



After 3 years your drives are living on borrowed time, statistically speaking.

After 4 years 80% of their drives are still alive.

IOwnCalculus
Apr 2, 2003





Yeah, the article is definitely worth a read.

The other part I want to see is the forthcoming article they mention where they compare 'enterprise' drives to the 'consumer' drives that make up the vast majority of their disks. I'm betting they don't see that much improvement.

Won't stop me from buying Reds due to the warranty, though.

SammichBacon
Nov 11, 2013



I've been toying around with a home built NAS. I failed at reading and bought parts incompatible for the distro I wanted to use and threw 6 x1TB drives together into a freenas zfs array. I found freenas had frequent issues in my case, so I switched to nas4free and will soon expand to something far greater in size and likely horsepower. This little side project was educational in that I learned stuff and things about FreeBSD and permissions that I'd never have otherwise. I'll read through some of the recommendations in this thread and jack off all over it with wetodddid questions and shitpostery.

drukqs
Oct 15, 2010

wank wank you're a pro vaper I'm not wooptiedoo...


Boss is gifting this to me, including the two 15k rpm enterprise drives still inside the two bays on the left to use for my OS.

NAS time? Paycheck on Friday so I could grab a few 3tb drives soonish. I'm kind of hesitant only because I would really only have use of 4-5 bays. Plus this thing would have to be kept in the garage somewhere or I'd have to do something drastic to quiet the fans down.



He says it has two quad core processors and 24 gigs of RAM. I would probably retire or gradually phase out the system I currently use to stream and serve media to the family, run PS3 Media Server directly on this machine.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!


drukqs posted:

Boss is gifting this to me, including the two 15k rpm enterprise drives still inside the two bays on the left to use for my OS.

NAS time? Paycheck on Friday so I could grab a few 3tb drives soonish. I'm kind of hesitant only because I would really only have use of 4-5 bays. Plus this thing would have to be kept in the garage somewhere or I'd have to do something drastic to quiet the fans down.



He says it has two quad core processors and 24 gigs of RAM. I would probably retire or gradually phase out the system I currently use to stream and serve media to the family, run PS3 Media Server directly on this machine.

Keep in mind it will consume shitloads more electicity than a lower end NAS, so if that's a concern you should just flip it on eBay.

drukqs
Oct 15, 2010

wank wank you're a pro vaper I'm not wooptiedoo...


deimos posted:

Keep in mind it will consume shitloads more electicity than a lower end NAS, so if that's a concern you should just flip it on eBay.

Yeah that is true... Hm...

SammichBacon
Nov 11, 2013



drukqs posted:

Boss is gifting this to me, including the two 15k rpm enterprise drives still inside the two bays on the left to use for my OS.

NAS time? Paycheck on Friday so I could grab a few 3tb drives soonish. I'm kind of hesitant only because I would really only have use of 4-5 bays. Plus this thing would have to be kept in the garage somewhere or I'd have to do something drastic to quiet the fans down.



He says it has two quad core processors and 24 gigs of RAM. I would probably retire or gradually phase out the system I currently use to stream and serve media to the family, run PS3 Media Server directly on this machine.


Yeah, that's gonna eat away at your power bill. Just toss some GPU's in there and farm some bitcoins to soundly murder your utility bill.

kiwid
Sep 30, 2013



Check out what this thread says about the Dell PowerEdges: http://forums.somethingawful.com/sh...hreadid=3561669

TL;DR: They're noisy and power hungry.

thebigcow
Jan 3, 2001

Bully!

keep it racked at work for

drukqs
Oct 15, 2010

wank wank you're a pro vaper I'm not wooptiedoo...


Disaster. The controller is incapable of running drives larger than 2tb. A replacement controller which can do 3tb is 500 clams.

All three servers being tossed have the same hamstrung PERC6/i controller.

SopWATh
Jun 1, 2000


I ran into the same problem.

Also, there are approximately 8 loud as fuck fans in those things and they make a shit ton of noise. We're talking "you can't put the server in a closet, it's got to go in the garage" loud.

IOwnCalculus
Apr 2, 2003





Having not ever actually torn apart a 2950 - what is the physical interface from the drives to the controller? Can you not just pick up a cheap IBM 1015 or Dell H200 on eBay instead?

But yes, that server is going to be louder than all hell. The fans are designed to move metric tons of air front to back, noise is literally the last thing they cared about when designing it.

drukqs
Oct 15, 2010

wank wank you're a pro vaper I'm not wooptiedoo...


IOwnCalculus posted:

Having not ever actually torn apart a 2950 - what is the physical interface from the drives to the controller? Can you not just pick up a cheap IBM 1015 or Dell H200 on eBay instead?

But yes, that server is going to be louder than all hell. The fans are designed to move metric tons of air front to back, noise is literally the last thing they cared about when designing it.

There's a backplane with a bunch of sata connectors on it

there's a special/bespoke cable which leads from the backplane to the stock controller, I'm a little doubtful that one of these add in cards would work

IOwnCalculus
Apr 2, 2003





I mean on the back side of the backplane, where it connects to the controller - is it still presenting six SATA connections, or some Dell proprietary bullshit?

If the backplane is just passing the SATA connection from the hot-swap side to the inside, you should be able to grab any M1015 or similar cheap controller + appropriate cables and hook it up in place of the old PERC, but I have a sneaky suspicion that the backplane uses some Dell-specific shit to connect to the rest of the server.

Could always gut the backplane and run direct cables instead!

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll

Nap Ghost

The PERC cards use SFF-8484 connectors it seems and can be seated in a PCI-E 4x slot. You'd need to have a way to breakout the 8484 connector to the various SATA connectors on the backplane. Unless... the backplane also accepts an SFF-8484 connector. It may be possible Dell mucked with the pinouts or something to keep people from using cheapo hardware on the 2950s.

It's really not worth it though for home use IMO unless you just hate the environment and money. In which case I can come over to your house, litter in your driveway, and take your money without you having to haul that server anywhere.

drukqs
Oct 15, 2010

wank wank you're a pro vaper I'm not wooptiedoo...


necrobobsledder posted:

The PERC cards use SFF-8484 connectors it seems and can be seated in a PCI-E 4x slot. You'd need to have a way to breakout the 8484 connector to the various SATA connectors on the backplane. Unless... the backplane also accepts an SFF-8484 connector. It may be possible Dell mucked with the pinouts or something to keep people from using cheapo hardware on the 2950s.

It's really not worth it though for home use IMO unless you just hate the environment and money. In which case I can come over to your house, litter in your driveway, and take your money without you having to haul that server anywhere.

Nah, plan aborted. I'll have to look into conventional NAS enclosures or custom build a machine for this purpose. I'm kind of leaning in that direction right now, I would really like to have more storage capacity and some redundancy.. The way I've been handling drive failures is... Add on another higher capacity drive, transfer stuff over, RMA dead drive, reinstall reman drive... But I'm now completely out of physical space for new drives.

I might transplant the Dell machine I use as a media server into a larger tower, get some kind of a RAID add-in card for it.

http://www.newegg.com/Product/Produ...N82E16816118128

This looks pretty promising...

drukqs fucked around with this message at 22:46 on Nov 12, 2013

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.


drukqs posted:

I might transplant the Dell machine I use as a media server into a larger tower, get some kind of a RAID add-in card for it.

http://www.newegg.com/Product/Produ...N82E16816118128

This looks pretty promising...

DO NOT purchase this card. I cannot advise against the purchase of this card any harder than this. I had two of these cards and had nothing but trouble with them. Seriously. Run far, far away.

The card is advertised as running advanced parity-based RAIDs out of the box (RAID-5 / 50), but you will get horrible performance with it, because there is no on-board cache to help buffer requests. You will get great performance on the card until the number of pending writes fills up your available RAM and then watch as your throughput drops to kilobytes per second.

On both cards I had enterprise-class SATA drives fall out of the array all the time, killing the volume with data loss. When I tried to open a ticket with LSI, they came back with "Your drives and motherboard are not on our approved hardware list so we aren't going to help you. Bye."

I went out and bought a bunch of approved drives from ebay, but they were the wrong firmware so again, "Sorry, wrong firmware. Thanks for calling!"

I managed to get the system working in JBOD mode, which defeats the purpose of buying a card with advanced RAID functionality. Save your money. Buy a M1015 instead and flash it with the LSI firmware and run that in JBOD mode for a third of the price.

Agrikk fucked around with this message at 23:55 on Nov 12, 2013

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.


build report: homegrown NAS / SAN with Windows Server 2012

I've been looking for a better solution for shared storage for my ESX / Hyper-V / Failover Clustering lab and I finally got it up and running and configured, so I thought I'd post it here for others to mock use as a data point on a particular route to grow your own storage.

I originaly had a NetApp FAS2050 for use but, while it delivered excellent performance, it was crazy loud, an energy hog and out of warranty so any hardware failure would leave me with degraded performance until I could figure out what was broken and then source the part from eBay. So I decided to create my own iSCSI-based SAN that I could also use for file shares and other miscellaneous storage needs in my lab while being built out of easily replaceable parts.

My requirements were as follows:

- Reasonably quiet
- Power efficient
- Built from "pro-sumer" parts
- Decent performance
- Able to run Folding@Home in "Low" mode 24/7


Here is the build:

(The 4U box on top of the shot)




The components:

Norco RPC-470 ($75)

2x SuperMicro 5-in-3 3.5" Hot-swap drive bay ($100 each)

HP SmartArray P410 512MB BBWC RAID Controller ($150)

Supermicro H8SGL-F Motherboard ($220)

Opteron 6128 8-core @ 2GHz CPU ($45)

16GB (4x4GB) DDR3 PC1333 ECC Ram ($40)

Rosewill 550w 80 plus gold PSU ($85)

10x Western Digital Caviar Blue 640GB SATA 7200rpm ($18 each)

Intel Pro/1000 dual port adapter ($30)

Total cost (excluding shipping) is $1,025 USD.

Since I had the parts around, I used what I used. But if I were to do this from scratch, I'd probably go with a Norco RPC-4020 4U case. It's roughly the same price, also has hot-swap bays, but it has 20 of them instead of the ten in my solution. It would be nice to be able to add a second P410 and another 8 SATA drives at a later date for increased storage and/or IOPS.

Also: I plan on folding with this machine, so that's where the 8-core CPU comes into play. But as a pure storage box, 8 cores is completely overkill and an Intel i3 processor would probably handle the iSCSI workload just fine.

I end up with a 600gb software RAID-1 volume using the native Adaptec SATA controllers on the motherboard, and a 2.4TB hardware RAID-10 storage volume for iSCSI use. The motherboard has two internal ethernet NICs and a dedicated port for out-of-band IPMI management (including remote KVM ability), so with the dual-port PCI-e NIC, I have two ports for LAN traffic and two ports for iSCSI traffic that I can configure for round-robin use on my ESX hosts. The P410 card makes expanding an array a breeze, so for those with raw storage requirements instead of performance, it would be a simple task to build a 4-drive RAID-6 array of 2TB disks (or 4TB disks moneybags. But the rebuild will take days). This would give ~4TB of usable space while providing an upgrade path to ~10TB when configured with a hot spare.

I went with the HP SmaryArray P410 controller because I have experience with it in HP servers and understand its idiosyncrasies in running in a non-HP server. In this case, the Supermicro motherboard has enough BIOS memory available to be able to load and run the raid controller BIOS. An example: on my other build for my home file server involving a Gigabyte board and two P410s, the motherboard doesn't have enough BIOS memory to be able to load the HP BIOSes. I had to disable a lot of features on the motherboard (IDE controllers, printer port, etc) and remove the P410 from the boot order to disable their boot BIOSes to get them to be recognized by the board and subsequently the OS.

The one thing I really like about this card is that it is crazy cheap for a RAID card capable of RAID-50 or RAID-60 with 512mb of battery-backed write cache. Even though I was planning on running the storage array in RAID-10, I like to have options and the performance boost of the 512mb of write cache comes into play on RAID-10 as well.


I'll have more details about the Windows Server 2012 OS installation and IOPS scores in a followup post (when I can get home and take some screenshots).

Agrikk fucked around with this message at 00:45 on Nov 13, 2013

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.


nevermind

Agrikk fucked around with this message at 22:45 on Nov 19, 2013

Bob Morales
Aug 18, 2006

I love the succulent taste of cop boots

What kind of #'s do you get out of it?

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.


Bob Morales posted:

What kind of #'s do you get out of it?

I'll post the numbers tonight.

edit-quickly:

Max IOPS 24-hour test:
4K blocks, 100% READ 0% RANDOM. 2 workers: 1 per 100GB LUN with 70GB full: 18,816 IOPS

"SQL Simulation" 24-hour test:
8K blocks, 67% READ 40% RANDOM. 2 workers: 1 per 100GB LUN with 70GB full: 698 IOPS

Agrikk fucked around with this message at 03:25 on Nov 13, 2013

IOwnCalculus
Apr 2, 2003





Agrikk posted:

DO NOT purchase this card. I cannot advise against the purchase of this card any harder than this. I had two of these cards and had nothing but trouble with them. Seriously. Run far, far away.



I managed to get the system working in JBOD mode, which defeats the purpose of buying a card with advanced RAID functionality. Save your money. Buy a M1015 instead and flash it with the LSI firmware and run that in JBOD mode for a third of the price.

They are actually pretty much the same card - if you decide you want to be crazy and set up a M1015 to do hardware RAID, it basically becomes a 9240-8i. So yeah, that card is very poorly specced if you want to use it to do a hardware RAID array; it should only be used as an HBA, and as you pointed out you can get the M1015 for pennies on the dollar by comparison.

If you really need a hardware controller that will do parity RAID and has significant onboard cache, you're pushing the limits of what can be considered 'home' storage anyway. Which I suppose we all are to one extent or another

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!


Do you guys run the LSI IT firmware on your M1015s or just use it with the stock IBM firmware?

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.


IOwnCalculus posted:

They are actually pretty much the same card - if you decide you want to be crazy and set up a M1015 to do hardware RAID, it basically becomes a 9240-8i.

My extreme hatred for this card stems from two things: the horrible stability I had with it in any configuration and the bullshit "tech support" I received from LSI. The cards were unreliable and their hardware compatibility list is very limited and their refusal to give help to people who don't use the proper hardware is simply not acceptable.

Oh, and don't flash these cards via the GUI tool thingy, ever. It will brick your card (but I did experience a prompt RMA process for a bricked card, so I guess that's something). You must use a DOS-formatted USB key to get any kind of reliable flash.

deimos posted:

Do you guys run the LSI IT firmware on your M1015s or just use it with the stock IBM firmware?

I cross-flashed it to the IR mode because I wanted the option to have on-board RAID, even if I never used it. The IBM firmware won't let you do a straight JBOD pass-through, so you'll get best results by flashing it.

Here's the process I used (from a great home storage site):

http://www.servethehome.com/ibm-serveraid-m1015-part-4/

Agrikk fucked around with this message at 03:36 on Nov 13, 2013

bacon!
Dec 10, 2003

The fierce urgency of now

Newegg has the 4TBs on sale for $180 today, but I'm not sure if it's email blast only or store-wide. Amazon doesn't seem to be interested in matching these daily deals; we're still positive Newegg hasn't learned its lesson about shipping drives right? I should stay away?

IOwnCalculus
Apr 2, 2003





deimos posted:

Do you guys run the LSI IT firmware on your M1015s or just use it with the stock IBM firmware?

LSI IT firmware for me, the very first time I booted up with the card I flashed it off of the IBM firmware.

Agrikk posted:

My extreme hatred for this card stems from two things: the horrible stability I had with it in any configuration and the bullshit "tech support" I received from LSI. The cards were unreliable and their hardware compatibility list is very limited and their refusal to give help to people who don't use the proper hardware is simply not acceptable.

On one hand, I can see where they're coming from because their primary customer base is people putting these cards into actual servers for corporate deployments, not home users throwing them into whatever we've got that we're trying to make work.

On the other hand, the only reason to spend $300 for that card is support, so that's fucking lame, and yet another reason you should just acknowledge the fact that LSI won't help you so you should just spend $50-$80 on a used M1015 or equivalent instead; at least then the price reflects the support you're not getting.

Agrikk posted:

Oh, and don't flash these cards via the GUI tool thingy, ever. It will brick your card (but I did experience a prompt RMA process for a bricked card, so I guess that's something). You must use a DOS-formatted USB key to get any kind of reliable flash.

I flashed my LSI 1064 based controllers successfully in Windows, actually - and it was even by using VT-d to pass them to a Windows VM. I did use a DOS USB stick for the M1015, though, just because it was quicker.

Gozinbulx
Feb 19, 2004


bacon! posted:

Newegg has the 4TBs on sale for $180 today, but I'm not sure if it's email blast only or store-wide. Amazon doesn't seem to be interested in matching these daily deals; we're still positive Newegg hasn't learned its lesson about shipping drives right? I should stay away?



not seeing it. Link?

Bedurndurn
Dec 4, 2008


Gozinbulx posted:

not seeing it. Link?

http://www.newegg.com/Product/Produ...em-_-22-236-599

Kreeblah
May 17, 2004

INSERT QUACK TO CONTINUE



Taco Defender

Gozinbulx posted:

not seeing it. Link?

http://www.newegg.com/Product/Produ...N82E16822236599 + EMCYTZT4832

Phone
Jul 30, 2005

ああ!彼からのメールだ!

College Slice

I still haven't stood up my NAS and I was told to look into NAS4Free and use a USB stick for the OS install. I want the Embedded x64 Image, right?

e: I got it up and the ZFS pool created. What misc stuff should I do to make sure that this sucker stays up?

Phone fucked around with this message at 22:04 on Nov 16, 2013

IOwnCalculus
Apr 2, 2003





Look into scripts / cronjobs to notify you if the array drops from online to degraded, and maybe use the built in email function to give yourself a two mile long update once a week.

Phone
Jul 30, 2005

ああ!彼からのメールだ!

College Slice

I remember something about doing weekly/monthly checks or something similar. Unfortunately that's as much as I can remember.

(I had this fucking box built for the past 4 months, and it took me less than an hour to get it up and running, ughhhhhh motivation and effort)

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!


What's a decent way to stress test hard drives, I want to drop my 4 new Reds in an array but want to give them a shakedown run before I do.

IOwnCalculus
Apr 2, 2003





Phone posted:

I remember something about doing weekly/monthly checks or something similar. Unfortunately that's as much as I can remember.

(I had this fucking box built for the past 4 months, and it took me less than an hour to get it up and running, ughhhhhh motivation and effort)

Ah yeah, I have mine set up to do a full zpool scrub weekly, just set it up in the crontab as well.

sleepy gary
Jan 11, 2006



deimos posted:

What's a decent way to stress test hard drives, I want to drop my 4 new Reds in an array but want to give them a shakedown run before I do.

run badblocks with write enabled a few times?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

I thought manufacturers do a burn-in? Why do it again at home?

sleepy gary
Jan 11, 2006



Combat Pretzel posted:

I thought manufacturers do a burn-in? Why do it again at home?

Because shipping is a huge variable.

IOwnCalculus
Apr 2, 2003





DNova posted:

Because shipping is a huge variable.

It really is, see the Backblaze bit above - they mention in that article that they actually see different failure rates depending on whether the drives are delivered from their assembly location to the datacenter (best to worst) by a Ford Transit Connect, an air-spring truck, or a leaf-spring truck.

I've had good luck with using DBAN to check if a drive is working or not, but usually I don't get a failure that fast, at least not unless I decide to let a box run it for days on end.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell



IOwnCalculus posted:

Ah yeah, I have mine set up to do a full zpool scrub weekly, just set it up in the crontab as well.

I have mine do it once a month, but that's because I made my pool super huge and it takes like two and a half days.

Wish I'd made it smaller.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »