«608 »
  • Post
  • Reply
apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

What's a recommended HBA PCI card to run about 4 to 6 SATA drives off?

It will be for home server use so nothing extreme, just something that will provide something better than what I've currently got, which is using the SATA ports on a consumer motherboard and running a ZFS array with them.

I'd like to take it up a notch, in terms of expandability. Maybe start running nested ZFS, like a striped pair of mirrors.

D. Ebdrup
Mar 13, 2009



apropos man posted:

What's a recommended HBA PCI card to run about 4 to 6 SATA drives off?

It will be for home server use so nothing extreme, just something that will provide something better than what I've currently got, which is using the SATA ports on a consumer motherboard and running a ZFS array with them.

I'd like to take it up a notch, in terms of expandability. Maybe start running nested ZFS, like a striped pair of mirrors.
A LSI Avago Broadcom SAS HBA 4i4e will give you 4 ports internally and 4 ports externally (more via port multiplication).
There are plenty of the older cards available used, in case you find the price for the new ones too steep.

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

D. Ebdrup posted:

A LSI Avago Broadcom SAS HBA 4i4e will give you 4 ports internally and 4 ports externally (more via port multiplication).
There are plenty of the older cards available used, in case you find the price for the new ones too steep.

Nice one! Thank you Sir!

Crunchy Black
Oct 24, 2017

CASTOR: Uh, it was all fine and you don't remember?
VINDMAN: No, it was bad and I do remember.




Yeah they've discontinued a lot of the older HBAs over the years but the 9207 just keeps trucking along. The 8i is what I use for my internal drives with freenas and an expander backplane, zero complaints. Been rock solid for many years.

Heners_UK
Jun 1, 2002


Sort of along those lines, in the Lenovo TS430 I bought I found an LSI MegaRAID 9240-8i, with 2 backplanes and cables. Before I found that (for reasons unknown), the old owner had disconnected the PSU from the backplanes, I was trying to find out why it didn't work and assumed a compatibility problem. I've seen a lot of information about flashing these cards with different firmwares to turn them into HBAs and assumed I needed too... until I connected the backplanes and it worked no problem at all in Unraid.

Thing is, now that I've seen page after page of people telling me flashing to HBA firmware/IT mode, I'm wondering if I need to anyway, and why?

mrking
May 27, 2006

There's No Limit To What We Can't Accomplish


I picked up an LSI Logic SAS 9207-8i off amazon, it was the cheapest of the LSI cards on amazon at the time. IT mode makes it so that drives show up just like plugging into the motherboard, instead of in some kind of raid controller thing (I think).

slidebite
Nov 6, 2005

Good egg


WD 8tb Easystores seem to be at amazon.ca for $202 ea, which according to Camelcamel is about as good as it gets other than black friday, which might be $20 cheaper each.

Any reason not to buy 4 and shuck them?

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

Is there a culture of counterfeit LSI HBA's appearing on eBay?

I remember someone in this thread showing a handy guide on buying a genuine network card.

Does the same thing tend to happen with the HBA cards?

8-bit Miniboss
May 24, 2005

CORPO COPS CAME FOR MY


apropos man posted:

Is there a culture of counterfeit LSI HBA's appearing on eBay?

I remember someone in this thread showing a handy guide on buying a genuine network card.

Does the same thing tend to happen with the HBA cards?

Not really. Since 12G cards and other storage solutions become more available, people have been tossing up 6G cards for fairly cheap.

Heners_UK
Jun 1, 2002


slidebite posted:

WD 8tb Easystores seem to be at amazon.ca for $202 ea, which according to Camelcamel is about as good as it gets other than black friday, which might be $20 cheaper each.

Any reason not to buy 4 and shuck them?

Haven't seen easystores on there but if you mean WD Elements (seemingly the same) then your assessment is correct.

If you have found actual easystores on Amazon.ca, good grief please post a link

slidebite
Nov 6, 2005

Good egg


ffs, of course you are right. Elements.

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

8-bit Miniboss posted:

Not really. Since 12G cards and other storage solutions become more available, people have been tossing up 6G cards for fairly cheap.

OK. Cool.

I remembered that it was the Home Networking thread where we'd been discussing counterfeit cards. It was early and I got my threads mixed up.

IOwnCalculus
Apr 2, 2003





Yeah, I paid $20 for a 6Gbps card a month or two ago. Even adding a SAS-to-SATA cable cost, it's starting to look cheaper than even the shitty Marvell controllers.

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

Is there a tiny battery in a MegaRAID/HBA adaptor? So if you get a complete power failure then there's, like 10ms worth of battery time to flush emergency writes to disk?

Is it somehow a better system than just relying on a consumer motherboard's onboard SATA ports?

Someone told me the PERC controllers in our Dell rack servers had something similar.

EDIT: I should add that we do use UPS at work, but the discussion was centered around sending a sysrq command to Linux to force an instant reboot . So that's why we were discussing the merits of disk controllers with a small on-board battery/capacitor.

apropos man fucked around with this message at 19:54 on Feb 13, 2019

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!

Grimey Drawer

Are there any relevant differences between regular and helium-filled drives in terms of reliability?

Heners_UK
Jun 1, 2002


apropos man posted:

Is there a tiny battery in a MegaRAID/HBA adaptor? So if you get a complete power failure then there's, like 10ms worth of battery time to flush emergency writes to disk?

Anecdotally I've seen capacitor like behaviour from my one. I can yank the power cord and still get a few seconds of power from it.

However, my card also has a true add-on battery option according to the manual. This maintains the cache etc for a few days.

H110Hawk
Dec 28, 2006


apropos man posted:

Is there a tiny battery in a MegaRAID/HBA adaptor? So if you get a complete power failure then there's, like 10ms worth of battery time to flush emergency writes to disk?

Is it somehow a better system than just relying on a consumer motherboard's onboard SATA ports?

Someone told me the PERC controllers in our Dell rack servers had something similar.

EDIT: I should add that we do use UPS at work, but the discussion was centered around sending a sysrq command to Linux to force an instant reboot . So that's why we were discussing the merits of disk controllers with a small on-board battery/capacitor.

You shouldn't be caching information if you do not have a formal BBU somewhere, whether it be a UPS or a battery on the card. Normal raid operations disable in-disk write cache to maintain consistency, and if you don't have a BBU your FS/RAID device should be set to "write through" not "write back."

I totally missed the part about sysrq. If you force a reboot you will get exactly one try to sync but otherwise the system can abandon buffers if I recall correctly. If you are using the sysrq that acts as a NMI then the buffers are dropped because the kernel just says "fuck this noise" and bails. No bbu will save you there.

H110Hawk fucked around with this message at 20:33 on Feb 13, 2019

D. Ebdrup
Mar 13, 2009



apropos man posted:

Is there a tiny battery in a MegaRAID/HBA adaptor? So if you get a complete power failure then there's, like 10ms worth of battery time to flush emergency writes to disk?

Is it somehow a better system than just relying on a consumer motherboard's onboard SATA ports?

Someone told me the PERC controllers in our Dell rack servers had something similar.

EDIT: I should add that we do use UPS at work, but the discussion was centered around sending a sysrq command to Linux to force an instant reboot . So that's why we were discussing the merits of disk controllers with a small on-board battery/capacitor.
Hardware RAID controllers do have backup batteries (called BBUs, short for backup battery units). The HBA units that we tend to recommend for ZFS don't, as their task is to get the data from the OS to the disk as quickly as possible and not touch it in any other way.
I don't know much about Linux but I assume it's not completely different from FreeBSD where sync(8) exists to, well, sync data to disk if you've just edited a file and need to make sure its been written to disk.

As for shutdown, part of shutdown.c and reboot.c contain a process that's designed to ensure all caches get flused (it's a bit more complex than that since FreeBSD has a unified buffer cache as part of its VM, but if you ever see "Syncing disk buffers remaining" followed by a number that counts down (and somtimes up, then down - because of how VMs work), you know you have to wait for at least that amount of time.
This is one of the areas where my knowledge about Linux is insufficient, because I feel like there has to be a lot of problems here that stem from the way Linux wrongly handles (handled?) fsync (known, among the postgres and general database crowd as fsyncgate).

On ZFS (regardless of platform, I believe) things are different, because there you have to contend with the ZIL unless your dataset has sync=always set (which is typically only used on datasets for databases or datasets which are zvols to provide block storage for hypervisors; and in both those cases, the system has a SLOG used to speed up all syncronous operations). For asyncronous writes however, since ZFS is atomic and uses NCQ and other commands to control the disk cache, the worst that can happen is that you need to revert to one transaction group previous to the one which wasn't finished properly (something that ZFS has been capable of doing automatically for long time, and which it's fairly recently gotten even better at).

D. Ebdrup fucked around with this message at 20:24 on Feb 13, 2019

IOwnCalculus
Apr 2, 2003





D. Ebdrup posted:

Hardware RAID controllers do have backup batteries (called BBUs, short for backup battery units).

To be fair - most of the ones on eBay are capable of this but do not actually have one with them.

D. Ebdrup
Mar 13, 2009



IOwnCalculus posted:

To be fair - most of the ones on eBay are capable of this but do not actually have one with them.
Most often when the RAID controllers get decommisioned from servers and sold on ebay, the BBUs get thrown out because they're not worth much after lying on a shelf for months after having been in use for a long time.

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

Interesting chat about BBU's. I've bought a 9207-8i, which I'm gonna hang a bunch of SATA disks from and try out a couple of ZFS configurations. So, although, there wasn't any practical point in bringing it up, it was interesting reading.


H110Hawk posted:

You shouldn't be caching information if you do not have a formal BBU somewhere, whether it be a UPS or a battery on the card. Normal raid operations disable in-disk write cache to maintain consistency, and if you don't have a BBU your FS/RAID device should be set to "write through" not "write back."

I totally missed the part about sysrq. If you force a reboot you will get exactly one try to sync but otherwise the system can abandon buffers if I recall correctly. If you are using the sysrq that acts as a NMI then the buffers are dropped because the kernel just says "fuck this noise" and bails. No bbu will save you there.

I once called the sysrq that immediately reboots Linux (echo 1 > /proc/sys/kernel/sysrq ; echo b > /proc/sysrq-trigger) on an unresponsive server that was starting to fail running extra SSH sessions. ELF interpretation was dying and the choice of binaries that were able to be called was getting more limited by the minute. Needless to say, it fucked something in the filesystem and when we got it rebooted the platform wasn't stable or trustworthy but we got what we needed from it.

I tried the same thing at home on a VM, just to see how often the immediate reboot sysrq caused damage, and a brand new CentOS 7 installation was also slightly borked after one reboot. So telling the kernel to bail is really a destructive last resort, unless you're running some kind of minimalist Linux that barely touches the disk, I guess.

Anyway, I digress majorly.

H110Hawk
Dec 28, 2006


apropos man posted:

Interesting chat about BBU's. I've bought a 9207-8i, which I'm gonna hang a bunch of SATA disks from and try out a couple of ZFS configurations. So, although, there wasn't any practical point in bringing it up, it was interesting reading.


I once called the sysrq that immediately reboots Linux (echo 1 > /proc/sys/kernel/sysrq ; echo b > /proc/sysrq-trigger) on an unresponsive server that was starting to fail running extra SSH sessions. ELF interpretation was dying and the choice of binaries that were able to be called was getting more limited by the minute. Needless to say, it fucked something in the filesystem and when we got it rebooted the platform wasn't stable or trustworthy but we got what we needed from it.

I tried the same thing at home on a VM, just to see how often the immediate reboot sysrq caused damage, and a brand new CentOS 7 installation was also slightly borked after one reboot. So telling the kernel to bail is really a destructive last resort, unless you're running some kind of minimalist Linux that barely touches the disk, I guess.

Anyway, I digress majorly.

Correct. sysrq `b` is the last step in the reboot process. If taken as the first step then the kernel just peaces out and resets the system. It is as close to pressing the reset button as you can get. The steps taken prior to it would normally be sync, then unmount. If you have dirty buffers when you sysrq `b` you're in for a bad day depending on what they were.

Moey
Oct 22, 2010

I LIKE TO MOVE IT


Welp, the 10tb Easystores (with a free 32gb thumbdrive) are down to $170 at BestBuy.

https://www.bestbuy.com/site/wd-eas...p?skuId=6290669

ChiralCondensate
Nov 13, 2007

what is that man doing to his colour palette?


Grimey Drawer

Moey posted:

Welp, the 10tb Easystores (with a free 32gb thumbdrive) are down to $170 at BestBuy.

https://www.bestbuy.com/site/wd-eas...p?skuId=6290669
fyi, I had to sign in to see the $170 price, it showed the regular $200 before.

slidebite
Nov 6, 2005

Good egg


Wow, that's a smoking deal.

Got my 4x 8tb Elements from Amazon today. They were in 2 different types of packaging, 2 of the boxes were shrink wrapped, 2 used those adhesive tape closures.

Not surprisingly all 4 white label, but 2 different styles. Left side is shrink wrapped. Same model number on all of them, all made in Aug/Sept 2018.

Just need to put in my RN2120 after they warm up a bit (-20C from shipping).

H110Hawk
Dec 28, 2006


Moey posted:

Welp, the 10tb Easystores (with a free 32gb thumbdrive) are down to $170 at BestBuy.

https://www.bestbuy.com/site/wd-eas...p?skuId=6290669

Fancy, glad you told me about signing in to get the deal. Turns out I had to disable my adblocker for their account management to work which is hilarious. Thank goodness for private browsing. I can pick it up in one hour from my local store!

topenga
Jul 1, 2003


Moey posted:

Welp, the 10tb Easystores (with a free 32gb thumbdrive) are down to $170 at BestBuy.

https://www.bestbuy.com/site/wd-eas...p?skuId=6290669

Mother. Fucker.

40TB NAS it is, I guess!

(thanks!)

Heners_UK
Jun 1, 2002


slidebite posted:

Got my 4x 8tb Elements from Amazon today.

Just curious, what shucking instructions did you use? I made a pigsear of shucking an Elements (well it was an older mybook but seemingly same design).

Heners_UK
Jun 1, 2002


Moey posted:

10tb Easystores (with a free 32gb thumbdrive) are down to [USD]$170 at BestBuy.

Eesh, $220 CAD + driving&effort to go and pick them up... tempting...

topenga
Jul 1, 2003


Heners_UK posted:

Just curious, what shucking instructions did you use? I made a pigsear of shucking an Elements (well it was an older mybook but seemingly same design).

If they are built like the Easystores I used this: https://www.youtube.com/watch?v=b6VCQ64DkfM#t=74s

This guy shucks an Elements: https://www.youtube.com/watch?v=i_Gv2eMla3w#t=319s
and he makes it look hard. Someone commented: "Don't open it like that! Pry from the rounded end and just slide it off. The clips are not meant to come apart the way he did it. They will break."

IOwnCalculus
Apr 2, 2003





Heners_UK posted:

Just curious, what shucking instructions did you use? I made a pigsear of shucking an Elements (well it was an older mybook but seemingly same design).

I cut up an old credit card and used it. Any of the Youtube videos on it will show where the clips are, you just need to shim the case open at those and then it just takes a little push on the whole enclosure to slide it.

edit: yes, the first video immediately above

H110Hawk
Dec 28, 2006


Heners_UK posted:

Just curious, what shucking instructions did you use? I made a pigsear of shucking an Elements (well it was an older mybook but seemingly same design).

I basically lever them until the clips all break.

Actually just got out a WD100EMAZ with 0 clips broken. 256MB cache, made in Thailand, P/N 2W10228.

H110Hawk fucked around with this message at 21:58 on Feb 14, 2019

slidebite
Nov 6, 2005

Good egg


Heners_UK posted:

Just curious, what shucking instructions did you use? I made a pigsear of shucking an Elements (well it was an older mybook but seemingly same design).
I just used the back of a knife to pry the back open and then worked around it. Broke 2 clips per I think, but I wasn't being terribly careful. Probably all of 30 seconds to do.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


Take an old credit card or gift card and cut it into 4 pieces, or grab 4 guitar picks. Shove them in at the right spots, and the case pops open easily with no breakage. Example video: https://www.youtube.com/watch?v=7XyUHcaJLLE

Irritated Goat
Mar 12, 2005

This post is pathetic.


Goons, I'm in the market to upgrade my NAS. I've currently got a Synology DS213J but would like to move all my Plex\Usenet junk off my personal PC and to a NAS.

Should I get a Synology again or build my own? I'm wanting to do something much larger than what I have, which is 2 JBOD drives at 4TB and 2TB.

I am open to suggestions. I don't mind learning ZFS or Unraid but if Synology is gonna handle it for me, I'm game. I would like to do at least RAID 1 but might do more if drive prices are still good when I can afford it.

H110Hawk
Dec 28, 2006


Irritated Goat posted:

Goons, I'm in the market to upgrade my NAS. I've currently got a Synology DS213J but would like to move all my Plex\Usenet junk off my personal PC and to a NAS.

Should I get a Synology again or build my own? I'm wanting to do something much larger than what I have, which is 2 JBOD drives at 4TB and 2TB.

I am open to suggestions. I don't mind learning ZFS or Unraid but if Synology is gonna handle it for me, I'm game. I would like to do at least RAID 1 but might do more if drive prices are still good when I can afford it.

There are two main camps: 1. Buy the largest synology you can afford. (This is my camp.) It can do limited Plex/Usenet/etc junk through their apps, and it can run Docker. Make sure you get an x86_64 one. (Maybe they all are now?) If it can't do it, get something like a NUC (or one of the many clones) to handle the heavy lifting part. 2. Unraid. It's just linux, and can do all of this stuff, but you have to build the server yourself.

The synology adds a significant markup but 0 parts searching. Regardless, disks are still going to be a significant chunk of your spending. You say you want to do "a lot bigger", you can play around here: https://www.synology.com/en-us/supp...tor?hdds=4%20TB|2%20TB|10%20TB and see what you would wind up with, green is usable, grey is waste. Click the disk in the chassis to remove it.

D. Ebdrup
Mar 13, 2009



There's also a camp that insists that a NAS should be capable of not fucking your data over, and which suggests you should use ZFS.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast


D. Ebdrup posted:

There's also a camp that insists that a NAS should be capable of not fucking your data over, and which suggests you should use ZFS.

Ooh... the rabid ZFS camp. Ahh, yes. You might also be the person that thinks any single box should take care of your data. Nope, you shouldn't trust them. Back all the shit up.
UnRAID is great in many ways, and flexible. Most home users have some pirated movies, and several copies of their data that actually matters. For them, an UnRAID system will do the job. The UnRAID box will allow them to add god-knows-what-size drives in at random, without mirroring, but still retaining parity protection (or indeed, as one should, double parity).

Checksums and snapshots are great things, and ZFS is clearly a better file system than B-Tree File System, but it's not as if UnRAID uses it as a) default (that'd be XFS), or b) trusts it to do anything other than store data on a single drive.

The truth of the matter is, most people don't want to give a crap about how the data is stored on the drives, and UnRAID's parity-by-file system allows you to remove drives at will, and have them be readable. It's just a great solution for the home user.

Edit: not that there's anything wrong with building a ZFS-based FreeNAS system. Hell no. Do that if you want to. Just be aware of the various options and limitations.

HalloKitty fucked around with this message at 20:24 on Feb 15, 2019

slidebite
Nov 6, 2005

Good egg


slidebite posted:

Wow, that's a smoking deal.

Got my 4x 8tb Elements from Amazon today. They were in 2 different types of packaging, 2 of the boxes were shrink wrapped, 2 used those adhesive tape closures.

Not surprisingly all 4 white label, but 2 different styles. Left side is shrink wrapped. Same model number on all of them, all made in Aug/Sept 2018.

Just need to put in my RN2120 after they warm up a bit (-20C from shipping).



My Netgear RN2120 recognized all 4 drives immediately (didn't need that 3.3V power thing I've heard about) and it's now syncing the drives. No idea how long that's going to take, but first if said 60 hours, but it's already down to 30 so

e: Any idea what to do with 4 5v power supplies and USB 3.0 cables?

slidebite fucked around with this message at 20:49 on Feb 15, 2019

H110Hawk
Dec 28, 2006


slidebite posted:

e: Any idea what to do with 4 5v power supplies and USB 3.0 cables?

Are you going to save the boxes for RMA? If so, leave em in there. Otherwise recycle em.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »