«608 »
  • Post
  • Reply
astral
Apr 26, 2004



Matt Zerella posted:

Friend. Don't do it. If you're using external drives just use them as JBOD.

My experience with trying to use a USB hard drive as 'intended' was that the Easystore enclosure tried to murder the hard drive inside with an overly aggressive attempt at power saving, completely ignoring the external drive hibernation settings of the Synology device it was connected to. If I recall correctly, the drive was starting and stopping a bit over twice per hour, which adds up rather quickly.

It's possible this was just a bug with the Synology, but it was enough to discourage me from trying that again for a long time.

Zorak of Michigan
Jun 10, 2006

Waiting for his chance

Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

bobfather
Sep 20, 2001

I will analyze your nervous system for beer money

Zorak of Michigan posted:

Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

Stablebit Drivepool will let you import data. In fact, there’s an advanced method to create your pool and manually move the files from each drive into the hidden folder that Drivepool uses, so duplication can begin immediately.

Drivepool is amazing.

Matt Zerella
Oct 7, 2002


I am 99% sure you can add a second parity drive to an existing array in the webui without needing to do anything.

D. Ebdrup
Mar 13, 2009



eames posted:

then please be so kind and fix the wikipedia article
Yeah, no, I just wrote a dumb thing that's untrue.
UnRAID is beyond quantification.

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!

Grimey Drawer

Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.

D. Ebdrup
Mar 13, 2009



Smashing Link posted:

Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.
Shot of using something that does checksumming, that's basically as good as is possible. The biggest thing I can think of is adding UPS' so that you can't get RAID write holes.
So, ZFS it is!

eames
May 9, 2009



Smashing Link posted:

Improve my backup strategy:

(home) Synology 1515+ / 5x 6TB WD Red Pro / SHR2 / ~ 5TB used
- backs up to (work) Synology 1515+ / 5x 3TB WD Red Pro / SHR2 / via HyperBackup using rsync, connected by OpenVPN served by the home NAS
- backs up to 2x 8TB WD EasyStore (non-shucked) by HyperBackup that are alternately kept at home and office
(work) Synology backs up to G-suite Drive via HyperBackup (encrypted)

Family photos and critical documents are also backed up on a couple of other random HDDs and USB disks on an occasional basis.

seems very hard to improve upon! 1515+ are prone to sudden death (due to the atom bug, usually starts with uncontrolled reboots) and I'd maybe do one of the backups with something other than HyperBackup so you don't lose your stuff in case that particular program decides to write garbage or deletes all previous backups because of a bug.
BTRFS does checksumming and seems to work fine with SHR2 (unlike the BTRFS RAID5/6 implementations).

Bob Morales
Aug 18, 2006

I love the succulent taste of cop boots

necrobobsledder posted:

Services like Backblaze B2 or perhaps Tarsnap are much cheaper than the usual cloud service providers like Google, Amazon, Microsoft, or even Apple because when you retrieve data back outside their cloud you get charged a metric ton.

Atomizer
Jun 24, 2007

Bote McBoteface. so what


D. Ebdrup posted:

You said this before but last time you also said it's for uptime so I didn't crawl down your throat about it.
RAID is only about redundancy when you're using nested levels of mirrors or striping with distributed parity. If there's any device striping without distributed parity, you're doing it for availability and speed.
Put another way: RAID 11, 15, 16, 17, 51, 55, 61, 65, 66, 71, 75, 76, or 77 provides you with redundancy, everything else is just for availability and/or speed.

I agree, aside from "redundant" being literally the first word in the initialism of RAID, I mischaracterized the "redundancy" component because there's no direct duplication (or more) of data, but technically any extra drives (i.e. ones that can fail without immediately compromising the array) are literally redundant, but we shouldn't argue semantics here. I think it's pretty clear I was emphasizing real backup solutions above all else and discouraging RAID (which I don't even use myself, at all) but trying to explain these topics to individuals without a proper foundation is not that simple.

Moey
Oct 22, 2010

I LIKE TO MOVE IT


Pull up, thread.

Schadenboner
Aug 15, 2011

I MEAN, TURN OFF YOURE MONITOR, MIGTH EXPLAIN YOUR BAD POSTS, HOPE THIS HELPS?!

Moey posted:

Pull up, thread.

Nah, fuck it, we'll just pull backups and restore.

Wait, the "B" in RAID stands for "Backup", right?

Smashing Link
Jul 8, 2003

I'll keep chucking bombs at you til you fall off that ledge!

Grimey Drawer

D. Ebdrup posted:

Shot of using something that does checksumming, that's basically as good as is possible. The biggest thing I can think of is adding UPS' so that you can't get RAID write holes.
So, ZFS it is!


eames posted:

seems very hard to improve upon! 1515+ are prone to sudden death (due to the atom bug, usually starts with uncontrolled reboots) and I'd maybe do one of the backups with something other than HyperBackup so you don't lose your stuff in case that particular program decides to write garbage or deletes all previous backups because of a bug.
BTRFS does checksumming and seems to work fine with SHR2 (unlike the BTRFS RAID5/6 implementations).

Thanks for the feedback -- I learned a lot from this thread!

Forgot to add that I do have a UPS. And although I am regretting that I bought two 1515+ I don't need to be tempted to buy anymore hardware at the moment. If one dies I will happily pull the trigger on an 1819+ or perhaps even roll my own.

Rexxed
May 1, 2010

Dis is amazing!
I gotta try dis!



Hey guys my raid array of independent disks is pretty redundant.

Heners_UK
Jun 1, 2002


Schadenboner posted:

Nah, fuck it, we'll just pull backups and restore.

The backup drive is part of our RAID array. Why would we even worry about including a tape drive in a striped array alongside our SSDs?!

sharkytm
Oct 9, 2003

Gimme Gimme Swedish Fish...



Fallen Rib

Heners_UK posted:

The backup drive is part of our RAID array. Why would we even worry about including a tape drive in a striped array alongside our SSDs?!

Where can I find a floppy interface expander? I've got an idea...

D. Ebdrup
Mar 13, 2009



Just stripe ZFS on top of 45 1.44MB floppies for the minimum 64MB required for a zpool.

HalloKitty
Sep 30, 2005

Adjust the bass and let the Alpine blast


Atomizer posted:

I agree, aside from "redundant" being literally the first word in the initialism of RAID, I mischaracterized the "redundancy" component because there's no direct duplication (or more) of data

Except in RAID1, which is literally that

D. Ebdrup
Mar 13, 2009



The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

Schadenboner
Aug 15, 2011

I MEAN, TURN OFF YOURE MONITOR, MIGTH EXPLAIN YOUR BAD POSTS, HOPE THIS HELPS?!

D. Ebdrup posted:

The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

P. sure it stands for "Backup" (because there is no R in backup and if you use RAID you will not have to worry about backups anymore because they are they best backup solution available)?

D. Ebdrup
Mar 13, 2009



Schadenboner posted:

P. sure it stands for "Backup" (because there is no R in backup and if you use RAID you will not have to worry about backups anymore because they are they best backup solution available)?
Eh, close enough definition for government work.

greazeball
Feb 4, 2003




Oven Wrangler

I want to upgrade my Plex server but I don't know what to look for or how much it should cost. Right now I've got a WD MyCloudMirror with 3.5/4TB on it. Video files are getting bigger so I'm going to need to do a big clean out soon (nah) or buy something new. I want something bigger and faster that won't throw up so many the server is not powerful enough errors. It would be great if it could also run Subsonic (or another app that's similar for playing a big mp3 library). I'd like to spend about $1000.

I'd then send the MyCloud to my mother-in-law's house and just use that for backing up photos and important docs.

Matt Zerella
Oct 7, 2002


You honestly don't need to spend that much. Have an old PC laying around?

greazeball
Feb 4, 2003




Oven Wrangler

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

mystes
May 31, 2006



greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.
Most of the NAS boxes have really slow arm processors. Your dell is going to be way faster than that.

Heners_UK
Jun 1, 2002


greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

Almost anything 3 years old will be fine.

Moey
Oct 22, 2010

I LIKE TO MOVE IT


Give us more details on your Plex usage.

Do you just stream inside your home or do you stream remotely as well?
What clients are going to be used to consume media?
Do you share with friends/family remotely?
How many concurrent users?
What resolution media do you hoard?

Beaucoup Haram
Jun 18, 2005



I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

H110Hawk
Dec 28, 2006


Beaucoup Haram posted:

I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

Your local electric utility thanks you.

Atomizer
Jun 24, 2007

Bote McBoteface. so what


HalloKitty posted:

Except in RAID1, which is literally that

D. Ebdrup posted:

The R in RAID is short for redundant because when RAID was made at Berkeley in California, it used simple mirroring for RAID1 or striping with parity for RAID2, with either Hamming or Reed-Solomon code which was meant to be done in hardware.

FYI to everyone who responded: the point is that "RAID" is complicated (and then you throw RAID0 in there) and isn't appropriate for the average user. Remember that this all came about in response to a guy who: dropped and broke an HDD without having any backups, begged for help in recovering data, then asked if RAID was backup for his media collection; he ended up opting not to implement any proper backup solution after all this. (And I'm not trying to criticize him by recapping this, just pointing out why this all came up.) The "proper" answer to all this is not to get into what RAID is and describe all the configurations: it's to insist that a) everyone needs to backup their important data, and b) you do not need RAID for your media library.

greazeball posted:

I want to upgrade my Plex server but I don't know what to look for or how much it should cost. Right now I've got a WD MyCloudMirror with 3.5/4TB on it. Video files are getting bigger so I'm going to need to do a big clean out soon (nah) or buy something new. I want something bigger and faster that won't throw up so many the server is not powerful enough errors. It would be great if it could also run Subsonic (or another app that's similar for playing a big mp3 library). I'd like to spend about $1000.

I'd then send the MyCloud to my mother-in-law's house and just use that for backing up photos and important docs.

greazeball posted:

I have a Dell something or other that's about 3 years old and I've been looking to upgrade that. The processor is not very powerful though.

You should post in the Plex thread. We're still going to ask what the hardware is on your Dell, though. And for reference, both of the PCs I've used for PMS are over 3 years old and work perfectly; you can almost certainly use that for PMS, Subsonic, etc. (I use my current PMS system for a Steam caching server and other stuff.)

Moey
Oct 22, 2010

I LIKE TO MOVE IT


Beaucoup Haram posted:

I've got a dual 2680v2 with 128gb of ECC ram running my plex server but please don't do this to yourself.

Hopefully it's doing something else besides Plex, because that's just wasted RAM.

IOwnCalculus
Apr 2, 2003





H110Hawk posted:

Your local electric utility thanks you.

I am so very glad that I don't have to pay for the energy mine uses.

greazeball
Feb 4, 2003




Oven Wrangler

Moey posted:

Give us more details on your Plex usage.

Do you just stream inside your home or do you stream remotely as well?
What clients are going to be used to consume media?
Do you share with friends/family remotely?
How many concurrent users?
What resolution media do you hoard?


Atomizer posted:

You should post in the Plex thread. We're still going to ask what the hardware is on your Dell, though. And for reference, both of the PCs I've used for PMS are over 3 years old and work perfectly; you can almost certainly use that for PMS, Subsonic, etc. (I use my current PMS system for a Steam caching server and other stuff.)


Thanks, I posted my replies in the Plex thread

Beaucoup Haram
Jun 18, 2005



Moey posted:

Hopefully it's doing something else besides Plex, because that's just wasted RAM.

Yeah it's a virt lab also and does a bunch of stuff but it's still ridiculous overkill.

Raldikuk
Apr 7, 2006

I'm bad with money and I want that meatball!

Zorak of Michigan posted:

Do any of these unRaid / SnapRAID / etc solutions let you import existing drives that have data on them? Taking a quick stroll through them, it looks like they're all designed so that you configure a new storage set and create new file systems, though you can then add new blank drives to grow capacity. That's part of what I meant when I told WWWWWWWWWWWWWWWWWW that he was going to have to put a lot of work into this- you can't just buy your sixteenth drive and make it hold parity data for the existing fifteen, you need to create a new file system using some newer technology and then start playing shell games with your data and drives. None of the steps involved are especially hard, but if you go into it with no plan, you can end up with a lot of confusion and frustration.

Snapraid lets you use existing drives since it doesn't touch the file system config itself. You can also remove drives at will. So you just need to add the parity drives and have it compute that. You can expand it at will too...just need to recalculate for your given parity level. It allows you to go up or down in parity level too without losing any data (other than the parity calc...which itself isn't lost until a successful rebuild).

Moey
Oct 22, 2010

I LIKE TO MOVE IT


RAM for my frankenstein Gen8 Microserver came in. So far so good.

I'll post some pictures when I finish up with it, but the plan is 4x3.5" and up to 4x2.5" internally.

Moey fucked around with this message at 18:55 on Mar 19, 2019

Atomizer
Jun 24, 2007

Bote McBoteface. so what


Not related to anything else, I broke down several old (~15 years), obsolete laptops (mainly removing & separating the batteries for disposal and the HDDs for wiping user data) earlier today. Despite their age, I was pleasantly surprised that 3 of the 8 HDDs were SATA (I thought they'd all be PATA/IDE.) Going back and dealing with PATA is a royal pain in the ass, but it makes me appreciate SATA that much more (from the ease-of-use alone.) The ~15-year-old 60 GB 2.5" HDD I pulled was identical to work with to the 2 TB FireCuda that I also got delivered today. No different connectors, no jumpers, no USB adapters not passing SMART diagnostics to the host, none of that bullshit to deal with.

D. Ebdrup
Mar 13, 2009



Atomizer posted:

Not related to anything else, I broke down several old (~15 years), obsolete laptops (mainly removing & separating the batteries for disposal and the HDDs for wiping user data) earlier today. Despite their age, I was pleasantly surprised that 3 of the 8 HDDs were SATA (I thought they'd all be PATA/IDE.) Going back and dealing with PATA is a royal pain in the ass, but it makes me appreciate SATA that much more (from the ease-of-use alone.) The ~15-year-old 60 GB 2.5" HDD I pulled was identical to work with to the 2 TB FireCuda that I also got delivered today. No different connectors, no jumpers, no USB adapters not passing SMART diagnostics to the host, none of that bullshit to deal with.
USB PATA/SATA adapters masking S.M.A.R.T data is one of the weirdest things I've had to deal with with computers, because sometimes a device that doesn't work on Windows works on FreeBSD, and other times it's the other way around.

Atomizer
Jun 24, 2007

Bote McBoteface. so what


D. Ebdrup posted:

USB PATA/SATA adapters masking S.M.A.R.T data is one of the weirdest things I've had to deal with with computers, because sometimes a device that doesn't work on Windows works on FreeBSD, and other times it's the other way around.

Oh, I didn't even know it had anything to do with the OS; I just figured that the janky, kinda-works PATA-USB3 cable I got had such a cheap controller that it didn't even pass SMART data. (I've a couple of USB2 enclosures that work just fine, but bottleneck anything >100 GB in capacity.) And that's pretty significant: of the 3 SATA drives I pulled, 2 were healthy but one had a ton of reallocated sectors and other warning flags thrown. I have to wait to peek into the PATA drives because I don't have a 40-44-pin cable and the janky USB3 thing I mentioned above only intermittently works.

I wrote about this a few months back, I think in another thread, that as an alternative to PATA-USB3 (to achieve better performance than USB2) I tried to use eSATA, but was never able to get it to work at all; I couldn't get a drive recognized, period, regardless of combination of: [apparently-properly-configured] host PC, internal drive, enclosure/adapter. I even tried a USB3-eSATA cable, and a SATA-eSATA enclosure (for the hell of it) with no luck. Anyone have tips for getting eSATA to work?

D. Ebdrup
Mar 13, 2009



Atomizer posted:

Oh, I didn't even know it had anything to do with the OS; I just figured that the janky, kinda-works PATA-USB3 cable I got had such a cheap controller that it didn't even pass SMART data. (I've a couple of USB2 enclosures that work just fine, but bottleneck anything >100 GB in capacity.) And that's pretty significant: of the 3 SATA drives I pulled, 2 were healthy but one had a ton of reallocated sectors and other warning flags thrown. I have to wait to peek into the PATA drives because I don't have a 40-44-pin cable and the janky USB3 thing I mentioned above only intermittently works.

I wrote about this a few months back, I think in another thread, that as an alternative to PATA-USB3 (to achieve better performance than USB2) I tried to use eSATA, but was never able to get it to work at all; I couldn't get a drive recognized, period, regardless of combination of: [apparently-properly-configured] host PC, internal drive, enclosure/adapter. I even tried a USB3-eSATA cable, and a SATA-eSATA enclosure (for the hell of it) with no luck. Anyone have tips for getting eSATA to work?
I suspect the core of it to be controller issues caused by the OEMs asking the ODMs to build down to a price rather than conforming to spec, because I have one USB3 to PATA/SATA adapter which works with S.M.A.R.T consistently across all OS' I've tried it on; it uses a NEC chip (before RENESAS took over their production line).
Unfortunately it's also a completely no-name/un-branded device which I have no clue where I bought, so I can't buy another one or recommend them.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »