«608 »
  • Post
  • Reply
Star War Sex Parrot
Oct 2, 2003



Muldoon

Internet Explorer posted:

Are there any consumer DASes that can be connected via SATA/eSATA/SAS that can hold more than 5 drives?
Something like this?

https://www.startech.com/HDD/Enclos...ure~S358BU33ERM

Teabag Dome Scandal
Mar 19, 2002





Smellrose

DrDork posted:

Well, the Plex plugin is occasionally a little behind the latest and greatest, but if anything it's usually behind by some minor revision or two. It's not like Plex drops major updates anymore, but if you really want to go the jail route you can. It's just easier to dick up.

Transmission will happily let you move completed torrents to wherever you want and seed them from there until the sun expires (on completion you just do Set Location -> new location and it will take care of moving the file for you, as well as seeding). It also supports a script-on-completion function, but I've never really dug into how easy or hard it would be to do complex sorting to ensure TV shows go into one directory and movies in another or whatnot. There seem to be several example scripts floating around.

SickBeard is actually pretty easy to get to work if you go through the plugin interface, which again I highly recommend because it makes it a lot harder to dick up.

Which brings me to the big problem with trying to do manual jails on FreeNAS: You almost certainly will fuck up the permissions at some point. When you do, it can be extremely difficult to un-fuck them, and half the time the easiest path to recovery is to simply delete the entire /jail directory and start fresh. It's obnoxious as fuck, but on the other hand the plugin subsystem works quite well.

You certainly can go the Linux + ZFS route if you want. I think uTorrent can do the type of sorting you want using labels, which you can either manually assign (if you do you're not really saving any effort over Transmission + Set Location), or try out some of the auto-labeling plugins, which seem to have mixed success.

I'll have to give Transmission another look then because I thought I'd checked out the settings and didn't really see anything that led me to believe it could do what you describe under FreeNAS as far as scripting goes. I thought I'd checked for the Set location function as well since I know thats a fairly ubiquitous option with torrenting software these days and didn't think I saw that either but will give it another look. I would have honestly been happy with set location at minimum since I'm not really against a little post completion fiddling within the program but the web interface seemed super barebones compared to their already minimalist Mac version. Maybe I already had my heart set on something different and didn't give it a close enough look. Thanks! I'm not dead set on abandoning FreeNAS so I'll see if ditching the Plex jail and setting it back up using the plugin resolves whatever issues I'm currently having and give Transmission another look since from what you're saying it should actually do what I want it to be able to do.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Set Location is accessable by right clicking on a torrent in the web-based GUI. You can select a bunch of torrents at once and set them all together, which is nice if you just downloaded an entire series or something.

Internet Explorer
Jun 1, 2005





Oven Wrangler


That's exactly what I was looking for. Seems expensive for what it is, but I guess it is cheaper than buying an 8 bay Synology. I currently have 2 bay Synology and a Zotac Z-box. The 2 bay Synology is JBOD and backs up to my PC, and the Zotac Zbox is starting to show it's age, having trouble keeping up with higher bitrate transcodes.

Think about getting a beefier PC and moving to something like Xpenology, but if I am going that route I'd like more than a few hard drives.

Thanks for the link!

Matt Zerella
Oct 7, 2002


So o spin up a test VM of FreeNAS last night. So far so good but I have one issue: Transmission kind of sucks.

Deluge or rtorrent would be preferred and it looks like I can set up jails for one of them.


I guess my main question is, how does FreeNAS handle custom jails if I upgrade? Are the preserved?

D. Ebdrup
Mar 13, 2009



I just upgraded from FreeNAS 9.3 to 9.10 (read: FreeBSD 9.3 to 10.3) a little while ago, and both my MySQL and my TVHeadend jails have survived the upgrade just fine.

D. Ebdrup fucked around with this message at 17:23 on May 21, 2016

Matt Zerella
Oct 7, 2002


D. Ebdrup posted:

I just upgraded from FreeNAS 9.3 to 9.10 a little while ago, and both my MySQL and my TVHeadend jails have survived the upgrade just fine.

Good to know, thanks!

phosdex
Dec 16, 2005



Tortured By Flan

D. Ebdrup posted:

I just upgraded from FreeNAS 9.3 to 9.10 (read: FreeBSD 9.3 to 10.3) a little while ago, and both my MySQL and my TVHeadend jails have survived the upgrade just fine.

Same. I did the 9.3 to 9.10 upgrade yesterday and my non-plugin jail was preserved.

corded ware culture
Jul 16, 2007
mean green

thought it'd be worth a shot in the dark in this thread-- i have a nas with ~15tb of data that i want to get into a zfs pool or raid 6 array, but i'd rather not have to purchase all the extra drives to do so outright. there aren't any kind of short term storage solutions / nas rentals / other methods where i can temporarily store my data while i create an array with my existing disks? i know if the data is important i should have backup already..

Internet Explorer
Jun 1, 2005





Oven Wrangler

Sounds like a good excuse to get a backup in place.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Yeah, 3x5TB external drives aren't all that expensive.

There really aren't any "rentals" that you could do that would store that much data reasonably, unfortunately. Best bet if you needed it to be free would be to try to borrow some drives from friends (or work, if that's viable).

Hughlander
May 11, 2005



LmaoTheKid posted:

So o spin up a test VM of FreeNAS last night. So far so good but I have one issue: Transmission kind of sucks.

Deluge or rtorrent would be preferred and it looks like I can set up jails for one of them.


I guess my main question is, how does FreeNAS handle custom jails if I upgrade? Are the preserved?

What's wrong with transmission? I use it with a docker container that only lets it use a VPN IP so never looked at other server based clients. If you have the API set up does the client really matter?

Matt Zerella
Oct 7, 2002


Hughlander posted:

What's wrong with transmission? I use it with a docker container that only lets it use a VPN IP so never looked at other server based clients. If you have the API set up does the client really matter?

No support for multiple watch folders is a deal breaker for me.

E: I'm kind of thinking of going with unRAID.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

Raymn posted:

I've been using FreeNAS for a bit and am not really a fan. As a file server it has been great but I want it to run Plex and other random software and its been a bitch getting that all working properly. I've been thinking of switching and trying some flavor of Linux and importing my existing pool. This should be fairly easy, right?
Uh, more recent versions of FreeNAS do encrypt the disks with GEOM, which sits under ZFS. I'm not entirely sure how easy it is to decrypt everything and have GEOM be gone. I specifically created the pool by hand on the console, to avoid all that bullshit and get a clean unencumbered ZFS pool.

Hughlander
May 11, 2005



LmaoTheKid posted:

No support for multiple watch folders is a deal breaker for me.

E: I'm kind of thinking of going with unRAID.

Which is used for what? Multiple output folders? I really never did a lot of research there and just been dumping sabbzbd and transmission in the same output folder that sick rage, couchpotato, headphones, and Mylar are looking to post process in...

Mr Shiny Pants
Nov 12, 2012


LmaoTheKid posted:

No support for multiple watch folders is a deal breaker for me.

E: I'm kind of thinking of going with unRAID.

Use two instances?

Arvid
Oct 9, 2005


I´ve had a Synology DS213J where I installed one WD red 3tb drive for a while. The drive´s been almost full for some time so I decided to add another drive of the same type. After I installed the new drive it showed up in the store manager and I added it to the existing volume, however after adding it I can see it in the list of drives but the total size of the volume is still the same and the manage button in the storage manager is greyed out so I´m unable to do anything about it. How do I fix this ?

The picture below shows the status right now in the storage manager.

Toast Museum
Dec 3, 2005

30% Iron Chef


Arvid posted:

I´ve had a Synology DS213J where I installed one WD red 3tb drive for a while. The drive´s been almost full for some time so I decided to add another drive of the same type. After I installed the new drive it showed up in the store manager and I added it to the existing volume, however after adding it I can see it in the list of drives but the total size of the volume is still the same and the manage button in the storage manager is greyed out so I´m unable to do anything about it. How do I fix this ?

The picture below shows the status right now in the storage manager.



Sounds like it's because you're using SHR.

quote:

Note:

For SHR volumes, storage capacity is only expanded when adding hard drives to a volume with two or more hard drives.

https://www.synology.com/en-us/know...expand_add_disk

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Synology posted:

Synology Hybrid Raid (SHR) (With data protection of 1 disk fault-tolerance)
It basically has you set to a pseudo RAID-1 (mirror), so you got no extra space, but you now could lose 1 disk and not lose any data. Unsure if Synology will let you swap from SHR to JBOD or whatever to exchange the data protection for space.

eightysixed
Sep 23, 2004

I always tell the truth. Even when I lie.


DrDork posted:

It basically has you set to a pseudo RAID-1 (mirror), so you got no extra space, but you now could lose 1 disk and not lose any data. Unsure if Synology will let you swap from SHR to JBOD or whatever to exchange the data protection for space.

99.999999% sure you can't go from SHR-1 to JBOD. You definitely can't go from SHR-1 to SHR-2.

DrDork
Dec 29, 2003
commanding officer of the Army of Dorkness

Yeah, I rather doubt it, as well. On the upside, 3TB external drives are pretty cheap these days.

I don't pretend to understand the low-level methodology of how SHR works, but I find it surprising that you can't go from SHR-1 to -2.

eightysixed
Sep 23, 2004

I always tell the truth. Even when I lie.


DrDork posted:

I don't pretend to understand the low-level methodology of how SHR works, but I find it surprising that you can't go from SHR-1 to -2.

SHR-1 is basically Raid 5. SHR-2 is basically Raid 6. The upside to SHR is you can use different size disks. With that being said, can you migrate from Raid 5 to Raid 6 without having to rebuild the whole array as well? I've never tried, so I have no idea.

Thanks Ants
May 21, 2004

Bless You Ants, Blants



Fun Shoe

I am pretty sure you can't because the parity is spread all over the disks. You can do RAID4 to RAID-DP but that's a NetApp thing.

Arvid
Oct 9, 2005


Alright, error on my part then. Is there an easy way to backup everything on the NAS to my computer, wipe the volume and start with a new larger volume spanning both disks and then restore my data and everything to be exactly as before ?
I have a lot of shares, permissions, services etc set up that I´d rather not lose since I can´t really remember how I set them up in the first place

Krailor
Nov 2, 2001
I'm only pretending to care

Taco Defender

Arvid posted:

Alright, error on my part then. Is there an easy way to backup everything on the NAS to my computer, wipe the volume and start with a new larger volume spanning both disks and then restore my data and everything to be exactly as before ?
I have a lot of shares, permissions, services etc set up that I´d rather not lose since I can´t really remember how I set them up in the first place

If you just want to run both disks as one big JBOD volume then you can do the following:

1. Remove the new drive from your current SHR volume
2. Create a new volume on the new drive with volume type of JBOD
3. Copy all of your data from the old SHR volume to the new JBOD volume
4. Delete the SHR volume
5. Add your old drive to the new JBOD volume.

Arvid
Oct 9, 2005


Krailor posted:

If you just want to run both disks as one big JBOD volume then you can do the following:

1. Remove the new drive from your current SHR volume
2. Create a new volume on the new drive with volume type of JBOD
3. Copy all of your data from the old SHR volume to the new JBOD volume
4. Delete the SHR volume
5. Add your old drive to the new JBOD volume.

Thank you, I'll give this try. As for point 1 removing the drive, can I just physically remove it or do I have to do something in the disk manager first ?

Krailor
Nov 2, 2001
I'm only pretending to care

Taco Defender

Arvid posted:

Thank you, I'll give this try. As for point 1 removing the drive, can I just physically remove it or do I have to do something in the disk manager first ?

Now that I think about it I'm not sure Synology actually supports removing a drive from a SHR volume. You might have to do step 1 the hard way:

1. Shut down your NAS
2. Pull the new drive
3. Plug it into a PC and format it.
4. Plug it back into the NAS

Then continue on with the other steps.

Your NAS will probably whine at you about a degraded volume but you can ignore it and start copying the data over into the new volume.

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Arvid posted:

Thank you, I'll give this try. As for point 1 removing the drive, can I just physically remove it or do I have to do something in the disk manager first ?

You can not physically remove it, the point being you want them both connected. You are making a new JBOD volume on the empty drive, copying all data from your SHR drive, and then clearing the SHR drive and adding it to your JBOD drive.

Edit: What he said.

eightysixed
Sep 23, 2004

I always tell the truth. Even when I lie.


You sirs, are Gods amongst people that try to overthink things. I currently have 5x1TB in an SHR-1 Array on an Xpenology install, and was in the very, very near future going to upgrade to a 4x4TB SHR-2 array, and had been thinking about making a JBOD on the old-as-dirt DNS-323 and transferring it over the network, which would have taken ages, because for whatever reason my DNS-323 caps out at like 3.9MB/sec. I totally forgot about the option of setting up a second volume and just storing it on that in the interim.

I'll just put those two disks on a separate volume in JBOD, and move the data over while I install the new SHR-2 disks and move it all back and save hours. However after thinking about it, I kind of wanted to install Xpenology in an ESXi environment - but that would require me to change all the hardware out, anyway (i3/i5 or something with an accompanying mobo) - Currently running an Athlon X2 240, which somehow has suited my needs perfectly even on that old crap. It'll stream 1080p video with DTS/AC3/AAC/whatever audio just fine to a Raspberry Pi running Kodi with literally zero hiccups (with multiple users, even). Plex works just fine as long as it's just one stream as well (granted everything peaks to 100% concurrently and consistently, but so what as long as it works flawlessly).

Maybe I'll just build a second NAS for my ESXi ideas, and leave this one alone and migrate the data to a separate volume, install the new SHR-2 volume and move it all back.

With all of that being said, where is the best place to monitor any sales on the 4TB drives, with Memorial Day coming up and everything, someone will probably throw some sort of deal out there.

edit: Just to make sure, what I just said is completely possible, right?

eightysixed fucked around with this message at 00:37 on May 26, 2016

D. Ebdrup
Mar 13, 2009



If it was me, I'd leave the current machine doing its thing - because who knows whether or not you'll run into a disk failure caused by messing around with things too much - and instead build a second machine for ESXi, possibly with a Xeon D-15x1 based motherboard like the ones found here on page 31 and 32.


So I'm planning a new server now that ASRockRack have brought out some Xeon D-15x1 motherboards giving me the option of building a server that will be running FreeBSD 10.x-RELEASE with 18TB raidz2-on-root and bhyve+iohyve, serving as a thin-provisioned hypervisor, network attached storage for multiple machines via SMB, PVR backend+frontend and HTPC (via tvheadend + kodi), and possibly as a router and L2TP/IPsec concentrator at some point. Case offers both a PCI-ex slot for a HBA with external mini SAS HD ports and 10 3.5" drives with the HDD bay, so I can expand my pool quite a bit.

Case: Lian-Li PC-A04
HDD bay: Lian-Li Ex-23N
Motherboard+CPU: D1541D4U-2T8R
Memory: 4x Kingston KVR24R17D4/16
HDD: 5x Hitachi DeskstarNAS 6TB in raidz2
GPU: To be determined, but probably a passively cooled card.

Possible upgrades, will judge storage preformance before buying: 128GB SATA6G TLC SSD (for ZFS L2ARC) + 2x 32GB M.2 SLC SSDs (mirrored, for ZFS ZIL).

Can anyone think of anything wrong with this, or think of ways to improve it?

D. Ebdrup fucked around with this message at 13:53 on May 27, 2016

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll

Nap Ghost

I have the same case and use a 1x2 hard drive adapter bracket that I had with my HP Microserver in the top 5.25" bays. I'm considering a build option that lets me use a mini ITX Xeon D board and use an external HBA to allow me to use something like the SGI / Rackable SE3016 for keeping drives separate from the main compute hardware and potentially save some space or allow some more expansion options. Unless you need gobs and gobs of IOPS an older DAS like that is fairly viable (you'd max out throughput around 750 MBps which is fine for almost everyone at home) it should be fine. The only reason I haven't sprung for it is because trying to silence one of those disk shelves for home use is pretty awkward and then you're back in custom tower land again.

I run VMware ESXi 6 on my machine running FreeNAS in a VM as well as OS X in a VM. pfsense works pretty swell and since every service is on a virtualized network they all run at several Gbps, I didn't even bother trying to prioritize the VM or anything. Getting inter-VM latencies down (probably need some guest OS drivers that can do zero copy in the network stack) and throughput up on bhyve isn't obvious to me though. But at a point it might just be easier to use jails instead of bhyve, but I suppose that's what's good about running FreeBSD given you can do it all.

D. Ebdrup
Mar 13, 2009



That's not that bad of an idea, but I'm not sure how small mini-ITX cases actually get - since most of them are designed to fit at least a few 3.5" disks.
It might be an idea to use a Silverstone DS380, filling up just 6 bays full of disks internally connected to the Intel PCH (available on all Xeon D-15x1 boards), and then using a HBA with 2 external ports and drive enclosures like these.
The cool thing about daisy-chaining with these (and similar cases), as I understand it, is that it's possible to always expand while keeping at least two cables connected to the HBA, thus maintaining full redundancy.

FreeBSD is definitely very flexible, and you only really need bhyve if you're planning to run something that CANNOT be run through the linux compatability layer or wine.

(Non-stealth)EDIT: Edited the link to point to actual SAS expander drive enclosures.

D. Ebdrup fucked around with this message at 18:06 on May 28, 2016

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll

Nap Ghost

The really tiny mini ITX cases like the Dan case one fitting an ATX PSU are filled mostly by space for the PSU and heatsink. You can fit two mini ITX boards into a 1U as well. Regardless, I'm a maniac running OS X in a VM and may be changing my software setup anyway.

But I might go insane and get / custom build a Lian Li desk with support for multiple machines instead of buying a rack mount setup because it'll save space and at least look cooler.

phosdex
Dec 16, 2005



Tortured By Flan

The smallest mini-itx case I know of is the Minibox.com m350. I use one for my pfsense router and its perfect for that role.

VulgarandStupid
Aug 5, 2003
I AM, AND ALWAYS WILL BE, UNFUCKABLE AND A TOTAL DISAPPOINTMENT TO EVERYONE. DAE WANNA CUM PLAY WITH ME!?


phosdex posted:

The smallest mini-itx case I know of is the Minibox.com m350. I use one for my pfsense router and its perfect for that role.

The ISK-110 is the same size and comes with a PSU.

Furism
Feb 21, 2006

Live long and headbang


Before committing my whole data to a custom NAS I wanted to run some tests in VMs. I find that I like Nas4Free the most, so I went with that. I created a VM with 5 disks: one for the OS, 4 to emulate a ZFS cluster. Each disk is 20 GB large.

So I created a RAID-Z1 with the 4x20GB drives and N4F shows me 80 GB available for storage. I expected about 60 GB, the remainder being used for the parity? Anyway I copied some files, then removed a drive from the VM. N4F correctly detected the missing drive, told me the volume was in a degrated state. I replaced the missing drive and resilvered and it's all back to normal with all the files. I think.

If my drives are full (let's say I used 70 GB on the volume), I would have lost some data right? What is the maximum I can use to make sure I can lose any given disk and not lose any data?

Mr Shiny Pants
Nov 12, 2012


Furism posted:

Before committing my whole data to a custom NAS I wanted to run some tests in VMs. I find that I like Nas4Free the most, so I went with that. I created a VM with 5 disks: one for the OS, 4 to emulate a ZFS cluster. Each disk is 20 GB large.

So I created a RAID-Z1 with the 4x20GB drives and N4F shows me 80 GB available for storage. I expected about 60 GB, the remainder being used for the parity? Anyway I copied some files, then removed a drive from the VM. N4F correctly detected the missing drive, told me the volume was in a degrated state. I replaced the missing drive and resilvered and it's all back to normal with all the files. I think.

If my drives are full (let's say I used 70 GB on the volume), I would have lost some data right? What is the maximum I can use to make sure I can lose any given disk and not lose any data?

You will not lose data if the disks are full. It will resilver everything.

IOwnCalculus
Apr 2, 2003





N4F seems to report the pool size before parity losses. Do it again, write some known amount of data, and then see what it says is used / available.

D. Ebdrup
Mar 13, 2009



The best way to get an actual idea of how much disk space you have available is not to look at zpool list, but instead use zfs list and add together the USED and AVAIL row for your pool name - here's an example (with irrelevant pools, vdevs, zvols and everything else snipped):
code:
shai-hulud.nerdheaven.local ~/ debdrup~$ sudo zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage        7.25T 6.23T   1.02T        -   23%     85%  1.00x  ONLINE  /mnt

shai-hulud.nerdheaven.local ~/ debdrup~$ sudo zfs list  
NAME                                                           USED  AVAIL  REFER  MOUNTPOINT
storage                                                        4.53T 592G   14.3G  /mnt/storage
As you can see, zpool list gives the raw disk size and the allocated size (that is, the size that's available to all datasets and internal metadata), whereas zfs list gives you the used and available amount that you can then add together to figure out how much actual diskspace you have available that isn't taken up by metadata or parity.

Interestingly, though not surprisingly, df is of absolutely no use if you collect your stuff in datasets according to content (so a dataset with compression for documents and ebooks, and one without compression for audio and video):
code:
shai-hulud.nerdheaven.local ~/ debdrup~$ df -h
Filesystem                                                       Size    Used   Avail Capacity  Mounted on
storage                                                          606G     14G   592G  2%        /mnt/storage

D. Ebdrup fucked around with this message at 10:04 on May 31, 2016

Shaocaholica
Oct 29, 2002

Fig. 5E


Can't seem to find any info on what internal drive is inside the 'WD 8TB My Book Desktop External Hard Drive'.

https://www.amazon.com/gp/product/B...=A2L77EE7U53NWQ

PMR?

edit: oh now I'm reading there's some sort of hardware encryption on the externals that renders them useless for harvesting....

edit2: Seems like a WD80EZZX is inside and the hardware encryption thing might just be a rumor

Shaocaholica fucked around with this message at 16:44 on Jun 1, 2016

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »