«608 »
  • Post
  • Reply
Gozinbulx
Feb 19, 2004


G-Prime posted:

For something that small, unless you're talking about seriously high log volume, that's probably overkill. But on the flip side, it's not all that expensive to do a build like that. The CPU on mine is just a little below that and is ample.

What cpu are you using?

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


code:
Build
FreeNAS-Corral-10.0.4
Uptime
3 weeks 26 days 3 hours 6 minutes 44 seconds
System Time
Platform
Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
8 Cores @ 3.3GHz
CPU VM Support
Full
Memory
31.92GB
SSD
111.79GB x 1
HDD
7.28TB x 8
Now, in fairness, I'm not doing constant log writes. But seriously, unless you're managing to saturate a gigabit connection with the logs, there's literally nothing to worry about there. Especially if the box isn't going to be doing any kind of analysis on them. If it's literally just taking a dump of log streams from two other boxes, pretty much any modern CPU is overkill. With a near-constant 5mbit stream (read from disk and transmitted out) I'm under 5% CPU on one core. Write will incur more CPU cost due to checksumming and generating parity, but not enough to be a concern.

Just for giggles, I just did a 1GB write from /dev/random to a file on my array, and one core spiked to 30% CPU, at ~133MB/s.

Gozinbulx
Feb 19, 2004


Sounds cool. Do you do any "docks" or whatever of web server or apps like sonarr or whatever?

necrobobsledder
Mar 21, 2005
Lay down your soul to the gods rock 'n roll

Nap Ghost

Even when I ran VMs for my different services individually I didn't really have a problem with anything besides possibly RAM, and with Docker your resources are even more efficiently used including RAM. So unless you have less than 16 GB of RAM you shouldn't be hitting memory saturation issues with almost any service you'd use in a SOHO situation. If you don't know if what you're doing will use a ton of CPU or memory, chances are that with your usage pattern you're not going to use much at all.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


What he said. I'm running two VMs on mine right now, one of which is running something like 8 docker containers. The box is definitely being exercised, all day, every day, and the CPU never once has been the bottleneck.

Gozinbulx
Feb 19, 2004


G prime, what mobo are using, out of curiosity. Also do you do anything special to cool, or is a nice fat heatsink+fan ok?

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


Supermicro X10SL7-F in a Fractal Design Define R5 with the stock fan for the CPU and 3 Fractal Design Dynamic GP-14 fans on the case. None of the optional vents on the case are open, and the fans are set up with two on the front, one at the rear. The case is very good for airflow, and decent for sound dampening.

Edit: Under normal load, my CPU is around 38 degrees C and my 8x8TB WD Reds run anywhere between 29 and 35 degrees C dependent on their position in the case. The ones closest to the center of the fan body run a little on the warmer side.

G-Prime fucked around with this message at 03:44 on Aug 30, 2017

Gozinbulx
Feb 19, 2004


Seems that my proposed build would only be mildly more expensive than mimicing yours...

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


And I wouldn't suggest that you change it if that's what you want. I just wanted to point out that it's overkill. But here's the thing: It's easy to add more load to an idle system. It's hard to add more capacity to a busy system. I, as a rule, always overspec. You can, and probably should, do the same. All I was making sure you knew is that you're going to have that headroom, in spades.

Furism
Feb 21, 2006

Live long and headbang


Can anybody vouch for Chenbro products, more specifically this unit? It'll have to fit this M/B, 4xHDDs, 2x SSD and at least one PCI extension card (risers are fine).

Would this chassis work? Any alternative? 1U or 2U are fine.

Furism fucked around with this message at 10:09 on Aug 30, 2017

Thanks Ants
May 21, 2004

Bless You Ants, Blants



Fun Shoe

Looks like the box that the ReadyNAS 3138 uses

Furism
Feb 21, 2006

Live long and headbang


Thanks Ants posted:

Looks like the box that the ReadyNAS 3138 uses

I think I might be SOL because they specifically say "without audio jacks." And I think that's because those are higher than the rest of the components (http://www.asrock.com/mb/photo/N3150B-ITX(L5).png) but my motherboard has got those.

Oh well, still looking for recommendations.

Thanks Ants
May 21, 2004

Bless You Ants, Blants



Fun Shoe

You could just desolder the jacks

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull


EVIL Gibson posted:

I bring up rsync doing block by block hashing several times and someone already said the original post can be understood as the program was doing weird things to the files.

OK I misunderstood you, sorry!

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


My father-in-law's company just upgraded their CAD workstations again, he uses all Apple products (lol) so now I have a HP Z420 with some Sandy/Ivy Xeon E5 and a Quadro K4000.

For now I'm gonna strip my current NAS into the new chassis, including doubling up the RAM (I'm guessing it has 16 GB so this'll get me to 32 GB total).

edit: it's an E5-1650 with 32 GB, I should be able to cram in the extra 16 GB to run 48 GB.

It has an actual liquid cooler setup on the CPU. Super weird, even for a workstation, but the cooling solution on the Z400 was actually pretty decent too.

Paul MaudDib fucked around with this message at 15:33 on Sep 4, 2017

Kristneder
Jul 21, 2006

This is my first post.


So, with Crashplan ending their Home products, their "computer-to-computer" backup is also coming to an end.

I have a Windows machine running 24-7 at work that I currently backup to using the Crashplan client. Is there any alternative now that won't be possible in the future?

Furism
Feb 21, 2006

Live long and headbang


Kristneder posted:

So, with Crashplan ending their Home products, their "computer-to-computer" backup is also coming to an end.

I have a Windows machine running 24-7 at work that I currently backup to using the Crashplan client. Is there any alternative now that won't be possible in the future?

This was covered several times in the last few pages mate.

Kristneder
Jul 21, 2006

This is my first post.


Furism posted:

This was covered several times in the last few pages mate.

I might be slightly retarded, but I can't find anything in the last few pages about this?

Can't use FreeNAS (because the machine runs Windows)
It's not Blackblaze/Carbonite, they don't offer what I seek.
AWS/Azure is not what I need.
And I don't have a synology NAS.
Rsync is some headless terminal linux thing that seems hard to manage? Unless I'm misunderstanding how it works.

Did I miss any?

Edit: I did miss one; https://duplicacy.com/ that might be the one you were referring to.

Kristneder fucked around with this message at 16:35 on Sep 1, 2017

Furism
Feb 21, 2006

Live long and headbang


restic, which I personally mentions, also works.

You didn't specify if CLI is okay or not, and we mentioned a lot of clients with a GUI I don't want to be an asshole it's just that we kind of circle around.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Furism posted:

restic, which I personally mentions, also works.

You didn't specify if CLI is okay or not, and we mentioned a lot of clients with a GUI I don't want to be an asshole it's just that we kind of circle around.

I'm still leaning towards restic + backblaze B2. I did a small backup as a trial (40GB) and I really like the ease of use and performance so far.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

For the desktop side of things (documents & photos) I was trying out Google Drive for Windows. I like the ease of access from my Android phone. The Windows app doesn't seem very good though. I get random errors that certain files cannot be synced, despite there being nothing unusual looking about the file.

There's also tons of duplicate files like "20150616_230734 (1).jpg" and "20150616_230734.jpg" and the filesizes are quite different, with the renamed one being larger. Maybe it's storing some compressed version for whatever reason and they are bastards for renaming my original files. edit: upon further inspection I think it's any photos that I've "shared" through Google Photos. It creates a web optimized version. Lovely

I've also got a video that is nowhere to be found on my local copy of the Google Drive\Google Photos folder, yet I can see it just find in the web version.

As usual with consumer software from Google, details are nowhere to be found, and Google searches bring up echo chamber threads dating back years.

fletcher fucked around with this message at 09:37 on Sep 2, 2017

8-bit Miniboss
May 24, 2005

CORPO COPS CAME FOR MY


Oh boy, I loaded up Portainer for the first time on my server to help manage my Docker containers since OpenMediaVault's OMV-Extras implementation of their Docker GUI is a bit more jank than I'd like. Everything seems to be laid out pretty well and easy to navigate. Best of all they recently added a function to easily recreate/duplicate a container while also having the option to grab the latest image tag. This is handy to update for containers that can't do in-place upgrades like Plex.

Alternatively you could use Watchtower of course if you just needed an easy way to update containers, but I really wanted to try a new UI.

Twobirds
Oct 17, 2000

He's just using space travel as a metaphor for beating his wife.




Dinosaur Gum

I thought this might be a better thread than the Enterprise thread, so here goes.

I work at a plant and mostly do long-term projects and research stuff. One project collects data from production samples and stores it in a MySQL database. Our IT department gave me half a TB for a 'live' server on a VM, but since I'm not really production and there isn't a VP screaming at them about it, they told me I'm on my own beyond that (I don't blame them, things aren't that great here to begin with). It's been about a year and the live server is full. The company that made the analysis software suggested a NAS setup, though their recommended rack system is waaaay outside our budget. I was thinking about getting a Synology DS916+, two 4 TB drives, run a RAID-1, and move old data as needed off the live server to this long-term setup. This seems to make sense to me since it's expandable, doesn't need high user volume, and is in our budget (<$1k).

I don't work in IT/IS, so I'm a little clueless about some of this stuff. Does this make sense to do?

unknown
Nov 16, 2002
Ain't got no stinking title yet!


Simple thing: ask the it department to allocate some additional storage for your vm? or allocate another drive to your your vm and just move the Sql storage directory to it.

your version is basically doing the above manually (mounting a second drive on the server) and without IT knowing about it.

Getting a small Nas device will sort of work - it's probably "just" fast enough for MySQL, but you're in for a world of hurt when something goes wrong or your processing / io needs slightly increase.

Steakandchips
Apr 30, 2009



Don't do anything like getting a NAS and setting it up at your work network.

Get the IT department to do their job and give you more server space to do your job.

Farmer Crack-Ass
Jan 2, 2001

~this is me posting irl~


Twobirds posted:

I thought this might be a better thread than the Enterprise thread, so here goes.

I work at a plant and mostly do long-term projects and research stuff. One project collects data from production samples and stores it in a MySQL database. Our IT department gave me half a TB for a 'live' server on a VM, but since I'm not really production and there isn't a VP screaming at them about it, they told me I'm on my own beyond that (I don't blame them, things aren't that great here to begin with). It's been about a year and the live server is full. The company that made the analysis software suggested a NAS setup, though their recommended rack system is waaaay outside our budget. I was thinking about getting a Synology DS916+, two 4 TB drives, run a RAID-1, and move old data as needed off the live server to this long-term setup. This seems to make sense to me since it's expandable, doesn't need high user volume, and is in our budget (<$1k).

I don't work in IT/IS, so I'm a little clueless about some of this stuff. Does this make sense to do?

How crucial is this data? If that Synology caught fire and burned up all the data on it, how hard would you be fucked?

silence_kit
Jul 14, 2011


I figure that the posters in this thread might be able to help me with my problem with performing a backup to an external hard drive using Macrium Reflect. Macrium seems to be getting confused about how much disk space I have on my external drive. If anybody would be willing to check out link to the tech support thread below and help me out, it would be much appreciated.

https://forums.somethingawful.com/s...hreadid=3832882

Twobirds
Oct 17, 2000

He's just using space travel as a metaphor for beating his wife.




Dinosaur Gum

Farmer Crack-Ass posted:

How crucial is this data? If that Synology caught fire and burned up all the data on it, how hard would you be fucked?

I have backups in the form of barebones data that strips out image info (and consequently takes up a lot less space). That goes on a network share that is backed up properly. Dealing with that if the Synology melts down would be non-optimal, but doable.

Can I just offer to buy a drive or something for IT if they simply don't want to find room on what they have? I have no idea if it's that easy, but getting more space on the VM was my first request, and they didn't seem willing to do it. This is the group that, when asked for a phone cable, walked into another office and pulled it off someone else's phone. I'd definitely prefer to do it the right way, but I figured there had to be a reason I couldn't.

Farmer Crack-Ass
Jan 2, 2001

~this is me posting irl~


If I was on your IT department I'd be thinking "great, we gave this guy a half terabyte and he's used it up in a year; where are we going to be a few years from now?" The kind of storage they're using to hold their virtual machines may well be more expensive (on both a total and per-gigabyte basis) than the vendor-recommended unit you said was way out of your budget.

Another thing I'd be worried about in their shoes is that you say it's going to be low-traffic and doesn't need high performance, but if something changes and suddenly it's a high-traffic application (or if you or someone else sees a big lump of cheap storage and throws a new data-intensive application on it), who's the heat actually going to come down on when the application falls over because a couple of mirrored drives weren't up to the task?

Also, even if they were receptive to the idea, they may not have any place they can just plug in a couple of extra hard drives. The storage system(s) they have may already have all of their drive bays fully populated.



Here's some more important questions: Has your storage consumption been linear, or has it been accelerating? In other words, have you been recording a steady ~38GB per month every month (and will you continue to do so for the foreseeable future), or did you start at like 10GB a month and now you're at 50GB a month?

How long does this data need to be retained for? Three years, five years, ten years? Will you be able to retire/delete old data before you completely fill whatever expansion you go with, or will you have to look at a second expansion at some point? Will there be money for that expansion too?

How secure does this data need to be? If someone walked off with the NAS, what kind of risk would your company be exposed to?



My suggestion is to try to figure out what your long-term needs will be, and then go talk to IT and say "here's how much data I need to be able to store over the next [x] years; could you look into what the best solution would be and let me know an estimate of what it would cost?"

Then when you get the estimate back, take it to your boss and say "this is what IT says it would cost to properly store the data for this project" and then have your boss make the decision as to whether the data is worth the money, or whether the increased risk of data loss is an acceptable trade-off for the reduced cost in going with a cheap NAS.

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


What is the actual failure mode if a vulnerable drive in default/spanned configuration (non-RAIDZ non-mirrored) shits itself in ZFS, do I lose the data on the other drives too?

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!

ZFS does load balancing over the vdevs. If there's at least one non-redundant vdev, that shits the bed, your pool is fucked.

D. Ebdrup
Mar 13, 2009



Yeah, ZFS can't do concatenation (sometimes known as SPAN or BIG). At the very least, it'll function like RAID0 (striping with no parity or mirroring) when there's more than one vdev with one disk in each in a pool that data is written to. Ditto blocks (zfs set copies=n where n is 2 or more) will protect you from silent data corruption (as it does with just one disk without any mirroring), but if any disk catastrophically fails you instantly lose all data.

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


I need to move drives from one system/chassis to another and reattach them to the new motherboard - do I need to pay attention to drive ordering, or will ZFS rebuild it automatically from metadata or w/e?

Steakandchips
Apr 30, 2009



I did not need to care re ordering when moving from freenas server 1 to my freenas server 2

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.


It depends on if the drives were added by ID or by the slot. I think you can run 'zpool status' and see which it is. If it's by ID, ordering shouldn't matter. If it's by slot, putting them in the wrong order should be a problem, and I honestly don't know what the impact of doing so would be.

D. Ebdrup
Mar 13, 2009



As long as the IDs don't change from reboot to reboot (which'll happen on Linux because of non-persistent /dev enumeration), it doesn't matter whether it's by path (as is the case when you have many HBAs) or through simple /dev enumeration. As long as the ZFS data (and the cache file, if you don't want to wait a little, on large arrays) on one disk can be read, ZFS can run on any number of interfaces.

Speaking of which, I've heard a little bird whisper something about FreeBSD soon getting a way that can enumerate disk drives by serial number, including in the installer when setting up a root on zfs setup.

D. Ebdrup fucked around with this message at 15:03 on Sep 4, 2017

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


G-Prime posted:

It depends on if the drives were added by ID or by the slot. I think you can run 'zpool status' and see which it is. If it's by ID, ordering shouldn't matter. If it's by slot, putting them in the wrong order should be a problem, and I honestly don't know what the impact of doing so would be.

It's "sdb, sdc, sdd, sde" (ZFS on Linux, ubuntu). What is the more proper way to create a pool?

D. Ebdrup
Mar 13, 2009



Paul MaudDib posted:

It's "sdb, sdc, sdd, sde" (ZFS on Linux, ubuntu). What is the more proper way to create a pool?
The proper way to do it on Linux is to use /dev/by-id, since - as I mentioned before - Linux doesn't have persistent device enumeration.

D. Ebdrup fucked around with this message at 15:04 on Sep 4, 2017

Furism
Feb 21, 2006

Live long and headbang


Paul MaudDib posted:

I need to move drives from one system/chassis to another and reattach them to the new motherboard - do I need to pay attention to drive ordering, or will ZFS rebuild it automatically from metadata or w/e?

I'm about to do the same (moving from one OS to another in my case). I'm told I should zfs export before booting on the other OS?

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


Furism posted:

I'm about to do the same (moving from one OS to another in my case). I'm told I should zfs export before booting on the other OS?

I was wondering the same thing. A proper shutdown should have everything flushed, is it necessary to explicitly export it or is a shutdown enough synchronization?

edit: maybe just a shutdown wouldn't flush ZIL/SLOG?

Paul MaudDib fucked around with this message at 15:36 on Sep 4, 2017

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »