«608 »
  • Post
  • Reply
Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


Is there a reason to do that on consumer gear, vs just having a UPS and being set to flush and shutdown as soon as it sees power is lost (NUT)? The only use-case I see the actual power-backed hardware solving is if someone physically unplugs the PC from the UPS, which... just don't do that.

D. Ebdrup
Mar 13, 2009



Paul MaudDib posted:

Is there a reason to do that on consumer gear, vs just having a UPS and being set to flush and shutdown as soon as it sees power is lost (NUT)? The only use-case I see the actual power-backed hardware solving is if someone physically unplugs the PC from the UPS, which... just don't do that.
The way to size a SLOG SSD is to look at the maximum syncronous data you're writing per second, multiple that by 5 (because that's the length of a transaction group), and then multiple it by 2 to ensure that it can completely finish writing to disk.
You'll quickly discover how small SSDs you can get away with buying, or buy them a bit bigger and over-provision them to extend their write-endurance, or go somewhere in between those and buy two so they're mirrored (as you should, since if you lose one, you lose data).

D. Ebdrup fucked around with this message at 19:46 on Jun 16, 2019

CommieGIR
Aug 22, 2006

If Godzilla can do it, you know I can deliver!

Pillbug

Yeah, I'm gonna see how many 128 or 240 GB ssds I can pick up for slog

I got the batch of 900gbs at next to nothing

Crunchy Black
Oct 24, 2017

CASTOR: Uh, it was all fine and you don't remember?
VINDMAN: No, it was bad and I do remember.




CommieGIR posted:

Yeah, I'm gonna see how many 128 or 240 GB ssds I can pick up for slog

I got the batch of 900gbs at next to nothing

Man, you guys up by the dam must get your power for nothing!

sharkytm
Oct 9, 2003

Gimme Gimme Swedish Fish...



Fallen Rib

Solar for me.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell



Crunchy Black posted:

Man, you guys up by the dam must get your power for nothing!

Not saying this is you but...

I find everyone overestimates how much power NAS hardware and drives actually use on an absolute basis.

I've got an old i5 server with 20+ hard drives in it and the last time I hooked up my kill-a-watt to measure its power draw it cost me like 4 or 5 bucks a month to run 24/7.

H2SO4
Sep 11, 2001

put your money in a log cabin




Buglord

fletcher posted:

If you want a synology accessible from anywhere, how are you guys typically doing that?

My friend has the Synology web interface exposed to the internet on a non-standard port to accomplish this. Is the Synology security trusted enough to do this? I would have stuck it behind a VPN.

Synology has a whole built in mechanism for remote access without going suicidal like this. They call it QuickConnect.

KOTEX GOD OF BLOOD
Jul 7, 2012



Fallen Rib

H2SO4 posted:

Synology has a whole built in mechanism for remote access without going suicidal like this. They call it QuickConnect.
There is no fucking way I trust that shit.

H110Hawk
Dec 28, 2006


H2SO4 posted:

Synology has a whole built in mechanism for remote access without going suicidal like this. They call it QuickConnect.

Yeah that is hot garbage and no one should use it.

H2SO4
Sep 11, 2001

put your money in a log cabin




Buglord

you're right you should totally port forward to the management IP instead

Setup a VPN and use dynamic DNS like a grownup then.

Heners_UK
Jun 1, 2002


Am I the only one who has an RPi acting as my remote access server? It used to run OpenVPN but I found myself just as happy connecting via SSH (key based auth) to it then tunneling through there to my home server?

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


SSH tunnelling is fine too, just make sure you're using key-auth, with password auth disabled.

jeeves
May 27, 2001

Deranged Psychopathic
Butler Extraordinaire


I have a ten year old HP Proliant with 4x drive bays (+5th drive in 5" bay) that is running all 2TB drives in ZFS.

This is a NAS so I don't need it to be fast, but I remember reading ZFS gobbles up RAM and thus I put in a ridiculous 16GB or RAM into it or something at the time for my ~8TB usable pool. If I replace the drives with 4TB or 6TB drive, I'm guessing the existing RAM amount will be fine.

Also, I am running FreeNAS and it always takes about 4-5 minutes to boot off of the internal USB2 drive. I'm thinking of plugging in an SSD into the unused eSATA port in the back and snaking in a eSATA to SATA cord for it. Would that actually boot up the system faster, or does FreeNAS take like 4-5 minutes to load regardless?

D. Ebdrup
Mar 13, 2009



jeeves posted:

I have a ten year old HP Proliant with 4x drive bays (+5th drive in 5" bay) that is running all 2TB drives in ZFS.

This is a NAS so I don't need it to be fast, but I remember reading ZFS gobbles up RAM and thus I put in a ridiculous 16GB or RAM into it or something at the time for my ~8TB usable pool. If I replace the drives with 4TB or 6TB drive, I'm guessing the existing RAM amount will be fine.

Also, I am running FreeNAS and it always takes about 4-5 minutes to boot off of the internal USB2 drive. I'm thinking of plugging in an SSD into the unused eSATA port in the back and snaking in a eSATA to SATA cord for it. Would that actually boot up the system faster, or does FreeNAS take like 4-5 minutes to load regardless?
The access time shouldn't be faster in theory, but probably is in practice. Bandwidth is also higher, so that might help some (I'm not sure if FreeNAS uses crunchgen (basically, making a whole OS into one binary) but a SSD would be significantly faster for sequential data than USB2 (20MBps read in practice) and even if they don't, the SSDs are still faster).
Also, once they upgrade to 11.3 or 12.0 (maybe 11.3, definitely 12.0; I don't know if the appropriate code has been merged to the stable branches), they should get a bunch of code changes from FreeBSD that make booting quicker (ie. speeding up how VT draws bitmapped text, and a whole series of other changes).

Also, this came up in an IRC channel I'm in, so while I'm not affected since I run FreeBSD, I just wanna relay it: iX has decided that anonymous usage stats is opt-out in FreeNAS 11.3.

D. Ebdrup fucked around with this message at 20:40 on Jun 17, 2019

Crunchy Black
Oct 24, 2017

CASTOR: Uh, it was all fine and you don't remember?
VINDMAN: No, it was bad and I do remember.




Are you really having to reboot your storage server that often?

The Milkman
Jun 22, 2003

No one here is alone,
satellites in every home


Lipstick Apathy

With how cheap they've gotten I've upgraded the SSDs in my desktop and laptop. Which now leaves me with an extra 500gb drive kicking around. Is it worth it to put it into my FreeNAS box? I'm still booting off dual USB sticks, which was the style at the time. Can I partition it for boot as well as a cache drive? I'm not dying for the cache part, but if I can why not right?

D. Ebdrup
Mar 13, 2009



The Milkman posted:

With how cheap they've gotten I've upgraded the SSDs in my desktop and laptop. Which now leaves me with an extra 500gb drive kicking around. Is it worth it to put it into my FreeNAS box? I'm still booting off dual USB sticks, which was the style at the time. Can I partition it for boot as well as a cache drive? I'm not dying for the cache part, but if I can why not right?
Sure, you can partition it any way you want. Do you want to, though?
L2ARC requires mapping memory segments with pointers to LBA sectors on the disk, which reduces the size of the ARC (330 bytes per LBA, so you can easily estimate how much when you know how many sectors your disk has; camcontrol identify <device> can tell you).
SLOG is a bad idea if you can't mirror it, because any data that's on the drive if it fails or experiences an URE, is lost data and will require you to mess around with zdb to back-track to the last known-good TXG.

D. Ebdrup fucked around with this message at 21:29 on Jun 17, 2019

H2SO4
Sep 11, 2001

put your money in a log cabin




Buglord

Don't attach a cache drive unless your stats show you need one. I think newer versions of FreeNAS let you detach them from a pool gracefully but that used to be a one way operation. That said, I did use old small SSDs as a mirrored boot pair in my freeNAS box, but with a 500Gb drive I'd probably let it sit until I found a better use for it.

IOwnCalculus
Apr 2, 2003





If you don't have one yet, consider using the SSD as a landing drive for new data. I have my server set up such that anything inbound lands on a SSD first and doesn't get moved to the ZFS array until after it's complete. For simple file downloads this is probably overkill, but it seems like this should help keep torrent downloads from getting fragmented as hell.

The Milkman
Jun 22, 2003

No one here is alone,
satellites in every home


Lipstick Apathy

H2SO4 posted:

Don't attach a cache drive unless your stats show you need one. I think newer versions of FreeNAS let you detach them from a pool gracefully but that used to be a one way operation. That said, I did use old small SSDs as a mirrored boot pair in my freeNAS box, but with a 500Gb drive I'd probably let it sit until I found a better use for it.

I almost certainly don't need it. It's 90% a home media server with a few projects on the side. I already took one of the drives and put into an enclosure as portable storage that I don't really need either. So it's either throw it in the server or see if a friend needs it I guess.

D. Ebdrup posted:

Sure, you can partition it any way you want. Do you want to, though?

Only insofar if it's Free Real Estate. Having a minimal benefit isn't a deterrent but over-complicating things definitely is.

IOwnCalculus posted:

If you don't have one yet, consider using the SSD as a landing drive for new data. I have my server set up such that anything inbound lands on a SSD first and doesn't get moved to the ZFS array until after it's complete. For simple file downloads this is probably overkill, but it seems like this should help keep torrent downloads from getting fragmented as hell.

This might be what I actually want? How do you set that up

jeeves
May 27, 2001

Deranged Psychopathic
Butler Extraordinaire


Crunchy Black posted:

Are you really having to reboot your storage server that often?

I keep my NAS off when I am not using it. I've had it for like 8-10 years and the hard drives are still a-ok.

I don't need it on all of the time, I use it for more archival stuff. I have a lower power NUC I use for my day to day 24/7 usage.

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Something I was thinking about was using a raspberry pi to wake up and sleep my nas so I can power it down when it isn’t being used and then turn it on so it can serve videos or whatever during the evening.

Is there any canned solutions for this? Something using wake on lan or a usb press.

Paul MaudDib
May 3, 2006

"Tell me of your home world, Usul"


priznat posted:

Something I was thinking about was using a raspberry pi to wake up and sleep my nas so I can power it down when it isn’t being used and then turn it on so it can serve videos or whatever during the evening.

Is there any canned solutions for this? Something using wake on lan or a usb press.

https://wiki.archlinux.org/index.php/Wake-on-LAN

https://wiki.debian.org/WakeOnLan

set up wol on a cron-job or create an alias that sends the packet you want. Never played with it myself and there may be limitations on what interfaces can receive magic packets while the system is down (it's Intel ME that does the reception afaik).

If you have a real NAS you can of course use the ipmitool utility and set a cron-job or whatever.

Paul MaudDib fucked around with this message at 22:35 on Jun 17, 2019

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Yeah hopefully my next nas build will have an ipmi, that would solve some hassle.. unless it requires some bullshit unlock license

Heners_UK
Jun 1, 2002


priznat posted:

Is there any canned solutions for this? Something using wake on lan or a usb press.

Just curious, do you have a home automation system by any chance?

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Heners_UK posted:

Just curious, do you have a home automation system by any chance?

I do but it is homekit. I want something easy as heck because I do similar stuff at work most days and I don’t want to have to do it at home if possible

It’s why I love pi-hole, so easy to setup and gooooo (I have another rpi for setting up automation on if there is a good option)

Heners_UK
Jun 1, 2002



Ahhh nuts. Had it been something that ran webcore for sure, and probably a few others, I could have suggested creating an automation that sent a WOL packet if tapped a button or triple tapped a lightswitch. Not sure if that can be done with homekit.

New idea, although one that assumes the same network: Some phone app that sends WOL packets?

priznat
Jul 7, 2009

Let's get drunk and kiss each other all night.

Heners_UK posted:

Ahhh nuts. Had it been something that ran webcore for sure, and probably a few others, I could have suggested creating an automation that sent a WOL packet if tapped a button or triple tapped a lightswitch. Not sure if that can be done with homekit.

New idea, although one that assumes the same network: Some phone app that sends WOL packets?

I do have a spare RPi sitting around so I can definitely do the cron job idea Paul posted earlier, which would work nicely. So I could check out the webcore!

What would be great would be having some kind of remote control web front end so I could manually toggle it and adjust the on/off times without having to edit crontab etc.

This is probably leading to me adding more home automation stuff beyond homekit at this point too since why not

Heners_UK
Jun 1, 2002


priznat posted:

I do have a spare RPi sitting around so I can definitely do the cron job idea Paul posted earlier

That's what I did for ages, but instead of a web interface, I just used juiceshh and remembered (pressed up until I found) the command & mac addr

xzzy
Mar 5, 2009

wakey wakey to
this bowl of tasty


Yams Fan

Paul MaudDib posted:

SSH tunnelling is fine too, just make sure you're using key-auth, with password auth disabled.

With fail2ban and laugh at the logs every so often.

eames
May 9, 2009



priznat posted:

Something I was thinking about was using a raspberry pi to wake up and sleep my nas so I can power it down when it isn’t being used and then turn it on so it can serve videos or whatever during the evening.

Is there any canned solutions for this? Something using wake on lan or a usb press.

I read about a solution for this where one would have a rPi and the NAS (in standby) on a shared virtual IP using some open source failover software.
Accessing the shared virtual IP would first contact the always-on rPi which would then automatically send a magic packet to the NAS to wake it. The NAS would then take priority over the rPi IP and receive all traffic until the next standby. Like Wake on LAN except completely automatic and hands off. Sadly it was pretty complicated to set up.
Somebody should make a prebuilt rPI image for this, it would probably save a ton of power.

D. Ebdrup
Mar 13, 2009



xzzy posted:

With fail2ban and laugh at the logs every so often.
Have you been introduced to blacklistd by Christos Zoulas over at NetBSD? It's like fail2ban, except it doesn't grep log files (or fail to, in the case of IPv6) - instead it relies on the daemons to inform it when a certain exceeds the allowed logins; so while it requires modifications, most daemons that you'll want to use it on (examples like sshd in base, postfix and sendmail from ports) already support it.

Progressive JPEG
Feb 19, 2003



If you're already relying on patching the daemon to tell you when to block something, why can't the daemon do the blocking itself?

Ika
Dec 30, 2004
Pure insanity



eames posted:

I read about a solution for this where one would have a rPi and the NAS (in standby) on a shared virtual IP using some open source failover software.
Accessing the shared virtual IP would first contact the always-on rPi which would then automatically send a magic packet to the NAS to wake it. The NAS would then take priority over the rPi IP and receive all traffic until the next standby. Like Wake on LAN except completely automatic and hands off. Sadly it was pretty complicated to set up.
Somebody should make a prebuilt rPI image for this, it would probably save a ton of power.

I heard that somebody did that but never found a good guide. Would be awesome. The pi could probably be bound to two IP addresses and also be used for other things as well, right?

eames
May 9, 2009



Ika posted:

I heard that somebody did that but never found a good guide. Would be awesome. The pi could probably be bound to two IP addresses and also be used for other things as well, right?

yeah, both devices would keep their regular IP addresses and you use the virtual ip to access the NAS during standby and operation.
I think the software used was keepalived and a custom script that parses firewall logs to the virtual IP to trigger the wake script.
I saw the idea in the comments of a NAS review and was fascinated by it but it seemed (and still is) beyond my scope to reproduce it without a guide.
Apple does something very similar with the Bonjour sleeproxy, so Apple TVs and Airport Express routers can wake macs on network access. I've been told new Synology NAS appliances can also go to sleep and wake on network access.

Atomizer
Jun 24, 2007

Bote McBoteface. so what


jeeves posted:

I keep my NAS off when I am not using it. I've had it for like 8-10 years and the hard drives are still a-ok.

I don't need it on all of the time, I use it for more archival stuff. I have a lower power NUC I use for my day to day 24/7 usage.

priznat posted:

Something I was thinking about was using a raspberry pi to wake up and sleep my nas so I can power it down when it isn’t being used and then turn it on so it can serve videos or whatever during the evening.

Is there any canned solutions for this? Something using wake on lan or a usb press.

Both of your use cases sounds more like they need a simple external drive (or multi-bay enclosure) as opposed to a NAS. I think it's assumed that one operates a NAS 24/7 so multiple clients can connect to it as needed. In jeeves' case, if you're manually powering on your "NAS" then you're basically just operating a file server, and one with, what, 8 or 10 TB of capacity currently? 10 TB external USB HDDs are typically ~$160 on sale, and you could have one or two of those connected to that NUC.

I use an SFF system (HP EliteDesk) as my multi-purpose server (including Plex) with an external drive for media storage, and although I leave it on 24/7, it's effectively doing exactly what you guys want to do; it even works perfectly with WoL.

D. Ebdrup
Mar 13, 2009



Progressive JPEG posted:

If you're already relying on patching the daemon to tell you when to block something, why can't the daemon do the blocking itself?
Patching the daemon is a one-time diff or pull-request to an upstream project, unless the project actively refuses to be portable.
You still need something to maintain the list of IPs that need to be blocked as well as keep track of how long they've been blocked.
The other advantage of blacklistd is that it's firewall-agnostic because it's intended to be widely ported (as one might expect from someone using NetBSD) - meaning it works with ipfw, freebsds pf, openbsds pf, ipf, npf (NetBSDs new firewall that's pretty amazing), iptables, bpf-tables (Linux finally getting a packet filter based on Berkeley Packet Filter, it only took them almost 3 decades), and anything else you wanna use (even Windows' firewall).

The other problem with sshguard, fail2ban, and the like - aside from the minor issue with dual-stacking which most of them don't handle right or took almost two decades after IPv6 to fix - is that by grepping log files like this means they're surprisingly susceptible to denial of service.

Eletriarnation
Apr 6, 2005

People don't appreciate the substance of things...
objects in space.



Oven Wrangler

Atomizer posted:

10 TB external USB HDDs are typically ~$160 on sale
Speaking of that, it's live right now.

Heners_UK
Jun 1, 2002


Damn I wish I had CAD$440 spare for two of them.

For those in Canada, 8TB WD My Books for CAD$195+tax

eames
May 9, 2009



Anandtech reviewed the Seagate hybrid SMR drives discussed here a few weeks ago

https://www.anandtech.com/show/1453...-portable-smr/4

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«608 »