|
Thermopyle posted:Crashplan. Should probably add it to the OP or something at this point ![]()
|
![]() |
|
IOwnCalculus posted:Should probably add it to the OP or something at this point Probably! So, Crashplan has a free version that you can use to backup to your own server? edit: Read FAQ, question answered. Awesome program. <3 Goon recommendations edit2: Reading about ZFS it appears that it plays nicely with Ubuntu, are there any other options for Linux distros that run nicely with it (like maybe CentOS)? blacksun fucked around with this message at 02:38 on Apr 22, 2013 |
![]() |
|
So inspired in part by this thread, and in part by the XBMC thread, I tried to setup a small linux fileserver. It ended pretty poorly: Nothing I could do over the course of 5 hours could make windows 7 talk to the samba shares on the fileserver. Is this the appropriate thread to ask for help with samba problems, or is that better suited to the general linux questions thread? Regardless, I'm curious about how this works with things like FreeNAS/unRaid/all the other things actually recommended in this thread. Do they come with the smb shares preconfigured via magic or would I have to go through all this again if I switched to a setup like that?
|
![]() |
|
I think this was covered before but I can't seem to find the post now - is it possible to run the Crashplan software on NAS4Free? Or do you have to run it from something that has a full OS installed?
|
![]() |
|
Giant Isopod posted:Regardless, I'm curious about how this works with things like FreeNAS/unRaid/all the other things actually recommended in this thread. Do they come with the smb shares preconfigured via magic or would I have to go through all this again if I switched to a setup like that? fletcher posted:I think this was covered before but I can't seem to find the post now - is it possible to run the Crashplan software on NAS4Free? Or do you have to run it from something that has a full OS installed?
|
![]() |
|
I think I'll install Ubuntu on my AD10 and have it be run Crashplan as well to back up the NFS mounts - that is kosher right? I really like OpenELEC, bummed I gotta ditch it. At least I'll be able to watch The Daily Show on my TV now since I can install Flash player.
|
![]() |
|
Hi, I am having a small problem with my NAS. I have a synology DS213 wtih 2x2 TB discs. The disk quota shows 1.3 TB are occupied, but when I go through my shared folders, only 500 GB are accounted for. 800 GB seem to be used up but don't show anywhere. How can I find out where those 800 gb are hiding?
|
![]() |
|
BabyFur Denny posted:The disk quota shows 1.3 TB are occupied, but when I go through my shared folders, only 500 GB are accounted for. 800 GB seem to be used up but don't show anywhere. How can I find out where those 800 gb are hiding?
|
![]() |
|
What are my options if I want to stop using FreeNAS and switch to Ubuntu or something? Do I basically have to get the space to copy everything off of my ZFS hard drives, do that, then reformat them all?
|
![]() |
|
http://howtounix.info/howto/zfs-on-ubuntu
|
![]() |
|
tarepanda posted:What are my options if I want to stop using FreeNAS and switch to Ubuntu or something? Do I basically have to get the space to copy everything off of my ZFS hard drives, do that, then reformat them all? You should be able to export you your zpool and reimport it once you have your linux system installed and configured. I am doing the same thing tomorrow when Ubuntu 13.04 comes out.
|
![]() |
|
Lowen SoDium posted:You should be able to export you your zpool and reimport it once you have your linux system installed and configured. Heh, that's exactly why I was asking. Let me know how it goes... a step-by-step walkthrough for dummies would be nice, too!
|
![]() |
|
tarepanda posted:Heh, that's exactly why I was asking. Let me know how it goes... a step-by-step walkthrough for dummies would be nice, too! I will probably write something up and post it here since I have time to kill this weekend. I will be using Ubuntu server, but the steps should be largely the same for any version of Ubuntu.
|
![]() |
|
Lowen SoDium posted:I will probably write something up and post it here since I have time to kill this weekend. I'm looking forward to reading this!
|
![]() |
|
Has anyone else used a flashed M1015 with SAS drives? I migrated my file server to this controller from a 8708EM2 (which has been working for over 2 years), and my existing SAS drives have been doing nothing but throwing fits the entire time. The RAID card will fail a disk, start rebuilding on another, then drop more disks during the rebuild. There are no useful reported events in the log and the configuration just says it is rebuilding. I thought it was something to do with RAID, so I broke the array and used a single disk (still on the SAS expander). It very quickly proved to not be fixed. Then, thinking it was something to do with the SAS expander/RAID controller combination, I used the second port on the RAID card to plug a single drive directly into it. This was doing fine until tonight, where it exploded while I was doing something completely unrelated. Currently, I'm imaging the SAS drive over to a spare SATA drive that I have laying around until the Samsung Pro I ordered earlier this week arrives. On a related note that is better, ZFS on linux works great. I just migrated my disks over from various RAID 10/6 setups into a single consolidated pool with sub-pools. This makes it much easier to manage. code:
code:
thideras fucked around with this message at 05:49 on Apr 25, 2013 |
![]() |
|
Why ZFS on Linux over FreeNAS or NAS4Free? I started my NAS setup using NAS4Free and while I liked that it had multiple installiation options, I was disapointed to find that their plugin system is not complete yet and installing and configuring programs is a manual process. I switched to FreeNAS which has a working plug in system and several fuctioning plug ins for standard apps like SABNZBD and Sickbeard. But there is no way to export your plug in settings and upgrading your plug-in jail means starting over. Also, I wanted to be able to run Crashplan with out resorting to a lot of work arounds or recompiling FreeNAS to include the required libraries. While I am comfortable with FreeBSD's command line, I am much more accustomed to Linux, specifically Ubuntu. I find Ubuntu's package management to be much more user friendly and reliable than FreeBSD/FreeNAS. Linux also has a wider varity of hardware support and more applications. If all you are doing is serving files, then there is probably not much use for using anything other than FreeNAS or NAS4Free since they are the simplest to get up and running. IF you want to run many applications, especially outside of the SABNAZD and Transmission varity, you might be better off using Linux. That is if you don't mind a little bit of work upfront. My Setup This guide is going to mirror my installation. Your configuration will likely be different, so you will need to adjust accordingly Here is the hardware I am using (Note that that CPU is no longer for sale and has been replaced with the Intel Celeron G1620 Ivy Bridge 2.7GHz which is about the same price) This motherboard: $100 This RAM: $125 This CPU: $60 This heatsink/fan: $48 This case: $120 This power supply $80 Short cables for that powersupply (optional) $25 USB3 header cable to use thumbdrive inside case (optional) $14 USB3 flash drive to install on $27 Intel NIC for better network performance (optional) $29 3 of these Harddrives for storage $479.97 Things I already had that are going to be used for this were SATA cables, an aditional USB drive to install Linux from, a 1TB SATA Seagate harddisk to use for downloading and unpacking files (optional), and a keyboard & monitor. The 1 TB drive is an older drive that I am going to set up for apps like Sickbeard and Transmission to use to download files and unpack them before moving them to their permanate home on the ZFS array. This is to reduce the work done on the expensive disk that hold all my important data to maybe reduce the chance of failure. This is probably completely unnessisary and if I didn't have this extra 1TB disk laying around, I don't think I would bother with it one bit. I am going to install Ubuntu on to the USB3 flash drive I listed above. I don't think this will be an issue since that flashdrive has decent reads and writes and since this system won't be doing much out. So far the system boots on it very quickly. I am also going to be exporting my exsisting ZPOOL array from FreeNAS, but I will also breifly cover creating a new ZPOOL if you are working from scratch. Anyways, most of that is not really important for this guide. What is important is that I am NOT going to be installing Linux on the ZFS Pool. I saw a few people talking about doing this in the ZFS discussion on Google Groups, but I personally think that is a bad design for this system and I will not be covering it here. To recreate what I am doing here, you will need your NAS drives plus at least one other disk, either SATA or a USB drive with decent speeds. Getting Started Go download your Ubuntu ISO. I am going to using the freshly released Ubuntu 13.04 Server. But these instructions should work for most versions. You will need a 64Bit version for this to work properly. Some of things you are going to need to do are a lot easier to do if you have a GUI, such as setting your IP static and creating SMB shares, etc. Its up to you how to handle this and I will not be going in to how to do several of those things. This guide is more about setting up ZFS. We need to make a bootable usb stick from that ISO. I suggest Universal-usb-installer. It's free and it works. Ubuntu Server 13.04 doesn't show up on the install list for some reason, but you pick the generic "Other Linux option and it works fine. Next, if you have an existing ZFS raid on FreeNAS, NAS4Free, or some other system, export it. You will only be able to import it in Linux if it is version 28 or lower. code:
Install Ubuntu on your system. Double check that you are installing it on the correct drive. Nothing is going to wreak your day worse than formating one of your exsisting data drives. Durring the install, if you are using Ubuntu Service, it will give you some options for componates to install. I went ahead and installed the openSSH service and the SAMBA service since we will be using both of those. Once you are finish, and the server is up and running, I recommend finding out your IP address using ifconfig and then using SSH to log in to the server from another machine. It's easier to copy and paste some of the commands we will be using. You should probably set your IP statically since this is a server. Once you are up and running and you have good network connectivity, you will need to grab these packages that are required for ZFS code:
code:
code:
Congradulations, you now have a Linux system that supports ZFS. If you have an exsisting ZPool that you exported from a previous system, use the command code:
When it finishes, it will mount in / So my pool1 is mounted in /pool1. You might need to fix permission on your files if they were owned by an account and group that do not exsist on your system. It is important that you do not upgrade your Zpool version if you think you will ever be going back to FreeNAS or NAS4Free ZFSonlinux upgrades your pool to version 5000 and uses a feature flag system. Upgrading to this version is a one way trip and can not be undone. Fortunately, ZFSonLinux can still read Zpool version 28 without any issues. If you need to create a new Zpool, I suggest you read this to see the various ways you can create a pool. The short of the long is that the easy quick way is to create it using device names, but the more proper way to do it us to use the /dev/disk/by-id name since those will not change if new disk are added or the system other wise changes configuration. At this point you are more or less on your own to configure the system the way that you want. If you are interested in a web based management system for managing users, groups, network shares, etc, I would recommend look in to Webmin. There is also a more detailed guide on it's set up on this page. You can replace the wget statement they give you with code:
If you are interested installing a gui, there are many guides on many different GUIs that you can install on Ubuntu server. I am not going to cover it in this guide but you can google "ubuntu server gui" and you will find dozens of guides covering all the major and several minor user interfaces. Past this, all you have to do is configure your shares. You can read more about that here or you can use Webmin if you installed it to manage your shares. There are also some tools that allow you manager shares in Unity/Gnome/KDE/etc can might be easier than the command line. From here, you can install any services you want such as SABNZBD, Sickbeard, Headphones, Transmission, Crashplan, Subsonic, Plex, and many many more. I will not be covering this in this guide as there are tons of guides for those as well. Other things you will want to consider is setting up cron jobs to scrub your Zpool. The command to scrub your pool is code:
Lowen SoDium fucked around with this message at 17:04 on May 2, 2013 |
![]() |
|
Lowen SoDium posted:Why Linux over FreeNAS or NAS4Free? Nice writeup. Definitely a lot easier than rolling an ESXi all-in-one!
|
![]() |
|
I'm more curious as to how ZFS4Linux proceeds. As in, freezing the on-disk format, go with what Illumos develops, or start cooking their own shit.
|
![]() |
|
My understanding is that with the introduction of feature flags they can do both. They can make their own changes, fold in Illumos changes if they want, and if they're really ambitious, push their changes into the main branch. What's the license like on ZFS, is it still CDDL?
|
![]() |
|
FISHMANPET posted:My understanding is that with the introduction of feature flags they can do both. They can make their own changes, fold in Illumos changes if they want, and if they're really ambitious, push their changes into the main branch. Feature flags let you know what stuff the filesystem supports so when you try mounting it on a different system which doesn't support those flags it will error out instead of trashing your data. The only entity that can relicense ZFS is Oracle and they closed sourced it. Luckily they can't do that retroactively so the open sourced Sun code will stay CDDL for eternity.
|
![]() |
|
Trying to put together a NAS on the cheap and looking over what spare parts I have. Got two questions: 1 - How much wiggle room is there on FreeNAS's 1GB RAM per TB storage suggestion? and 2 - How much RAM does unRaid really need? Their faq says 512 to 1 gig but in light of how much FreeNAS recommends I thought I'd ask since those numbers are pretty dramatically different. If those numbers are accurate I'd pretty much have to build a whole new machine for FreeNAS but have 2-3 that could conceivably run unRaid.
|
![]() |
|
FISHMANPET posted:My understanding is that with the introduction of feature flags they can do both. They can make their own changes, fold in Illumos changes if they want, and if they're really ambitious, push their changes into the main branch.
|
![]() |
|
It looks like it does support feature flags, and version 5000 is the default: https://github.com/zfsonlinux/zfs/issues/778
|
![]() |
|
Giant Isopod posted:1 - How much wiggle room is there on FreeNAS's 1GB RAM per TB storage suggestion? and
|
![]() |
|
adorai posted:A lot, especially if you don't dedupe and don't need a large arc cache. I run Solaris 11, but I've got a 9 TB pool with 4 GB of RAM, no problems. I don't use dedupe and since most of my access is media, it's pretty sequential, so I'm fine with a small cache in memory. I'm going to be replacing 5 of my 750GB disks with 3TB disks in the very near future and that'll bump me up to 18 TB, and I have no plans to up memory either.
|
![]() |
|
adorai posted:A lot, especially if you don't dedupe and don't need a large arc cache. I don't think I'll be doing those, but I hadn't really heard of either until I just googled them. I was planning on doing 3x 3TB but my free machines are older than I thought. One apparently maxes out at 1 GB of ram, which is why I was looking at unRaid Edit: This is where I was getting some of my numbers from. They sound pretty high. http://doc.freenas.org/index.php/Hardware_Recommendations posted:ZFS typically requires a minimum of 8 GB of RAM in order to provide good performance. Anecdotal posts on forums I see people saying you can run it with much less, which is why I'm asking here. Also it looks like NAS4Free has lower reqs too Giant Isopod fucked around with this message at 21:42 on Apr 26, 2013 |
![]() |
|
I thought 1G/TB was the recommendation only if you dedupe? Which is stupid on a FreeBSD/ZFS system anyway.
|
![]() |
|
ZFS was built for enterprise, so a lot of the RAM recommendations are based around that - you're going to see a lot of random access in that environment, and RAM caching really helps a lot there. At home, however, most loads are sequential in nature and you don't have a huge number of concurrent users, so caching doesn't help nearly as much. That said, never use dedup on ZFS. Even with a lot of RAM the performance is utter shit.
|
![]() |
|
Giant Isopod posted:Trying to put together a NAS on the cheap and looking over what spare parts I have. Got two questions: Unraid can run on 1gb fine vanilla, but the minute you start putting in addons you'll be asking for trouble. Unraid itself is very light weight, but if you want to put on let's say sabnzbd, you now need to take into account your 1gb will be taxed with both roles. I run it with 2gb and run all sorts of shit, so it's not like you're walking in the danger zone at 1gb or anything.
|
![]() |
|
What's been people's experience with sata port multipliers? How reliable are they and how much of a performance hit should I expect if I use them? You can get a 5:1 multiplying backplane (no trays, just a circuitboard) for Alternately PCIe extension cables and locating the SATA cards directly in the external box, but that seems both excessively hacky and anything over 4-port cards goes in the multi-hundred dollar range pretty quickly, which would chew through PCIe slots rapidly. Edit: Two corrections, $45 is the bulk price, so my small setup would be the $60ish retail, and also expanding the question to SAS JBOD cards + expanders if they're cheap as well. Harik fucked around with this message at 03:27 on Apr 27, 2013 |
![]() |
|
The posted ZFS on Linux guide is great, but I'm running into a problem after rebooting. The zfs pool mounts, but I don't seem to see my datasets. When trying to re-create them they say they are already there? Very confused.code:
kill your idols fucked around with this message at 04:35 on Apr 27, 2013 |
![]() |
|
Harik posted:What's been people's experience with sata port multipliers? How reliable are they and how much of a performance hit should I expect if I use them? Not sure how many ports you're aiming for, but an IBM M1015 + 2 break out cables will drive 8 drives for around $130, they're 8x PCIe so you should be able to fit a couple on any board.
|
![]() |
|
Is anyone using 3TB drives in Solaris? I just got some and I'm not sure how to make it see the whole disk. When I run "format" it says this for my drives:code:
|
![]() |
|
FISHMANPET posted:Is anyone using 3TB drives in Solaris? I just got some and I'm not sure how to make it see the whole disk. When I run "format" it says this for my drives:
|
![]() |
|
Well that message comes up as an error, and googling that exact number tells me that 2794.52 is the actual "2TB drive" limit. But you're right, that's the actual size when I was using these in Windows. But the fact that the "configured with capacity" message doesn't come up for any other drives, I want to make sure I'm doing this right.
|
![]() |
|
FISHMANPET posted:Well that message comes up as an error, and googling that exact number tells me that 2794.52 is the actual "2TB drive" limit. But you're right, that's the actual size when I was using these in Windows. But the fact that the "configured with capacity" message doesn't come up for any other drives, I want to make sure I'm doing this right. ![]()
|
![]() |
|
As far as I know, OpenSolaris supported LBA48 for a long time. So it should be in regular Solaris, too.
|
![]() |
|
One of the drives on my home server is on its way out, but before I replace the drive I thought I'd ask if there was a better way I could be doing the raid setup. Currently the (Centos) linux install is on 2 x 500GB in RAID 1, with a 3 x 1TB RAID 5 array for storage. All mdraid. The drive that is going is from the RAID 1 array. Is it worth me keeping 2 separate arrays like this, or should I just move the whole thing onto a single array of some kind? The server is used as a low volume mail and web server, but mostly for media storage/streaming.
|
![]() |
|
Combat Pretzel posted:As far as I know, OpenSolaris supported LBA48 for a long time. So it should be in regular Solaris, too. Yeah, I decided to just make a zpool with a single 3TB disk, and after I did that I stopped seeing the message when using the format command, and the actual reported capacity was 2.68T, which seems reasonable after a little ZFS overhead. Now to unfuck the drive that I wrote the wrong disk label to ![]()
|
![]() |
|
I'm looking to rebuild my server, but I'm completely new to eSATA and SAS, so I come to this thread for some sanity checking. At the moment, it's the remains of an old gaming PC with five SATA disks in it. It's big, it's awkward, it's inefficient, and it's a pain to work inside. Ideally, I want to replace it with something smaller and more power-efficient, using uATX or mITX, and then get an external drive enclosure and connect that to it rather than trying to keep all of the disks in the main case. I also want to use software RAID - I'm currently using mdadm, but might take this opportunity to upgrade to ZFS - so however I attach the disks, they need to be visible to the OS as individual disks rather than as a single hardware RAID device. As I understand it, this means I want: - an external drive enclosure that supports an external SAS connection + an SAS controller in the server (builtin or, more likely, a PCIe card with SAS support - would the IBM M1015 be what I'm looking for here?), - or an external drive enclosure that supports port multiplied eSATA + a plain SATA controller in the server, - or an external drive enclosure that supports plain eSATA + a port multiplied eSATA controller in the server. Does this make sense? If so, is there any hardware people would recommend in particular for this, stuff in the 4-6 disk range in particular? If not, where have I screwed up? The last few pages posted:Crashplan! Apropos of this, is there any way to get Crashplan storage to manifest as, say, an SFTP server, or are you stuck using their client to back stuff up?
|
![]() |