Category Archives: BSD*

Home Server – What do I want?

What service do I want to run on my Home Server?

I do have a NAS already which has the following services: File Sharing (Samba, AFS and NFS), Media Streaming Server (DLNA), VPN Server, Cloud Sync Repository. So I do not intend to have redundant services on my Home Server. What is left?

My Home Server could support:

  • Backup: Having a proper backup of all important files from the NAS and our laptop. Implementations: rdiff-backup, Box Backup, fwbackups*, duplicity*, rsnapshot or storeBackup.
  • (N)-IDS: As I have services open to the internet, I want to take some precautions and check that no exploits is taken advantage of. I am not sure this is enough, but it is the least I can do. Implementations: AIDE or Suricata.
  • DNS cache/server: I am thinking of hosting my own DNS server to perform some caching and hopefully enhance a bit the browsing experience in terms of performance. Though I would need to benchmark this to make sure I have any gain as I suspect my old router to do some caching. Implementation: dnsmasq.
  • DHCP server: My home router is a Netgear WG614 and its features for what concern DHCP are fairly limited, having my home server addressing this issue is a nice idea (until we get a better router). I could be even tightly coupled with the DNS server (see earlier bullet point) so that one could use hostname within the local network. Implementation: dnsmasq.
  • Syslog server
  • Maybe – ownCloud: maybe one day I would prefer to use an open source solution for Cloud Sync rather than the closed source one from my NAS vendor.

*: FreeBSD support is uncertain.

As one can see, I could use Linux or BSD based OS or a mixture. However, ZFS is so compelling that I am seriously considering to go for FreeBSD+jails and basta cosi! February will be the month where I try to set-up a FreeBSD server.

My Future Home Server – Part 2

I am experimenting with different OS to find the right settings for my Home Server. I was interested by Fedora especially because there are several “Red Hat” technology which I would like to use on my server, namely: oVirt and virt-manager. Furthermore it sports a recent Linux Kernel (3.7 as of this writing) which could be beneficial if I choose Btrfs for the underlying file system.

However, testing the upgrade path from Fedora 17 to Fedora 18, I am not so thrilled by the robustness of this OS. I have managed after painfully hitting 3 different blocking bugs to recover from the upgrade and have a nice Fedora 18 up and running. But this gave me little trust in the Q&A of the community. It seems that it is not the first time such problems happen (see Fedora 11).

I am still willing to give a go to Fedora. But out of precaution, I am going to experiment first with Ubuntu (for which I had since 2006 only once an upgrade problem). I want to see the state of oVirt and virt-manager on this OS before I am making any choice.

Or maybe I forget entirely about Linux based OS, and I go for FreeBSD with several jails instead of using virtualisation. Though I would need to check the state of technologies like ownCloud, (n)IDS, etc. on this OS.

My Future Home Server – Part 1

I have finally my Home Server built, it has its first storage hard drive and I upgraded the memory to something decent. Time to install the operating system.

I am not yet fully decided which operating system to implement on my Home Server, I would love ZFS as a file system for managing my storage, but I would still want to use Linux and not make the full switch to BSD. I decided to go for Fedora as the main OS, and install BSD in a virtual machine and see how this setup performs.

I had tried for a few month Fedora 17 in a virtual machine, I liked it, although I prefer the Debian package manager over yum, but this is really based on my own feelings and not on technical grounds.

So let’s go and install Fedora 18 (just released) on my server.

Continue reading

Btrfs – Linux answer to ZFS

Sadly ZFS on Linux is not at the same maturity level than on FreeBSD (or even Solaris). There is a FUSE implementation but it is now more than 16 month since anything happen there, and in my opinion not yet stable. Regarding native ZFS port, only one ZFS implementation for Linux is still developed by the Lawrence Livermore National Laboratory but it is still a release candidate version.
The state of ZFS on Linux is perhaps not too good today, but there is another file system in development and good support that could soon compete with ZFS, its name is btrfs (pronounce ‘butter-fs‘). Btrfs is still experimental
Yesterday, one of my virtual machines running Oracle Linux 6.3 got its root file system full, as it was configured with LVM it was not so much trouble but I wanted to try btrfs. I decided to move the /var to another partitions using btrfs. I have created a new hard disk in my VM and started it. Here is the rest of the story.

Warning: following these instructions might break your system. As an advice, create a virtual machine and experience with it before doing so on a real system.

Continue reading

SSH goodies

Today I stumble upon an article regarding SSH on Oracle’s blogs.

One interesting feature I did not know is the SSH escape character ‘~’. So many times I had to open another terminal just to do one command locally before returning to the SSH session. Now this is over, just type ~^Z (a tilde followed by Ctrl-Z). Example

malmur :: ~ » uname -a
FreeBSD malmur 9.0-RELEASE-p3 FreeBSD 9.0-RELEASE-p3 #0: [...]
malmur :: ~ » ssh 192.168.78.10
unbreakable :: ~ » uname -a
Linux unbreakable.linux 2.6.39-200.29.1.el6uek.x86_64 #1 [...]
unbreakable :: ~ » ~^Z [suspend ssh]
[1]  + 2540 suspended  ssh 192.168.78.10
malmur :: ~ » fg
[1]  + 2540 continued  ssh 192.168.78.10
unbreakable :: ~ »

Securing ZFS data by mirroring them

This article is a follow-up of an earlier post about ZFS on FreeBSD. We have created a ZFS pool with one disk and put some data on it. Now we want to mirror the data to safeguard them from disk failure.

In my virtual machine I created a new disk of the same size than previous ZFS dedicated disk and fire-up the machine.

Creating a dataset with 2 internal copies for each file

But before I added the second disk, I decided to create a dataset (of the file system type) inside the pool I have created in previous article. The dataset will be configured to replicate internally the data for safety. This is an entirely optional step which I did just to experiment with ZFS.

The reader should notice that my pool had only 1 drive which means that each file in this dataset will appear twice on the same drive. If the drive fails, everything is lost. It just help if one version of the file gets corrupted, ZFS will detect it and use the (hopefully) uncorrupted copy to restore the file.

# zfs create -o copies=2 laug/safe

Note about: mirror/striped pools and dataset copies

Dataset copies are in addition to any pool configuration such as mirroring or RAID-Z. In case of a stripped pool (the case if you use zpool add command), ZFS will try to use different disks in the pool for each copy, if it can! In case of mirrors (the case if you use zpool attach command) or RAID-Z, in addition to the pool duplication of data, ZFS will try to keep extra copies on different drives.

Preparing the second ZFS drive and adding it as a mirror to the existing pool

As the hard disk is exactly of the same size (same disk space and number of sectors) I can reuse the commands from the previous articles:

gpart create -s gpt ada2
gpart add -b 2048 -s 41932733 -t freebsd-zfs -l disk01 ada2

But now we are going to add the new disk to the existing pool in a mirror configuration. For this we use zpool attach:

# zpool attach laug ada1p1 ada2p1
# zpool status
  pool: laug
 state: ONLINE
 scan: resilvered 1.37M in 0h0m with 0 errors on Tue Jul 31 18:16:43 2012
config:

        NAME        STATE     READ WRITE CKSUM
        laug        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada1p1  ONLINE       0     0     0
            ada2p1  ONLINE       0     0     0

errors: No known data errors

As I don’t have much data on my pool, the resilvering was fast (see the scan message). In addition, one can see that the 2 disk partitions are now inside a mirror.

I really like ZFS, the command line interface is clean, it is easy to manage and it is powerful.

ZFS on FreeBSD 9

I have created a VM with 2 hard disks. Did a standard installation of FreeBSD on the first hard disk (ada0) and decided to play around with ZFS on the second hard disk.

First of all, I destroyed any existing partitions on the second disk (warning the next command is dangerous, it will destroy all the data on the hard disk):

# gpart destroy -F ada1

I then went on to create a ZFS partition and pool. Note: I did not use the full disk size, in case I want to switch to RAID*. A prerequisite for ZFS in this case is that any newer disk should be at least the same size of the existing ones. This is not guaranteed by hard disk manufacturers that two 2TB hard disks have the same exact size.

# gpart create -s gpt ada1
# gpart show ada1
=>      34  41942973  ada1  GPT  (20G)
        34  41942973        - free -  (20G)
# camcontrol identify ada1
...
protocol              ATA/ATAPI-6 SATA 2.x
device model          VBOX HARDDISK
...
sector size logical 512, physical 512, offset 0
DMA supported         WDMA2 UDMA6

The 2 last commands gave me the partition size and sector size (in bold). I use this information to leave a bit of space after the ZFS partition. In addition, it is recommended to try to align the partitions to the sectors correctly. When using a physical hard disk for which you know the real sector size (note: sometimes the hardware is lying to you, which is the case of the 4K 512e HDD!!) you can directly use the corresponding alignment (i.e. 512 or 4k), but when using virtual HDD either as files (e.g. vdi, qcow2, etc.) or partitions (e.g. an LVM logical volume), it is better to use a 1MB aligment (1m), so adapt the “-a” option in the following command.

# gpart add -b 2048 -s 41932733 -a 1m -t freebsd-zfs -l disk00 ada1
ada1p1 added
# gpart show ada1
=>      34  41942973  ada1  GPT  (20G)
        34      2014        - free -  (1M)
      2048  41932733     1  freebsd-zfs  (20G)
  41934781      8226        - free -  (4.0M)

# zpool create laug ada1p1
# zfs set compression=lzjb laug
# zpool status
  pool: laug
 state: ONLINE
 scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        laug        ONLINE       0     0     0
          ada1p1    ONLINE       0     0     0

errors: No known data errors
# df -Th
Filesystem   Type     Size    Used   Avail Capacity  Mounted on
/dev/ada0p2  ufs       18G    2.6G     14G    15%    /
devfs        devfs    1.0k    1.0k      0B   100%    /dev
laug         zfs       19G     31k     19G     0%    /laug

Et voilà, a nice ZFS pool which is using compression (lzjb algorithm).

Ubuntu Server remote administration – Ajenti

Ajenti LogoToday, I will present Ajenti, a remote administration tool for Linux and FreeBSD.

This article is part of a series about remote administration of Linux-based server (actually some of the tool work on *BSD systems too). You can find all article pertaining to this series using the Tag remote-server-admin.

Continue reading

Ubuntu Server remote administration – Munin

Munin LogoToday, I will present Munin, a tool to monitor various health measurement of a system (essentially resources).

This article is part of a series about remote administration of Linux-based server (actually some of the tool work on *BSD systems too). You can find all article pertaining to this series using the Tag remote-server-admin.

Continue reading

Ubuntu Server remote administration – Monit

Monit logoI have recently tested some remote administration tools for Ubuntu server (or any other Linux-based server). I have recorded here my findings and installation steps.

Today I present Monit, a monitoring and control tool for Unix and Unix-like systems.

This article will be followed by others with different tools. Stay tune, and you can find them all using the following tag remote-server-admin.

Continue reading