Everyone knows probably the `netstat’ command already, so I will only give my favourite options with it (`sudo’ is optional here, but if you want to see the process attached to the socket, you will want to use it):
List all (TCP) listening sockets:
$ sudo netstat -tlpen
List all TCP listening sockets and all UDP sockets:
$ sudo netstat -tulpen
List all sockets (TCP, UDP, Unix, etc.):
$ sudo netstat -apen
Now, you may not know it, but `netstat’ is deprecated. It is still installed on most Linux distribution that I know of, but you should move to its replacement command `ss’. The good thing is that all of the above options are still working with this command for the same results, the output is however differently formatted, hence script using the output of netstat command might get broken.
$ sudo ss -tlpen
$ sudo ss -tulpen
$ sudo ss -apen
Finally because everything (or almost) is a file on Unix/Linux, you can use the `lsof’ command – which provides listing of open files – to query the list of open sockets. It is yet another method, but I like it because the output is more condensed and contains usually what I want. Thanks to Michael Crosby (@crosbymichael) for sharing his knowledge.
So the first command is listing all IPv4 TCP and UDP sockets:
$ sudo lsof -Pnl +M -i4
The next one does the same thing for IPv6 TCP and UDP sockets:
$ sudo lsof -Pnl +M -i6
The combined output of the previous commands is equivalent to the output of `sudo ss -atupen’.
If you have a Synology NAS that supports BTRFS (mostly the intel based NASes) and that you decided to use BTRFS, there are a couple of shared folders automatically created for you (like the “homes” or “video”) but they don’t have the “compression” option set, and trying to edit the shared folder in the administration GUI does not help, the check box is grayed out, meaning it is not possible.
BTRFS compression is quite “clever”. It has some heuristics that evaluate if a file is worth being compressed or not so it won’t try to compress the 1GB video of your toddlers playing together which is a waste of time given that the compression achieved might not be visible. But anyway, even if BTRFS is “clever” it does not mean that if you have a folder named video that you should consider using compression. Simply just don’t do it.
For folders with mixed data like “homes” (which is the shared folder for all user home directory) you might have wished Synology would have activated the compression. Or if you forgot to tick the check box once creating the volume, you might want to change it. But there is a way to change that. It is not guaranteed that it won’t break your NAS, especially if you do execute the wrong command, but if you don’t mind the risk then follow on.
BTRFS allows you to change the option on a live system without troubles. However, existing data on the shared folder won’t be compressed after activating the option, you would need to copy again the existing data to take benefits for it or defragment it using the compression option (-c see man btrfs-filesystem), however depending on your amount of data this might take a while.
To do it, you will need to activate SSH remote connection (try to limit it to your local LAN and do not open it to the internet unless you know what you are doing). You will need to connect via SSH using the administrator account (admin by default, but you would be wise to change the default name). I trust you know how to activate SSH on your NAS box, if not I would recommend you don’t try to do the rest of this article, ask someone who might know it! From a Linux or macOS (OS X) system, just open a terminal and type:
$ ssh <admin>@<hostname>
(and replace admin by the correct user account and hostname by your NAS hostname or IP address)
On Windows, you could use putty and achieve a similar fate.
Once connected, you need to know your BTRFS volume path:
$ mount -t btrfs
/dev/mapper/vg1-volume_1 on /volume1 [...]
In the above example, it is /volume1. Now you should have a BTRFS subvolume (think of it as a BTRFS internal sub partition which Synology uses to define shared folders) called “homes” (or whatever other shared folder you would like to tweak):
$ sudo btrfs subvolume list /volume1
ID 259 gen 1688 top level 257 path homes
ID 264 gen 1686 top level 257 path video
So here we have made sure that the “homes” shared folder is located on /volume1/homes. Now let us check its properties:
$ sudo btrfs property get /volume1/homes
Here we can confirm that compression is not set (note that compression was not set as a mount option, nor at the volume root). To activate is, you need to create the “compression” property, you can choose either zlib or lzo. The former compress better but is slower, the latter is fast but as much lower compression ratio. I personnaly choose lzo:
$ sudo btrfs property set /volume1/homes compression lzo
You can use again the previous command to get the properties for the volume and see if it was set. Now you can copy your files to the shared folder, and BTRFS will try to compress them if it thinks it makes sense.
I’ve been toying around with containers using LXC, and I decided to use this technology to do some performance technique for a PHP web application. So I set-up a VM with Ubuntu 14.04 LTS and decided to use containers to test various stacks, e.g. MySQL 5.5 + PHP 5.6 + Nginx 1.4.6 vs MariaDB 10 + PHP 7 RC3 + Nginx 1.9 (and all possible combinations), and HTTP/2, latencies, etc. On top of this experiment I wanted to learn more about Ansible.
Therefore my needs were the following:
One account to rule them all: one user account on the VM with sudo privileges. Used by Ansible to administer the VM and its containers.
Run each container as unprivileged ones.
Each container is bound to one unique user.
As of Ansible 1.8, LXC containers are supported, but as second class citizen: this extra module needs some dependency to work (which are not in the default repositories) and it would hide me how to configure the LXC containers, so I discarded this solution.
Unprivileged containers using the LXC command line UI
Standard containers usually run as root and the root user within that container maps to the root user outside of the container. This is not exactly how I think of a container.
In my view, a container is meant to hold something which I don’t want to leak out of the container, a chroot on steroids! So although I was really looking forward for containers on Linux, the first implementation were not matching my expectations or use cases.
Being able to run unprivileged containers is one of the great thing which finally decided me to check those containers on Linux, they finally cut the mapping between the privileged users on the host and those in the container.
Setting up LXC unprivileged containers require a few more packages, especially for the user and group IDs mapping, some preliminary account setup and giving LXC proper access to where you store the LXC containers data in your home folder. I did all of this, including setting up ACLs for the access. But when I hit lxc-start (...) it all failed!!!
What I did after all preliminary configurations was:
$ sudo -H -u dbusr lxc-create -t download -n mysql55 -- --dist ubuntu --release trusty --arch amd64
$ sudo -H -u dbusr lxc-start -n mysql55 -d
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options
And the lxc-start command failed miserably.
Why? A bit of background first, I will try to describe at a high level how containers “contain” on Linux. LXC should not really be compared to Solaris Zones or FreeBSD Jails. LXC uses Linux Control Groups (cgroups) to contain (allow/restricting/limiting) access by some process to some resources (e.g. CPU, memory, etc.). When one creates a “local” session on Ubuntu 14.04 LTS (and this is still valid on the up-coming Ubuntu 15.10 as of writing), such as login through the console or via SSH, Ubuntu allocates for the user different control groups controllers which can be viewed by doing:
Each line is a type of control group controller assigned to the user. For example the line about memory concerns the Memory Resource Controller which can be used for things such as limiting the amount of memory a group of process can use.
So when I run my script via Ansible, Ansible first establish a SSH session using my “rule-them-all” user, an SSH session is considered “local” and PAM is triggering systemd-logind to create automatically cgroups for my user shell process. Yes, even-though Ubuntu 14.04 LTS is still using upstart for init system, it has already a few dependencies on systemd! Now my “rule-them-all” user, when he is using sudo to execute commands as another user (be it root or one of my container users), the executed command is not considered as a “local” session for the sudo-ed user. So no cgroups are created for the new process, and it actually inherit the cgroups of the callee. This is easily visible by doing, you can see that the username and UID did not change despite the command being run as another user:
This usually does not really matter, unless you are LXC and you use cgroups heavily!! So what happened is that lxc-start wanted to write to the various cgroups to create the container. But lxc-start was called by the user dbuser (via the sudo command), however the cgroups it inherited were from my “rule-them-all” user and obviously (and thankfully) dbuser does not have the right to change the cgroups of my “rule-them-all” user. So lxc-start failed due to some permission denied:
lxc_container: cgmanager.c: lxc_cgmanager_create: 299 call to cgmanager_create_sync failed: invalid request
lxc_container: cgmanager.c: lxc_cgmanager_create: 301 Failed to create hugetlb:mysql55
lxc_container: cgmanager.c: cgm_create: 646 Error creating cgroup hugetlb:mysql55
lxc_container: start.c: lxc_spawn: 861 failed creating cgroups
lxc_container: start.c: __lxc_start: 1080 failed to spawn 'mysql55'
lxc_container: lxc_start.c: main: 342 The container failed to start.
Getting Ubuntu 14.04 LTS ready to run our Unprivileged Containers
I did not manage to solve the problem on Ubuntu 14.04 LTS in a first attempt. But after some extra steps (which I will details below), I made it work on the up-coming Ubuntu 15.10 (still alpha) release! So revisiting my Ubuntu 14.04 LTS setup, I identified the required packages needing an upgrade: kernel, lxc and cgmanager (and a few dependencies). For the Kernel, I’ve used the latest Ubuntu LTS Enablement Stack and upgraded to kernel 3.19. For the other packages, despite my aversion for 3rd party repositories, I decided to trust the Ubuntu LXC team (they are the ones who do the work to get LXC/LXD in Ubuntu in the first place) and their LXC PPA. So the commands were:
The lxc-templates package are necessary for me as I’m using the “download” template. The uidmap is mandatory for unprivileged containers. And libpam-cgm is necessary to resolve my problem, it is a PAM module for the cgmanager which is the Control Group Manager daemon (installed as a dependency to LXC on Ubuntu).
Now armed with this updated kernel and container stack it is time to present you the extra setup steps I was talking about. We need first to update either (or both depending which method you use) the PAM configuration for sudo or su. I will show the extra line for the former, they should apply for the later if you wish to use it. You need to edit as root the file /etc/pam.d/sudo and add the following lines after the line ‘@include common-session-noninteractive‘:
The above 2 lines will register the new session created by sudo with the systemd login manager (systemd-logind). That’s the guy we wanted notified so that the creation of the cgroups for our user can now work. I’m still scouring the internet for the exact explanation of how this work. If I find it, I will probably write another post with the information.
If I would simply use su -l userdb or sudo -u userdb -H -s, I would just have to execute the following:
$ sudo cgm create all $USER
$ sudo cgm chown all $USER $(id -u) $(id -g)
$ cgm movepid all $USER $$
This will create all cgroups under the user $USER which is userdb. It will then set the owner of these new cgroups to the UID and GID of the userdb. And the last command move the SHELL process ($$) within these new cgroups. Then, the rest is trivial:
$ lxc-start -n mysql55 -d
And it is working. And if you want to run those commands using sudo, this is how you do it:
If you close later the SSH session and you reconnect to it, you have to run the 2 commands again, even though it might display some warning that the paths or what-not are already existing. I still have those pesky warning and extra commands which I’m not sure why I still have.
While trying to run unprivileged containers on Ubuntu 14.04 LTS I’ve met several problems which I could only solved by using the latest LXC and CGManager packages and some special PAM configuration for which I’m not 100% sure of the impact. So unprivileged LXC containers on Ubuntu 14.04 LTS is still quite rough.
All of this make me really look forward to the next Ubuntu LTS due to next Spring, the newer LXC and the under-heavy-development LXD will be part of Ubuntu 16.04 LTS and this could give a really great experience out-of-the-box with Linux Containerisation powers.
I bought a USB Wireless dongle from AVM called Fritz!Wlan USB Stick N v2. The wireless chipset of this dongle is usually Ralink RT5572 (this device has had a few revisions, hence the v2 should be this Ralink chipset) which is supported under Linux by the rt2800usb module.
This dongle is particular, it is first seen as a CD drive. This is meant for Windows users so that the system will automatically install the correct driver, eject the CD and then transform itself into a Wireless dongle. On other platform, it also shows up as a CD drive, if you eject it (automatically or manually) it becomes a Wireless dongle.
Once you plug it, you can view it on Linux directly running: lsusb -d 057c:. However depending on your version of Linux, it might shows as a CD, broken, or potentially in the future as a Wireless device.
On Debian Wheezy, it was seen as a CD drive. A simple manual eject command and it is working. Slightly annoying, but fine.
On Debian Jessie (which uses systemd) and on Fedora 22 (also using systemd), the device is not mounting as a CD not as a Wireless dongle. So you can’t use it.
I don’t know why, but there is a package (already installed on both distribution by default) named usb-modeswitch which tries to be clever and detects USB devices like the one from AVM and do automatically for you the necessary kirks to make it work as intended. It seems that an Wheezy, it was not triggered. But on Jessie and Fedora 22, it was triggered but wrongly configured.
I have the solution which work flawlessly on both distributions and will allow you to just plug your USB dongle and see it as a wireless device (as expected! Thanks AVM x-( )
You either need to create or modify the file /etc/usb_modeswitch.d/057c:62ff so that it contains exactly the following text:
# AVM Fritz!Wlan USB Stick N v2
On Debian Jessie, this file did not exist. Adding solve the problem!
Side note: this device is pretty cool if you need to do advanced wireless stuff. For example, it is possible to build a WiFi rogue AP detector with this device and some tools.
Picture credits: Picture was created by me using elements from the KDE project. The original materials were licensed under GNU LGPLv3, and the picture is also provided under this license terms and conditions.
Our computers are not really good at providing random numbers because they are quite deterministic (unless you count these pesky random bugs that make working on a computer so “enjoyable”). So we created different ways to generate pseudo-random numbers of various qualities depending on the use. For cryptography, it is paramount to have excellent random numbers, or an attacker could predict our next move!
Getting unpredictable is a difficult task, Linux tries to provide it by collecting environmental noise (e.g. disk seek time, mouse movement, etc.) in a first entropy pool which feeds a first Cryptographically Secure Pseudo-Random Number Generator (CSPRNG) which then output “sanitised” random numbers to different pools, one for each of the kernel output random device: /dev/random and /dev/urandom.
Our goal is to help our Raspberry Pi to have more entropy, so we will provide it with a new entropy collector based on its on-board hardware random number generator (HW RNG).
This article is an update for all of you who upgraded their Raspbian to Jessie (Debian 8). The new system uses SystemD for the init process rather than Upstart for previous release.
The Raspberry Pi has an integrated hardware random number generator (HW RNG) which Linux can use to feed its entropy pool. The implication of using such HW RNG is debatable and I will discuss it in a coming article. But here is how to activate it.
It is still possible to load the kernel module using $ sudo modprobe bcm2708-rng. But I know recommend using the Raspberry Pi boot configuration, as it is more future proof: if there is a newer module for the BCM2709 in the Raspberry Pi 2 (or any newer model), using Raspberry Pi Device Tree (DT) overlays should always work. DT are a mean to set-up your Raspberry Pi for certain tasks by selecting automatically the right modules (or drivers) to load. It is possible to activate the HW RNG using this methods.
Actually, we do not need to load any DT overlays, but only to set the random parameter to ‘on‘. You can achieve this by editing the file /boot/config.txt, find the line starting with ‘dtparam=(...)‘ or add a new one starting with it. The value of dtparam is a comma separated list of parameters and value (e.g random=on,audio=on), see part 3 of the Raspberry Pi documentation for further info. So at least, you should have:
With this method, you have to reboot so that the bootloader can pick-up automatically the right module for you.
Now install the rng-tools (the service should be automatically activated and started, default configuration is fine, but you can tweak/amend it in /etc/default/rng-tools), and set it to be enable at next boot:
I was too young to have witness this and I had never heard of the internet, e-mails or usenet at that time. But it’s amasing to look once back and see how much technology and connectivity is so much more accessible, and that this simple hobby which was Linux at that time is now everywhere, even in space!
Hello everybody out there using minix –
I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I’d like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).
PS. Yes – it’s free of any minix code, and it has a multi-threaded fs. It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that’s all I have :-(.
2018-03: update instructions for Raspbian Stretch.
I’m going to explain how to install Raspbian (latest is based on Debian Stretch, dated 2018-03) on a Raspberry Pi which is only connected to the network using a Ethernet cable (of course your Raspberry Pi should be connected to a power source).
This guide should work for all Raspberry Pi supported version with an Ethernet interface (B, B+, 2, 3 and 3-B+). It could potentially work without an Ethernet interface if you have a WiFi USB dongle or a Raspberry Pi with an integrated WiFi which is supported out-of-the box by the RPi. I’ve tested the following on a Raspberry Pi 2 and a Raspberry Pi 3 B+. This guide assumes that you are doing all steps from a Unix machine (I’ve done them from Fedora Linux but it should work on any Linux, BSD or OS X with potential adaptation). It is possible to do it on Windows I suppose, but I don’t have Windows and can’t explain.
The steps are:
Flash the Raspbian image to a SD card;
Sync everything and mount the second partition created;
Modify files on the SD card to allow SSH (and optionally configure WiFi);
Put the SD card in your RPi and plug the power;
Wait a couple of minute for thing to settle;
Either check your DHCP server (e.g. your router) for the RPi IP address, or scan your network);
Connect to your RPi using SSH and finish the setup;
(option) Upgrade to Debian Jessie (you will have to view the full article)!
Flash the Raspbian image to a SD card
There are already numerous guides online on how to do that. So I will be brief, but refer to those guides for your exact configuration. I assume a new SD card which has already a Windows FAT partition with a size bigger than 4GB (I recommend 16GB). When I plugged that SD card on my laptop it was recognised as /dev/mmcblk0p1 and automatically mounted. First we need to unmount this partition and then to remember the device name (without the partition suffix, so /dev/mmcblk0 in my case). Note that if you are on BSD or OS X those steps are slightly different, check online. Using the device name we can flash the Raspbian image (after downloading the zip file, check that the SHA-256 checksum match, you can either pick-up the normal or the Lite version of Raspbian, but if your goal is a headless server, then the Lite is more suitable).
After doing the above you should have now 2 new partitions on your SD card. The first partition (suffix p1) is the boot one and is formatted as FAT (was <60MB for me). The second one (suffixed p2) is the root partition and is formatted as ext4 (roughly 3GB). That’s the partition /dev/mmcblk0p2 which is interested for the next step.
Allow SSH access
For pre-systemd Raspbian releases (up to Wheezy, or Debian 7), you need to mount the second partition now, So create first a mount point and then mount the partition to it.
Optional: if you do not have an Ethernet interface and need to use WiFi, you need to add the WiFi configuration (your SSID – or WiFi network name – and your WiFi password). Assuming you have something like WPA2-PSK, you simply need to edit the file /mnt/rpi/etc/wpa_supplicant/wpa_supplicant.conf and add the following at the end of the file:
The network configuration on Raspbian is set to use DHCP, so after booting the system will use whatever network interface it has available and make DHCP requests in order to get an IP address.
You will also have to specify your country code (in ISO/IEC alpha2 code, DE for Germany, US for USA, JP for Japan, etc.) by either modifying the existing line or by adding it. Here is an example for Iceland:
The country code is mandatory (at least on recent Raspbian) in order to have WiFi activated. So you will need to set it up correctly if you want a headless installation using WiFi and no Ethernet.
Note: please don’t use both the Ethernet and WiFi interfaces if you they are both on the same network. Such a configuration is possible, but it won’t work out-of-the-box. So set up first one interface, make it work and then add the other one (look for online resources about how to configure Linux for 2 NIC on the same subnets).
Start Raspbian and connect to it
Now sync all changes to the SD card:
$ cd ~ ; sudo umount /mnt/rpi
Before removing the SD card from your computer, be sure that you properly unmounted it.
Now remove the SD card from the computer, insert it in the Raspberry Pi, plug the Ethernet cable (or the WiFi USB dongle) and then the USB power plug. The LEDs of your RPi should light up (PWR – red one – means that your power supply is good enough; ACT – green one – should not be steady green, it means your SD card is not readable or was wrongly flashed, it should blink). After the RPi has completed boot up, the ACT LED should be off (mostly), meaning that there are no more I/O activities going on. That’s when you can be sure that the RPi has sent its DHCP probes and should have an IP address assigned.
If you have a proper router which register a DNS entry for each DHCP clients, you should be able to directly login to your Raspberry Pi by doing:
$ ssh pi@raspberrypi
If you router does not support such feature, you will need to know your Raspberry Pi IP address, you can either go to your DHCP server (e.g. your router) and check which address was assigned to it, or simply scan your network. To scan your network you need to know your subnet (e.g. 192.168.1.0 with a netmask of 255.255.255.0) and have nmap installed on your computer (sudo dnf install nmap will work for Fedora, and it is as easy for Debian/Ubuntu-based distros as well, just replace dnf by apt-get).
$ sudo nmap -sP 192.168.1.0/24
Of course you need to adapt the above command to your subnet. The “/24” part is the netmask equivalent of 255.255.255.0. I recommend running the above command with sudo because it will display the MAC address of all the discovered devices which will help you spot your Raspberry Pi as nmap is displaying the vendor attached to each MAC address. See for yourself in the example output:
Starting Nmap 6.47 ( http://nmap.org ) at 2015-07-19 20:12 CEST
Nmap scan report for raspberrypi.lan (192.168.1.9)
Host is up (0.0060s latency).
MAC Address: B8:27:EB:1E:42:18 (Raspberry Pi Foundation)
Nmap done: 256 IP addresses (8 hosts up) scanned in 2.05 seconds
Now you can simply connect to your RPi by using the user pi and password raspberrypi (which are default on Raspbian):
$ ssh email@example.com
The authenticity of host 'raspberrypi (192.168.1.9)' can't be established.
ECDSA key fingerprint is SHA256:WSF9Rpmh0Mr/JYUye8r69nXzwZtYbdH0xJ5M4AFYxYY.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'raspberrypi,192.168.1.9' (ECDSA) to the list of known hosts.
Linux raspberrypi 4.9.80-v7+ #1098 SMP Fri Mar 9 19:11:42 GMT 2018 armv7l
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
NOTICE: the software on this Raspberry Pi has not been fully configured. Please run 'sudo raspi-config'
pi@raspberrypi ~ $
It was advised, that you run the command sudo raspi-config especially to increase the partition size so it uses the complete SD card space available. This step is however done automatically for you on recent Raspbian.
However, you should really do one extra step: change the default pi password or create a specific account for you. You can use the command passwd to change the password on the command line.
Picture credits: The Debian Logo belongs to the Debian project. Raspberry Pi and its logo are trademarks of the Raspberry Pi Foundation. The OpenSSH logo is copyrighted by OpenBSD. The wireless device is an icon from the Oxygen set of the KDE project provided under GNU LGPLv3.
Update: I forgot the icing on the cake: upgrading to Debian Jessie. Continue reading to learn a quick way to do it.
On my laptop, I’m running Fedora 22 which was shipped initially with a Linux 4.0 kernel. It was difficult to get 4h of battery life (3h30 was usually enough to deplete the battery down to 5%). Recently, the kernel was changed to 4.1 and because after 5h working on my laptop I got notified that I still had 10% power I got curious,
Therefore with a fully charged battery, I booted with Fedora 22 Linux kernel 4.0.4-301. I used powertop to measure the battery power usage in Watt in graphical (ex-init 5 level) and multi-user target (ex-init 3 level). I then rebooted using Linux kernel 4.1.3-201 and did the same measurement. I waited each time that the system settled down and that successive measurements where constant. Nothing was running, WiFi was ON and connected (Link Quality=64/70), screen brightness at 30%, Graphical target is using Gnome 3.
SystemD Target (~init level)
Linux 4.0 (in W)
Linux 4.1 (in W)
Multi-User (init 3)
Graphical (init 5)
Wow! That’s great. And the estimated battery power is now up from 4h to 6h30 with WiFi ON. But with light browsing usage and some Arduino development, I got a bit more than 5h15 without requiring a wall power connection!! That close to 50% more battery life than with earlier kernels.
Did you also witness improvement when switching to Linux kernel 4.1? Let me know using any social media means!
Note to self: telinit is now deprecated in favour of systemd targets. Runlevel 3 can be reached by invoking sudo systemctl isolate multi-user.target and the switch to the “runlevel 5” can be triggered using sudo systemctl isolate graphical.target.
Picture credits: Picture was created by me using elements from the KDE project. The original materials were licensed under GNU LGPLv3, and the picture is also provided under this license terms and conditions.
My goal is to use it to prototype something I want to make: a small network of temperature and humidity sensors running for months on batteries.
Why? First because I can (I need to learn a lot first regarding electronic or microcontrollers, but I’m sure I can). Second because we have add a few problems with humidity in our basement and I want to be able to have a better idea when this is happening to find a proper solution. My idea would be to correlate the measurements to others done externally and which would include more environmental data (e.g. pressure and amount of rain) and if possible with some events (e.g. gutters are overflowing water).
I already started the prototyping based on a tutorial from Adafruit – a Wifi Weather Station. The results works well as one could expect. So I validated my first part: yes I can do some basic electronic and microcontroller programming, the Raspberry Pi is doing the web server side.
The next step is reprogramming the microcontroller to push data periodically using plain UDP and either a Graphite, Statsd or Fluentd syntax. This would be pushed to a multicast address which my Raspberry Pi would listen to, it would run the Graphite/Statsd/Fluentd stack (I’m going to start with Graphite alone and see how good it is). I want to keep historical data of at least one month, perhaps even a bit more and to be able to visualise the data in real time.
Once this is done, then I want to get rid of the WiFi module and use a transceiver in the 433 or 868 MHz band (I’m in Europe). The Raspberry Pi will be the gateway between this radio-protocol and the more standard computer network stack. So I would need to adapt whatever I had chosen for stack on my Raspberry Pi to be able to cope with the new type of input. Either I contribute to a project if it is well architectured, or I build a bridge interface.
Final step of the prototyping will be to go low power. I’ve already approximately calculated how long I could run on 2 AA rechargeable batteries with the latest Arduino prototype and my “guestimates” is that it won’t last more than a week (probably less). So far from ideal. The solution is to build your own “Arduino”, meaning taking the microcontroller only (plus a few required components for it to run) and the needed components for the sensor. It seems that the power draw from the battery in this case would be sub mA. So I should be able to run many months on a 2 AA batteries
A headless server by definition has no input devices such as a keyboard or a mouse which provides a great deal of external randomness to the system. Thus, on such a server, even if using rotational hard disks, it can be difficult to avoid the depletion of the Linux kernel’s random entropy pool. A simplified view of this situation is if the entropy pool is deemed too low, one of the “dices” which generates random numbers is getting biased. This is can be even more exacerbate on server hosted in virtual machines, but this article won’t help you in this case.
Update 2015-08-24: the article was updated to provide some more information on TPM and commands adapted to the new systemd-based Ubuntu 15.04 and newer. When marked, use either the classic commands from Ubuntu 14.04 LTS and older or the newer ones.
The level of the kernel entropy pool can be checked with the following command:
(Note depending of the workload of your server the above result could be considered adequate or not)
What means a depleted entropy pool? It means that any call to /dev/random would be blocking until enough entropy is available. A blocking call is not usually wished, it means the application could be considered frozen until the call is freed. On the other side, calls to /dev/urandom would not be blocking in such situation, but there is a higher risk that the randomness quality of the output decreases. Such a higher risk could mean giving a higher chance for an attacker to predict your next dice roll. This could be exploitable or not and it is hard to tell, at least for me. Therefore, I tend to try avoiding having a depleted entropy pool especially for certain workload.
There are several mechanism to provide randomness sources to the entropy pool. The haveged daemon uses some CPU clock timers variation to achieve that, but it is highly dependant of the CPU being used. Other approaches are using sound cards, etc. And finally there are hardware random number generators (RNG). In my previous article, I talked briefly about the hardware RNG from the Raspberry Pi. In this article, I will present another hardware RNG which is available in many computers and servers: Trusted Platform Module (TPM).
I found the name somewhat marketing, and I’m not even sure we should trust it that much. But we will activate it only to provide a new source of entropy and nothing more. I would advise to use other source of entropy as well. Anyway, if interested the following paragraphs are describing how to achieve this, and if you decide to implement them, be reminded that I give no warranty that it will work for you nor that it won’t break things. I can only guarantee you that it worked on my machine running Ubuntu 14.04.2 LTS.
Let’s install the necessary tools and deactivate the main services (the tools launch a daemon which we don’t want to use as we will use rngd to get and verify the randomness of the TPM RNG before feeding the entropy pool).
Then, you will need to go into the BIOS/EFI settings of your computer/server and activate TPM, and possibly clear the ownership of the TPM if it happened to be owned by someone else. Of course, don’t do that if it is not your computer.
I found out that my particular BIOS option only allow clearing from the BIOS settings. Trying to do so from the OS results in the following:
$ sudo tpm_clear --force
Tspi_TPM_ClearOwner failed: 0x0000002d - layer=tpm, code=002d (45), Bad physical presence value
I also found out that my particular BIOS when clearing the ownership, deletes also the Endorsement Key (EK). When I was trying to take ownership of the TPM device (see further) I was getting the following error:
$ sudo tpm_takeownership
Tspi_TPM_TakeOwnership failed: 0x00000023 - layer=tpm, code=0023 (35), No EK
So I had to do an extra step, to generate a new EK:
$ sudo tpm_createek
After having creating a new EK, I was able to successfully take ownership of the TPM device.
$ sudo tpm_takeownership
Enter owner password:
Enter SRK password:
Once all this is done, we are ready to load the TPM RNG kernel module and launch the user space tool that will use this source to feed the Linux kernel entropy pool. The user space tool are the rng-tools suite (with the rngd daemon).
Load the kernel module (to load it permanently add tpm_rng as a new line to /etc/modules):
$ sudo modprobe tpm_rng
And then install the user space tool.
$ sudo apt install rng-tools
The default configuration should be good enough. But you can check it by editing the file /etc/default/rng-tools. The default settings for rngd are to not fill more than half of the pool. I don’t advise to set it to higher, unless you really trust blindly those TPM chips. If you have installed the tool, it should already be running but if you modified the default settings then a restart is necessary.
Ubuntu 14.04 LTS
$ sudo service rng-tools restart
Ubuntu 15.04 and newer
$ sudo systemctl restart rng-tools.services
Now you can check again the available entropy with the command that I gave at the beginning of this post.