Minimalist Raspberry Pi Server Image

The following tutorial will help you generate a 367MB Raspbian image (uncompressed).

The standard 2012-12-16 version of Raspbian “Wheezy” fills up 1.5GB (or 88%) of a 2GB SD card, which has a formatted capacity of only 1.8GB. While I have a larger capacity SDHC card for this particular RPi (I have five “pies”), one of the projects I’m considering is potentially quite data intensive, so I want to reserve as much disk space as possible.

My previous article began addressing the removal of unnecessary packages but its primary focus was on reducing I/O usage, however this article is focused on conserving disk space and reducing the operating system fingerprint. This optimizes the RPi for a server-type environment, rather than its use as a desktop workstation.

This minimalist image is different from others since it is significantly smaller than most images, yet it retains sound and touch-screen support.

Remove the GUI Modules

Let’s start by getting rid of over 325 optional packages (about 700MB) used by the graphical user interface:

sudo apt-get purge xserver.* x11.* xarchiver xauth xkb-data console-setup xinit lightdm lxde.* python-tk python3-tk scratch gtk.* libgtk.* openbox libxt.* lxpanel gnome.* libqt.* libxcb.* libxfont.* lxmenu.* gvfs.* xdg-.* desktop.* tcl.* shared-mime-info penguinspuzzle omxplayer gsfonts
sudo apt-get --yes autoremove
sudo apt-get upgrade

Remove Development Packages (Optional)

My personal minimalist RPi image retains most of the default installed development packages.

sudo apt-get purge gcc-4\.[0-5].*

However, you can optionally recover almost 150MB of space by removing the majority of compiler and development packages from your image.

sudo apt-get purge `sudo dpkg --get-selections | grep "\-dev" | sed s/install//` gcc-4\.[0-6].* python.*
sudo apt-get --yes autoremove

Disable Memory Swapping

Memory swapping allows part of the usable memory of your computer to be stored on disk, effectively allowing the system to have more RAM than it does physically. Disabling swap files helps free up valuable disk space.

sudo swapoff -a
sudo apt-get purge dphys-swapfile
sudo rm /var/swap

Delete Unnecessary Extras

A few stray files and folders slice up more than 125MB of the Raspberry Pi.

sudo rm -rf /usr/share/doc/* /opt/vc/src/hello_pi/hello_video/test.h264 /home/pi/python_games
find /usr/share/locale/* -maxdepth 0 -type d |grep -v en |xargs sudo rm -rf
find /usr/share/man/* -maxdepth 0 -type d |grep -Pv 'man\d' |xargs sudo rm -rf

Replace SSHd with Dropbear

Dropbear is a lightweight SSH v2 server which lacks backwards compatability support for SSH v1 (translation: smaller code base, uses less memory and fewer resources) and is secure enough for general use — ideal for the Raspberry Pi.

sudo apt-get install dropbear openssh-client
sudo /etc/init.d/ssh stop
sudo sed -i 's/NO_START=1/NO_START=0/' /etc/default/dropbear
sudo /etc/init.d/dropbear restart

After testing that you can still connect to your RPi via SSH, remove the default ssh server.

sudo apt-get purge openssh-server.* openssh-blacklist.*

Eliminate Rarely Used Caches

Prevent aptitude from caching package data by creating /etc/apt/apt.conf.d/02_nocache. This will slow down package updates, but will save about 35MB of disk space in the final image. You could remove the aptitude package altogether, but I find its presence more useful than not, especially when the bulky cache files are removed.

Dir::Cache {
   srcpkgcache "";
   pkgcache "";
}

Since we’re done using apt for package updating, after you reboot your Pi with sudo reboot, you can remove the orphaned cache files to recover another 35MB of disk space:

sudo rm -f /var/cache/apt/*cache.bin
sudo apt-get --yes autoclean
sudo apt-get --yes clean

Prepare for Final Imaging

Numerous files and logs are safely re-created or re-fetched as needed. Removing them will shrink the overall size of the finalized minimalist distro.

sudo find / -type f -name "*-old" |xargs sudo rm -rf
sudo rm -rf /var/backups/* /var/lib/apt/lists/* ~/.bash_history
find /var/log/ -type f |xargs sudo rm -rf
sudo cp /dev/null /etc/resolv.conf

Having also incorporated some drive-I/O-reduction techniques, running df -hlP on my Raspberry Pi shows disk usage of only 367MB, a significant reduction from the original 1.5GB. Your mileage may vary. Shutdown the Raspberry Pi with sudo halt, remove the SD card from the Raspberry Pi board and access it from a Mac or Linux computer to create the final image.

Author’s Note: If you’re just cleaning up your Raspbian installation and have no need to create a distributable image, you can stop now.

To determine which drive designation to use on a Mac, type diskutil list and locate the Identifier of the Linux disk type that ranges around 1.9 GB. On my MacBook Pro, mine is “disk1s2”.

sudo dd if=/dev/disk1s2 of=2012-12-16-wheezy-raspbian-rr-minimalist.img count=3788800

An archive image file is transferred to your Linux PC or Mac, ready for the next steps to generate a zero-byte-filled, compressed final distribution image. Since I’m using a Mac, for which the steps are not very straightforward because the Mac will not natively mount the RPi ext4 partition, I’ll cover that in a future article.

In the meantime, for native Linux users, you can generate a compressed image with the following generic steps.

mkdir mnt
sudo mount -o loop,offset=$((512*122880)) 2012-12-16-wheezy-raspbian-rr-minimalist.img mnt
sudo sfill -z -l -l -f mnt
sudo umount mnt
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on 2012-12-16-wheezy-raspbian-rr-minimalist.img.7z 2012-12-16-wheezy-raspbian-rr-minimalist.img

If you’ve gotten this far, congratulations! The original Raspbian installation image uncompressed to 1.5GB, while the minimalist version is well under 600MB — or even sub-400MB, depending on the options chosen. Use your image any way you want in-house, however I wouldn’t start distributing your image quite yet; wait for a future article where we cover the final topics of security and key re-generation. And no one, not even the builders of Raspbian, seems to have properly addressed the licensing concerns of the included non-free hardware drivers.



Extend the Life of Your RPi SD Card

Many of the modern single-board or System-on-Chip computers (such as Raspberry Pi or BeagleBone) use standard SD memory cards for drive volumes. Since frequent writes eventually damage flash memory systems, in order to extend the life of the SD card, it is important to reduce as many superfluous writes as practical. Shutting down the Pi improperly while it is writing to the card can also induce errors and corruption to open volumes, so reducing writes can have the added benefit of prolonged stability of the file systems.

Author’s Note: This tutorial assumes you have the 2012-12-16 Raspbian “wheezy” distribution installed on your Raspberry Pi, and you have not begun additional customization. While the techniques described below generally apply to other common Linux distros, the instructions, code, and commands may not apply or function in those other cases. You have been warned.

While I’ll be using one of my Pi’s as a built-in intranet web server installed into my powered structured media cabinet (mainly as a wiki and general file server), the goal of significantly fewer periodic writes should be obtainable because I’m hoping that much of the information on the wiki will be fairly static. Worst case, the Pi could buffer logs in memory and then dump them to a NAS or to an Amazon EC2 instance — which would be another article for another time.

Install Sysstat

In order to measure the success of your efforts, you will need a method of monitoring disk activity — one of the most widely used commands of which is iostat, part of the sysstat package.

sudo apt-get install sysstat

Running iostat with no arguments displays aggregate usage information gathered since the system was last booted. Adding interval and count arguments first displays usage since boot, then displays usage during each time period since the previous report.

In the following example, three iostat reports are displayed; the first report on usage since boot is displayed instantly, followed by a usage report gathered over 300 seconds (or five minutes), followed by another five-minute report. Because the focus of this examination is on device usage and not CPU usage, the “-d” (or “device”) option is added; the opposite would require a “-c” (or “cpu”) option.

Run the command before you go any further in order to test the impact of the optimizations below.

$ iostat -d 300 3
Linux 3.2.27+ (raspberrypi) 	2013/02/07 	_armv6l_	(1 CPU)

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.13         2.87         0.95     138189      45840

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.05         0.00         0.35          0        104

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.06         0.00         0.49          0        144

According to the above initial results, during the first five minutes of examination, 104 kilobytes of data was written to disk and 144 KB was written during the second five minutes, for a total of 248 KB over ten minutes. No reads were performed during the entire test.

Disable Swapping

Memory swapping allows part of the usable memory of your computer to be stored on disk, effectively allowing the system to have more RAM than it does physically. Even on high-power Windows or Mac computers, running applications with extensive memory usage beyond the hardware RAM will result in the computer slowing down while the hard drive light blinks incessantly, indicating that the OS is swapping in and out available virtual RAM.

Raspbian utilizes a swap file (managed by dphys-swapfile) in lieu of a more standard swap partition. Removing the functionality is a better long-term solution that simply turning it off.

sudo apt-get remove dphys-swapfile

Remove Unused Packages

In order to build a distribution that satisifies the needs of the majority, Raspbian includes many packages that many RPi users won’t use, especially those that solely intend their Pi to be headless. Removing packages needed for the X Window System graphical user interface will recover a significant amount of disk space, as well as remove always-on daemons that continually generate log messages and other disk I/O (even if you never use the desktop).

sudo apt-get --yes purge xserver.* x11-.* xarchiver xauth xkb-data console-setup xinit lightdm lxde.* python-tk python3-tk scratch gtk.* libgtk.* openbox libxt.* lxpanel gnome.* libqt.* gvfs.* xdg-.* desktop.*
sudo apt-get --yes autoremove
sudo apt-get --yes autoclean
sudo apt-get --yes clean

Author’s Note: Use the above purge command with caution if you have already begun customization of your Raspberry Pi. You may delete packages that are unintended.

Purely to recover more disk space (and not neceesarily in consideration of the SD card life expectancy), I used sudo ncdu from the root partition to remove unused locales from within /usr/share/locale/ as well as everything under /tmp/.

Also consider removing the following files:

  • /var/swap (100 MB)
  • /opt/vc/src/hello_pi/hello_video/test.h264 (30 MB)

Before I started purging, disk usage on my Pi was 1.5 GB (as determing by running df -hlP). After performing all of above suggestions, total disk usage was reduced to only 609 MB, recovering almost 60% of installed space. Your milege may vary.

Reduce Unnecessary Logging

Every couple of minutes cron jobs tend to add log entries to /var/log/auth.log similar to the following:

pam_unix(cron:session): session opened for user root by pi(uid=0)
pam_unix(cron:session): session closed for user root

Edit the /etc/pam.d/common-session-noninteractive configuration file and add the following line near the bottom of the file just before the pam_unix.so call to prevent cron jobs that complete successfully from adding the authorization logs.

session [success=1 default=ignore] pam_succeed_if.so service in cron quiet use_uid

Comment out the log settings line in the /etc/dpkg/dpkg.cfg configuration file. Set the DHCP lease time on your router to no less than one day.

Rotate Logs Less Frequently

The default logrotate configuration files tend to rotate logs daily or weekly, unnecessary on a low-usage server. Edit the /etc/logrotate.d/rsyslog configuration file and replace the “daily” and “weekly” directives with “monthly”, and update the various “rotate nn” options to “rotate 12”.

For your (and my) reference, here’s a copy of my refactored main syslog logrotate configuration file:

monthly
size 10m
rotate 12
missingok
notifempty
compress
delaycompress

/var/log/syslog {
	postrotate
		invoke-rc.d rsyslog rotate > /dev/null
	endscript
}

/var/log/mail.info /var/log/mail.warn /var/log/mail.err
/var/log/*.log /var/log/debug /var/log/messages {
	sharedscripts
	postrotate
		invoke-rc.d rsyslog rotate > /dev/null
	endscript
}

Use Temporary Filesystems

Older Raspbian distributions once seem to have enabled temporary filesystems for the /tmp/ folder, but the latest “Wheezy” has the feature disabled by default, only mounting /run/ and its subfolders as tmpfs filesystems. Reboot after re-enabling the capability.

sudo sed -i 's/\#\s*RAMTMP\s*=\s*no/RAMTMP=yes/i' /etc/default/tmpfs

Verify Improvements

Run the same iostat command as above again in order to compare before and after results. Hopefully you should see some improvement.

$ iostat -d 300 3
Linux 3.2.27+ (raspberrypi) 	2013/02/08 	_armv6l_	(1 CPU)

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.44         1.74         7.84     177045     795692

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.00         0.00         0.00          0          0

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
mmcblk0           0.03         0.00         0.24          0         72

A small sampling of data isn’t truly sufficient to gauge long-term effectiveness of these procedures. That said, according to the latest results above, only 72 kilobytes of data was written to disk during the entire ten-minute test, a reduction of approximately 70% as compared to the initial usage test results.

Given similar usage over an extended period of time, the longevity of the above-tested SD card could be extended by more than three times simply by eliminating memory swaps, uninstalling unneeded modules, and reducing a few unnecessary logs and their subsequent maintenance.


Raspberry Pi Intranet Server

This article will help you set up a simple, fast, light-weight web server on your home network intranet. Please note that by definition the end result is not intended to be suitable for an HTPC, a publicly accessible web site, or a media server, etc.

Author’s Note: Disclaimer for the anticipated wrath of the inevitable Raspberry Pi Fanbois — Yes, an RPi can absolutely be used in those capabilities. I, myself, use one of my Pi’s as a surprisingly capable XBMC-based media center, but support for those heavy-duty applications is the exact opposite focus of this article!

Background Scenario

As a web professional, business owner, and web host provider for any friend or family member who needs free hosting, I have several server deployments in various data centers of Amazon, GoDaddy, Google, DreamHost, A Small Orange, and probably a few others. So access to web-based hosting isn’t exactly an issue for me.

Sometimes I just need access to certain information only while I’m at home. Since the majority of our home network is hard-wired into a central patch panel — along with phone lines, and central video and audio — and we have a photovoltaic (solar) power-generation system, I have accumulated a lot of configuration and operating data that I refer to every now and then while carrying an iPod or netbook around my home. The information could also be useful for the next homeowner; serving it up from a permanent intranet web server that I can just leave hard-wired in the home when we move out would be ideal — and it could be a great sales bullet point for the Realtor listing!

A $35 low-power Raspberry Pi fits the requirements perfectly. Accessories may cost you more, but I initially built a tight-fitting case for free with Legos in order to retain the ability to easily access on-board pin-outs for other development projects. I eventually added an inexpensive Pi case for my “production” server.

Configure the Operating System

Plug in your network cable and micro USB power source. When first booting with the recommended Raspbian operating system, a raspi-config setup menu allows you to easily make some necessary configuration changes. The initial configuration will require a TV with composite video or HDMI input, as well as an appropriate RCA or HDMI cable, plus a basic USB keyboard.

You should minimally perform the following tasks with raspi-config before you begin the rest of the setup of the intranet server.

  1. Expand your root partition on your SD card to its fullest. Alternatively, you can create multiple volumes on the drive, but that’s beyond the scope of this article.
  2. Ensure the keyboard layout settings match your USB keyboard, if you need one. Note that by default, Raspbian installs the UK keyboard layout which is different than the US layout.
  3. Change the default password.
  4. Set your locale and timezone.
  5. Enable the SSH server

Log into your Raspberry Pi with user ‘pi’ and the password you set above, either by an attached keyboard and monitor, or via SSH (enabled above). On my Mac, I type the following to access my Pi:

ssh raspberrypi -l pi

If you have already gone through the setup once before and you wish to run it again, type the following once you’ve logged in:

sudo raspi-config

The default Raspbian distribution is rather bloated for headless systems. At this point you should elect to remove some of the unnecessary cruft and extend the life of your SD card.

Install Lighttpd

Because of their power, reliability, and feature sets, the proven pairing of Apache and MySQL has become the defacto standard for web applications, so much so that WAMP and LAMP have become common acronyms for Apache and MySQL installations on Windows and Linux machines, respectively. I generally use Apache and/or Tomcat for my professional projects because I need that power and reliability.

That said, you don’t need a sledgehammer to crack open a nut. Because my simple project doesn’t need the power or complexity of Apache, I choose to install lighttpd, a small light-weight, single-threaded, single-process web server which performs well on the low-power Raspberry Pi. For the same reasons, if my Pi project needed a database, I would have used something small and light-weight like SQLite over MySQL.

sudo apt-get install lighttpd

Install PHP

If being light-weight was my primary concern for the Pi project, the inclusion of Perl within the default Raspbian installation may have been sufficient. However, for expediency, simplicity, convenience, and reuse of existing code, installing PHP opens up access to myriad readily available software packages and libraries.

sudo apt-get install php5-common php5-cgi php5

Enable FastCGI in order to allow lighttpd to handle PHP requests:

sudo lighty-enable-mod fastcgi-php

When the installation is complete, restart the lighttpd server:

sudo service lighttpd force-reload

Update Permissions

If you want easier, non-root access to modify your web server files, change the permissions on the www directory and add the pi user to the www-data user group created by default in Raspbian:

sudo chown www-data:www-data /var/www
sudo chmod 775 /var/www
sudo usermod -a -G www-data pi

Test the Web Server

Assuming a vanilla Raspbian installation and depending on your network/router configuration, you might be able to access your new web server by simply typing “http://” followed by the hostname (usually ‘raspberrypi’) into your web browser address bar. Alternatively, type “http://” followed by the actual IP address of the network interface on your Raspberry Pi. Examples:

http://raspberrypi/
http://192.168.1.101/

You should see a standard welcome placeholder page. It will be the same page no matter which URL you use to access it. Please note that your IP address is probably not 192.168.1.101.

Find the IP Address

You might not know the IP address of your RPi. Configuring your network is outside the scope of this document, so the assumption is that the Pi is connected via Ethernet cable to a DHCP-enabled router. To obtain the assigned IP address of the built-in network interface of the Raspberry Pi, type the following:

grep -h fixed-address /var/lib/dhcp/*

You should see something like the following:

fixed-address 192.168.1.101;

Alternatively, if you manually configured your IP address, are using a USB WiFi adapter, or would just rather type less and read more, simply hunt through the output of the following command for your IP address, usually in section “eth0” (the built-in network interface), “eth1” (a USB NIC), or even “wlan0” (a USB wireless network adapter):

ifconfig

Most of the time on a home network, the IP address of a computer or device usually starts with “192.168.”, and the last set of the four sets of numbers usually are not “0”, “1”, or “255”. Computers on networks using an Apple Time Capsule or Airport Extreme as a router often have IP addresses that start with “10.0.” or “172.16.”, the other two common private network ranges.

Next Steps

Delete the default lighttpd web page and replace it with your own:

sudo rm /var/www/index.lighttpd.html

Personalize your private, intranet web server with any number of available software packages. For my project, I installed DokuWiki, a mid-level, file-based, collaborative web text editor; somewhere between little-too-simple PmWiki and the complex, database-driven MediaWiki.

Other ideas include:

The rest is up to you.