Living Room Cluster

As a fun side project with a small amount of utility, I tried connecting a few old Dell Optiplex 320 small form-factor PCs together into a cluster. They are rather weak machines without much of anything, but they were cheap surplus, and in a group they have a bit of clout (albeit with a horrendous energy footprint).

I decided to go the simple route (not necessarily the easy route) in terms of software. I loaded them each with Debian Jessie netinst (currently testing), and installed puppet. I install nothing more than the netinst and puppet, and puppet will take care of the rest - it has a list of Debian packages to install, as well as an nfs export to mount. I put together a few small scripts to manage the nodes (send tasks to the node with the smallest load), which are very simple because the things are on single core processors. Typically I will just send the task to a node that isn't currently running a task. For things like compiling code, I'm currently testing out distcc. I should just be able to give a list of the nodes to distcc and have them all work on compiling a large C++ toolkit, for example.

Regarding the network, these surplus machines have gigabit ethernet since they were enterprise systems, which is quite fortunate. They have static IPs over the unmanaged switch in the 192.168.3.* range. The master node uses a list of these IPs to issue wake-on-lan commands to nodes that are currently not running, which cuts down a lot on the overhead! In this regime, the idle nodes will eventually shutdown, and they will wake only as needed.

Bill of Materials

  1. Computers Dell Optiplex 320.

  2. Switch Gigabit Trendnet 8-port.

The total cost was $20/node + $15 for the switch. A 4-node cluster would be under $100.

Software

The puppet modules will be posted here shortly (unless I forget - send me an email if so). A puppet configuration was set up that would automatically install a set of packages from a file directory. I had to manually rename the packages for ease of install, but simply downloaded them from the Debian Jessie repository and moved them in, then added the name to my puppet configuration file. It was split into two configurations - one required site.pp file (necessary for each puppet install) and one module loaded from modules/scientific/manifests/init.pp. 


import 'scientific'
# as a test: ensure permissions on sudo
class sudo {
    file { "/etc/sudoers":
        owner => root,
        group => root,
        mode => 440,
    }
}
class datamounted {
	file { "/home/wilson/public":
		ensure => "directory",
		owner => "wilson",
		group => "wilson",
	}
	mount { "/home/wilson/public":
		ensure => "mounted",
		device => "node1:/home/wilson/public",
		options => "rw,intr",
		fstype => "nfs",
		atboot => "true",
	}
}

node default {
	include datamounted
	include debpackages
	include logins
	include ntp
	include idleshutdown
}


class idleshutdown {

	file { "/home/wilson/shutdownonidle":
		ensure => "file",
		source => "puppet:///modules/scientific/shutdownonidle",
		owner => "wilson",
		group => "wilson",
	}

	cron { shutdownonidle:
		command => "/home/wilson/shutdownonidle",
		user => "root",
		minute => [15,45],
	}

}
class logins {
	file { "/home/wilson/.ssh":
		ensure => "directory",
		owner => "wilson",
		group => "wilson",
		mode => "700",
	}
	file { "/home/wilson/.ssh/authorized_keys":
		ensure => "file",
		owner => "wilson",
		group => "wilson",
		mode => "600",
		source => "puppet:///modules/scientific/authorizedkeys",
	}
}

class ntp {
	file { "/etc/ntp.conf":
		ensure => "file",
		source => "puppet:///modules/scientific/ntp",
		owner => "root",
		group => "root",
	}
}

class debpackages {
	file { "/home/wilson/.bashrc":
		ensure => "file",
		source => "puppet:///modules/scientific/bashrc",
		owner => "wilson",
		group => "wilson",
	}

	file { "/opt/dpackages":
		ensure => "directory",
		source => "puppet:///modules/scientific/dpackages",
		recurse => "true",
		owner => "root",
		group => "root",
	}

	define instdpkg ( ) {
		file { "/opt/dpackages/${name}.deb":
			ensure => "present",
			owner   => "root",
			group   => "root",
			mode    => "644",
			source  => "puppet:///modules/scientific/dpackages/${name}.deb",
		}
		package { "${name}":
			provider => "dpkg",
			ensure => "latest",
			source => "/opt/dpackages/${name}.deb",
		}
	}

	instdpkg{ "ethtool":}

	instdpkg{ "vim-runtime":}
	instdpkg{ "vim":}

	instdpkg{ "libc-dev-bin":}
	instdpkg{ "linux-libc-dev":}
	instdpkg{ "libc6-dev":}
	instdpkg{ "make":}
	instdpkg{ "screen":}

	instdpkg{ "libfile-copy-recursive-perl":}
	instdpkg{ "libfile-fcntllock-perl":}
	instdpkg{ "update-inetd":}
	instdpkg{ "openbsd-inetd":}
	instdpkg{ "rstatd":}

	instdpkg{ "openssh-server":}

	instdpkg{ "libopts25":}
	instdpkg{ "ntp":}
}