Beowoof: a Beowulf Cluster with Power Mac 6100s running Yellow Dog Linux
Notes

Introduction

Motivation

Hardware

Software

Configuration

Installing Linux

Progress

Projects

Notes

Background

Specific Advice

• Glibc is not required, but it can be used.
• Each node should be identically configured.
• Benchmark, profile, find the bottleneck, fix it, repeat.
• Or, as Greg Lindahl points out, if you just want to run the same program a few thousand times with different input files, a shell script will suffice.
• Allow root to rlogin on the nodes.
• /home is shared across all the nodes. The login node shares /home and all the cluster nodes log on to it.

General Stuff

There is no "standard" Beowulf installation. Every cluster has its own requirements and approaches to solving them.

Security tends to get in the way of what nodes in a Beowulf cluster do, so a standard Linux installation needs to be tweaked before it will work. Diskless booting sounds appealing but makes setup and configuration more difficult.

For a CoolCS Beowulf, there should be two installations, one for the login node and one for all of the cluster nodes. The cluster nodes are configured with root rlogins allowed; the login root will be configured with routing tables and such to act as a firewall to the cluster nodes. The work nodes should have default configurations as Beowulf nodes; the installer should ask the number of the node it's installing, and set up all the required network addresses and host names based on that.

We should set up a group of administrators and a group of users for the nodes. The idea is to allow Beowoof to be open on the Internet so remote subscribers can use it.

Can an install program access a file on a server somewhere? If the login node is set up first, then it can start right away by serving up certain cluster configuration files. The installer could refer to those files during its setup. Specifically, every cluster node needs to have the same user list as the login node.

SSH

A correspondent writes...

I thought I'd share with you something interesting related to the problem you were having with identical machines not behaving identically as far as SSH without passwords goes... I had the exact same problem, and I found that when I....
a.) generated keys on each machine and sent every machine's key to a single machine where I concatenated them and sent a copy of the concatenated key list to each machine....
it failed but when I ...
b.) sent a copy of each key to each machine and did the concatenation *separately* on each machine....
it worked fine. I don't know why.

I see that the ~/.ssh/authorized_keys file needs chmod go-rwx. That seems to fix the problem. So maybe the difference wasn't what machine I made the file on but rather who I was logged in as at the time. I was playing with the password-free ssh as root, so I guess maybe it wouldn't have worked for any other user the way I was doing things. But I never tried it as anybody else.