The ANTS Load Balancing System - Sample Setup

Simple two-node setup

We have two nodes, node1 and node2. On both nodes you are expected to have installed the ANTS system.

On both nodes, you should now create the file /etc/antsd.hosts, which tells the antsd which hosts are allowed to join the ANTS network (the JobNet). The file looks like hosts.equiv and similar files, and is simply a list of hostnames. Your file should look like: 

node1
node2
...and that's it.

Now you should define what job-types you want to run on each host. Jobs are given types so that you can limit the number of eg. linkers and compilers you wish to run on each host individually. We define three job-types on host1, and two job-types on host2, in this example.
On host1 write the following in the /etc/antsd.conf file: 

gcc 8 3
g++ 4 2
This tells the job-daemon that:
On host2 we also define a ld job type. This is for linkers. Perhaps it's host2 that holds the shared home-directory, and since linkers are really heavy on disk I/O, it may make sense to only define the linker job-type on the host physically holding the disks for the home directory. We write a /etc/antsd.conf on host2: 
gcc 4 3
g++ 4 2
ln  6 2
The numbers mean the same as before.

Using this example, a rant -t gcc hostname should return either node1 or node2, meaning the hostname command was executed on that host. But a rant -t ln hostname will always return host2 since host2 is the only host defining the ln job type.

I hope this example gave you an idea about what this is all about.

The Makefile changes

Although you can use this system for other things than parallel make jobs, this was actually what the system was built for. A simple change in a makefile will allow your compilation to make use of the ANTS functionality.

The idea is simple: Change the variables defining the name of your compiler, to the rant -t'ified version of the same. Example: 

CC = rant -t gcc gcc
LD = rant -t ld  ld
CPP = rant -t g++ g++
Putting this in your makefile should allow normal compile rules that use those variables, to use the ANTS system. You should also run the make command with a -j switch, to allow the make program to start up multiple compilations concurrently. See the make documentation for more info on the switches.

The beautiful solution would of course be to integrate GNU Make with the ANTS system so these things happened automatically.

Compiling the Linux kernel

Change the following in the main Makefile for the kernel: 
AS              = rant -t ld $(CROSS_COMPILE)as
LD              = rant -t ld $(CROSS_COMPILE)ld
CC              = rant -t gcc $(CROSS_COMPILE)gcc
CPP             = rant -t gcc $(CC) -E
AR              = rant -t ld $(CROSS_COMPILE)ar
NM              = rant -t ld $(CROSS_COMPILE)nm
STRIP           = rant -t ld $(CROSS_COMPILE)strip
OBJCOPY         = rant -t ld $(CROSS_COMPILE)objcopy
OBJDUMP         = rant -t ld $(CROSS_COMPILE)objdump
And create the ld and gcc job types in your cluster, as descibed earlier. When compiling, you should specify the -j switch for make, to tell make that it should spawn multiple concurrent jobs. Example: 
] make clean ; make dep ; time make -j16 
....
real    1m34.156s
user    0m29.120s
sys     0m10.640s
1.5 minutes for a full 2.4.2 kernel compilation - that's not too bad :-) The above was run on a cluster consisting of