The general security policy for Beowulf clusters should be such that all the nodes within the cluster fully trust each other. The reason you can relax the security inside the cluster is because none of the client nodes are directly connected to the outside world, and all nodes are basically the same. If someone hacks into the gateway node they will not get any more information from any of the client nodes, therefore you don't have to worry about the security at this level. It is practically impossible for anyone to access any of your client nodes without actually sitting at the console, or going via the server node first. The main advantages of relaxing the security within the cluster are flexibility and ease of use and administer. The server node on the other hand should trust its client nodes but not the outside world. There are few things you can do to relax the security within the cluster and to protect your self from outside.
The tcpd daemon, commonly known as TCP wrapper, is the first line of
defense, and is the simplest way of limiting access to your machine
and therefore increasing security. It comes as part of Red Hat
installation and is simple to configure. There are three
configuration files: /etc/hosts.allow which checks for hosts
which are allowed connections, /etc/hosts.deny which is read
if the host was not found in /etc/hosts.allow and checks for
hosts which are to be refused connection, and /etc/inetd.conf
which you should not have to modify to configure
tcpd. hosts_access(5) man page provides good source
of information on the syntax of both /etc/hosts.allow and
/etc/hosts.deny.
/etc/hosts.allowThe example file below will allow access to any port from any host
with IP address 10.0.0.x, 10.0.1.x, or 10.0.2.x. It will also allow
telnet access from host myworkstation.usq.edu.au. All other
connections will be blocked by the /etc/hosts.deny file,
assuming the service is listed in /etc/inetd.conf and
configured to be launched via tcpd.
# # hosts.allow This file describes the names of the hosts which are # allowed to use the local INET services, as decided # by the '/usr/sbin/tcpd' server. # # we fully trust ourself and all the other nodes within the cluster ALL : localhost, 10.0.0., 10.0.1., 10.0.2. in.telnetd : myworkstation.usq.edu.au
/etc/hosts.denyThe /etc/hosts.deny file is checked for matches when no match
was found in /etc/hosts.allow. The best way of using the TCP
wrappers is to deny everything that has not been allowed or matched by
/etc/hosts.allow. In our cases we not only match ALL in
/etc/hosts.deny, and therefore deny everything, but for every
denied connection we send an e-mail with details to the administrator.
ALL: ALL: spawn ( \ echo -e "\n\ TCP Wrappers\: Connection Refused\n\ By\: $(uname -n)\n\ Process\: %d (pid %p)\n\ User\: %u\n\ Host\: %c\n\ Date\: $(date)\n\ " | /bin/mail -s "From tcpd@$(uname -n). %u@%h -> %d." root)
If a connection is attempted from a host not listed in
/etc/hosts.allow the match will occur in
/etc/hosts.deny, so connection will be closed and I will
receive an e-mail with notification. An example of such an e-mail is
shown below.
From root Fri Apr 16 23:33:50 1999
Return-Path: <root>
Received: (from root@localhost)
by topcat.beowulf.usq.edu.au (8.8.7/8.8.7) id XAA19278
for root; Fri, 16 Apr 1999 23:33:50 +1000
Date: Fri, 16 Apr 1999 23:33:50 +1000
From: TOPCAT Admin <root@topcat.beowulf.usq.edu.au>
Message-Id: <199904161333.XAA19278@topcat.beowulf.usq.edu.au>
To: root@topcat.beowulf.usq.edu.au
Subject: From tcpd@topcat.beowulf.usq.edu.au. jacek@lamport.comp.usq.edu.au -> in.telnetd.
Status: O
TCP Wrappers: Connection Refused
By: topcat.beowulf.usq.edu.au
Process: in.telnetd (pid 19270)
User: jacek
Host: jacek@lamport.comp.usq.edu.au
Date: Fri Apr 16 23:33:50 EST 1999
/etc/inetd.confA very simple, but effective way of improving your server security is
to disable unwanted services. The rule of thumb is to disable every
thing you don't need. Most daemons are started by the inetd
super server and should be turned off by commenting out lines in
inetd.conf. Example below shows part of inetd.conf
with login, exec, talk, and ntalk disabled.
shell stream tcp nowait root /usr/sbin/tcpd in.rshd #login stream tcp nowait root /usr/sbin/tcpd in.rlogind #exec stream tcp nowait root /usr/sbin/tcpd in.rexecd #talk dgram udp wait root /usr/sbin/tcpd in.talkd #ntalk dgram udp wait root /usr/sbin/tcpd in.ntalkd
inetd daemon. The simplest way to do it on Linux is to send
a hang up signal to the daemon which will force it to re-read its
configuration file.
[root@topcat root]# killall -HUP inetdDo not try this on other Unix system without reading the killall man page first!
You can check which daemons are running by getting a list of all listening ports. You can easily get this list by running:
[root@topcat root]# netstat -a | grep "LISTEN" | grep -v "^unix"
Servers like Web server (httpd) and Samba (smbd)
start as rc scripts. Normally each should be disabled by deleting the
corresponding link in /etc/rc.d/rc.3d directory. These links
point to startup scripts in /etc/rc.d/init.d. For
example, to stop httpd, samba, and sendmail (another potential
security problem) from automatically starting in run levels 3 and 5
you would do :
[root@topcat samba]# rm -f /etc/rc.d/rc3.d/S*httpd [root@topcat samba]# rm -f /etc/rc.d/rc5.d/S*httpd [root@topcat samba]# rm -f /etc/rc.d/rc3.d/S*smb [root@topcat samba]# rm -f /etc/rc.d/rc5.d/S*smb [root@topcat samba]# rm -f /etc/rc.d/rc3.d/S*sendmail [root@topcat samba]# rm -f /etc/rc.d/rc5.d/S*sendmail
ipfwadmipfwadm program allows blocking packets from specific IP
addresses to specific ports and is the most flexible way of
controlling security. The example firewall (see
firewall_script) rc script should be started
automatically at boot time. This can be achieved by:
[root@topcat init.d]# cp /home/jacek/firewall /etc/rc.d/init.d [root@topcat init.d]# chmod u+rx firewall [root@topcat init.d]# ln -s /etc/rc.d/init.d/firewall /etc/rc.d/rc3.d/S05firewall [root@topcat init.d]# ln -s /etc/rc.d/init.d/firewall /etc/rc.d/rc5.d/S05firewallNOTE: You should modify my script to suit your requirements.
One of the things you will want to allow your users to do, is to login and execute remote shells between the nodes without entering their password. Most of the Beowulf software and utilities assume that you can execute a remote shell (rsh) to at least all of your client nodes without the need to enter the password.
There are two ways to eliminate passwords within the cluster. You can
either add an entry to the /etc/hosts.equiv file or add a
.rhosts in each users home directory.
The /etc/hosts.equiv is preferable because the information in
this file can applied to the whole node, where .rhosts is per
user.
The format of .rhosts file is simply a list of hosts:
# must be read/writable by user only! node1 node2 node3 node4 node5 node6
The format of the hosts.equiv file is
#node name optional user name node1 node2 node3 node4 node5 node6
To allow root to rlogin to any node in the cluster, add a .rhosts file
in the root directory on each node. The .rhosts file should list all
the nodes in the cluster. IMPORTANT: The .rhosts must be only
read/writable by the owner. (chmod go-rwx .rhosts) Again this
should not be done for the gateway node.
In addition, swap the first two lines of /etc/pam.d/rlogin:
#original /etc/pam.d/rlogin auth required /lib/security/pam_securetty.so auth sufficient /lib/security/pam_rhosts_auth.so auth required /lib/security/pam_pwdb.so shadow nullok auth required /lib/security/pam_nologin.so account required /lib/security/pam_pwdb.so password required /lib/security/pam_cracklib.so password required /lib/security/pam_pwdb.so shadow nullok use_authtok session required /lib/security/pam_pwdb.so #first two lines are swapped /etc/pam.d/rlogin auth sufficient /lib/security/pam_rhosts_auth.so auth required /lib/security/pam_securetty.so auth required /lib/security/pam_pwdb.so shadow nullok auth required /lib/security/pam_nologin.so account required /lib/security/pam_pwdb.so password required /lib/security/pam_cracklib.so password required /lib/security/pam_pwdb.so shadow nullok use_authtok session required /lib/security/pam_pwdb.so
NOTE: I do not know if there is a better way to do this, but it seems to work.
On every node except the gateway, the following has been added to
the /etc/securetty file:
ttyp0 ttyp1 ttyp2 ttyp3 ttyp4
This change will allow remote telnet to any node in the cluster.
On any system that needs root ftp access, /etc/ftpusers file has to have the
entry for root commented out:
Comment out root to allow other systems ftp access as root
#root bin daemon adm lp sync shutdown halt mail news uucp operator games nobody