Torque Configuration: User Access

Getting a simple installation of Torque up and running is relatively easy. But to take advantage of its many features and creating a useful cluster can be a challenge. The online documentation can be pretty cryptic for the beginning and even the advanced Torque user. I am going to start a new series of blog posts which will focus in on different aspects of Torque configuration and hopefully, make it easier to understand how different parameters work and work together.

In this post I am going to talk about the server parameters acl_host_enable, acl_hosts, allow_node_submit, and submit_hosts. These parameters serve to give the administrator control of what nodes and what users can have access to their cluster. We have recently found there are some inaccuracies in the online Torque documentation which has been the source of some confusion.

Following is what the new documentation will be for these parameters.

acl_host_enable

Format <Boolean>
Default False
Description When acl_host_enable is true hosts not in the $TORQUE_HOME/server_priv/nodes file must be added to the acl_hosts list in order to get access to pbs_server

acl_hosts

Format <HOST>[,<HOST>]… or <HOST>[range] or
<HOST*> where the asterisk (*) can appear anywhere in the
host name
Default Not set
Description Specifies a list of hosts which can have access to pbs_server
when acl_host_enable is set to true. This does not enable a node
to submit jobs. To enable a node to submit jobs use submit_hosts. Hosts which are in the $TORQUE_HOME/server_priv/nodes file do
not need to be added to the acl_hosts list.

Qmgr: set queue batch acl_hosts=”hostA,hostB”
Qmgr: set queue batch acl_hosts+=hostC
Qmgr: set server acl_hosts=”hostA,hostB”
Qmgr: set server acl_hosts+=hostC

In Torque version 2.5 and later the wildcard (*) character can appear anywhere in the host name, and ranges are supported.
Qmgr: set server acl_hosts = “galaxy*.tom.org”
Qmgr: set server acl_hosts += “galaxy[0-50].tom.org”

submit_hosts

Format “<HOSTNAME>[,<HOSTNAME>]…”
Default Not set
Description Hosts in this list are able to submit jobs. This applies to any
node whether within the cluster or outside of the cluster. If
acl_host_enable  is set to true and the host is not in the
$TORQUE_HOME/server_priv/nodes file then the host must also be in
the acl_hosts list.

allow_node_submit

Format <Boolean>
Default False
Description When set to True this allows all hosts that are in the
$TORQUE_HOME/server_priv/nodes file (MOM nodes) to submit jobs to
pbs_server

 

I could explain what was inaccurate in the previous documentation but I think it will be more instructive and less confusing just to write about how these parameters really work.

acl_host_enable

The acl_host_enable parameter lets the administrator decide if the cluster is to be open to any host that can reach the pbs_server host or if it will be restricted to only the nodes found in $TORQUE_HOME/server_priv/nodes (these are the MOM nodes) and any hosts enumerated in the acl_hosts parameter.

By default acl_host_enable is set to false. With acl_host_enable set to false any user from any host can run any query type command against pbs_server and get a response. That is any host whether inside the cluster as designated by the pbs_server nodes file or any other host that can reach pbs_server can run query commands.  Examples of query commands would be qstat and pbsnodes.

When acl_host_enable is set to true only trusted hosts can have access to the pbs_server host. By default all of the MOM nodes are trusted hosts. But all other  hosts outside of the cluster are not trusted. In order to be trusted a host must be in the acl_hosts list. Whether acl_host_enable is set to true or false users cannot submit jobs except from the pbs_server host. In order to submit jobs from other nodes they must be configured to be submit hosts which I will discuss later.

acl_hosts

The acl_hosts parameter is only effective when acl_host_enable is set to true. In order to get access to pbs_server a host must be trusted. All of the MOM nodes are trusted by default so the acl_host list is simply a way to designate trusted hosts that are not part of the cluster. The names of the hosts can be the full host name or the ‘*’ wildcard can be used to add a group of similarly named hosts in one entry. The ‘*’ can be used anywhere within the string and will match all characters. For example:

qmgr -c ‘set server acl_hosts=row*.edu

With this one line all hosts that start with row and end with .edu will be allowed to access pbs_server.

acl_hosts also allows ranges for numbered hosts. So if your data center uses numbers to name hosts such as qt01.data thru qt50.data then you could add all 50 hosts using the following:

qmgr -c ‘set server acl_hosts+=”qt[1-50].data

The documentation has claimed you can be granular and allow specific users access from a host by using something like [email protected] However, in my testing I have found that this is not working. At least as a way to grant access to the system for the individual user.

acl_host can be set on a per queue basis. So instead of allowing access to all queues in the system the administrator can restrict the access of a host to a single queue. The following is an example of how you would do this with qmgr.

qmgr -c ‘set queue batch acl_hosts=qt01.data

This allows the host qt01.data access to the batch queue only. If users from this host try to submit jobs to other queues they will fail.

Creating Submit Nodes

There are three ways any host can be turned into a submit node. Using allow_node_submit, submit_hosts and using the /etc/hosts.equiv file.

/etc/hosts.equiv

Torque is able to grant job submission access to the server by adding a host and user name to the /etc/hosts.equiv file. Internally Torque calls ruserok to authorize users based on what is in the /etc/hosts.equiv file. But as Joshua Bernstein liked to say “ruserok is not ok”. While this method is supported we recommend using the other configuration options to control user access if possible.

submit_hosts

When a host is in the submit_hosts list then it becomes a submit node. The exception is the pbs_server host which is always a submit node. Unless the allow_node_submit parameter is set to true even MOM nodes must be added to the submit_hosts list in order to be a submit node.

When acl_host_enable is false all hosts listed in submit_hosts are submit nodes. But when acl_host_enable is true a node must also be in the acl_hosts list in order to be a submit node.

allow_node_submit

allow_node_submit is a shorthand way to make all MOM nodes submit nodes. Often users have a need to submit jobs from the scripts they run so that means the MOM must be a submit node. Because it is not known ahead of time where a job may run it is likely that the administrator will need of the the MOM nodes to be submit nodes. Rather than adding all of the MOM nodes the the submit_hosts list you can set allow_node_submit to true and immediately all MOM nodes are submit nodes.

 

 

 

Facebook Twitter Email

Speak Your Mind

*