Context

Func, Fedora Unified Network Controller, allows us to send commands for remote execution in a set of pre-configured hosts. More, it allows us to choose which machines we want to address using shell globs. This, and more, is done securely over SSL. Func facilitates remote administration.

Func is a Fedora project whose general architecture is based on two parties: Func itself and Certmaster. Certmaster is responsible for authentication and encryption using SSL certificates. Certificates are identity proofs. And, from my knowledge, (correct me) they are little more than a signature (private key encryption of an hash) attached to a cryptographic public key. Certmaster application is composed by a client and a daemon capable of requesting and serving SSL certificates. This enables a master machine to communicate with a slave over SSL.

On top of this comes Func: it is also a built on client-server style and enables administrators to build an infrastructure of slave machines running the daemon funcd. This daemon listens on a port for requests from a certain master machine. In fact, the SSL scheme behind it lets us sleep tight at night knowing that those slaves will only reply to the master and not some fake.

The master is where the all-powerful sysadmin is logged in From that machine, the sysadmin can issue commands for execution by the slaves at the same time. Even cooler, he can address those machines in the following manner:

"*fe.up.pt"

Meaning all the machines in the “fe.up.pt” domain under my control. How cool is that?

Func is python, and has a python API so, besides using Func directly, we can even build our own customized scripts.

Installation…

I tested this in CentOS 5. I think Func and Certmaster are in the EPEL repo.

This, in fact, little more than what can be found in the official Func page.

On the master:

Install Func and Certmaster. Certmaster gets pushed by Func on installation.

Certmaster daemon will be running on the master machine, making it possible for the slaves (called minions) to request certificates.

$ yum install func
$ /sbin/chkconfig --level 345 certmaster on
$ /sbin/service certmaster start

On the slave machines:

On the minions, the funcd will be running, accepting requests from func client and running commands. Besides this, func will try to request certificate signing from the master. For this to work, we’ll need to indicate the master’s address on the certmaster configuration file.

$ yum install func

Edit the /etc/certmaster/minion.conf file and change the address of your master machine. For example:

# configuration for minions
[main]
certmaster = master.fe.up.pt
certmaster_port = 51235
log_level = DEBUG
cert_dir = /etc/pki/certmaster

Now, put the funcd running at boot and start it.

$/sbin/chkconfig --level 345 funcd on
$/sbin/service funcd start

Also, you need to open ports on iptables. The master machine must have port 51235 open, and the minions must have port 51234 open.

Working

Now we have the environment set.

Before starting to send commands to slaves, we need to “sign” them first. Issue the following command from the master to list the slave machines. If everything is ok, you’ll see all the slave machines where you’ve just installed func.

$ certmaster-ca --list

Once you’ve seen the machines that are ready to be signed, you can sign them:

$ certmaster-ca --sign slave1.fe.up.pt
$ certmaster-ca --sign slave2.fe.up.pt
$ ...

Example

Now, you’re ready to send commands. Let’s imagine you want to send a message to all logged users on the slaves, using wall command:

$ func "*.fe.up.pt" call command run "wall 'Hello users! Machine will go down for reboot in 1 minute! Save you work!'"

This is a simple example. You can use more complex globbing to address specific machines. See https://fedorahosted.org/func/wiki/CommandLineGlobbing.

About provisioning

If you want to integrate minion installation with kickstart for example, it’s fairly simple, as you only need to install packages and change a line of the config file.

You can run into some problems if you happen to kickstart a minion machine, both in the master and slave. This is because in the master there will be a CSR inconsistent with the new machine. And in the minion, because funcd automatically retrieves the certificate at boot, funcd will have a wrong certificate, one referring to the previous state of the machine before re installation. You can solve this by cleaning SSL keys and such in the minion and cleaning the certificate request on the master:

# on the new, kickstarted machine
$ service funcd stop
$ rm -rf /etc/pki/certmaster/*
# on the master
$ certmaster-ca --clean <hostname_of_the_kickstarted_machine>
# again, on the slave
$ service funcd start
# on the master again
$ certmaster-ca --sign <hostname_of_the_kickstarted_machine>

All this is just to clean certificates and certificate requests form both ends.

This can be avoided if, immediately before restarting the minion to a re installation,  we clean its respective certificate request in the master with:

$ certmaster-ca --clean <hostname>

This way, when the minion boots up with the funcd daemon running, it will reach a consistent state.

Still, you’ll have to issue the –sign command.

References

• https://fedorahosted.org/func/