Introduction

Fabric Roles

In standard Fabric roles provide a many-to-many mapping between tasks and hosts. Once roles are decided, roledefs map hosts to roles. For example, in the case of a “dev” web server, all roles might collapse onto a single server. Install and deploy tasks are nicely related to hosts via roledefs.

For a production cluster, a role is often applied to more than one host. For example, a pool of webapp or memcache servers.

In the first use-case, where roles are collapsed onto a single server, Fabric comes up a little short because roles are only used to construct host lists for tasks. A role has no “configuration state”; once hosts are assigned to tasks, what role instigated that task is out of the picture. More importantly, a single task will only be invoked once per host, even if that host matches more than one role.

An example: the following fabfile runs install on each server:

from fabric.api import *

env.roledefs.update({
    'webserver': ['www1', 'www2'],
    'dbserver': ['db1']
})

env.roles = ('webserver', 'dbserver')

@roles('webserver', 'dbserver')
def install():
    print("{command} invoked on host {host}".format(**env))

# outputs...
# [www1] Executing task 'install'
# install invoked on host www1
# [www2] Executing task 'install'
# install invoked on host www2
# [db1] Executing task 'install'
# install invoked on host db1

That looks like what we’d want. But the next fabfile, which tries to combine webserver and dbserver onto a single host runs install only once.

from fabric.api import *

env.roledefs.update({
    'webserver': ['dev'],
    'dbserver': ['dev']
})

env.roles = ('webserver', 'dbserver')

@roles('webserver', 'dbserver')
def install():
    print("{command} invoked on host {host}".format(**env))

# outputs...
# [dev] Executing task 'install'
# install invoked on host dev

But, what if we really want install to run once per role? Yes, we could split out a install_web and install_db. If they both run the same "apt-get install " + env.packages it sure would be more DRY not to split them.

Even if we did invoke install for each role, that would just install env.packages multiple times. Then again, what if we could also inject role specific settings into the Fabric env, for each role invocation of a task?

Fabex Task Roles

What if we could write

@roles('webserver', 'dbserver')
def install():
    packages = { 'webserver': 'nginx',
                 'dbserver': 'postgres' }
    with settings(packages=packages[env.role]):
        sudo("apt-get install " + env.packages)

and have install execute once per role. Wouldn’t that be convenient? That’s exactly what the Fabex task_roles decorator gives you: one invocation per role, with a role specific env. The above task becomes

@task_roles('webserver', 'dbserver')
def install():
    sudo("apt-get install " + " ".join(env.packages))

Where does env.packages get set? In a “target configuration” yaml file.

roledefs:
    webserver: [server]
    dbserver: [server]

hostenvs:
        server: {ip: 127.0.0.1, ssh_host: 127.0.0.1, ssh_user: ubuntu}

roleenvs:
    webserver:
        packages: [nginx]
    dbserver:
        packages: [postgres]

domain: example.com

The first attribute, roledefs is the standard Fabric role<->host relation. To this, we add top level hostenvs and roleenvs. The task_roles decorator pulls attributes from hostenvs for the matching host and from roleenvs from the invoking role and updates the env dict with those values.

In addition, any top level attributes are injected, e.g., domain in the above example. As this configuration information comes from a file separate from the fabfile, we’d like to be able to point at that file “on the fly”, in the fab invocation. That allows different settings for different installations: dev, stage, production, etc.

Where do we keep these yaml targets? Fabex provides a fabex_config function to initialize and point at a few things. Here’s a complete, working Fabex fabfile.

from fabex.api import *

fabex_config({'target_dir': '.'})

@task_roles('webserver', 'dbserver')
def install():
    print("installing {packages} on host {host} for {domain}".format(**env))

Save this snippet into fabfile.py, copy the yaml above into a test.yaml file. Then run

fab target:test install

and you should see

[server] Executing task 'install'
[server:webserver] Executing role task 'install'
installing ['nginx'] on host server for example.com
[server:dbserver] Executing role task 'install'
installing ['postgres'] on host server for example.com