.. _example: ********************************* A Django Build and Deploy Example ********************************* The Example Project =================== This project brings up a Django site on a Ubuntu 14 server. The site itself is trivial, just what ``django-admin startproject`` produces. The cluster consists of an Nginx http proxy in front of a uwsgi server, the latter launched by supervisord. The uwsgi server is running the Django app in a virtualenv that we build, configure and install. The deploy will copy Django static files into a path accessible to Nginx for static delivery. Static file processing will only execute once per deploy, as will database migrations, even if we add multiple Django application server instances. We will deploy local settings to the Python app, only if those settings change. The Django site ``SECRET_KEY`` will be kept encrypted in our deploy configuration, and only decrypted in memory while the deploy runs. Project Source and DIY ====================== If you want to play along at home, and try this project yourself, the source can be found in the Fabex repo at https://bitbucket.org/rmorison/fabex/src under the fabex-example tree. Once you pull the source, and pip install Fabex, check and change the ``hostenvs`` section in ``targets/example.yaml`` .. code-block:: yaml hostenvs: server: {ip: 127.0.0.1, ssh_host: 127.0.0.1, ssh_user: ubuntu} You might want to build an Ubuntu VM and point at that, or you can run the example on localhost. Note, the build does install the apt packages listed in example.yaml, so decide if that's ok for your local machine. After you're all installed and configured, you can build the "whole enchilada" with .. code-block:: bash fab password:123 target:example install setup deploy Target Yaml & Encrypting Sensitive Settings =========================================== Here's the yaml configuration our Fabex fabfile will use: .. code-block:: yaml domain: example.com timezone: America/Los_Angeles locale: en_US.UTF-8 roledefs: proxy: [server] app: [server] hostenvs: server: {ip: 192.168.56.33, ssh_host: 192.168.56.33, ssh_user: ubuntu} roleenvs: proxy: packages: [ntp, nginx] server_names: "*.example.com" httpd_user: www-data static_path: /opt/html static_user: ubuntu project_name: example app: packages: [ntp, supervisor, python-dev, python-pip, virtualenvwrapper] user: ubuntu virtualenvwrapper: /usr/share/virtualenvwrapper/virtualenvwrapper.sh workon_home: /home/ubuntu/pyves project_name: example requirements: [django==1.9 uwsgi] debug: yes secret_key: !decrypt DeZkYMRlEbWoMre7waFBoKFTxB0AFIBCsDxCzuttAkEeVHL1Xivbuc16OaPc7HmCVi2fOXUWEZp8ZDbFRXSCGjHG903lcAyhhdtuv70GdIU= wsgi_port: 10000 Note the ``!decrypt`` on the ``secret_key``. ``!decrypt`` is a Fabex specific yaml macro to decrypt on the fly. That encrypted text was generated by the ``fxcrypt`` utility provided by Fabex. .. code-block:: bash fxcrypt 123 'el!qznto#i@v=v+b-(3^3*%nv=kzx!j3+%j6h*7or95anju#uc' which outputs .. code-block:: bash DeZkYMRlEbWoMre7waFBoKFTxB0AFIBCsDxCzuttAkEeVHL1Xivbuc16OaPc7HmCVi2fOXUWEZp8ZDbFRXSCGjHG903lcAyhhdtuv70GdIU= Note: your output will differ due to the `AES initialization vector`_. See ``fabex/crypt.py`` for more info. We will, however, need the ``123`` encryption key, else Fabex will except when it reads the target yaml. As a check .. code-block:: bash fxcrypt 123 --decrypt 'DeZkYMRlEbWoMre7waFBoKFTxB0AFIBCsDxCzuttAkEeVHL1Xivbuc16OaPc7HmCVi2fOXUWEZp8ZDbFRXSCGjHG903lcAyhhdtuv70GdIU=' yields .. code-block:: bash el!qznto#i@v=v+b-(3^3*%nv=kzx!j3+%j6h*7or95anju#uc Fabfile Walkthrough =================== A Fabex ``fabfile.py`` should import from fabex.api, instead of fabric.api. Fabex wraps many of the Fabric functions and adds several of its own, e.g., the ``dryrun``, ``target``, and ``password`` tasks. Again, you'll find this complete example at https://bitbucket.org/rmorison/fabex/src under the fabex-example tree. Import and Initialize ^^^^^^^^^^^^^^^^^^^^^ .. code-block :: python """A Fabex example deploy of Django with an Nginx proxy""" import os from contextlib import contextmanager ## get fabric from fabex, with addons from fabex.api import * from fabex.contrib.files import * ## fabex specific config fabex_config(config={'target_dir': 'targets', 'template_dir': 'templates', 'template_config': 'templates.yaml'}) The ``fabex_config`` sets up the * target directory, where site specific deploy values are stored in yaml files * template directory, where Jinja2 templates for upload to the server are stored * template config, with per template upload directives. More on the templating subsystem as we proceed. Helper Functions ^^^^^^^^^^^^^^^^ Next, a ``virtualenv`` context manager, for running under a Python virtualenv, and a ``ssh`` task, to run a cmd on a host via ssh. These helpers are used later. .. code-block :: python ## utility functions and helper tasks @contextmanager def virtualenv(workon=None, cd=None): """Source .virtualenvrc for virtualenvwrapper; workon and cdvirtualenv, if given""" pre = ["source ~/.virtualenvrc"] if workon: pre.append("workon {}".format(workon)) pre.append("cdvirtualenv") if cd: pre.append("cd {}".format(cd)) with prefix(" && ".join(pre)): yield @task def ssh(cmd, use_sudo=''): """Run a command remotely over ssh. Use sudo if use_sudo=y""" return sudo(cmd) if use_sudo.startswith('y') else run(cmd) Install Tasks ^^^^^^^^^^^^^ Next up, we have our first block of tasks, for the install step. We'll update existing packages, install per role packages, and setup ssh keys. In the ``task_roles`` decorators, note the ``group='install'`` setting, along with the ``runs_once_per_host`` decorator. The ``group`` argument allows several tasks to be bundled into an "uber" task. The ``runs_once_per_host`` decorator is a Fabex add on, similar to Fabric's ``runs_once``. A task so decorated will run once, and only once, on each host it matches up with. .. code-block :: python ## install @task_roles(['proxy', 'app'], group='install') @runs_once_per_host def apt_upgrade(): """apt update package source listings and update installed packages""" sudo("DEBIAN_FRONTEND=noninteractive apt-get -y update") sudo("DEBIAN_FRONTEND=noninteractive apt-get -y upgrade") @task_roles(['proxy', 'app'], group='install') def apt_install(): """apt install role required packages""" sudo('DEBIAN_FRONTEND=noninteractive apt-get install --yes {}' .format(' '.join(env.packages))) @task_roles(['proxy', 'app'], group='install') @runs_once_per_host def setup_sshkey(): """ssh-keygen a key""" run("[ -e ~/.ssh/id_dsa ] && echo id_dsa exists" " || ssh-keygen -t dsa -N '' -f ~/.ssh/id_dsa") In fact, if you look at the ``fab -l`` for this fabfile you'll see .. code-block:: bash install Fabex group: apt_upgrade, apt_install, setup_sshkey in the listed tasks. Fabex added this task when it saw the ``group`` arg in ``task_roles``, and built a list of tasks to invoke. Setup Tasks ^^^^^^^^^^^ Here things get interesting, as setting up Nginx and a Python virtualenv are more involved. In the Nginx setup, we handle the tricky business of getting app server ssh public keys and adding then to the proxy's ``.ssh/allowed_hosts`` file. (Remember, there could be many app servers.) We use ``env.roledefs`` to find all the app servers, and the Fabex ``host_settings`` context manager run the ``ssh`` task on an app server instead of the proxy. With that, we get that app server's ssh key, and apply it to the proxy. ## setup .. code-block :: python @task_roles('proxy', group='setup') def setup_nginx(): """Build an (nginx) static server/load balancer/appserver proxy""" # make sure dirs exist and any default site file is gone sudo("mkdir -p /etc/nginx/conf.d /etc/nginx/sites-enabled /etc/nginx/sites-available") sudo("rm -f /etc/nginx/sites-enabled/default") # landing path for django static files sudo("mkdir -p {static_path} && chown {static_user} {static_path}".format(**env)) # get app host ssh keys, so app can sync static files to proxy for app_server in env.roledefs['app']: with host_settings(app_server): try: pubkey = execute(ssh, cmd="cat ~/.ssh/id_dsa.pub").values()[0] except (ValueError, IndexError): abort("{} could not get ssh key from {}".format(env.host, app_host)) append('~/.ssh/authorized_keys', pubkey) Now we setup the Django application server. The virtualenvwrapper package was installed by the previous install task, so we'll use that to build up a Python virtualenv and pip install to it. .. code-block :: python @task_roles('app', group='setup') def setup_virtualenv(): """Setup virtualenvwrapper and make a virtualenv for the project""" # virtualenv setup upload_project_template('virtualenvrc') append(".bashrc", "source $HOME/.virtualenvrc") if not exists(env.workon_home): run("mkdir -v -p {workon_home}".format(**env)) # mkvirtualenv with virtualenv(): run("lsvirtualenv | grep -q '^{project_name}$'" " && echo virtualenv {project_name} exists" " || mkvirtualenv {project_name}".format(**env)) @task_roles('app', group='setup') def pip_install(): """Install requirements""" # clone repo, setup django project, pip install with virtualenv(env.project_name): run('pip install {}'.format(' '.join(env.requirements))) Here, we meet Fabex's use of Jinja templates. Fabric contrib's section has a `template_upload`_. Fabex extends that with a ``upload_project_template`` facility. "Project templates" are listed in the ``templates.yaml`` we referenced in the ``fabex_config`` init. Here is ``templates.yaml`` from the example project: .. code-block:: yaml nginx-site.conf: local_path: nginx-site.conf remote_path: /etc/nginx/sites-available/{{project_name}} reload_command: > cd /etc/nginx/sites-enabled && ln -s -f -v ../sites-available/{{project_name}} . && service nginx restart owner: root:root # app virtualenvrc: local_path: virtualenvrc remote_path: .virtualenvrc supervisord-uwsgi: local_path: supervisord-uwsgi.conf remote_path: "/etc/supervisor/conf.d/uwsgi.conf" reload_command: supervisorctl update owner: root local_settings: local_path: local_settings.py remote_path: "{{settings_dir}}/local_settings.py" uwsgi.ini: local_path: uwsgi.ini remote_path: "{{settings_dir}}/uwsgi.ini" The top level key is what we reference in the ``upload_project_template`` call. The sub attributes tell Fabex where to find the source file, where to put the templated output file, any command to run after placing (optional), and an owner for a chmod after placing (optional). Note especially, we can inject ``env`` values into attribute values using Jinja2 syntax. This feature turns out to be `extremely` useful in practical applications. Of course, the templates themselves are Jinja2 processed, with the ``env`` context, as provided by Fabex. And finally, templated output is built and compared to that on the host. If there's no diff, there's no file uploaded, nor ``reload_command`` run. Kudos to the Mezzanine_ project for inspiring this approach. Deploy Tasks ^^^^^^^^^^^^ And finally, deploy. We use a number of "cross role tricks". One in ``config_proxy`` is to build the list of app server IP addresses for Nginx's ``upstream`` proxy directive. Another in ``deploy_app`` gets the proxy's IP address to rsync Django's collectstatic output into an Nginx accessable tree. The remainder I'll leave as an exercise for the reader ;-) .. code-block :: python ## deploy @task_roles('proxy', group='deploy') def config_proxy(): """Deploy config changes to load balancer""" wsgi_port = env.roleenvs['app']['wsgi_port'] appserver_ips = [env.hostenvs[host]['ip'] for host in env.roledefs['app']] wsgi_servers = ["server {}:{};".format(ip, wsgi_port) for ip in appserver_ips] upstream_servers = '\n '.join(wsgi_servers) with settings(upstream_servers=upstream_servers): upload_project_template('nginx-site.conf') @task_roles('app', group='deploy') def deploy_app(): """Deploy webapp""" with virtualenv(env.project_name): # pull and update code in real app run("[ -e {project_name} ] && echo app {project_name} exists" " || django-admin startproject {project_name}".format(**env)) # uploads, appends don't obey cd django_path = os.path.join(env.workon_home, *[env.project_name]*2) app_settings = {'django_path': django_path, 'settings_dir': os.path.join(django_path, env.project_name), 'settings_py': os.path.join(django_path, env.project_name, 'settings.py'), 'static_root': os.path.join(django_path, 'static')} with settings(**app_settings): upload_project_template('local_settings', reload=False) upload_project_template('uwsgi.ini', reload=False) append(app_settings['settings_py'], "from local_settings import *") upload_project_template('supervisord-uwsgi') start_or_restart = sudo("supervisorctl status uwsgi | grep -q '^uwsgi *RUNNING'" " && echo restart || echo start") sudo("supervisorctl {} uwsgi".format(start_or_restart)) @task_roles('app', group='deploy') @runs_once def once_after_app(): """Deploy actions that should run only once per deploy; should follow deploy_app""" with virtualenv(env.project_name, cd=env.project_name): # update db and statics run("./manage.py migrate --noinput") run("mkdir -p static && ./manage.py collectstatic --noinput --clear") # rsync statics to proxy proxy_ip = env.hostenvs[env.roledefs['proxy'][0]]['ip'] with role_settings('proxy', proxy_ip=proxy_ip): run("rsync -e 'ssh -o StrictHostKeyChecking=no' -avz" " static {proxy_ip}:{static_path}".format(**env)) Target for a "Real" Cluster =========================== Finally, let's look at what it would take to build a cluster with 3 application servers. The top of our target yaml becomes .. code-block:: yaml roledefs: proxy: [server] app: [app1 app2 app3 app4] hostenvs: server: {ip: 192.168.56.33, ssh_host: 192.168.56.33, ssh_user: ubuntu} app1: {ip: 192.168.56.34, ssh_host: 192.168.56.34, ssh_user: ubuntu} app2: {ip: 192.168.56.35, ssh_host: 192.168.56.35, ssh_user: ubuntu} app3: {ip: 192.168.56.36, ssh_host: 192.168.56.36, ssh_user: ubuntu} That's it, not a line of change in the fabfile! .. _AES initialization vector: http://eli.thegreenplace.net/2010/06/25/aes-encryption-of-files-in-python-with-pycrypto .. _template_upload: http://docs.fabfile.org/en/1.10/api/contrib/files.html .. _Mezzanine: http://mezzanine.jupo.org/