A Primer on virtualenv

May 7, 2009 – 4:11 pm

What is virtualenv and Why Should You Use It?

Put simply (and copied from the project page), virtualenv is a tool to create isolated Python environments. Why is this good? You can create a new Python environment to run a Python/Django/whatever app and install all package dependencies into the virtualenv without affecting your system's site-packages. Need to upgrade a package to see how it affects your app? Create a new virtualenv, install/copy your app into it, run your tests, and delete it when you are done. Just like git makes branching inexpensive and easy, virtualenv makes creating and managing new Python environments inexpensive and easy.

Show Me Some Code!

First, you need to install virtualenv. The typical way to install is from PyPi using easy_install. You can also use pip or install from source. We'll use easy_install since it is…easy:

$ sudo easy_install virtualenv
Password:

Searching for virtualenv
Best match: virtualenv 1.3.3
Downloading http://pypi.python.org/packages/2.5/v/virtualenv/virtualenv-1.3.3-py2.5.egg#md5=fae350c941cd9eadf5e9a407c37a2e03Processing virtualenv-1.3.3-py2.5.egg
Adding virtualenv 1.3.3 to easy-install.pth file
Installing virtualenv script to /usr/local/bin

Using /Library/Python/2.5/site-packages/virtualenv-1.3.3-py2.5.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

Now, you're ready to create your virtualenv. BUT FIRST…you have a decision to make. Do you want this virtualenv to use packages from your system site-packages on install them in the virtualenv's site-packages? By default, virtualenv will symlink to your system's site-packages if you install a package in the virtualenv that is already installed on your system. If you want a totally (well, as much as is possible) isolated virtualenv, you'll want to do the latter. To do this, you pass in the --no-site-packages switch when creating your virtualenv.

So, let's create a virtualenv named for the project we are working on (mycoolproject) and tell it not to use the system site-packages but keep it's own copy:

$ pwd
/Users/chris/Documents/clients

$ virtualenv --no-site-packages mycoolproject
New python executable in mycoolproject/bin/python
Installing setuptools…………done.

$ ls mycoolproject
bin     include lib

I told virtualenv to not use the system site-packages and to create the virtualenv in a directory named mycoolproject. This directory didn't exist so it created it, however, if it did exist virtualenv would install it's files within it. virtualenv created three directories to use for running Python and installing packages into the virtualenv.

So, we have a new virtualenv, now what? Activate your virtualenv. To do this, change to the virtualenv directory you just created and source the activation script from the bin directory in your virtualenv:

$ cd mycoolproject
chris@chris.local [~/Documents/clients/mycoolproject]
$ source bin/activate
(mycoolproject)chris@chris.local [~/Documents/clients/mycoolproject]
$

I've included my full command prompt here so you can see that when you activate a virtualenv, it modifies your current prompt to prepend the name of the virtualenv in parentheses so that you know at a glance you have it active.

So, what happened here? Other than the prompt change, the main changes the activate script makes are:

VIRTUAL_ENV="/Users/chris/Documents/clients/mycoolproject"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

This sets up an environment variable with the virtualenv path and also modifies your path so the bin directory in the virtualenv is first. This is important since running python from the command line when your virtualenv is active runs it from there instead of your system path. This means through the magic of the Python site module the site-packages in your virtualenv will be in sys.path and not your system's site-packages.

Installing Packages in a virtualenv

So, this is where the fun part starts. If you look at the bin directory in your virtualenv, you'll see easy_install which has been modified to put eggs and packages in the virtualenv's site-packages directory. So, once you activate your virtualenv installing a package in it is as easy as always:

$ easy_install ipython

Notice that I didn't have to sudo this time since the files will all be installed in the virtualenv's lib/python2.5/site-packages directory which was created as my user account so I have the appropriate permissions.

You can also use pip to install packages in a virtualenv since it is virtualenv-aware. This is very handy since you can easily create a pip requirements file and use that to easily install all the various packages the software you are running in your virtualenv will need. Check out the Pinax project's installation docs for an example of this.

Tips for Working With virtualenv

  • Use the virtualenvwrapper script to make working with and changing to/from multiple virtualenvs easy.
  • For Django, after you install Django in your virtualenv, copy the django-admin.py from the Django package to the bin directory of your virtualenv so that you can still type django-admin.py startproject without specifying the full path to django-admin.py.
  • If you use the --no-site-packages option you are starting with a bare Python install and will need to install all the things you have forgotten about since the first time you set your system up like PIL, ipython, ipdb, readline, etc. This is where a pip requirements file is a very good thing.
  • If you are on OS X installing PIL from pip or easy_install may not work. To get around this just symlink the PIL directory from /Library/Python/2.5/site-packages to lib/python2.5/site-packages in your virtualenv.

Questions, Corrections, Tips?

I'm not a virtualenv expert, I just play one on the internet. If you see anything wrong with what I've said or have any suggestions or tips, please let me know in the comments or email chris@ this domain.

By Chris Scott | Posted in Django, Python, Tech | Tagged | Comments (11)

Word Movement in iTerm on OS X

April 15, 2009 – 8:54 am

Prompted by a question from Tim Rosenblatt and answer from Matt Williams about how to move forward and backward by words in OS X Terminal.app, I wanted to see how to do this in iTerm. Referencing the article Matt sent made it easy.

In iTerm, choose Manage Profiles from the Bookmarks menu. Expand the Keyboard Profiles tree, choose the Global profile and add a new mapping. I want to use Option+Left Arrow for back and Option+Right Arrow for forward. Here's what my settings look like for back:

iTerm Keyboard Mapping

Since these are Global, I had to choose the High interception priority option to get them to work. Do the same thing for forward, but use f for the escape sequence to send.

By Chris Scott | Posted in Other Stuff | Comments (0)

Local to External Django Docs Bookmarklet

April 7, 2009 – 8:53 am

As I'm learning Django, I've found it helpful to have a local copy of the Django docs to refer to. This is quicker than going to the Django docs site and also helps when I don't have a connection. The only drawback to this is that when I find something I want to share with someone I have to go find it again in the online docs and send them a link.

I finally got tired of this and created a simple bookmarklet that will take me from my local copy to the same place in the online docs. Here's it is: Django Docs Link. Just drag that to your browser's toolbar. When you are browsing your local Django docs, click on it to be taken to the same page online.

If you find any problems with this or have any suggestions, please let me know in the comments or email me at chris@ this domain.

By Chris Scott | Posted in Django | Comments (0)

Rex

December 23, 2008 – 1:21 pm

Rex
We'll miss you, buddy.

By Chris Scott | Posted in Other Stuff | Comments Off

Customizing Django's Admin Templates and Media When Using the Built-In Development Web Server

December 11, 2008 – 3:12 pm

At work today, Sean was playing around with creating a Django app and wanted to use the django-grappelli project to get a cool look for Django’s admin. I warned him against using the instructions included in the project since if you are using Django from an svn checkout then it will break any future svn updates with an error message something like this:

working copy does not exist in repository

So, we started working on a way around this for a local dev environment. We knew we could override the admin templates easily but hadn’t really seen anything about how to override the admin media. Between the two of us, we got it working and figured putting it into writing for future reference would be a good thing.

Step 1

First, check out the django-grappelli project. I’m putting mine in the src directory in my home directory:

$ cd ~/src
$ pwd
/Users/chris/src
$ svn checkout http://django-grappelli.googlecode.com/svn/trunk/ django-grappelli

Step 2

In your project’s settings.py add a template directory to the templates directory in the django-grappelli directory you created above:

TEMPLATE_DIRS = (
'/Users/chris/src/django-grappelli/templates',
)

Also in settings.py change the ADMIN_MEDIA_PREFIX to use the full URL to the server that manage.py’s runserver command will create and append that with a path you want to use. This doesn’t need to relate to any file system path since you’re just going to use it in a urlconf with a static view.

ADMIN_MEDIA_PREFIX = 'http://127.0.0.1:8000/media/admin/'

Step 3

In your project’s urls.py comment out the lines to enable the admin and add a urlpattern to serve the admin media using the web path you used above from the file system path of the media directory in the django-grappelli project you created in the first step.

In my case, I’m using http://127.0.0.1:8000/media/admin/ as my ADMIN_MEDIA_PREFIX which is a web path of /media/admin/. Since Django strips off the initial slash, your regex to match it would be r'^media/admin/(?P<path>.*)$'. You’ll send these requests to the django static serve view and pass in the document_root which is the file system path to django-grappelli’s media directory. Your urls.py should now look something like this:

from django.conf.urls.defaults import *

from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns(”,
(r'^admin/(.*)', admin.site.root),
(r'^media/admin/(?P<path>.*)$', 'django.views.static.serve', {'document_root': '/Users/chris/src/django-grappelli/media'}),
)

Finally

Now, just do ./manage.py runserver in your project directory and browse to http://127.0.0.1:8000/admin/ and you should see Grappelli’s sweet styles looking back at you.

Notes

The key to making this work is using a full URL instead of an absolute or relative URL for your ADMIN_MEDIA_PREFIX. This allows Django to use your normal urlconf to serve up the media with the static serve view.

This is designed to work in a local dev environment using Django’s built-in server. Don’t use this in production—serve your media using nginx, Apache, etc.

There is probably a better way to do this and suggestions are welcome.

By Chris Scott | Posted in Django | Comments (9)