michael@0: michael@0: Introduction michael@0: ------------ michael@0: michael@0: ``virtualenv`` is a tool to create isolated Python environments. michael@0: michael@0: The basic problem being addressed is one of dependencies and versions, michael@0: and indirectly permissions. Imagine you have an application that michael@0: needs version 1 of LibFoo, but another application requires version michael@0: 2. How can you use both these applications? If you install michael@0: everything into ``/usr/lib/python2.7/site-packages`` (or whatever your michael@0: platform's standard location is), it's easy to end up in a situation michael@0: where you unintentionally upgrade an application that shouldn't be michael@0: upgraded. michael@0: michael@0: Or more generally, what if you want to install an application *and michael@0: leave it be*? If an application works, any change in its libraries or michael@0: the versions of those libraries can break the application. michael@0: michael@0: Also, what if you can't install packages into the global michael@0: ``site-packages`` directory? For instance, on a shared host. michael@0: michael@0: In all these cases, ``virtualenv`` can help you. It creates an michael@0: environment that has its own installation directories, that doesn't michael@0: share libraries with other virtualenv environments (and optionally michael@0: doesn't access the globally installed libraries either). michael@0: michael@0: michael@0: Installation michael@0: ------------ michael@0: michael@0: .. warning:: michael@0: michael@0: We advise installing virtualenv-1.9 or greater. Prior to version 1.9, the michael@0: pip included in virtualenv did not not download from PyPI over SSL. michael@0: michael@0: .. warning:: michael@0: michael@0: When using pip to install virtualenv, we advise using pip 1.3 or greater. michael@0: Prior to version 1.3, pip did not not download from PyPI over SSL. michael@0: michael@0: .. warning:: michael@0: michael@0: We advise against using easy_install to install virtualenv when using michael@0: setuptools < 0.9.7, because easy_install didn't download from PyPI over SSL michael@0: and was broken in some subtle ways. michael@0: michael@0: To install globally with `pip` (if you have pip 1.3 or greater installed globally): michael@0: michael@0: :: michael@0: michael@0: $ [sudo] pip install virtualenv michael@0: michael@0: Or to get the latest unreleased dev version: michael@0: michael@0: :: michael@0: michael@0: $ [sudo] pip install https://github.com/pypa/virtualenv/tarball/develop michael@0: michael@0: michael@0: To install globally from source: michael@0: michael@0: :: michael@0: michael@0: $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz michael@0: $ tar xvfz virtualenv-X.X.tar.gz michael@0: $ cd virtualenv-X.X michael@0: $ [sudo] python setup.py install michael@0: michael@0: michael@0: To *use* locally from source: michael@0: michael@0: :: michael@0: michael@0: $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz michael@0: $ tar xvfz virtualenv-X.X.tar.gz michael@0: $ cd virtualenv-X.X michael@0: $ python virtualenv.py myVE michael@0: michael@0: .. note:: michael@0: michael@0: The ``virtualenv.py`` script is *not* supported if run without the michael@0: necessary pip/setuptools/virtualenv distributions available locally. All michael@0: of the installation methods above include a ``virtualenv_support`` michael@0: directory alongside ``virtualenv.py`` which contains a complete set of michael@0: pip and setuptools distributions, and so are fully supported. michael@0: michael@0: Usage michael@0: ----- michael@0: michael@0: The basic usage is:: michael@0: michael@0: $ virtualenv ENV michael@0: michael@0: This creates ``ENV/lib/pythonX.X/site-packages``, where any libraries you michael@0: install will go. It also creates ``ENV/bin/python``, which is a Python michael@0: interpreter that uses this environment. Anytime you use that interpreter michael@0: (including when a script has ``#!/path/to/ENV/bin/python`` in it) the libraries michael@0: in that environment will be used. michael@0: michael@0: It also installs `Setuptools michael@0: `_ into the environment. michael@0: michael@0: .. note:: michael@0: michael@0: Virtualenv (<1.10) used to provide a ``--distribute`` option to use the michael@0: setuptools fork Distribute_. Since Distribute has been merged back into michael@0: setuptools this option is now no-op, it will always use the improved michael@0: setuptools releases. michael@0: michael@0: A new virtualenv also includes the `pip `_ michael@0: installer, so you can use ``ENV/bin/pip`` to install additional packages into michael@0: the environment. michael@0: michael@0: .. _Distribute: https://pypi.python.org/pypi/distribute michael@0: michael@0: activate script michael@0: ~~~~~~~~~~~~~~~ michael@0: michael@0: In a newly created virtualenv there will be a ``bin/activate`` shell michael@0: script. For Windows systems, activation scripts are provided for CMD.exe michael@0: and Powershell. michael@0: michael@0: On Posix systems you can do:: michael@0: michael@0: $ source bin/activate michael@0: michael@0: This will change your ``$PATH`` so its first entry is the virtualenv's michael@0: ``bin/`` directory. (You have to use ``source`` because it changes your michael@0: shell environment in-place.) This is all it does; it's purely a michael@0: convenience. If you directly run a script or the python interpreter michael@0: from the virtualenv's ``bin/`` directory (e.g. ``path/to/env/bin/pip`` michael@0: or ``/path/to/env/bin/python script.py``) there's no need for michael@0: activation. michael@0: michael@0: After activating an environment you can use the function ``deactivate`` to michael@0: undo the changes to your ``$PATH``. michael@0: michael@0: The ``activate`` script will also modify your shell prompt to indicate michael@0: which environment is currently active. You can disable this behavior, michael@0: which can be useful if you have your own custom prompt that already michael@0: displays the active environment name. To do so, set the michael@0: ``VIRTUAL_ENV_DISABLE_PROMPT`` environment variable to any non-empty michael@0: value before running the ``activate`` script. michael@0: michael@0: On Windows you just do:: michael@0: michael@0: > \path\to\env\Scripts\activate michael@0: michael@0: And type `deactivate` to undo the changes. michael@0: michael@0: Based on your active shell (CMD.exe or Powershell.exe), Windows will use michael@0: either activate.bat or activate.ps1 (as appropriate) to activate the michael@0: virtual environment. If using Powershell, see the notes about code signing michael@0: below. michael@0: michael@0: .. note:: michael@0: michael@0: If using Powershell, the ``activate`` script is subject to the michael@0: `execution policies`_ on the system. By default on Windows 7, the system's michael@0: excution policy is set to ``Restricted``, meaning no scripts like the michael@0: ``activate`` script are allowed to be executed. But that can't stop us michael@0: from changing that slightly to allow it to be executed. michael@0: michael@0: In order to use the script, you have to relax your system's execution michael@0: policy to ``AllSigned``, meaning all scripts on the system must be michael@0: digitally signed to be executed. Since the virtualenv activation michael@0: script is signed by one of the authors (Jannis Leidel) this level of michael@0: the execution policy suffices. As an administrator run:: michael@0: michael@0: PS C:\> Set-ExecutionPolicy AllSigned michael@0: michael@0: Then you'll be asked to trust the signer, when executing the script. michael@0: You will be prompted with the following:: michael@0: michael@0: PS C:\> virtualenv .\foo michael@0: New python executable in C:\foo\Scripts\python.exe michael@0: Installing setuptools................done. michael@0: Installing pip...................done. michael@0: PS C:\> .\foo\scripts\activate michael@0: michael@0: Do you want to run software from this untrusted publisher? michael@0: File C:\foo\scripts\activate.ps1 is published by E=jannis@leidel.info, michael@0: CN=Jannis Leidel, L=Berlin, S=Berlin, C=DE, Description=581796-Gh7xfJxkxQSIO4E0 michael@0: and is not trusted on your system. Only run scripts from trusted publishers. michael@0: [V] Never run [D] Do not run [R] Run once [A] Always run [?] Help michael@0: (default is "D"):A michael@0: (foo) PS C:\> michael@0: michael@0: If you select ``[A] Always Run``, the certificate will be added to the michael@0: Trusted Publishers of your user account, and will be trusted in this michael@0: user's context henceforth. If you select ``[R] Run Once``, the script will michael@0: be run, but you will be prometed on a subsequent invocation. Advanced users michael@0: can add the signer's certificate to the Trusted Publishers of the Computer michael@0: account to apply to all users (though this technique is out of scope of this michael@0: document). michael@0: michael@0: Alternatively, you may relax the system execution policy to allow running michael@0: of local scripts without verifying the code signature using the following:: michael@0: michael@0: PS C:\> Set-ExecutionPolicy RemoteSigned michael@0: michael@0: Since the ``activate.ps1`` script is generated locally for each virtualenv, michael@0: it is not considered a remote script and can then be executed. michael@0: michael@0: .. _`execution policies`: http://technet.microsoft.com/en-us/library/dd347641.aspx michael@0: michael@0: The ``--system-site-packages`` Option michael@0: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ michael@0: michael@0: If you build with ``virtualenv --system-site-packages ENV``, your virtual michael@0: environment will inherit packages from ``/usr/lib/python2.7/site-packages`` michael@0: (or wherever your global site-packages directory is). michael@0: michael@0: This can be used if you have control over the global site-packages directory, michael@0: and you want to depend on the packages there. If you want isolation from the michael@0: global system, do not use this flag. michael@0: michael@0: michael@0: Environment variables and configuration files michael@0: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ michael@0: michael@0: virtualenv can not only be configured by passing command line options such as michael@0: ``--python`` but also by two other means: michael@0: michael@0: - Environment variables michael@0: michael@0: Each command line option is automatically used to look for environment michael@0: variables with the name format ``VIRTUALENV_``. That means michael@0: the name of the command line options are capitalized and have dashes michael@0: (``'-'``) replaced with underscores (``'_'``). michael@0: michael@0: For example, to automatically use a custom Python binary instead of the michael@0: one virtualenv is run with you can also set an environment variable:: michael@0: michael@0: $ export VIRTUALENV_PYTHON=/opt/python-3.3/bin/python michael@0: $ virtualenv ENV michael@0: michael@0: It's the same as passing the option to virtualenv directly:: michael@0: michael@0: $ virtualenv --python=/opt/python-3.3/bin/python ENV michael@0: michael@0: This also works for appending command line options, like ``--find-links``. michael@0: Just leave an empty space between the passsed values, e.g.:: michael@0: michael@0: $ export VIRTUALENV_EXTRA_SEARCH_DIR="/path/to/dists /path/to/other/dists" michael@0: $ virtualenv ENV michael@0: michael@0: is the same as calling:: michael@0: michael@0: $ virtualenv --extra-search-dir=/path/to/dists --extra-search-dir=/path/to/other/dists ENV michael@0: michael@0: - Config files michael@0: michael@0: virtualenv also looks for a standard ini config file. On Unix and Mac OS X michael@0: that's ``$HOME/.virtualenv/virtualenv.ini`` and on Windows, it's michael@0: ``%APPDATA%\virtualenv\virtualenv.ini``. michael@0: michael@0: The names of the settings are derived from the long command line option, michael@0: e.g. the option ``--python`` would look like this:: michael@0: michael@0: [virtualenv] michael@0: python = /opt/python-3.3/bin/python michael@0: michael@0: Appending options like ``--extra-search-dir`` can be written on multiple michael@0: lines:: michael@0: michael@0: [virtualenv] michael@0: extra-search-dir = michael@0: /path/to/dists michael@0: /path/to/other/dists michael@0: michael@0: Please have a look at the output of ``virtualenv --help`` for a full list michael@0: of supported options. michael@0: michael@0: Windows Notes michael@0: ~~~~~~~~~~~~~ michael@0: michael@0: Some paths within the virtualenv are slightly different on Windows: scripts and michael@0: executables on Windows go in ``ENV\Scripts\`` instead of ``ENV/bin/`` and michael@0: libraries go in ``ENV\Lib\`` rather than ``ENV/lib/``. michael@0: michael@0: To create a virtualenv under a path with spaces in it on Windows, you'll need michael@0: the `win32api `_ library installed. michael@0: michael@0: PyPy Support michael@0: ~~~~~~~~~~~~ michael@0: michael@0: Beginning with virtualenv version 1.5 `PyPy `_ is michael@0: supported. To use PyPy 1.4 or 1.4.1, you need a version of virtualenv >= 1.5. michael@0: To use PyPy 1.5, you need a version of virtualenv >= 1.6.1. michael@0: michael@0: Creating Your Own Bootstrap Scripts michael@0: ----------------------------------- michael@0: michael@0: While this creates an environment, it doesn't put anything into the michael@0: environment. Developers may find it useful to distribute a script michael@0: that sets up a particular environment, for example a script that michael@0: installs a particular web application. michael@0: michael@0: To create a script like this, call michael@0: ``virtualenv.create_bootstrap_script(extra_text)``, and write the michael@0: result to your new bootstrapping script. Here's the documentation michael@0: from the docstring: michael@0: michael@0: Creates a bootstrap script, which is like this script but with michael@0: extend_parser, adjust_options, and after_install hooks. michael@0: michael@0: This returns a string that (written to disk of course) can be used michael@0: as a bootstrap script with your own customizations. The script michael@0: will be the standard virtualenv.py script, with your extra text michael@0: added (your extra text should be Python code). michael@0: michael@0: If you include these functions, they will be called: michael@0: michael@0: ``extend_parser(optparse_parser)``: michael@0: You can add or remove options from the parser here. michael@0: michael@0: ``adjust_options(options, args)``: michael@0: You can change options here, or change the args (if you accept michael@0: different kinds of arguments, be sure you modify ``args`` so it is michael@0: only ``[DEST_DIR]``). michael@0: michael@0: ``after_install(options, home_dir)``: michael@0: michael@0: After everything is installed, this function is called. This michael@0: is probably the function you are most likely to use. An michael@0: example would be:: michael@0: michael@0: def after_install(options, home_dir): michael@0: if sys.platform == 'win32': michael@0: bin = 'Scripts' michael@0: else: michael@0: bin = 'bin' michael@0: subprocess.call([join(home_dir, bin, 'easy_install'), michael@0: 'MyPackage']) michael@0: subprocess.call([join(home_dir, bin, 'my-package-script'), michael@0: 'setup', home_dir]) michael@0: michael@0: This example immediately installs a package, and runs a setup michael@0: script from that package. michael@0: michael@0: Bootstrap Example michael@0: ~~~~~~~~~~~~~~~~~ michael@0: michael@0: Here's a more concrete example of how you could use this:: michael@0: michael@0: import virtualenv, textwrap michael@0: output = virtualenv.create_bootstrap_script(textwrap.dedent(""" michael@0: import os, subprocess michael@0: def after_install(options, home_dir): michael@0: etc = join(home_dir, 'etc') michael@0: if not os.path.exists(etc): michael@0: os.makedirs(etc) michael@0: subprocess.call([join(home_dir, 'bin', 'easy_install'), michael@0: 'BlogApplication']) michael@0: subprocess.call([join(home_dir, 'bin', 'paster'), michael@0: 'make-config', 'BlogApplication', michael@0: join(etc, 'blog.ini')]) michael@0: subprocess.call([join(home_dir, 'bin', 'paster'), michael@0: 'setup-app', join(etc, 'blog.ini')]) michael@0: """)) michael@0: f = open('blog-bootstrap.py', 'w').write(output) michael@0: michael@0: Another example is available `here michael@0: `_. michael@0: michael@0: michael@0: Using Virtualenv without ``bin/python`` michael@0: --------------------------------------- michael@0: michael@0: Sometimes you can't or don't want to use the Python interpreter michael@0: created by the virtualenv. For instance, in a `mod_python michael@0: `_ or `mod_wsgi `_ michael@0: environment, there is only one interpreter. michael@0: michael@0: Luckily, it's easy. You must use the custom Python interpreter to michael@0: *install* libraries. But to *use* libraries, you just have to be sure michael@0: the path is correct. A script is available to correct the path. You michael@0: can setup the environment like:: michael@0: michael@0: activate_this = '/path/to/env/bin/activate_this.py' michael@0: execfile(activate_this, dict(__file__=activate_this)) michael@0: michael@0: This will change ``sys.path`` and even change ``sys.prefix``, but also allow michael@0: you to use an existing interpreter. Items in your environment will show up michael@0: first on ``sys.path``, before global items. However, global items will michael@0: always be accessible (as if the ``--system-site-packages`` flag had been used michael@0: in creating the environment, whether it was or not). Also, this cannot undo michael@0: the activation of other environments, or modules that have been imported. michael@0: You shouldn't try to, for instance, activate an environment before a web michael@0: request; you should activate *one* environment as early as possible, and not michael@0: do it again in that process. michael@0: michael@0: Making Environments Relocatable michael@0: ------------------------------- michael@0: michael@0: Note: this option is somewhat experimental, and there are probably michael@0: caveats that have not yet been identified. michael@0: michael@0: .. warning:: michael@0: michael@0: The ``--relocatable`` option currently has a number of issues, michael@0: and is not guaranteed to work in all circumstances. It is possible michael@0: that the option will be deprecated in a future version of ``virtualenv``. michael@0: michael@0: Normally environments are tied to a specific path. That means that michael@0: you cannot move an environment around or copy it to another computer. michael@0: You can fix up an environment to make it relocatable with the michael@0: command:: michael@0: michael@0: $ virtualenv --relocatable ENV michael@0: michael@0: This will make some of the files created by setuptools use relative paths, michael@0: and will change all the scripts to use ``activate_this.py`` instead of using michael@0: the location of the Python interpreter to select the environment. michael@0: michael@0: **Note:** scripts which have been made relocatable will only work if michael@0: the virtualenv is activated, specifically the python executable from michael@0: the virtualenv must be the first one on the system PATH. Also note that michael@0: the activate scripts are not currently made relocatable by michael@0: ``virtualenv --relocatable``. michael@0: michael@0: **Note:** you must run this after you've installed *any* packages into michael@0: the environment. If you make an environment relocatable, then michael@0: install a new package, you must run ``virtualenv --relocatable`` michael@0: again. michael@0: michael@0: Also, this **does not make your packages cross-platform**. You can michael@0: move the directory around, but it can only be used on other similar michael@0: computers. Some known environmental differences that can cause michael@0: incompatibilities: a different version of Python, when one platform michael@0: uses UCS2 for its internal unicode representation and another uses michael@0: UCS4 (a compile-time option), obvious platform changes like Windows michael@0: vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C michael@0: libraries on the system, if those C libraries are located somewhere michael@0: different (either different versions, or a different filesystem michael@0: layout). michael@0: michael@0: If you use this flag to create an environment, currently, the michael@0: ``--system-site-packages`` option will be implied. michael@0: michael@0: The ``--extra-search-dir`` option michael@0: --------------------------------- michael@0: michael@0: This option allows you to provide your own versions of setuptools and/or michael@0: pip to use instead of the embedded versions that come with virtualenv. michael@0: michael@0: To use this feature, pass one or more ``--extra-search-dir`` options to michael@0: virtualenv like this:: michael@0: michael@0: $ virtualenv --extra-search-dir=/path/to/distributions ENV michael@0: michael@0: The ``/path/to/distributions`` path should point to a directory that contains michael@0: setuptools and/or pip wheels. michael@0: michael@0: virtualenv will look for wheels in the specified directories, but will use michael@0: pip's standard algorithm for selecting the wheel to install, which looks for michael@0: the latest compatible wheel. michael@0: michael@0: As well as the extra directories, the search order includes: michael@0: michael@0: #. The ``virtualenv_support`` directory relative to virtualenv.py michael@0: #. The directory where virtualenv.py is located. michael@0: #. The current directory. michael@0: michael@0: If no satisfactory local distributions are found, virtualenv will michael@0: fail. Virtualenv will never download packages. michael@0: michael@0: michael@0: Compare & Contrast with Alternatives michael@0: ------------------------------------ michael@0: michael@0: There are several alternatives that create isolated environments: michael@0: michael@0: * ``workingenv`` (which I do not suggest you use anymore) is the michael@0: predecessor to this library. It used the main Python interpreter, michael@0: but relied on setting ``$PYTHONPATH`` to activate the environment. michael@0: This causes problems when running Python scripts that aren't part of michael@0: the environment (e.g., a globally installed ``hg`` or ``bzr``). It michael@0: also conflicted a lot with Setuptools. michael@0: michael@0: * `virtual-python michael@0: `_ michael@0: is also a predecessor to this library. It uses only symlinks, so it michael@0: couldn't work on Windows. It also symlinks over the *entire* michael@0: standard library and global ``site-packages``. As a result, it michael@0: won't see new additions to the global ``site-packages``. michael@0: michael@0: This script only symlinks a small portion of the standard library michael@0: into the environment, and so on Windows it is feasible to simply michael@0: copy these files over. Also, it creates a new/empty michael@0: ``site-packages`` and also adds the global ``site-packages`` to the michael@0: path, so updates are tracked separately. This script also installs michael@0: Setuptools automatically, saving a step and avoiding the need for michael@0: network access. michael@0: michael@0: * `zc.buildout `_ doesn't michael@0: create an isolated Python environment in the same style, but michael@0: achieves similar results through a declarative config file that sets michael@0: up scripts with very particular packages. As a declarative system, michael@0: it is somewhat easier to repeat and manage, but more difficult to michael@0: experiment with. ``zc.buildout`` includes the ability to setup michael@0: non-Python systems (e.g., a database server or an Apache instance). michael@0: michael@0: I *strongly* recommend anyone doing application development or michael@0: deployment use one of these tools. michael@0: michael@0: Contributing michael@0: ------------ michael@0: michael@0: Refer to the `pip development`_ documentation - it applies equally to michael@0: virtualenv, except that virtualenv issues should filed on the `virtualenv michael@0: repo`_ at GitHub. michael@0: michael@0: Virtualenv's release schedule is tied to pip's -- each time there's a new pip michael@0: release, there will be a new virtualenv release that bundles the new version of michael@0: pip. michael@0: michael@0: Files in the `virtualenv_embedded/` subdirectory are embedded into michael@0: `virtualenv.py` itself as base64-encoded strings (in order to support michael@0: single-file use of `virtualenv.py` without installing it). If your patch michael@0: changes any file in `virtualenv_embedded/`, run `bin/rebuild-script.py` to michael@0: update the embedded version of that file in `virtualenv.py`; commit that and michael@0: submit it as part of your patch / pull request. michael@0: michael@0: .. _pip development: http://www.pip-installer.org/en/latest/development.html michael@0: .. _virtualenv repo: https://github.com/pypa/virtualenv/ michael@0: michael@0: Running the tests michael@0: ~~~~~~~~~~~~~~~~~ michael@0: michael@0: Virtualenv's test suite is small and not yet at all comprehensive, but we aim michael@0: to grow it. michael@0: michael@0: The easy way to run tests (handles test dependencies automatically):: michael@0: michael@0: $ python setup.py test michael@0: michael@0: If you want to run only a selection of the tests, you'll need to run them michael@0: directly with nose instead. Create a virtualenv, and install required michael@0: packages:: michael@0: michael@0: $ pip install nose mock michael@0: michael@0: Run nosetests:: michael@0: michael@0: $ nosetests michael@0: michael@0: Or select just a single test file to run:: michael@0: michael@0: $ nosetests tests.test_virtualenv michael@0: michael@0: michael@0: Other Documentation and Links michael@0: ----------------------------- michael@0: michael@0: * James Gardner has written a tutorial on using `virtualenv with michael@0: Pylons michael@0: `_. michael@0: michael@0: * `Blog announcement michael@0: `_. michael@0: michael@0: * Doug Hellmann wrote a description of his `command-line work flow michael@0: using virtualenv (virtualenvwrapper) michael@0: `_ michael@0: including some handy scripts to make working with multiple michael@0: environments easier. He also wrote `an example of using virtualenv michael@0: to try IPython michael@0: `_. michael@0: michael@0: * Chris Perkins created a `showmedo video including virtualenv michael@0: `_. michael@0: michael@0: * `Using virtualenv with mod_wsgi michael@0: `_. michael@0: michael@0: * `virtualenv commands michael@0: `_ for some more michael@0: workflow-related tools around virtualenv. michael@0: michael@0: Status and License michael@0: ------------------ michael@0: michael@0: ``virtualenv`` is a successor to `workingenv michael@0: `_, and an extension michael@0: of `virtual-python michael@0: `_. michael@0: michael@0: It was written by Ian Bicking, sponsored by the `Open Planning michael@0: Project `_ and is now maintained by a michael@0: `group of developers `_. michael@0: It is licensed under an michael@0: `MIT-style permissive license `_.