python/virtualenv/docs/virtualenv.rst

Thu, 22 Jan 2015 13:21:57 +0100

author
Michael Schloh von Bennewitz <michael@schloh.com>
date
Thu, 22 Jan 2015 13:21:57 +0100
branch
TOR_BUG_9701
changeset 15
b8a032363ba2
permissions
-rw-r--r--

Incorporate requested changes from Mozilla in review:
https://bugzilla.mozilla.org/show_bug.cgi?id=1123480#c6

     2 Introduction
     3 ------------
     5 ``virtualenv`` is a tool to create isolated Python environments.
     7 The basic problem being addressed is one of dependencies and versions,
     8 and indirectly permissions. Imagine you have an application that
     9 needs version 1 of LibFoo, but another application requires version
    10 2. How can you use both these applications?  If you install
    11 everything into ``/usr/lib/python2.7/site-packages`` (or whatever your
    12 platform's standard location is), it's easy to end up in a situation
    13 where you unintentionally upgrade an application that shouldn't be
    14 upgraded.
    16 Or more generally, what if you want to install an application *and
    17 leave it be*?  If an application works, any change in its libraries or
    18 the versions of those libraries can break the application.
    20 Also, what if you can't install packages into the global
    21 ``site-packages`` directory?  For instance, on a shared host.
    23 In all these cases, ``virtualenv`` can help you. It creates an
    24 environment that has its own installation directories, that doesn't
    25 share libraries with other virtualenv environments (and optionally
    26 doesn't access the globally installed libraries either).
    29 Installation
    30 ------------
    32 .. warning::
    34     We advise installing virtualenv-1.9 or greater. Prior to version 1.9, the
    35     pip included in virtualenv did not not download from PyPI over SSL.
    37 .. warning::
    39     When using pip to install virtualenv, we advise using pip 1.3 or greater.
    40     Prior to version 1.3, pip did not not download from PyPI over SSL.
    42 .. warning::
    44     We advise against using easy_install to install virtualenv when using
    45     setuptools < 0.9.7, because easy_install didn't download from PyPI over SSL
    46     and was broken in some subtle ways.
    48 To install globally with `pip` (if you have pip 1.3 or greater installed globally):
    50 ::
    52  $ [sudo] pip install virtualenv
    54 Or to get the latest unreleased dev version:
    56 ::
    58  $ [sudo] pip install https://github.com/pypa/virtualenv/tarball/develop
    61 To install globally from source:
    63 ::
    65  $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
    66  $ tar xvfz virtualenv-X.X.tar.gz
    67  $ cd virtualenv-X.X
    68  $ [sudo] python setup.py install
    71 To *use* locally from source:
    73 ::
    75  $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
    76  $ tar xvfz virtualenv-X.X.tar.gz
    77  $ cd virtualenv-X.X
    78  $ python virtualenv.py myVE
    80 .. note::
    82     The ``virtualenv.py`` script is *not* supported if run without the
    83     necessary pip/setuptools/virtualenv distributions available locally. All
    84     of the installation methods above include a ``virtualenv_support``
    85     directory alongside ``virtualenv.py`` which contains a complete set of
    86     pip and setuptools distributions, and so are fully supported.
    88 Usage
    89 -----
    91 The basic usage is::
    93     $ virtualenv ENV
    95 This creates ``ENV/lib/pythonX.X/site-packages``, where any libraries you
    96 install will go. It also creates ``ENV/bin/python``, which is a Python
    97 interpreter that uses this environment. Anytime you use that interpreter
    98 (including when a script has ``#!/path/to/ENV/bin/python`` in it) the libraries
    99 in that environment will be used.
   101 It also installs `Setuptools
   102 <http://peak.telecommunity.com/DevCenter/setuptools>`_ into the environment.
   104 .. note::
   106   Virtualenv (<1.10) used to provide a ``--distribute`` option to use the
   107   setuptools fork Distribute_. Since Distribute has been merged back into
   108   setuptools this option is now no-op, it will always use the improved
   109   setuptools releases.
   111 A new virtualenv also includes the `pip <http://pypi.python.org/pypi/pip>`_
   112 installer, so you can use ``ENV/bin/pip`` to install additional packages into
   113 the environment.
   115 .. _Distribute: https://pypi.python.org/pypi/distribute
   117 activate script
   118 ~~~~~~~~~~~~~~~
   120 In a newly created virtualenv there will be a ``bin/activate`` shell
   121 script. For Windows systems, activation scripts are provided for CMD.exe
   122 and Powershell.
   124 On Posix systems you can do::
   126     $ source bin/activate
   128 This will change your ``$PATH`` so its first entry is the virtualenv's
   129 ``bin/`` directory. (You have to use ``source`` because it changes your
   130 shell environment in-place.) This is all it does; it's purely a
   131 convenience. If you directly run a script or the python interpreter
   132 from the virtualenv's ``bin/`` directory (e.g. ``path/to/env/bin/pip``
   133 or ``/path/to/env/bin/python script.py``) there's no need for
   134 activation.
   136 After activating an environment you can use the function ``deactivate`` to
   137 undo the changes to your ``$PATH``.
   139 The ``activate`` script will also modify your shell prompt to indicate
   140 which environment is currently active. You can disable this behavior,
   141 which can be useful if you have your own custom prompt that already
   142 displays the active environment name. To do so, set the
   143 ``VIRTUAL_ENV_DISABLE_PROMPT`` environment variable to any non-empty
   144 value before running the ``activate`` script.
   146 On Windows you just do::
   148     > \path\to\env\Scripts\activate
   150 And type `deactivate` to undo the changes.
   152 Based on your active shell (CMD.exe or Powershell.exe), Windows will use
   153 either activate.bat or activate.ps1 (as appropriate) to activate the
   154 virtual environment. If using Powershell, see the notes about code signing
   155 below.
   157 .. note::
   159     If using Powershell, the ``activate`` script is subject to the
   160     `execution policies`_ on the system. By default on Windows 7, the system's
   161     excution policy is set to ``Restricted``, meaning no scripts like the
   162     ``activate`` script are allowed to be executed. But that can't stop us
   163     from changing that slightly to allow it to be executed.
   165     In order to use the script, you have to relax your system's execution
   166     policy to ``AllSigned``, meaning all scripts on the system must be
   167     digitally signed to be executed. Since the virtualenv activation
   168     script is signed by one of the authors (Jannis Leidel) this level of
   169     the execution policy suffices. As an administrator run::
   171         PS C:\> Set-ExecutionPolicy AllSigned
   173     Then you'll be asked to trust the signer, when executing the script.
   174     You will be prompted with the following::
   176         PS C:\> virtualenv .\foo
   177         New python executable in C:\foo\Scripts\python.exe
   178         Installing setuptools................done.
   179         Installing pip...................done.
   180         PS C:\> .\foo\scripts\activate
   182         Do you want to run software from this untrusted publisher?
   183         File C:\foo\scripts\activate.ps1 is published by E=jannis@leidel.info,
   184         CN=Jannis Leidel, L=Berlin, S=Berlin, C=DE, Description=581796-Gh7xfJxkxQSIO4E0
   185         and is not trusted on your system. Only run scripts from trusted publishers.
   186         [V] Never run  [D] Do not run  [R] Run once  [A] Always run  [?] Help
   187         (default is "D"):A
   188         (foo) PS C:\>
   190     If you select ``[A] Always Run``, the certificate will be added to the
   191     Trusted Publishers of your user account, and will be trusted in this
   192     user's context henceforth. If you select ``[R] Run Once``, the script will
   193     be run, but you will be prometed on a subsequent invocation. Advanced users
   194     can add the signer's certificate to the Trusted Publishers of the Computer
   195     account to apply to all users (though this technique is out of scope of this
   196     document).
   198     Alternatively, you may relax the system execution policy to allow running
   199     of local scripts without verifying the code signature using the following::
   201         PS C:\> Set-ExecutionPolicy RemoteSigned
   203     Since the ``activate.ps1`` script is generated locally for each virtualenv,
   204     it is not considered a remote script and can then be executed.
   206 .. _`execution policies`: http://technet.microsoft.com/en-us/library/dd347641.aspx
   208 The ``--system-site-packages`` Option
   209 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   211 If you build with ``virtualenv --system-site-packages ENV``, your virtual
   212 environment will inherit packages from ``/usr/lib/python2.7/site-packages``
   213 (or wherever your global site-packages directory is).
   215 This can be used if you have control over the global site-packages directory,
   216 and you want to depend on the packages there. If you want isolation from the
   217 global system, do not use this flag.
   220 Environment variables and configuration files
   221 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   223 virtualenv can not only be configured by passing command line options such as
   224 ``--python`` but also by two other means:
   226 - Environment variables
   228   Each command line option is automatically used to look for environment
   229   variables with the name format ``VIRTUALENV_<UPPER_NAME>``. That means
   230   the name of the command line options are capitalized and have dashes
   231   (``'-'``) replaced with underscores (``'_'``).
   233   For example, to automatically use a custom Python binary instead of the
   234   one virtualenv is run with you can also set an environment variable::
   236       $ export VIRTUALENV_PYTHON=/opt/python-3.3/bin/python
   237       $ virtualenv ENV
   239   It's the same as passing the option to virtualenv directly::
   241       $ virtualenv --python=/opt/python-3.3/bin/python ENV
   243   This also works for appending command line options, like ``--find-links``.
   244   Just leave an empty space between the passsed values, e.g.::
   246       $ export VIRTUALENV_EXTRA_SEARCH_DIR="/path/to/dists /path/to/other/dists"
   247       $ virtualenv ENV
   249   is the same as calling::
   251       $ virtualenv --extra-search-dir=/path/to/dists --extra-search-dir=/path/to/other/dists ENV
   253 - Config files
   255   virtualenv also looks for a standard ini config file. On Unix and Mac OS X
   256   that's ``$HOME/.virtualenv/virtualenv.ini`` and on Windows, it's
   257   ``%APPDATA%\virtualenv\virtualenv.ini``.
   259   The names of the settings are derived from the long command line option,
   260   e.g. the option ``--python`` would look like this::
   262       [virtualenv]
   263       python = /opt/python-3.3/bin/python
   265   Appending options like ``--extra-search-dir`` can be written on multiple
   266   lines::
   268       [virtualenv]
   269       extra-search-dir =
   270           /path/to/dists
   271           /path/to/other/dists
   273 Please have a look at the output of ``virtualenv --help`` for a full list
   274 of supported options.
   276 Windows Notes
   277 ~~~~~~~~~~~~~
   279 Some paths within the virtualenv are slightly different on Windows: scripts and
   280 executables on Windows go in ``ENV\Scripts\`` instead of ``ENV/bin/`` and
   281 libraries go in ``ENV\Lib\`` rather than ``ENV/lib/``.
   283 To create a virtualenv under a path with spaces in it on Windows, you'll need
   284 the `win32api <http://sourceforge.net/projects/pywin32/>`_ library installed.
   286 PyPy Support
   287 ~~~~~~~~~~~~
   289 Beginning with virtualenv version 1.5 `PyPy <http://pypy.org>`_ is
   290 supported. To use PyPy 1.4 or 1.4.1, you need a version of virtualenv >= 1.5.
   291 To use PyPy 1.5, you need a version of virtualenv >= 1.6.1.
   293 Creating Your Own Bootstrap Scripts
   294 -----------------------------------
   296 While this creates an environment, it doesn't put anything into the
   297 environment. Developers may find it useful to distribute a script
   298 that sets up a particular environment, for example a script that
   299 installs a particular web application.
   301 To create a script like this, call
   302 ``virtualenv.create_bootstrap_script(extra_text)``, and write the
   303 result to your new bootstrapping script. Here's the documentation
   304 from the docstring:
   306 Creates a bootstrap script, which is like this script but with
   307 extend_parser, adjust_options, and after_install hooks.
   309 This returns a string that (written to disk of course) can be used
   310 as a bootstrap script with your own customizations. The script
   311 will be the standard virtualenv.py script, with your extra text
   312 added (your extra text should be Python code).
   314 If you include these functions, they will be called:
   316 ``extend_parser(optparse_parser)``:
   317     You can add or remove options from the parser here.
   319 ``adjust_options(options, args)``:
   320     You can change options here, or change the args (if you accept
   321     different kinds of arguments, be sure you modify ``args`` so it is
   322     only ``[DEST_DIR]``).
   324 ``after_install(options, home_dir)``:
   326     After everything is installed, this function is called. This
   327     is probably the function you are most likely to use. An
   328     example would be::
   330         def after_install(options, home_dir):
   331             if sys.platform == 'win32':
   332                 bin = 'Scripts'
   333             else:
   334                 bin = 'bin'
   335             subprocess.call([join(home_dir, bin, 'easy_install'),
   336                              'MyPackage'])
   337             subprocess.call([join(home_dir, bin, 'my-package-script'),
   338                              'setup', home_dir])
   340     This example immediately installs a package, and runs a setup
   341     script from that package.
   343 Bootstrap Example
   344 ~~~~~~~~~~~~~~~~~
   346 Here's a more concrete example of how you could use this::
   348     import virtualenv, textwrap
   349     output = virtualenv.create_bootstrap_script(textwrap.dedent("""
   350     import os, subprocess
   351     def after_install(options, home_dir):
   352         etc = join(home_dir, 'etc')
   353         if not os.path.exists(etc):
   354             os.makedirs(etc)
   355         subprocess.call([join(home_dir, 'bin', 'easy_install'),
   356                          'BlogApplication'])
   357         subprocess.call([join(home_dir, 'bin', 'paster'),
   358                          'make-config', 'BlogApplication',
   359                          join(etc, 'blog.ini')])
   360         subprocess.call([join(home_dir, 'bin', 'paster'),
   361                          'setup-app', join(etc, 'blog.ini')])
   362     """))
   363     f = open('blog-bootstrap.py', 'w').write(output)
   365 Another example is available `here
   366 <https://github.com/socialplanning/fassembler/blob/master/fassembler/create-venv-script.py>`_.
   369 Using Virtualenv without ``bin/python``
   370 ---------------------------------------
   372 Sometimes you can't or don't want to use the Python interpreter
   373 created by the virtualenv. For instance, in a `mod_python
   374 <http://www.modpython.org/>`_ or `mod_wsgi <http://www.modwsgi.org/>`_
   375 environment, there is only one interpreter.
   377 Luckily, it's easy. You must use the custom Python interpreter to
   378 *install* libraries. But to *use* libraries, you just have to be sure
   379 the path is correct. A script is available to correct the path. You
   380 can setup the environment like::
   382     activate_this = '/path/to/env/bin/activate_this.py'
   383     execfile(activate_this, dict(__file__=activate_this))
   385 This will change ``sys.path`` and even change ``sys.prefix``, but also allow
   386 you to use an existing interpreter. Items in your environment will show up
   387 first on ``sys.path``, before global items. However, global items will
   388 always be accessible (as if the ``--system-site-packages`` flag had been used
   389 in creating the environment, whether it was or not). Also, this cannot undo
   390 the activation of other environments, or modules that have been imported.
   391 You shouldn't try to, for instance, activate an environment before a web
   392 request; you should activate *one* environment as early as possible, and not
   393 do it again in that process.
   395 Making Environments Relocatable
   396 -------------------------------
   398 Note: this option is somewhat experimental, and there are probably
   399 caveats that have not yet been identified.
   401 .. warning::
   403     The ``--relocatable`` option currently has a number of issues,
   404     and is not guaranteed to work in all circumstances. It is possible
   405     that the option will be deprecated in a future version of ``virtualenv``.
   407 Normally environments are tied to a specific path. That means that
   408 you cannot move an environment around or copy it to another computer.
   409 You can fix up an environment to make it relocatable with the
   410 command::
   412     $ virtualenv --relocatable ENV
   414 This will make some of the files created by setuptools use relative paths,
   415 and will change all the scripts to use ``activate_this.py`` instead of using
   416 the location of the Python interpreter to select the environment.
   418 **Note:** scripts which have been made relocatable will only work if
   419 the virtualenv is activated, specifically the python executable from
   420 the virtualenv must be the first one on the system PATH. Also note that
   421 the activate scripts are not currently made relocatable by
   422 ``virtualenv --relocatable``.
   424 **Note:** you must run this after you've installed *any* packages into
   425 the environment. If you make an environment relocatable, then
   426 install a new package, you must run ``virtualenv --relocatable``
   427 again.
   429 Also, this **does not make your packages cross-platform**. You can
   430 move the directory around, but it can only be used on other similar
   431 computers. Some known environmental differences that can cause
   432 incompatibilities: a different version of Python, when one platform
   433 uses UCS2 for its internal unicode representation and another uses
   434 UCS4 (a compile-time option), obvious platform changes like Windows
   435 vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C
   436 libraries on the system, if those C libraries are located somewhere
   437 different (either different versions, or a different filesystem
   438 layout).
   440 If you use this flag to create an environment, currently, the
   441 ``--system-site-packages`` option will be implied.
   443 The ``--extra-search-dir`` option
   444 ---------------------------------
   446 This option allows you to provide your own versions of setuptools and/or
   447 pip to use instead of the embedded versions that come with virtualenv.
   449 To use this feature, pass one or more ``--extra-search-dir`` options to
   450 virtualenv like this::
   452     $ virtualenv --extra-search-dir=/path/to/distributions ENV
   454 The ``/path/to/distributions`` path should point to a directory that contains
   455 setuptools and/or pip wheels.
   457 virtualenv will look for wheels in the specified directories, but will use
   458 pip's standard algorithm for selecting the wheel to install, which looks for
   459 the latest compatible wheel.
   461 As well as the extra directories, the search order includes:
   463 #. The ``virtualenv_support`` directory relative to virtualenv.py
   464 #. The directory where virtualenv.py is located.
   465 #. The current directory.
   467 If no satisfactory local distributions are found, virtualenv will
   468 fail. Virtualenv will never download packages.
   471 Compare & Contrast with Alternatives
   472 ------------------------------------
   474 There are several alternatives that create isolated environments:
   476 * ``workingenv`` (which I do not suggest you use anymore) is the
   477   predecessor to this library. It used the main Python interpreter,
   478   but relied on setting ``$PYTHONPATH`` to activate the environment.
   479   This causes problems when running Python scripts that aren't part of
   480   the environment (e.g., a globally installed ``hg`` or ``bzr``). It
   481   also conflicted a lot with Setuptools.
   483 * `virtual-python
   484   <http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_
   485   is also a predecessor to this library. It uses only symlinks, so it
   486   couldn't work on Windows. It also symlinks over the *entire*
   487   standard library and global ``site-packages``. As a result, it
   488   won't see new additions to the global ``site-packages``.
   490   This script only symlinks a small portion of the standard library
   491   into the environment, and so on Windows it is feasible to simply
   492   copy these files over. Also, it creates a new/empty
   493   ``site-packages`` and also adds the global ``site-packages`` to the
   494   path, so updates are tracked separately. This script also installs
   495   Setuptools automatically, saving a step and avoiding the need for
   496   network access.
   498 * `zc.buildout <http://pypi.python.org/pypi/zc.buildout>`_ doesn't
   499   create an isolated Python environment in the same style, but
   500   achieves similar results through a declarative config file that sets
   501   up scripts with very particular packages. As a declarative system,
   502   it is somewhat easier to repeat and manage, but more difficult to
   503   experiment with. ``zc.buildout`` includes the ability to setup
   504   non-Python systems (e.g., a database server or an Apache instance).
   506 I *strongly* recommend anyone doing application development or
   507 deployment use one of these tools.
   509 Contributing
   510 ------------
   512 Refer to the `pip development`_ documentation - it applies equally to
   513 virtualenv, except that virtualenv issues should filed on the `virtualenv
   514 repo`_ at GitHub.
   516 Virtualenv's release schedule is tied to pip's -- each time there's a new pip
   517 release, there will be a new virtualenv release that bundles the new version of
   518 pip.
   520 Files in the `virtualenv_embedded/` subdirectory are embedded into
   521 `virtualenv.py` itself as base64-encoded strings (in order to support
   522 single-file use of `virtualenv.py` without installing it). If your patch
   523 changes any file in `virtualenv_embedded/`, run `bin/rebuild-script.py` to
   524 update the embedded version of that file in `virtualenv.py`; commit that and
   525 submit it as part of your patch / pull request.
   527 .. _pip development: http://www.pip-installer.org/en/latest/development.html
   528 .. _virtualenv repo: https://github.com/pypa/virtualenv/
   530 Running the tests
   531 ~~~~~~~~~~~~~~~~~
   533 Virtualenv's test suite is small and not yet at all comprehensive, but we aim
   534 to grow it.
   536 The easy way to run tests (handles test dependencies automatically)::
   538     $ python setup.py test
   540 If you want to run only a selection of the tests, you'll need to run them
   541 directly with nose instead. Create a virtualenv, and install required
   542 packages::
   544     $ pip install nose mock
   546 Run nosetests::
   548     $ nosetests
   550 Or select just a single test file to run::
   552     $ nosetests tests.test_virtualenv
   555 Other Documentation and Links
   556 -----------------------------
   558 * James Gardner has written a tutorial on using `virtualenv with
   559   Pylons
   560   <http://wiki.pylonshq.com/display/pylonscookbook/Using+a+Virtualenv+Sandbox>`_.
   562 * `Blog announcement
   563   <http://blog.ianbicking.org/2007/10/10/workingenv-is-dead-long-live-virtualenv/>`_.
   565 * Doug Hellmann wrote a description of his `command-line work flow
   566   using virtualenv (virtualenvwrapper)
   567   <http://www.doughellmann.com/articles/CompletelyDifferent-2008-05-virtualenvwrapper/index.html>`_
   568   including some handy scripts to make working with multiple
   569   environments easier. He also wrote `an example of using virtualenv
   570   to try IPython
   571   <http://www.doughellmann.com/articles/CompletelyDifferent-2008-02-ipython-and-virtualenv/index.html>`_.
   573 * Chris Perkins created a `showmedo video including virtualenv
   574   <http://showmedo.com/videos/video?name=2910000&fromSeriesID=291>`_.
   576 * `Using virtualenv with mod_wsgi
   577   <http://code.google.com/p/modwsgi/wiki/VirtualEnvironments>`_.
   579 * `virtualenv commands
   580   <https://github.com/thisismedium/virtualenv-commands>`_ for some more
   581   workflow-related tools around virtualenv.
   583 Status and License
   584 ------------------
   586 ``virtualenv`` is a successor to `workingenv
   587 <http://cheeseshop.python.org/pypi/workingenv.py>`_, and an extension
   588 of `virtual-python
   589 <http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_.
   591 It was written by Ian Bicking, sponsored by the `Open Planning
   592 Project <http://openplans.org>`_ and is now maintained by a
   593 `group of developers <https://github.com/pypa/virtualenv/raw/master/AUTHORS.txt>`_.
   594 It is licensed under an
   595 `MIT-style permissive license <https://github.com/pypa/virtualenv/raw/master/LICENSE.txt>`_.

mercurial