Thu, 22 Jan 2015 13:21:57 +0100
Incorporate requested changes from Mozilla in review:
https://bugzilla.mozilla.org/show_bug.cgi?id=1123480#c6
2 Introduction
3 ------------
5 ``virtualenv`` is a tool to create isolated Python environments.
7 The basic problem being addressed is one of dependencies and versions,
8 and indirectly permissions. Imagine you have an application that
9 needs version 1 of LibFoo, but another application requires version
10 2. How can you use both these applications? If you install
11 everything into ``/usr/lib/python2.7/site-packages`` (or whatever your
12 platform's standard location is), it's easy to end up in a situation
13 where you unintentionally upgrade an application that shouldn't be
14 upgraded.
16 Or more generally, what if you want to install an application *and
17 leave it be*? If an application works, any change in its libraries or
18 the versions of those libraries can break the application.
20 Also, what if you can't install packages into the global
21 ``site-packages`` directory? For instance, on a shared host.
23 In all these cases, ``virtualenv`` can help you. It creates an
24 environment that has its own installation directories, that doesn't
25 share libraries with other virtualenv environments (and optionally
26 doesn't access the globally installed libraries either).
29 Installation
30 ------------
32 .. warning::
34 We advise installing virtualenv-1.9 or greater. Prior to version 1.9, the
35 pip included in virtualenv did not not download from PyPI over SSL.
37 .. warning::
39 When using pip to install virtualenv, we advise using pip 1.3 or greater.
40 Prior to version 1.3, pip did not not download from PyPI over SSL.
42 .. warning::
44 We advise against using easy_install to install virtualenv when using
45 setuptools < 0.9.7, because easy_install didn't download from PyPI over SSL
46 and was broken in some subtle ways.
48 To install globally with `pip` (if you have pip 1.3 or greater installed globally):
50 ::
52 $ [sudo] pip install virtualenv
54 Or to get the latest unreleased dev version:
56 ::
58 $ [sudo] pip install https://github.com/pypa/virtualenv/tarball/develop
61 To install globally from source:
63 ::
65 $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
66 $ tar xvfz virtualenv-X.X.tar.gz
67 $ cd virtualenv-X.X
68 $ [sudo] python setup.py install
71 To *use* locally from source:
73 ::
75 $ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz
76 $ tar xvfz virtualenv-X.X.tar.gz
77 $ cd virtualenv-X.X
78 $ python virtualenv.py myVE
80 .. note::
82 The ``virtualenv.py`` script is *not* supported if run without the
83 necessary pip/setuptools/virtualenv distributions available locally. All
84 of the installation methods above include a ``virtualenv_support``
85 directory alongside ``virtualenv.py`` which contains a complete set of
86 pip and setuptools distributions, and so are fully supported.
88 Usage
89 -----
91 The basic usage is::
93 $ virtualenv ENV
95 This creates ``ENV/lib/pythonX.X/site-packages``, where any libraries you
96 install will go. It also creates ``ENV/bin/python``, which is a Python
97 interpreter that uses this environment. Anytime you use that interpreter
98 (including when a script has ``#!/path/to/ENV/bin/python`` in it) the libraries
99 in that environment will be used.
101 It also installs `Setuptools
102 <http://peak.telecommunity.com/DevCenter/setuptools>`_ into the environment.
104 .. note::
106 Virtualenv (<1.10) used to provide a ``--distribute`` option to use the
107 setuptools fork Distribute_. Since Distribute has been merged back into
108 setuptools this option is now no-op, it will always use the improved
109 setuptools releases.
111 A new virtualenv also includes the `pip <http://pypi.python.org/pypi/pip>`_
112 installer, so you can use ``ENV/bin/pip`` to install additional packages into
113 the environment.
115 .. _Distribute: https://pypi.python.org/pypi/distribute
117 activate script
118 ~~~~~~~~~~~~~~~
120 In a newly created virtualenv there will be a ``bin/activate`` shell
121 script. For Windows systems, activation scripts are provided for CMD.exe
122 and Powershell.
124 On Posix systems you can do::
126 $ source bin/activate
128 This will change your ``$PATH`` so its first entry is the virtualenv's
129 ``bin/`` directory. (You have to use ``source`` because it changes your
130 shell environment in-place.) This is all it does; it's purely a
131 convenience. If you directly run a script or the python interpreter
132 from the virtualenv's ``bin/`` directory (e.g. ``path/to/env/bin/pip``
133 or ``/path/to/env/bin/python script.py``) there's no need for
134 activation.
136 After activating an environment you can use the function ``deactivate`` to
137 undo the changes to your ``$PATH``.
139 The ``activate`` script will also modify your shell prompt to indicate
140 which environment is currently active. You can disable this behavior,
141 which can be useful if you have your own custom prompt that already
142 displays the active environment name. To do so, set the
143 ``VIRTUAL_ENV_DISABLE_PROMPT`` environment variable to any non-empty
144 value before running the ``activate`` script.
146 On Windows you just do::
148 > \path\to\env\Scripts\activate
150 And type `deactivate` to undo the changes.
152 Based on your active shell (CMD.exe or Powershell.exe), Windows will use
153 either activate.bat or activate.ps1 (as appropriate) to activate the
154 virtual environment. If using Powershell, see the notes about code signing
155 below.
157 .. note::
159 If using Powershell, the ``activate`` script is subject to the
160 `execution policies`_ on the system. By default on Windows 7, the system's
161 excution policy is set to ``Restricted``, meaning no scripts like the
162 ``activate`` script are allowed to be executed. But that can't stop us
163 from changing that slightly to allow it to be executed.
165 In order to use the script, you have to relax your system's execution
166 policy to ``AllSigned``, meaning all scripts on the system must be
167 digitally signed to be executed. Since the virtualenv activation
168 script is signed by one of the authors (Jannis Leidel) this level of
169 the execution policy suffices. As an administrator run::
171 PS C:\> Set-ExecutionPolicy AllSigned
173 Then you'll be asked to trust the signer, when executing the script.
174 You will be prompted with the following::
176 PS C:\> virtualenv .\foo
177 New python executable in C:\foo\Scripts\python.exe
178 Installing setuptools................done.
179 Installing pip...................done.
180 PS C:\> .\foo\scripts\activate
182 Do you want to run software from this untrusted publisher?
183 File C:\foo\scripts\activate.ps1 is published by E=jannis@leidel.info,
184 CN=Jannis Leidel, L=Berlin, S=Berlin, C=DE, Description=581796-Gh7xfJxkxQSIO4E0
185 and is not trusted on your system. Only run scripts from trusted publishers.
186 [V] Never run [D] Do not run [R] Run once [A] Always run [?] Help
187 (default is "D"):A
188 (foo) PS C:\>
190 If you select ``[A] Always Run``, the certificate will be added to the
191 Trusted Publishers of your user account, and will be trusted in this
192 user's context henceforth. If you select ``[R] Run Once``, the script will
193 be run, but you will be prometed on a subsequent invocation. Advanced users
194 can add the signer's certificate to the Trusted Publishers of the Computer
195 account to apply to all users (though this technique is out of scope of this
196 document).
198 Alternatively, you may relax the system execution policy to allow running
199 of local scripts without verifying the code signature using the following::
201 PS C:\> Set-ExecutionPolicy RemoteSigned
203 Since the ``activate.ps1`` script is generated locally for each virtualenv,
204 it is not considered a remote script and can then be executed.
206 .. _`execution policies`: http://technet.microsoft.com/en-us/library/dd347641.aspx
208 The ``--system-site-packages`` Option
209 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
211 If you build with ``virtualenv --system-site-packages ENV``, your virtual
212 environment will inherit packages from ``/usr/lib/python2.7/site-packages``
213 (or wherever your global site-packages directory is).
215 This can be used if you have control over the global site-packages directory,
216 and you want to depend on the packages there. If you want isolation from the
217 global system, do not use this flag.
220 Environment variables and configuration files
221 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
223 virtualenv can not only be configured by passing command line options such as
224 ``--python`` but also by two other means:
226 - Environment variables
228 Each command line option is automatically used to look for environment
229 variables with the name format ``VIRTUALENV_<UPPER_NAME>``. That means
230 the name of the command line options are capitalized and have dashes
231 (``'-'``) replaced with underscores (``'_'``).
233 For example, to automatically use a custom Python binary instead of the
234 one virtualenv is run with you can also set an environment variable::
236 $ export VIRTUALENV_PYTHON=/opt/python-3.3/bin/python
237 $ virtualenv ENV
239 It's the same as passing the option to virtualenv directly::
241 $ virtualenv --python=/opt/python-3.3/bin/python ENV
243 This also works for appending command line options, like ``--find-links``.
244 Just leave an empty space between the passsed values, e.g.::
246 $ export VIRTUALENV_EXTRA_SEARCH_DIR="/path/to/dists /path/to/other/dists"
247 $ virtualenv ENV
249 is the same as calling::
251 $ virtualenv --extra-search-dir=/path/to/dists --extra-search-dir=/path/to/other/dists ENV
253 - Config files
255 virtualenv also looks for a standard ini config file. On Unix and Mac OS X
256 that's ``$HOME/.virtualenv/virtualenv.ini`` and on Windows, it's
257 ``%APPDATA%\virtualenv\virtualenv.ini``.
259 The names of the settings are derived from the long command line option,
260 e.g. the option ``--python`` would look like this::
262 [virtualenv]
263 python = /opt/python-3.3/bin/python
265 Appending options like ``--extra-search-dir`` can be written on multiple
266 lines::
268 [virtualenv]
269 extra-search-dir =
270 /path/to/dists
271 /path/to/other/dists
273 Please have a look at the output of ``virtualenv --help`` for a full list
274 of supported options.
276 Windows Notes
277 ~~~~~~~~~~~~~
279 Some paths within the virtualenv are slightly different on Windows: scripts and
280 executables on Windows go in ``ENV\Scripts\`` instead of ``ENV/bin/`` and
281 libraries go in ``ENV\Lib\`` rather than ``ENV/lib/``.
283 To create a virtualenv under a path with spaces in it on Windows, you'll need
284 the `win32api <http://sourceforge.net/projects/pywin32/>`_ library installed.
286 PyPy Support
287 ~~~~~~~~~~~~
289 Beginning with virtualenv version 1.5 `PyPy <http://pypy.org>`_ is
290 supported. To use PyPy 1.4 or 1.4.1, you need a version of virtualenv >= 1.5.
291 To use PyPy 1.5, you need a version of virtualenv >= 1.6.1.
293 Creating Your Own Bootstrap Scripts
294 -----------------------------------
296 While this creates an environment, it doesn't put anything into the
297 environment. Developers may find it useful to distribute a script
298 that sets up a particular environment, for example a script that
299 installs a particular web application.
301 To create a script like this, call
302 ``virtualenv.create_bootstrap_script(extra_text)``, and write the
303 result to your new bootstrapping script. Here's the documentation
304 from the docstring:
306 Creates a bootstrap script, which is like this script but with
307 extend_parser, adjust_options, and after_install hooks.
309 This returns a string that (written to disk of course) can be used
310 as a bootstrap script with your own customizations. The script
311 will be the standard virtualenv.py script, with your extra text
312 added (your extra text should be Python code).
314 If you include these functions, they will be called:
316 ``extend_parser(optparse_parser)``:
317 You can add or remove options from the parser here.
319 ``adjust_options(options, args)``:
320 You can change options here, or change the args (if you accept
321 different kinds of arguments, be sure you modify ``args`` so it is
322 only ``[DEST_DIR]``).
324 ``after_install(options, home_dir)``:
326 After everything is installed, this function is called. This
327 is probably the function you are most likely to use. An
328 example would be::
330 def after_install(options, home_dir):
331 if sys.platform == 'win32':
332 bin = 'Scripts'
333 else:
334 bin = 'bin'
335 subprocess.call([join(home_dir, bin, 'easy_install'),
336 'MyPackage'])
337 subprocess.call([join(home_dir, bin, 'my-package-script'),
338 'setup', home_dir])
340 This example immediately installs a package, and runs a setup
341 script from that package.
343 Bootstrap Example
344 ~~~~~~~~~~~~~~~~~
346 Here's a more concrete example of how you could use this::
348 import virtualenv, textwrap
349 output = virtualenv.create_bootstrap_script(textwrap.dedent("""
350 import os, subprocess
351 def after_install(options, home_dir):
352 etc = join(home_dir, 'etc')
353 if not os.path.exists(etc):
354 os.makedirs(etc)
355 subprocess.call([join(home_dir, 'bin', 'easy_install'),
356 'BlogApplication'])
357 subprocess.call([join(home_dir, 'bin', 'paster'),
358 'make-config', 'BlogApplication',
359 join(etc, 'blog.ini')])
360 subprocess.call([join(home_dir, 'bin', 'paster'),
361 'setup-app', join(etc, 'blog.ini')])
362 """))
363 f = open('blog-bootstrap.py', 'w').write(output)
365 Another example is available `here
366 <https://github.com/socialplanning/fassembler/blob/master/fassembler/create-venv-script.py>`_.
369 Using Virtualenv without ``bin/python``
370 ---------------------------------------
372 Sometimes you can't or don't want to use the Python interpreter
373 created by the virtualenv. For instance, in a `mod_python
374 <http://www.modpython.org/>`_ or `mod_wsgi <http://www.modwsgi.org/>`_
375 environment, there is only one interpreter.
377 Luckily, it's easy. You must use the custom Python interpreter to
378 *install* libraries. But to *use* libraries, you just have to be sure
379 the path is correct. A script is available to correct the path. You
380 can setup the environment like::
382 activate_this = '/path/to/env/bin/activate_this.py'
383 execfile(activate_this, dict(__file__=activate_this))
385 This will change ``sys.path`` and even change ``sys.prefix``, but also allow
386 you to use an existing interpreter. Items in your environment will show up
387 first on ``sys.path``, before global items. However, global items will
388 always be accessible (as if the ``--system-site-packages`` flag had been used
389 in creating the environment, whether it was or not). Also, this cannot undo
390 the activation of other environments, or modules that have been imported.
391 You shouldn't try to, for instance, activate an environment before a web
392 request; you should activate *one* environment as early as possible, and not
393 do it again in that process.
395 Making Environments Relocatable
396 -------------------------------
398 Note: this option is somewhat experimental, and there are probably
399 caveats that have not yet been identified.
401 .. warning::
403 The ``--relocatable`` option currently has a number of issues,
404 and is not guaranteed to work in all circumstances. It is possible
405 that the option will be deprecated in a future version of ``virtualenv``.
407 Normally environments are tied to a specific path. That means that
408 you cannot move an environment around or copy it to another computer.
409 You can fix up an environment to make it relocatable with the
410 command::
412 $ virtualenv --relocatable ENV
414 This will make some of the files created by setuptools use relative paths,
415 and will change all the scripts to use ``activate_this.py`` instead of using
416 the location of the Python interpreter to select the environment.
418 **Note:** scripts which have been made relocatable will only work if
419 the virtualenv is activated, specifically the python executable from
420 the virtualenv must be the first one on the system PATH. Also note that
421 the activate scripts are not currently made relocatable by
422 ``virtualenv --relocatable``.
424 **Note:** you must run this after you've installed *any* packages into
425 the environment. If you make an environment relocatable, then
426 install a new package, you must run ``virtualenv --relocatable``
427 again.
429 Also, this **does not make your packages cross-platform**. You can
430 move the directory around, but it can only be used on other similar
431 computers. Some known environmental differences that can cause
432 incompatibilities: a different version of Python, when one platform
433 uses UCS2 for its internal unicode representation and another uses
434 UCS4 (a compile-time option), obvious platform changes like Windows
435 vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C
436 libraries on the system, if those C libraries are located somewhere
437 different (either different versions, or a different filesystem
438 layout).
440 If you use this flag to create an environment, currently, the
441 ``--system-site-packages`` option will be implied.
443 The ``--extra-search-dir`` option
444 ---------------------------------
446 This option allows you to provide your own versions of setuptools and/or
447 pip to use instead of the embedded versions that come with virtualenv.
449 To use this feature, pass one or more ``--extra-search-dir`` options to
450 virtualenv like this::
452 $ virtualenv --extra-search-dir=/path/to/distributions ENV
454 The ``/path/to/distributions`` path should point to a directory that contains
455 setuptools and/or pip wheels.
457 virtualenv will look for wheels in the specified directories, but will use
458 pip's standard algorithm for selecting the wheel to install, which looks for
459 the latest compatible wheel.
461 As well as the extra directories, the search order includes:
463 #. The ``virtualenv_support`` directory relative to virtualenv.py
464 #. The directory where virtualenv.py is located.
465 #. The current directory.
467 If no satisfactory local distributions are found, virtualenv will
468 fail. Virtualenv will never download packages.
471 Compare & Contrast with Alternatives
472 ------------------------------------
474 There are several alternatives that create isolated environments:
476 * ``workingenv`` (which I do not suggest you use anymore) is the
477 predecessor to this library. It used the main Python interpreter,
478 but relied on setting ``$PYTHONPATH`` to activate the environment.
479 This causes problems when running Python scripts that aren't part of
480 the environment (e.g., a globally installed ``hg`` or ``bzr``). It
481 also conflicted a lot with Setuptools.
483 * `virtual-python
484 <http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_
485 is also a predecessor to this library. It uses only symlinks, so it
486 couldn't work on Windows. It also symlinks over the *entire*
487 standard library and global ``site-packages``. As a result, it
488 won't see new additions to the global ``site-packages``.
490 This script only symlinks a small portion of the standard library
491 into the environment, and so on Windows it is feasible to simply
492 copy these files over. Also, it creates a new/empty
493 ``site-packages`` and also adds the global ``site-packages`` to the
494 path, so updates are tracked separately. This script also installs
495 Setuptools automatically, saving a step and avoiding the need for
496 network access.
498 * `zc.buildout <http://pypi.python.org/pypi/zc.buildout>`_ doesn't
499 create an isolated Python environment in the same style, but
500 achieves similar results through a declarative config file that sets
501 up scripts with very particular packages. As a declarative system,
502 it is somewhat easier to repeat and manage, but more difficult to
503 experiment with. ``zc.buildout`` includes the ability to setup
504 non-Python systems (e.g., a database server or an Apache instance).
506 I *strongly* recommend anyone doing application development or
507 deployment use one of these tools.
509 Contributing
510 ------------
512 Refer to the `pip development`_ documentation - it applies equally to
513 virtualenv, except that virtualenv issues should filed on the `virtualenv
514 repo`_ at GitHub.
516 Virtualenv's release schedule is tied to pip's -- each time there's a new pip
517 release, there will be a new virtualenv release that bundles the new version of
518 pip.
520 Files in the `virtualenv_embedded/` subdirectory are embedded into
521 `virtualenv.py` itself as base64-encoded strings (in order to support
522 single-file use of `virtualenv.py` without installing it). If your patch
523 changes any file in `virtualenv_embedded/`, run `bin/rebuild-script.py` to
524 update the embedded version of that file in `virtualenv.py`; commit that and
525 submit it as part of your patch / pull request.
527 .. _pip development: http://www.pip-installer.org/en/latest/development.html
528 .. _virtualenv repo: https://github.com/pypa/virtualenv/
530 Running the tests
531 ~~~~~~~~~~~~~~~~~
533 Virtualenv's test suite is small and not yet at all comprehensive, but we aim
534 to grow it.
536 The easy way to run tests (handles test dependencies automatically)::
538 $ python setup.py test
540 If you want to run only a selection of the tests, you'll need to run them
541 directly with nose instead. Create a virtualenv, and install required
542 packages::
544 $ pip install nose mock
546 Run nosetests::
548 $ nosetests
550 Or select just a single test file to run::
552 $ nosetests tests.test_virtualenv
555 Other Documentation and Links
556 -----------------------------
558 * James Gardner has written a tutorial on using `virtualenv with
559 Pylons
560 <http://wiki.pylonshq.com/display/pylonscookbook/Using+a+Virtualenv+Sandbox>`_.
562 * `Blog announcement
563 <http://blog.ianbicking.org/2007/10/10/workingenv-is-dead-long-live-virtualenv/>`_.
565 * Doug Hellmann wrote a description of his `command-line work flow
566 using virtualenv (virtualenvwrapper)
567 <http://www.doughellmann.com/articles/CompletelyDifferent-2008-05-virtualenvwrapper/index.html>`_
568 including some handy scripts to make working with multiple
569 environments easier. He also wrote `an example of using virtualenv
570 to try IPython
571 <http://www.doughellmann.com/articles/CompletelyDifferent-2008-02-ipython-and-virtualenv/index.html>`_.
573 * Chris Perkins created a `showmedo video including virtualenv
574 <http://showmedo.com/videos/video?name=2910000&fromSeriesID=291>`_.
576 * `Using virtualenv with mod_wsgi
577 <http://code.google.com/p/modwsgi/wiki/VirtualEnvironments>`_.
579 * `virtualenv commands
580 <https://github.com/thisismedium/virtualenv-commands>`_ for some more
581 workflow-related tools around virtualenv.
583 Status and License
584 ------------------
586 ``virtualenv`` is a successor to `workingenv
587 <http://cheeseshop.python.org/pypi/workingenv.py>`_, and an extension
588 of `virtual-python
589 <http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python>`_.
591 It was written by Ian Bicking, sponsored by the `Open Planning
592 Project <http://openplans.org>`_ and is now maintained by a
593 `group of developers <https://github.com/pypa/virtualenv/raw/master/AUTHORS.txt>`_.
594 It is licensed under an
595 `MIT-style permissive license <https://github.com/pypa/virtualenv/raw/master/LICENSE.txt>`_.