Sunday, March 29, 2009

Managing projects using buildout

Directories in the buildout
Before we dive into buildout.cfg, let us take a quick look at the directories that buildout has created for us:
bin/

Contains various executables, including the buildout command, and the instance Zope control script.
eggs/

Contains eggs that buildout has downloaded. These will be explicitly activated by the control scripts in the bin/ directory.
downloads/

Contains non-egg downloads, such as the Zope source code archive.
var/

Contains the log files (in var/log/) and the file storage ZODB data (in var/filestorage/Data.fs). Buildout will never overwrite these.
src/

Initially empty. You can place your own development eggs here and reference them in buildout.cfg. More on that later.
products/

This is analogous to a Zope instance's Products/ directory (note the difference in capitalisation). If you are developing any old-style Zope 2 products, place them here. We will see how buildout can automatically download and manage archives of products, but if you want to extract a product dependency manually, or check one out from Subversion, this is the place to do so.
parts/

Contains code and data managed by buildout. In our case, it will include the local Zope installation, a buildout-managed Zope instance, and Plone's source code. In general, you should not modify anything in this directory, as buildout may overwrite your changes.

The main [buildout] section
The [buildout] section is the starting point for the file. It lists a number of "parts", which are configured in separate sections later in the file. Each part has an associated recipe, which is the name of an egg that knows how to perform a particular task, e.g. build Zope or create a Zope instance. A recipe typically takes a few configuration options.

Our global settings are as follows:
[buildout]
parts =
plone
zope2
productdistros
instance
zopepy
find-links =
http://dist.plone.org
http://download.zope.org/ppix/
http://download.zope.org/distribution/
http://effbot.org/downloads
eggs =
elementtree
develop =

This specifies that the parts plone, zope2, productdistros, instance and zopepy will be run, in that order. Then, we tell buildout that it can search one of a number of URLs when it is looking for eggs to download. In addition, it will always search the Cheese Shop.

Next, we can list any eggs that buildout should download and install for us. This may include version specifications. For example, if you want sqlalchemy 0.3, but not 0.4, you could list;
eggs =
elementtree
sqlalchemy>=0.3,<0.4dev

Finally, we can list development eggs, by specifying a directory where the egg is extracted in source format. For example:
eggs =
elementtree
my.package
develop =
src/my.package

This presumes that there is an egg called my.package in the src/ directory. We will learn how to create such eggs a little later in this tutorial. Notice how we must also list my.package as an actual egg dependency: development eggs are not automatically added to the "working set" of eggs that are installed for Zope.

The [plone] section
This is very simple - it just uses plone.recipe.plone to download Plone's products and eggs.
[plone]
recipe = plone.recipe.plone

It will use the latest release available. Version numbers for plone.recipe.plone correspond to version numbers for Plone itself. Therefore, to make sure you always get a 3.0.x release, but not a 3.1, you can do:
[plone]
recipe = plone.recipe.plone>=3.0,<3.1dev

When the recipe is run, Plone's products will be installed in parts/plone. The eggs are made available via buildout variable ${plone:eggs}, which we will reference in the [instance] section later, and the URL of a "known good" version of Zope is available in the variable ${plone:zope2-url}.

The [zope2] section
This part builds Zope 2, using plone.recipe.zope2install. If you specified an existing Zope installation, you will not have this part. Otherwise, it looks like this:
[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

Here, we reference the download location for Zope as emitted by the [plone] part. This ensures that we always get the recommended version of Zope. You could specify a download URL manually instead, if you wanted to use a different version of Zope.

When the recipe is run, Zope 2 is installed in parts/zope2. The Zope software home becomes parts/zope2/lib/python.

The [productdistros] section
This uses the plone.recipe.distros recipe, which is able to download distributions (archives) of Zope 2 style products and make them available to Zope. It is empty to begin with:
[productdistros]
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages =

However, you can list any number of downloads. The recipe is also able to deal with archives that contain a single top-level directory that contains a bundle of actual product directories (nested-packages), or packages that have a version number in the directory name and thus need to be renamed to get the actual product directory (version-suffix-packages).

Consider the following distributions:

# A typical distribution

ExampleProduct-1.0.tgz
|
|- ExampleProduct
| |
| |- __init__.py
| |- (product code)

# A version suffix distribution

AnotherExampleProduct-2.0.tgz
|
|- AnotherExampleProduct-2.0
| |
| |- __init__.py
| |- (product code)

# A nested package distribution

ExampleProductBundle-1.0.tgz
|
|- ExampleProductBundle
| |
| |- ProductOne
| | |- __init__.py
| | |- (product code)
| |
| |- ProductTwo
| | |- __init__.py
| | |- (product code)

Here is what the part would look like if we try to install the three distributions above:
[productdistros]
recipe = plone.recipe.distros
urls =
http://example.com/dist/ExampleProduct-1.0.tgz
http://example.com/dist/AnotherExampleProduct-2.0.tgz
http://example.com/dist/ExampleProductBundle-1.0.tgz
nested-packages = ExampleProductBundle-1.0.tgz
version-suffix-packages = AnotherExampleProduct-2.0.tgz

You can specify multiple downloads on separate lines. When the recipe is run, the product directories for downloaded products are found in parts/productdistros.

The [instance] section
The instance section pulls it all together: It configures a Zope instance using the plone.recipe.zope2instance script. Here is how it looks:
[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
debug-mode = on
verbose-security = on
eggs =
${buildout:eggs}
${plone:eggs}
zcml =
products =
${buildout:directory}/products
${productdistros:location}
${plone:products}

Here, we reference the Zope 2 installation from the [zope2] part - if you specified a location yourself when creating the buildout, you would see that one here. Then, we specify the initial admin user and password, and the port that Zope will be bound to. We also turn on debug mode and verbose security. These options are used to generate an appropraite zope.conf file for this instance. See the recipe page in the Cheese Shop for more details on the options available.

Next, we specify which eggs that will be made available to Zope. This references the "global" eggs from the [buildout] section, as well as the eggs specified by Plone. You could add additional eggs here, though it is generally easier to specify these at the top of the file, so that they get included in the ${buildout:eggs} working set.

As explained previously, Zope 3 configure.zcml files are not loaded automatically for eggs or packages not the Products namespace. To load ZCML files for a regular package, we can make buildout create a ZCML slug by listing the package under the zcml option:
zcml =
my.package
my.package-overrides

This assumes that my.package was previously referenced in the buildout. This would load both the main configure.zcml and the overrides.zcml file from this package.

Finally, we list the various directories that contain Zope 2 style products - akin to the Products/ directory in a traditional instance. Notice how the products/ directory in the main buildout directory comes first, followed by the products downloaded with the [productdistros] part, followed by the products downloaded by the [plone] part. This means that even if Plone ships with a product, you could override it (e.g. with a newer product) by putting a product with the same name in the top-level products/ directory.

When the recipe is run, the Zope instance home will be parts/instance, and a control script is created in ./bin/instance.

The [zopepy] section
This final section creates a Python interpreter that has all the eggs and packages (but not Zope 2 style products) that Zope would have during startup. This can be useful for testing purposes.
[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

Here, we copy the eggs from the [instance] section, and include in the pythonpath the Zope instance home.

When the recipe is run, the script will be created in ./bin/zopepy.

Managing ZCML files
It is important to realize that Zope will not load configure.zcml files automatically for packages that are not in the Products.* namespace. Instead, you must explicitly reference the package. Buildout can create such a reference (known as a ZCML slug) with the zcml option under the [instance] part. Here is how to ensure that borg.project is available to Zope:
[buildout]
...
eggs =
elementtree
borg.project
...
[instance]
...
zcml =
borg.project

Should you need to load an overrides.zcml or a meta.zcml, you can use a syntax like:
zcml =
some.package
some.package-overrides
some.package-meta


Resources:
http://plone.org/documentation/tutorial/buildout/tutorial-all-pages

No comments: