In zc.buildout's configuration files, you can extend several files. Looks like a powerful feature, doesn't it? I tried it and started wondering what it is designed for. In fact, using this feature made me wonder what is buildout designed for...
In this article, I will try explain buildout's multiple inheritance, then point out problems I encountered, and finally suggest solutions whenever I can.
#1 use case for "extend" is configuration reuse:
If you do this, you are certainly going to copy-paste-adapt your favorite base configuration file. So you'll appreciate:
Also consider the following advices:
The motivations for the two advices above are detailed in buildout's inheritance mechanism below.
#2 use case for "extend" is management of multiple components:
Well, you can extend several components, but, as of december 2012, I feel buildout is not designed for this purpose:
Deeper explanations below...
Let's consider the following example, with 3 files: main "a.cfg" extends both "b.cfg" and "c.cfg".
What will happen when you run buildout -c buildout.cfg?
With buildout implementation, "A extends B and C" is handled exactly the same as "C extends B, then A extends C".
It means that, in case of clashes, = in "c.cfg" overrides value from "b.cfg". And += in "c.cfg" concatenates value from "a.cfg" and "b.cfg".
For buildout 1.6.3, I reported a bug with multiple inheritance and +=.
As a summary, you may encounter issues if you extend multiple files and make intensive use of += operator in sections.
Several solutions:
Yes, you can use += in bases. But don't you feel it is a bit strange? I mean, why would you use += in a file that don't inherit from other files?
The only reason I know is that the buildout inheritance mechanism makes bases extend each other. But I believe this is a bad reason.
So, as of version 1.6.3, my recommendation is to avoid += operator in bases, unless themselves have bases (multi-level inheritance).
Here are two designs I'd prefer, but they introduce backward incompatibilities...
Given:
I would find it more comprehensive if:
Here is something that could happen if you use one of the popular version manager in buildout (buildout.dumppickedversions or buildout-versions).
buildout.cfg:
[buildout]
extends = frontend.cfg backend.cfg
extensions = buildout-versions
versions = versions
parts = frontend backend
frontend.cfg:
[frontend]
recipe = z3c.recipe.scripts
eggs = Django frontend
[versions]
Django == 1.3
backend.cfg:
[backend]
recipe = z3c.recipe.scripts
eggs = Django backend
[versions]
Django == 1.2
Here, buildout performs implicit overrides, and you get Django==1.2 as a result. It means that the "frontend" won't work as expected. You don't get warnings or errors about it, so it's quite hard to debug if you aren't aware that implicit overrides occur.
When extending multiple bases (and because the "strange" inheritance implementation), you should use explicit overrides.
As an example, in some "a.cfg", you'd better write:
[buildout]
extends = b.cfg c.cfg
parts = b1 b2 c1 c2
... instead of using += in c[parts] and += again in a[parts].
The easiest way to avoid inheritance problems is to maintain a single buildout configuration file, i.e. don't use inheritance.
Try to keep your buildout.cfg small, simple and readable.
If you can't get a simple file, or want to reuse parts of your work, then what about generating your buildout configuration from templates? Template engines provide all you need to create buildout configuration: includes, inheritance, variables, and even more helpers like loops, formatters...
Consider buildout's inheritance and variables features as simple implementations. Use them for simple needs. If you need more, use powerful template engines. Buildout is good at managing execution of recipes ; templates engines are good at generating files.
You can apply the pattern below to deploy multiples components in a row, but keep in mind this has a major limitation: all components share the same versions for dependencies. When you run buildout, you build an environment with a consistent set of dependencies.
If you want components to use distinct versions for dependencies, you must run several buildouts.
In the following example, we build an environment with a frontend and a backend.
frontend.cfg:
[buildout]
parts = ${frontend:parts}
[frontend]
parts =
frontend-1
frontend-2
frontend-3
...
frontend-42
You can run buildout with standalone frontend.cfg. It works and installs frontend-a and frontend-b.
backend.cfg:
[buildout]
parts = ${backend:parts}
[backend]
parts =
backend-1
backend-2
You can run buildout with backend.cfg. It works and installs backend-a and backend-b.
Warning
If you run buildout -c frontend.cfg then buildout -c backend.cfg, the second command will start with uninstall of frontend!
Now let's compose the main buildout.cfg file explicitely:
[buildout]
extends =
frontend.cfg
backend.cfg
parts =
${frontend:parts}
${backend:parts}
Executing buildout on buildout.cfg installs frontend-a, frontend-b, backend-a and backend-b.
Note
Using this pattern, we didn't had to rewrite the complete (potentially long) list of parts from frontend.cfg and backend.cfg.
As told in previous sections, there are several reasons why you would want to run several buildouts:
There are several techniques. The ones I know customize directories.
Use directory options and -c argument to deploy each component in its own directory, i.e. alter buildout:directory option.
Note
By default, buildout:directory option is "the directory where lives configuration file".
As an example:
bin/buildout -c frontend.cfg buildout:directory=frontend
bin/buildout -c backend.cfg buildout:directory=backend
zc.buildout est un outil de déploiement puissant : isolation, reproductibilité, extensibilité... Il semble pourtant relativement méconnu parmi les développeurs Django. Souffrirait-il de son apparente complexité ?
Voici une présentation de zc.buildout, un lightning-talk proposé lors de DjangoCon Toulouse 2012.
Le contenu de cette présentation a été intégré avec landslide + un …