$url="http://10.190.200.87:8080/hudson/api/xml"
$path="c:\Temp\jobs.xml"
$expectedJobsNb=41
$client = new-object System.Net.WebClient
 #Download the hudson home page in xml, through the web service
$client.DownloadFile( $url, $path )
[xml] $content=get-Content $path
#Get the jobs list
$jobs=$content.SelectNodes("/hudson/job")

if ( $jobs.count -ne $expectedJobsNb ) {
    Write-Error "We do not have the good number of jobs $($jobs.count)"
    exit 1;
} else {
    Write-Output "We have the good number of jobs : $($jobs.count)"
}

When we look at evolution of tools, techniques, requirement, we saw the distributed tools having lot of success ? So, does build-master over design their build framework ? or do they already work distributed ?

In this article I’ll summarize why I think the CI is already distributed.

What’s distributed today ?

• Teams … yep,  collocated team, became more and more uncommon. Industry requirements, specific knowledge make collocation of brains more and more difficult, even for Giant ….
• Source Repository, as soon as your team is distributed, you need to distribute your source repository.  Of course, network speed, huge servers may give you the feeling you don’t need distributed source repository. But do you have really distributed administration ? does the team really use the power of your source repository ? Do you use pre/post build event ? how often they commit ?
• Dependencies … of course if the team are not co-located, your build process should be able to generate a partially your application, and download up-to-date (or the version you want)  of required libraries. So shared directory, and this kind of stupid stuff, are difficult to sync across the world.  Java community have understand that with artifacts repository that you can setup in your enterprise. Indeed, some language, provide similar functionality (perl,php,ruby, python, …) but how difficult it is to setup a proxy/mirror locally (to have the process under control).
• The build process, should allow dynamic download of dependencies. Does European, US, Indian, will download dependencies at the same location of course not ! Imagine in Visual Studio the ability to define a binaries repository for all your projects ? able to handle binaries not generated by visual ? or artefacts no generated by VS ….
• The programming languages are more and more different. Once upon a time, project were C or ++,  Borland even Java. Now, some part are in Java, some in PHP, some in Ruby, …..  Even if you have only one programming language (C#as example), you may have to handle different version in your application ( .Net 1.0 up to .Net 4.0 as example). On large project, it’s may not make sens to keep all your libraries to the last level of your programming language. So it’s common today to have at least 3 programming languages, that you want to test on a least 5 major OS ( and if you want to test 32/64b, you double this number)
• You clients became more and more heterogeneous, and so, you test platform is more and more complex.  If you write an application, you have to test it on at least 5 , 5 major OS, 5 major language, 5 major Web Browser, …. more than 125 combinations. Of course, if you minimized and combined some tests, you have at least  5 tests  platforms.
• The two previous point, encourage the continuous integration framework to scale easily, and to distribute build on a farm of servers, and that, at the job level. Indeed, all your libraries, applications, doesn’t have the same constraints.
• Static code analysis, syntax checker. As your team are distributed, don’t have the same knowledge, all your programmers can not follow the same rules at the same time. So, every thing should be configurable by libraries or project.
• Reporting tools, as all the stuff is distributed, asynchronous, reporting tools should take that into account. Does it make sens to compare code coverage of a new project under ruby, and a legacy one in C ? Does it make sens to compare an experimented team in Java, and a team of beginners ?

So, if your build master take all these elements into account, instead of using centralized tools, you may end with a distrusted platform, allowing team to work at their speed, with their constraints, their knowledge, ….

I’m not the only one in the world that belived that’s the mandatory next step !

Citons par exemple, la présence de démos régulière qui impose aux équipes de maitriser complètement leur processus de compilation/ déploiement et de configuration.

Nous avons donc décidé de présenter, non pas une session, mais 4 sessions, avec pour objectif de couvrir tous les aspects de la mise en place technique de l’agilité dans une équipe. Cerise sur le gâteau, nous avons essayer que chacune des sessions puisse se suivre indépendamment. Etant un peu fou, je me suis lancé dans 2 sessions.

1. Build et Intégration Continue : nous essayerons de présenter les pratiques pour maitriser et tirer profit de ces outils.
2. Déploiement et configuration : que serait l’intégration continue sans déploiement automatique de l’application ? une voiture sans volant ? un vase sans fleur ? un bébé sans couche (lavable of course:) ?  Nous présenterons donc les bonnes pratiques pour éviter de nombreux murs.

Tout cela évidement, en l’illustrant sur une application avec une base de donnée, des web services et un petit client. En évitant la facilité du monde linux, en faisant tourner cela sous windows.

Les principes vus au cours de ses séances s’applique a tous les langages, toutes les applications et tous les OS. Soyons fou !

Pour faire un peu de pub a nos camarades TDD et BBD sur du legacy, que serait une application qui compile, s’installe mais qui ne marche pas ou qui ne répond pas aux besoins des utilisateurs .

So, first of all, defined the priority of the issues. Mister Build, organize/prioritize the issue as :

1. Does compiling trunk and a branch at the same time on the computer lead to a random result ? yep … i try it … and discover some tests not able to run in parallel … In his mind mister build expect to be able to mount a second build box in parallel.
2. Do you have a problem if your build server is down ? yes, even if I did a standard installation, lot of plugin, lot of configuration file to recreate … In is mind, mister build expect to mount / automatize the creation of the second box.
3. Does changing your source code repository will cost you some (too much) money ? yep … too many location for configuration, some (too much) of the build configuration information are defined on the CI scheduler … Too far away to have a clean vision
4. Do you need to change your setup generation, each time you change a dependencies ? yep, even if setup package collect automatically all sub dependencies, i need a manual step to put them in the setup configuration file. Too far away to have a clean vision

So, we have a list of issues, a battle plan … so let solve the issues.

Mister build got his first car ! he’s level One

It’s current limits are :

• Do you have a problem if   your build server is down ? yes, even if I did a standard installation, lot of plugin, lot of configuration file to recreate …
• Does compiling trunk and a branch at the same time on the computer lead to a random result ? yep … i try it … and discover some tests not able to run in parallel …
• Does changing your source code repository will cost you some (too much) money ? yep … too many location for configuration, some (too much) of the build configuration information are defined on the CI scheduler …
• Do you need to change your setup generation, each time you change a dependencies ? yep, even if setup package collect automatically all sub dependencies, i need a manual step to put them in the setup configuration file

Hopfully no so bad In the next post, I’ll describe mister build battle plan to address these problems.

Bad Smell list for a build

Not Automated

1. Do you have a problem if   your build server is down ?
2. Do you use a build script different for the developers boxes  ?
3. Does your build is working on some computers but not on the other ones ?

Not Deterministic

1. Do you have some resources not packaged in your application ?
2. Does building twice the same code version, with the same inputs, produce a different result  (version number, … ) ?
3. Do you use a build script different for the release ?
4. Does your build ran on two computers produce something slightly different ?

Not Scalable

1. Does compiling two libraries at the same time on the computer, lead to a failing build ?
2. Does compiling trunk and a branch at the same time on the computer lead to a random result ?

Not Simple

1. Does your setup is difficult to maintain ?
2. Do you have some developers not able to update the build mechanism ?

Highly Coupled

1. Does changing your source code repository will cost you some (too much) money ?
2. Does changing your binary storage will cost you some (too much) money ?
3. Do you have conflicts between your IDE and your build process ?

Bad Execution time

1. Does your tests run time is too long ?
2. Does the build take too much time ?
3. Does your build time take time because you need to package for several environment ?
4. Does your build time is linked to the number of environment you deploy on ?
5. Does your build time is linked to the number of  language of your application ?
6. Does your application used several programming languages, and you don’t know how to link them ?

Bad Dependencies management

1. For some libraries included, do you have no clue where they come from ?
2. Do you have a library , you don’t know the version ?
3. Does your configuration files are included in your libraries ?
4. Does some dependencies among your modules are out of control ?
5. Is there any transitives dependencies you don’t know ?
6. Do you need to change your setup generation, each time you change a dependencies ?

Not Helpful

1. Does your code contains some bugs ?
2. Do you have a developer not using the same coding practices than the rest of the team ?
3. Do you have a developer not using the same coding standard than the rest of the team ?
4. Do you have a developer frightening of doing refactoring ?
5. Do you have a developer who don’t know (don’t have a vision of) the quality of the code he work on ?

If you answered yes, to one of this question, you have a problem like Mister build. In the next posts we’ll see how a simple and automated build process, following less coupling rules, coupled with a continuous integration framework can help you.

As mister build answered yes to some of the questions … he will stay as a pedestrian for some more days …