Agile Grenoble : Le programme est la !!!

Leave a comment

http://agile-grenoble.org/programme

Advertisements

1/8 Mister Build has problems

1 Comment

Mister build is part of a team, having some issues on its project. It was difficult for him to name/qualify its issues, so he decided to define a smell list.

Bad Smell list for a build

Not Automated

  1. Do you have a problem if   your build server is down ?
  2. Do you use a build script different for the developers boxes  ?
  3. Does your build is working on some computers but not on the other ones ?

Not Deterministic

  1. Do you have some resources not packaged in your application ?
  2. Does building twice the same code version, with the same inputs, produce a different result  (version number, … ) ?
  3. Do you use a build script different for the release ?
  4. Does your build ran on two computers produce something slightly different ?


Not Scalable

  1. Does compiling two libraries at the same time on the computer, lead to a failing build ?
  2. Does compiling trunk and a branch at the same time on the computer lead to a random result ?


Not Simple

  1. Does your setup is difficult to maintain ?
  2. Do you have some developers not able to update the build mechanism ?

Highly Coupled

  1. Does changing your source code repository will cost you some (too much) money ?
  2. Does changing your binary storage will cost you some (too much) money ?
  3. Do you have conflicts between your IDE and your build process ?

Bad Execution time

  1. Does your tests run time is too long ?
  2. Does the build take too much time ?
  3. Does your build time take time because you need to package for several environment ?
  4. Does your build time is linked to the number of environment you deploy on ?
  5. Does your build time is linked to the number of  language of your application ?
  6. Does your application used several programming languages, and you don’t know how to link them ?

Bad Dependencies management

  1. For some libraries included, do you have no clue where they come from ?
  2. Do you have a library , you don’t know the version ?
  3. Does your configuration files are included in your libraries ?
  4. Does some dependencies among your modules are out of control ?
  5. Is there any transitives dependencies you don’t know ?
  6. Do you need to change your setup generation, each time you change a dependencies ?

Not Helpful

  1. Does your code contains some bugs ?
  2. Do you have a developer not using the same coding practices than the rest of the team ?
  3. Do you have a developer not using the same coding standard than the rest of the team ?
  4. Do you have a developer frightening of doing refactoring ?
  5. Do you have a developer who don’t know (don’t have a vision of) the quality of the code he work on ?

If you answered yes, to one of this question, you have a problem like Mister build. In the next posts we’ll see how a simple and automated build process, following less coupling rules, coupled with a continuous integration framework can help you.

As mister build answered yes to some of the questions … he will stay as a pedestrian for some more days …

IMG_5135

Code review checklist

2 Comments

Starting a new code review process, trying to make it simple. After searching the web among my old links (No More Hacks), I’m sumarizing here my conclusion.

Review/Checklist guidelines (for short review, focused on one topic):

  • specific to the review : avoid having a generic checklist to long
  • Measurable point : ensure the decision is black or white, avoiding conflicting decision
  • Agreed among the team : ensure guidelines/checklist have team agreement
  • time box approach : avoid endless meeting, do them short/focused, we’ll try a 30minutes approach per topic.
  • No more than 6  items : 30 minutes is short … to many item is not realistic
  • Nothing automated : everything that can be automated, must be automated. To use human brain for that.
  • updated on a regular basis : review, team skill, application issues, change over time, so review and update your checklist to match team priority.

Having that in minds, a quick meeting among the team help us to define 3 checklist : Java, Unit test, static code analysis.

Java :

  • Documentation : does the reviewer can easily understand the code
  • Correct handling of try/catch code
  • justify visibility of method (private/public/….)
  • Test policy of the code described

Unit test :

  • does all bugs fixed are illustrated by a unit test ?
  • does mock objects are used
  • does tests are deterministic and don’t required external dependencies

Static code analysis tool

  • Focus on top priority issues : memory leak and JDBC connection management
  • For each fix, ensure a test exist to ensure behavior is kept. If you change behavior ensure impact at application level (add functional tests)
  • Investigate to ensure it’s not a false/positive, if you decide it’s a false positive, make it reviewed by someone else.

Code quality

Leave a comment

I’ve not post for some time, i was quite bust to set up some CI / metrics tools on a new project.

This raise me some questions / comments.

1) Complexity of the tools

When you read (even in my posts) it’s simple …  I’ve forget a bit the ‘open community’ and in its bad aspects.

Let me illustrate a bit. I am working in a team using mainly Java / Php.

So, using standard tools :

  • Source revision control : svn
  • IDE : eclipse
  • Build : Maven
  • CI : hudson

So, up to now, everything looks fine. I used to use some static code anlysis, in order to know the code quality :

Jdepend, PMD, findbugs, Javancss, cobertura, sonar, Coverity, …

So, nothing really exotic/fancy and most of them are ‘classical’ static code analysis tools.

So,  doing  my ‘Yoda’ show, i’ve encourage the team to use them as soon as possible in their development process so in the IDE.

I’ve plug them in their pom file (maven configuration file), update my hudson installation to have the trend, ….

AND … you know what …. 6 tools,  16 versions needed, because of incompatibility in maven, hudson,  pmd, ….

1 tool , 4 plugin needed …. most of the time just to parse an XML file, or to draw a simple curve with linear data …..

What’s a mess ! where is simplicity ? easy to use ? why coupling a CI tools with build specific plugin ??? why mixing build tools with Source control tools ???

Why not keeping the principle of less coupling to build/CI/static code analysis tools ????

Doing dcode is important, but application design should not be forget.

2) the second point, is some comments : it’s expensive to fix issues …

How can someone justify that fixing an issues at developing time … 5 .. 10 minutes … is more expensive that discovering the issue in production (something like 1$ a minute ? ) , requiring some dev again, some QA, some deployment …..

Of course, the bug may never be seen in production …. for how long ?  Don’t forget the Murphy law : if you can have a problem, you’ll have it !

How can Static Code Analysis tools may help you ?

Leave a comment

I love Static Code Analysis tools (SCA) because they are easy to use, easy to run, and most of the time very valuable.

You have of course, a learning step to know each families of tools (syntax checker, tools ensuring rules compliance, tools finding bugs,..) and to know which one to use and when to use it. But, as soon as you have this knowledge, you are very efficient and you can use them for example, during code review (even on large project) to have a good idea of the issues (it’s not easy to discover manually) or the origin of a problem. For example, you may have scalability issues on a project, but the root cause may be fully different from one project to an other one,…, the tools will help you to spot the origin.

So, I’m currently playing with several ‘bug finder’ tools, some commercial one, and some open source ones, on languages like Java/C/C++/C#,…The results ‘after some analysis’ give a good overview of a Project Quality.

So, like I am currently trying to explain to a friend of mine how to do Continuous Integration on top of Mysql (patches, plugin, specific hook, …) , I’m trying to plug some of these tools inside is Mysql CI line.

And … surprise …. more than 2000 potential bugs in Mysql Source code.

Lot of errors are due to memory handling, synchronization lock,  ‘some’ function return null, and the result of the call is used without any tests, some errors with static/non static field, ….., …..

That’s impressive …. there was a lot of time I’ve not seen so many errors per line of code. Also, the errors are very heterogeneous in the code … (side effect of open source ? )

In the following table I give some metrics :
Module Name                        Number of errors

client code 134
cmd-line-utils 80
core 1034
example 6
libmysql 211
mysys 61
server-tools 34
storage/archive 38
storage/blackhole 1
storage/csv 9
storage/federated 5
storage/heap 8
storage/innobase 262
storage/myisam 147
storage/ndb 785
system 0

For the experience I have,  I can say than 85% of the time there is a real bugs when the tool launched raise a warning.
Sometime, it took time to discover but, it was right 🙂

Some examples extracted from Myisam storage engine :

* storage/myisam/mi_check.c

Return code not check : everywhere the return code is checked, and an error is raised … my checker assume the return code is critical. So why at this line … no check ???

=> ligne 1185 :i_pack_get_block_info(info, &info->bit_buff, &block_info, &info->rec_buff, file, filepos)

* storage/myisam/mi_key.c

NPE :
–> ligne 252 : char_length= (!is_ft && cs && cs->mbmaxlen > 1) ? length/cs->mbmaxlen : length;
//so assuming cs is null
–> ligne 268
FIX_LENGTH(cs, pos, length, char_length); //which dereference cs without any checks ….

* storage/myisam/mi_rkey.c

Lock error :
—-> ligne 78 : rw_rdlock(&share->key_root_lock[inx]); // take a lock
if (!(nextflag & (SEARCH_FIND | SEARCH_NO_FIND | SEARCH_LAST))) use_key_length=USE_WHOLE_KEY;
……
if (rtree_find_first(info,inx,key_buff,use_key_length,nextflag) line 146
DBUG_ENTER(“table2myisam”);
if (!(my_multi_malloc(MYF(MY_WME, …..) ///// the allocation is not stored … and never free
DBUG_RETURN(HA_ERR_OUT_OF_MEM);

Static code analysis

Leave a comment

My current hobby is to find efficient / easy to use / and well package tools to analyse code.

For those of you who don’t know these technics,  I’ll summarize them as : ‘tools which analyse your source code, in order to help you to find potentiel bugs, or it try to convince you to change your way of programming and may give you some new programming hits, ….,’.

If your build process is clean, or if you use a standard IDE, adding these tools is very cheap, and the benefice is great.