Saturday, June 13, 2015

Example of Cyclomatic complexity analysis over real projects

Processed by:
1) launch of https://github.com/checkstyle/contribution/tree/master/checkstyle-tester generate HTML report

2) html2text target/site/index.html | grep "Cyclomatic Complexity is" | cut -d' ' -f7 | sort | uniq --count | sort -h -k 2

3) % is calculated as "=TRUNC(((B2)/B$47)*100)"

So amount of methods the complicated then 10 (from all method that complicated than 5) in the project is about 5%.  Below is tables for: Checkstyle , Spring Framework, Open JDK 7.

Checkstyle 6.7 Cynclomatic complexity analysis:

               CC           Count                  %
5 130 15
6 83 12
7 66 9
8 52 10
9 53 4
10 22 4
11 23 4
12 25 1
13 7 0
14 3 1
15 8 0
16 4 0
17 2 0
18 5 1
19 10 0
20 1 2
21 13 0
22 1 0
23 5 0
24 1 0
27 1 0
29 1 0
30 1 0
32 1 0
34 2 0
44 2 0
47 2 0
81 1 100

Total: 525


Spring frameworks v4.1.6.RELEASE Cyclomatic complexity analysis:
               CC           Count                 %
5 746 30
6 450 18
7 312 12
8 215 8
9 157 6
10 109 4
11 80 3
12 55 2
13 52 2
14 34 1
15 33 1
16 25 1
17 18 0
18 19 0
19 14 0
20 17 0
21 11 0
22 8 0
23 9 0
24 6 0
25 7 0
26 11 0
27 1 0
28 2 0
29 1 0
31 3 0
32 2 0
33 5 0
34 3 0
35 2 0
36 3 0
37 1 0
38 1 0
40 2 0
41 1 0
44 1 0
47 1 0
48 1 0
50 2 0
55 1 0
60 1 0
61 1 0
81 2 0
127 1 0
184 1 0

Total:2427

OpenJdk7 Cyclomatic complexity analysis: 

               CC           Count                 %
5 4650 25
6 2947 16
7 2234 12
8 1542 8
9 1161 6
10 932 5
11 706 3
12 574 3
13 505 2
14 383 2
15 332 1
16 261 1
17 230 1
18 200 1
19 183 1
20 128 0
21 119 0
22 121 0
23 102 0
24 78 0
25 58 0
26 60 0
27 57 0
28 47 0
29 30 0
30 29 0
31 27 0
32 30 0
33 22 0
34 30 0
35 11 0
36 19 0
37 27 0
38 12 0
39 17 0
40 10 0
41 16 0
42 12 0
43 11 0
44 14 0
45 6 0
46 11 0
47 14 0
48 9 0
49 7 0
50 12 0
51 11 0
52 6 0
53 6 0
54 8 0
55 6 0
56 5 0
57 2 0
58 4 0
59 1 0
60 3 0
62 6 0
63 3 0
64 4 0
66 4 0
67 6 0
68 2 0
69 1 0
70 2 0
71 1 0
72 2 0
73 3 0
74 7 0
75 4 0
76 3 0
77 3 0
78 2 0
79 1 0
80 4 0
81 1 0
82 5 0
83 1 0
85 1 0
86 2 0
87 1 0
88 3 0
93 2 0
95 3 0
96 1 0
97 1 0
100 1 0
101 1 0
103 1 0
109 1 0
110 1 0
111 1 0
121 1 0
124 1 0
125 1 0
130 2 0
140 1 0
145 1 0
149 1 0
164 1 0
186 1 0
219 1 0
256 1 0
281 1 0

Total:18136

Understanding of Cyclomatic Complexity for measuring code quality

But does that mean a program with a high Cyclomatic Complexity (CC) has a bad quality ? For sure not !

It is just another metrics and some people think it is better to use Line numbers instead as more simple and more robust metric. Very good paper  - http://www.leshatton.org/Documents/TAIC2008-29-08-2008.pdf#17 (page 17) , see also Conclusion - page 45.

It is common when people say " ... reducing the cyclomatic complexity of the code can help reducing the number of test cases ... " - that is not always true!!
Blind following of Cyclomatic metric will lead you to code that have a lot of small methods that will be hard to maintain as encapsulation might be damaged. It is not a rare case there you should put ignore to some case where complexity is high as you defined for the rest of the project.
To keep all that logic(even a bit complicated) in one method to not let other methods use it and what is more problematic change them to their needs. Not everything should be decomposed to smaller parts.
Finally if your complicated method is "public" and you will decompose it internals for several "private" method - you will not reduce amount of tests !!! as in tests you will still check your public method with all possible values in arguments.

The main point of CC is to measure complexity to measure amount of tests, so it suppose to be used as (from wiki) :
 branch coverage \leq cyclomatic complexity \leq number of paths

from wiki:
McCabe showed that the cyclomatic complexity of any structured program with only one entrance point and one exit point is equal to the number of decision points (π) (i.e., "if" statements or conditional loops) contained in that program plus one. However, this is true only for decision points counted at the lowest, machine-level instructions. Decisions involving compound predicates like those found in high-level languages like IF cond1 AND cond2 THEN ... should be counted in terms of predicate variables involved, i.e. in this examples one should count two decision points, because at machine level it is equivalent to  IF cond1 THEN IF cond2 THEN ... 
Cyclomatic complexity may be extended to a program with multiple exit points; in this case it is equal to:
π − s + 2, 
where π is the number of decision points in the program, and s is the number of exit points

So amount of decision point is good measure of complexity ,  but as you can see in definition it could be reduced by adding more exit points. That is true for amount of test calculation  but that is not true to measure of code quality and readability.
So "multi exits/returns" could be a incentive for engineers to calm down Checkstyle or PMD . But that conflict with best practice to have one entry and one exit point in method.

Conclusion: Cyclomatic Complexity is for measuring of Tests amount and NOT about code quality. But CC could be used as quality measure with custom to project level/threshold selection. Do not be ashamed to have complexity level 15 or even more, but keep it below 20 to catch really bad designed code automatically.

I highly recommend to read https://www.cqse.eu/en/blog/mccabe-cyclomatic-complexity/ to see on code examples how weird complexity calculation could be.

Checkstyle's HTML documentation was updated for this metric: http://checkstyle.sourceforge.net/config_metrics.html#CyclomaticComplexity

I updated Chekstyle code to follow level less then 11, I can confirm that about 54 cases were real ugly and messy code and it  was good that that metrics is pointed to that code. But in non library code level 11 is too demanding but reachable, 15 is ok.

Example of Cyclomatic complexity calculation over code base of Checkstyle, Spring, OpenJDK.

Sources:
https://en.wikipedia.org/wiki/Cyclomatic_complexity (especially links to external and other ref)
http://damienlepage.com/cyclomatic-complexity/
http://www.sonarqube.org/discussing-cyclomatic-complexity/


Wednesday, June 10, 2015

Sync tags from Zimbra to Thunderbird

To Sync tags from Zimbra to Thunderbird please make sure that your tags are named exactly as it on Zimbra (manual sync).
If you rename a tag please update ~/.thunderbird/rnwdyrj0.default/prefs.js file manually.

incorrect variant (after successful renaming "work" to "todo" in Thunderbird):

user_pref("mailnews.tags.work.color", "#3333FF");
user_pref("mailnews.tags.work.tag", "todo");

incorrect variant :

user_pref("mailnews.tags.todo.color", "#3333FF");
user_pref("mailnews.tags.todo.tag", "todo");

Wednesday, May 20, 2015

Install java plugin in FireFox on Ubuntu


Based on http://www.oracle.com/technetwork/java/javase/manual-plugin-install-linux-136395.html
I presume that JAVA_HOME is defined , if not substitute it with real path.

On Ubuntu:

cd /usr/lib/mozilla/plugins
sudo ln -s $JAVA_HOME/jre/lib/amd64/libnpjp2.so .

recheck: 

/usr/lib/mozilla/plugins $ ls -la
total 8
drwxr-xr-x 2 root root 4096 May 20 11:57 .
drwxr-xr-x 3 root root 4096 Oct 29  2014 ..
lrwxrwxrwx 1 root root   37 May 13 12:43 flashplugin-alternative.so -> /etc/alternatives/mozilla-flashplugin
lrwxrwxrwx 1 root root   42 May 20 11:57 libnpjp2.so -> /opt/jdk1.8.0_45/jre/lib/amd64/libnpjp2.so


Launch Java app from web and on jnlp usage please select javaws:





ATTENTION: if no javaws is selected - use that window by option "Other ..." to select application at $JAVA_HOME/bin/javaws (example: /opt/jdk1.8.0_45/bin/javaws)

Saturday, May 16, 2015

Sevntu Checkstyle release 1.13.0

Braking backward compatibility:
VariableDeclarationUsageDistanceCheck was moved to Checkstyle library . Author: Roman Ivanov
AbbreviationAsWordInNameCheck is transfered to Checkstyle library . Author: Roman Ivanov
 
 
New:
New Check RequiredParameterForAnnotationCheck. Fixes #321 . Author: Andrew Uljanenko
New Check UselessSuperCtorCallCheck implemeted. Fixes #186 . Author: Alexey
New Check EmptyPublicCtorInClassCheck implemented. fixes #291 . Author: Alexey
New Check NameConventionForJunit4TestClassesCheck implemented. Fixes #80 . Author: Alexey
New Check UselessSingleCatchCheck implemented. issue #267 fixed . Author: Alexey
New Check: AvoidConditionInversion . #112 . Author: alexkravin
New Check: NestedSwitchCheck - Checking if there is no switch block inside other switch block.  Issue #249. Author: Damian Szczepanik
 
Bugs:
fix for NPE in ConfusingConditionCheck . Author: melodicwing
Fixed false positive in check ForbidReturnInFinallyBlockChech. Fixes #269 . Author: Andrew Uljanenko
Diamond operator - fixed violations on anonymous classes . Issue #270 . Author: alexkravin
 
 
Miscellaneous:
Refactored prefixes, changed indentation of switch-case, issue #298 . Author: alexkravin
Moved test-input folder from src/test/java to src/test/resources . Author: alexkravin
no java6 support since 6.2 release . Author: Roman Ivanov
version 1.13.0 bump, java6, fixes for errors of m2e . Author: Roman Ivanov
Updated maximum line length to 100 symbols . Author: Daniil Yaroslavtsev
Small code changes and extended UT's. Fixes #317 . Author: Andrew Uljanenko
New test case added. Fixes #275 . Author: Andrew Uljanenko
New test case added. Fixes #274 . Author: Andrew Uljanenko
Extended javadoc, UTs. Fixed #264 . Author: Andrew Uljanenko
 
Code coverage improvement . Author: Damian Szczepanik
Short-circuit logic is used in boolean expressions, issue #169 . Author: Michal Kordas
Code coverage improvements for ReturnCountExtendedCheck . Author: Damian Szczepanik
Improved AvoidHidingCauseExceptionCheck code coverage . Author: Damian Szczepanik
Improved UselessSingleCatchCheck code coverage . Author: Damian Szczepanik
Improved AvoidDefaultSerializableInInnerClasses code coverage to 100% . Author: Damian Szczepanik
Fixes several Sonar violations . Author: Damian Szczepanik
Remove unused input for AvoidHidingCauseExceptionCheck . Author: Damian Szczepanik
Increase AvoidNotShortCircuitOperatorsForBooleanCheck code coverage Pull #324 . Author: Damian Szczepanik
Moved static duplicated methods into Utils class . Author: Damian Szczepanik
Sonar violation: Tabulation characters should not be used . Author: Damian Szczepanik
improved InterfaceTypeParameterNameCheck code coverage . Author: Damian Szczepanik
Improved UnnecessaryParenthesesExtendedCheck test coverage . Author: Damian Szczepanik
Fixed Sonar critical and major violations: - "static final" arrays should be "private" - "for" loop incrementers should modify the variable being tested in the loop's stop condition - Declarations should use Java collection interfaces such as "List" rather than specific implementation classes such as "LinkedList" . Author: Damian Szczepanik
Run cobentura with mvn check or mavn install and fail if code coverage criteria are not met. For new files we expect to have 100% code coverage, for older one that do not pass this rule we added some rules . Author: Damian Szczepanik
Added Utils class that has one method for reporting unsupported AST token . Author: Damian Szczepanik
Removed duplicate check for the empty list: first condition isEmpty() but then for-each which also skips iteration if list is empty . Author: Damian Szczepanik
Increase OverridableMethodInConstructorCheck code coverage: - Added more test cases (examples) - Replaced to Serializable/Clonable stirngs to Cloneable.class.getSimpleName() - Increase coverage by using true and false values of checkCloneMethod and checkReadObjectMethod . Author: Damian Szczepanik
MapIterationInForEachLoopCheck - Converted strings to .class.getName() - moved initialization to beginTree method . Author: Damian Szczepanik
Improved test coverity of EitherLogOrThrowCheck by: - adding new test with inner class - invoking setters method by setting attributes in test scenario . Author: Damian Szczepanik
Improved test coverity in ConfusingConditionCheck by setting default parameters in test case . Author: Damian Szczepanik
Improved coverage of AvoidDefaultSerializableInInnerClasses by importing full qualified class name Serializable (use java.io.Serializable) . Author: Damian Szczepanik
Improved test coverity of class ChildBlockLengthCheckCheck by adding test for tested class . Author: Damian Szczepanik
Improved CustomDeclarationOrderCheck - Added tests to increase coverage - Fixed rule in test method - Fixed formatting (indents) . Author: Damian Szczepanik

Saturday, April 11, 2015

Should logger be private static or not

Just read - http://wiki.apache.org/commons/Logging/StaticLog

decision is full of trade off but, that article is the best, so I am referencing it in my blog to let me quickly find it, as that question re-appear again and again.

My position:
Logger should never be public or protected and even share-able with or accessible by others classes, eventually will be enforced by Checkstyle - https://github.com/sevntu-checkstyle/sevntu.checkstyle/issues/286

Tuesday, March 24, 2015

Oracle listener is stopped unexpectedly on Xubuntu 14.04

Oracle listener is always launched on startup, I did not nothing special to stop it, but during work I found that listener is not started ....

15:01 $ lsnrctl status

LSNRCTL for Linux: Version 11.2.0.2.0 - Production on 24-MAR-2015 15:01:40

Copyright (c) 1991, 2011, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
TNS-12541: TNS:no listener
 TNS-12560: TNS:protocol adapter error
  TNS-00511: No listener
   Linux Error: 111: Connection refused
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rivanov)(PORT=1521)))
TNS-12541: TNS:no listener
 TNS-12560: TNS:protocol adapter error
  TNS-00511: No listener
   Linux Error: 111: Connection refused



15:01 $ lsnrctl start

LSNRCTL for Linux: Version 11.2.0.2.0 - Production on 24-MAR-2015 15:12:35

Copyright (c) 1991, 2011, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.2.0 - Production
Start Date                24-MAR-2015 15:02:20
Uptime                    0 days 0 hr. 10 min. 15 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Default Service           XE
Listener Parameter File   /u01/app/oracle/product/11.2.0/xe/network/admin/listener.ora
Listener Log File         /u01/app/oracle/product/11.2.0/xe/log/diag/tnslsnr/vdu-rivanov/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=rivanov)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=rivanov)(PORT=8088))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
  Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "XE" has 1 instance(s).
  Instance "XE", status READY, has 1 handler(s) for this service...
Service "XEXDB" has 1 instance(s).
  Instance "XE", status READY, has 1 handler(s) for this service...
The command completed successfully


BUT SQLDeveloper still have no ability to connect to local instance.
Java still cannot connect to ORACLE XE on "mvn clean test"


15:10 $ sudo /etc/init.d/oracle-xe restart

after that command all works fine