Saturday, August 31, 2013

Exception logging: 'log and rethrow' vs 'wrap and throw' vs 'log context and re-throw'

If Exception happen during any SQL call to DB server - Where it is good to catch and log it and how to do it?

There is no exact answer, but there are few ways to do it, and some of them have problems that you need to consider.

Case "log and re-throw":
} catch (SQLException e) {
               log.error("Exception occurred ,for sql [" + sql + ]", e);
               throw e;
}

Case "wrap and throw":
} catch (SQLException e) {
           throw new XXXXXDataAccessException("Exception occurred ,for sql [" + sql + ]", e) ;
}

Case "log context and rethrow":

} catch (SQLException e) {
               log.warn("Exception is appear, stack-trace see further in logs, failed sql is [" + sql + ]");
               throw e;
}

Case "log and re-throw":
In log and throw - log with Error or with Warning ?
If we do it with ERROR level - you will brake good pattern "one event - one ERROR message in logs", link to read.
If you will log with WARN - you will spoil logs with duplicated print of stacktrace. On moment of logs review it will not be clear (without review of sources) either exception happen twice or it was logged twice or ....   

General rules for logging and re-throwing:
 - if you develop Library - never log with ERROR level - library does not know how it is used so he have to return all to application and application decide how and when to log it, or even ignore it. Such problem existed in Hibernate, Hibernate, Hibernate, Hibernate, and looks like resolved in Hibernate 4.X only.
- if you are in Application - do logging on level that do know how to treat/translate/process that exception and make exception part of business logic(your algorithm).

Case "wrap and throw":
If you do not log sql and trow exception up, upper level does not have information what sql was in call. You can not log sql with ERROR level at place where you have SQL as is SQL is not an ERROR , exception is a ERROR event - log and rethrow is also bad practice (link to read). Wrapping to your exception make you have custom exception objects and ask yourself and Google why there is no such standard exception - it is so obvious (ones you end up in creation set of custom exceptions in each application).

if you so like to follow "big brothers" - non of these exceptions classes store SQL:
for 15 year off massive SQL usage in java non of them wanted to have it in class as member, because SQL is only one part of problem not a exact reason. 
Non of them have sql as member in class:
http://docs.oracle.com/javase/6/docs/api/java/sql/SQLException.html
https://github.com/hibernate/hibernate-orm/tree/master/hibernate-core/src/main/java/org/hibernate/exception


Case "log context and rethrow":
So logging with WARN is ok, but smb could yell "Error should encapsulate all details of it", and here is nuance - you will never put in Exception object all context, even previous events in logs is a context as you need to see what application did before ERROR. So I recommend to log context of SQL and parameters if any with WARN level and do rethrow exception without wrap.

Main idea is - as you got exception and that level can not log it or process it properly - just print context of exception with WARN level an re-throw it further. Context could be anything: all set of class members, any local variables, whatever will be useful to know state of class when exception happen.

Make a rule for yourself to search for exception in logs and then view events before ERROR event to see a context of problem not only be cause you did logging of exception context but due to the fact that previous events are also context , and they could be even more informational and helpful than smth that you did.

Attention: You can not fix a problem by just looking at exception stacktrace and its message - you need wide context of what happen in application before!

Friday, August 30, 2013

Moving Checkstyle sources from sourceforge to github

Sources: http://sourceforge.net/p/checkstyle/code/ci/default/tree/
Destination: https://github.com/checkstyle

Follow installation steps from "How to convert mercurial to git" manual:  http://codeimpossible.com/2011/12/29/Moving-your-mercurial-repository-to-git/

Command to do clone and convert in same time:
git-hg clone http://hg.code.sf.net/p/checkstyle/code checkstyle-git

Review all participants to find duplicates:
git log --raw | grep "^Author: " | sort | uniq -c

2 Author: Bewied <BenWiederhake.......de>
4 Author: BurnOl <BurnOl@KAZ-027469>
1 Author: convert-repo <devnull@localhost>
1 Author: Lars Kühne <lkuehne>
683 Author: lkuehne <devnull@localhost>
43 Author: mstudman <devnull@localhost>
1054 Author: oburn <devnull@localhost>
9 Author: Oleg Sukhodolsky <o_sukhodolsky>
3 Author: Oliver Burn Cygwin <devnull@localhost>
135 Author: Oliver Burn <devnull@localhost>
288 Author: o_sukhodolsky <devnull@localhost>
410 Author: rickgiles <devnull@localhost>
14 Author: tschneeberger <devnull@localhost>


Create bash file that do squash of authors duplicates - change_names.sh:
#!/usr/bin/bash
   
git filter-branch --tag-name-filter cat --commit-filter '
      if [ "$GIT_AUTHOR_NAME" = "BurnOl" ] && [ "$GIT_AUTHOR_EMAIL" = "BurnOl@KAZ-027469" ]; then
              GIT_AUTHOR_NAME="Oliver Burn";
              GIT_AUTHOR_EMAIL="oliver@.......com";
              GIT_COMMITTER_NAME="Oliver Burn";
              GIT_COMMITTER_EMAIL="oliver@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "convert-repo" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Oliver Burn";
              GIT_AUTHOR_EMAIL="oliver@.......com";
              GIT_COMMITTER_NAME="Oliver Burn";
              GIT_COMMITTER_EMAIL="oliver@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "lkuehne" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Lars Kühne";
              GIT_AUTHOR_EMAIL="lakuehne@.......com";
              GIT_COMMITTER_NAME="Lars Kühne";
              GIT_COMMITTER_EMAIL="lakuehne@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "Lars Kühne" ] && [ "$GIT_AUTHOR_EMAIL" = "lkuehne" ]; then
              GIT_AUTHOR_NAME="Lars Kühne";
              GIT_AUTHOR_EMAIL="lakuehne@.......com";
              GIT_COMMITTER_NAME="Lars Kühne";
              GIT_COMMITTER_EMAIL="lakuehne@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "mstudman" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Michael Studman";
              GIT_AUTHOR_EMAIL="mstudman@.......com";
              GIT_COMMITTER_NAME="Michael Studman";
              GIT_COMMITTER_EMAIL="mstudman@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "Oliver Burn" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Oliver Burn";
              GIT_AUTHOR_EMAIL="oliver@.......com";
              GIT_COMMITTER_NAME="Oliver Burn";
              GIT_COMMITTER_EMAIL="oliver@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "Oliver Burn Cygwin" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Oliver Burn";
              GIT_AUTHOR_EMAIL="oliver@.......com";
              GIT_COMMITTER_NAME="Oliver Burn";
              GIT_COMMITTER_EMAIL="oliver@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "oburn" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Oliver Burn";
              GIT_AUTHOR_EMAIL="oliver@.......com";
              GIT_COMMITTER_NAME="Oliver Burn";
              GIT_COMMITTER_EMAIL="oliver@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "Oleg Sukhodolsky" ] && [ "$GIT_AUTHOR_EMAIL" = "o_sukhodolsky" ]; then
              GIT_AUTHOR_NAME="Oleg Sukhodolsky";
              GIT_AUTHOR_EMAIL="os97673@.......com";
              GIT_COMMITTER_NAME="Oleg Sukhodolsky";
              GIT_COMMITTER_EMAIL="os97673@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "o_sukhodolsky" ] && [ "$GIT_AUTHOR_EMAIL" = "o_sukhodolsky" ]; then
              GIT_AUTHOR_NAME="Oleg Sukhodolsky";
              GIT_AUTHOR_EMAIL="os97673@.......com";
              GIT_COMMITTER_NAME="Oleg Sukhodolsky";
              GIT_COMMITTER_EMAIL="os97673@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "o_sukhodolsky" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Oleg Sukhodolsky";
              GIT_AUTHOR_EMAIL="os97673@.......com";
              GIT_COMMITTER_NAME="Oleg Sukhodolsky";
              GIT_COMMITTER_EMAIL="os97673@.......com";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "rickgiles" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Rick Giles";
              GIT_AUTHOR_EMAIL="rickgiles";
              GIT_COMMITTER_NAME="Rick Giles";
              GIT_COMMITTER_EMAIL="rickgiles";
              git commit-tree "$@";
      elif [ "$GIT_AUTHOR_NAME" = "tschneeberger" ] && [ "$GIT_AUTHOR_EMAIL" = "devnull@localhost" ]; then
              GIT_AUTHOR_NAME="Travis Schneeberger";
              GIT_AUTHOR_EMAIL="leo.herbie@.......com";
              GIT_COMMITTER_NAME="Travis Schneeberger";
              GIT_COMMITTER_EMAIL="leo.herbie@.......com";
              git commit-tree "$@";
      else
              git commit-tree "$@";
      fi' HEAD



explanation to script:
" --tag-name-filter cat" - required to not loose tags after history rewrite.
"GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL" - to let github show commits correctly at page of commits history, in other case you have invalid-email-address in details for each commmit.

Do changes to authors and do push to remote:
./change_names.sh
git remote add origin git@github.com:checkstyle/checkstyle.git
git push -u origin master --tags

Sources of wisdom:
In filter-branch shell syntax for IF-ELSE should be used - http://www.dreamsyssoft.com/unix-shell-scripting/ifelse-tutorial.php
usage of bash syntax for IF- ELSE cause errors - http://blog.linuxacademy.com/linux/conditions-in-bash-scripting-if-statements/
http://git-scm.com/book/ch6-4.html (chapter "Changing E-Mail Addresses Globally")
http://ayende.com/blog/4532/extracting-a-list-of-committers-from-git

Monday, August 26, 2013

Rest api development best practices resources

Here is just list of resources that I found and do not want search one more time:

Разработка web API / Хабрахабр
http://habrahabr.ru/post/181988/

Designing a Beautiful REST+JSON API (and other video around)
http://www.youtube.com/watch?v=5WXYw4J4QOU

steps toward the glory of REST
http://martinfowler.com/articles/richardsonMaturityModel.html

API creation blog:
https://blog.apigee.com/
http://apigee.com/about/api-best-practices/all/ebook

API creation mail list:
https://groups.google.com/forum/#!forum/api-craft

Tuesday, August 13, 2013

Krusader have problem with unpacking tag.gz

It is not a first time I noticed that after unzip/unarchive/unpacksome archive some files are missed. But when you do it from terminal or ubuntu befault archive manager - all works fine.

issue was reported: https://bugs.kde.org/show_bug.cgi?id=323472