Monday, 22 December 2014

Meditating about the self directed I.T company

It is probably this times that we are living now that the "Agile method" to develop software has become in my "modest" opinion, one of the most important topics that all the I.T professionals without exception, need to understand if we want to build a successful, prosperous, rational, healthy, ethic, diverse... software development industry.

The eleventh principle of the "Agile manifesto" says:

"The best architectures, requirements, and designs emerge from self-organizing teams"

Self organization is such a broad topic, that covering it in a blog post, or even on a book would probably not be enough.

What I want to do in this brief post, is just share some thoughts that hopefully will transmit to the readers some curiosity about the huge potential i believe self-organized teams have, for not just building great software, but also for building great self-directed companies.

                                                                       

Given an stimulus of some sort(e.g. challenge,threat,desire,problem,need...), either from within or from the outside, a self-organized team will increase its awareness and will react to it:

  • Feelings for gathering information related to the stimulus will arise.
  • Need for requirements will start to exist.
  • Interesting doubts and questions both technical and non-technical will bloom.
  • Debate will take place.
  • Priorities will be decided in consensus.
  • Interaction with other teams will occur if necessary(more stimulus will be created).
  • Actions will be suggested by team/s.
  • Team/s decisions will be made.
  • Slowly but unstoppable, a self directed organization will start moving in as many directions as its collective mind considers and software will start emerging.
  • Feedback will arrive, the self-directed organization will use it and will keep moving.

                                                                       

The company that is composed of self-organized teams is capable of moving in multiple directions at the same time, without the need of central management or a central financing bodies. We say that the company is self-directed.

Self-organized teams are also self-created, the individuals can choose to join or leave a team whenever they want, and even hiring  is their responsibility. In fact the teams keep changing shape continuously. Exactly the same principle applies to every single aspect, even vacations. Imagine having as much vacation as you would like... We work for living we don't live for work!

In this type of company every individual has a salary and also an additional reward upon completion of team goals which is determined by the teams gentlemen agreements. This reward is not necessarily cash, it will also be equity ownership. The company will end up being owned by their employees.

If a team fails for whatever reasons its goals, the overall impact for the company would be minimal and for the individual it would not be harmful at all and even in the worse case scenario, the team members can either decide to build something else, or incorporate themselves to other teams.

This is for me a self-directed company and in my opinion, the company of the future.
Just for finishing, one beautiful quote that I think describes very well the spirit of teamwork, and also is useful to lower big individualistic egos ;)

"None of us would be something without the rest and the rest would not be something without each of us"


Saturday, 20 December 2014

Acceptance Testing

Acceptance testing its an important part of the software development process.
This types of tests are focused in testing the scenarios that are valuable for the business.  Often acceptance tests are written with a life specification framework such as JBehave, Fittnesse, Cucumber...

When testing business value the developer, needs to make sure that has understood the acceptance criteria that the business has interested in having tested.

In some companies, the acceptance criterias/specifications are prepared in a planning session prior to the development cycle, in others it is up to developers,testers and business analysts to on spot decide what needs to be acceptance tested and why.

The important thing when acceptance testing, is to express "the whats" and not "the hows". In other words focus on the overall functionality under the part of the system that is under test and not in the deep detail.

about their scope
Sometimes development teams forget that the acceptance tests are there not just to be evaluated automatically at build time, at the end of the day it will be Business Analysts, Quality Assurance teams or other development teams who will read them, to understand what the software does. That's why they need to be concise.

When it comes to the scope of an acceptance test, there might be different points of view among different developers(in occasions technical debt could also be acceptance tested regardless of not directly hold business value), but in my opinion, acceptance tests should focus on the functionality of a valuable module(not every part of the system is business valuable). When I say module, I refer not necessarily to a hold application, but to a minimum part of the system that holds some business value.

The topic of the scope of acceptance tests can in occasions be a bit controversial to debate in many teams and this is often due to flaws in the working process or the monolithic nature in their software. In ideal scenarios we should find acceptance tests tackling service layers rather than system addons(e.g GUI, Databases, etc...)

In my opinion acceptance testing should not involve remove iteractions, instead this tests should serve themselves from plenty of Mocks, Stubs, Primers, Emulators, etc... in order to be able to focus in the main functionality described in the specification, that needs to be tested.

cons
Acceptance testing  requires dextry for develop, maintain and enhance our own domain specific text harness.

 Also, just in my opinion, as per my personal professional experience(part of it in the gambling industry) in many occasions the non deterministic nature(e.g probabilities & statistics) in which software behaves could make acceptance testing very complex. That is why, it is key to pick wisely the scenarios to test and also the edge cases.

an example
Next I will show a trivial example where I will isolate and acceptance test just a part of an application which is believed to hold some business valuable. To do so I will stub all its external dependencies. We should not integration test dependencies, we should stub them and assume they work.

Let's first look at the project structure and understand what is that we are testing:

In this example it is "SomeServiceAdapter", that holds business value and we decided to write an acceptance test for it. As we will soon see the other two adapters, represent access to remote systems which are out of our control.

The "UpstreamSystemAdapter", could for example be a controller for a GUI or maybe a Rest endpoint that is used to gather data for processing.
The "TargetSystemAdapter" could for example be the entrance to a persistence layer or rest client that forwards the result of processing to another system... Whatever those dependencies are, we don't care.

Initially when we write an acceptance test nothing exists, and we need to draft our requirements by creating new classes that will represent what we want to test, and also what we want to stub.

Many developers and also frameworks, like expressing the acceptance test in a common format, known as "The Given, When, Then format". It is just a more visually friendly way of understanding a well known testing pattern called "Arrange, Act, Assert". In other words, this pattern what they try to do is helping the developer writing the test, think about the Inputs/Premises(Given/Arrange) that are passed to some action in the code under test(When/Act) and the expected results(Then/Assert).
But we not necessarily need to follow that pattern, the important thing is that we make a concise and readable test. By the way, note that I am doing this in plain Java, without any framework, my goal is just to show a demo of how acceptance test could be written, but in real life you probably would like to write that code using your favourite live spec tool so you can get a beautiful output in some html page(e.g frameworks: Yaspec, Cucumber, Spock, JBehave, Fit, Fitnesse...). Also if you use a build tool such as Jenkins or TeamCity you should be able to nicely visualise your tests.

In the following simple example, we can see an acceptance test that tests "SomeServiceAdapter" and at the same time, stubs the dependencies.

public class SomeServiceAcceptanceTest {

    private UpstreamSystemStub upstreamSystemStub = new UpstreamSystemStub();
    private TargetSystemStub targetSystem = new TargetSystemStub();
    private SomeServiceAdapter someService = new SomeServiceAdapter(upstreamSystemStub, targetSystem);

    @Test
    public void shouldCalculateTheResultGivenTheReceivedDataAndPassItToTheTargetSystem() throws Exception{
        upstreamSystemStub.sends(asList(1, 2, 3));
        someService.calculate();
        targetSystem.hasReceived(6);
    }
}

Note how the class under test has the dependencies passed to its constructor. Also note that the types defined as parameters in the constructor are also interfaces, which are implemented by both the real classes that represent the dependencies, and their respective stubs(This way we make sure that the stub fulfils the contract with what the dependency does in reality).

One of the stubs:
 public class TargetSystemStub implements TargetSystem {  
   private Integer result;  
   @Override  
   public void receivesData(Integer result) {  
     this.result = result;  
   }  
   public void hasReceived(int expected) {  
     assertThat(result, is(expected));  
   }  
 }  

The other stub:
 public class UpstreamSystemStub implements UpstreamSystem {  
   private List<Integer> data = new ArrayList<Integer>();  
   @Override  
   public List<Integer> data() {  
     return data;  
   }  
   public void sends(List<Integer> values) {  
     data.addAll(values);  
   }  
 }  

Once the test is red, we can start implementing the production code. Important to mention, that this is just a very trivial example where the production code is so simple that the production code does not require to enter a TDD cycle  but in many cases, getting to see the acceptance test green, would require also to TDD each of the bits and pieces that enable the function called from the "when" to be properly tested. Just as a side note, when that is the case we also refer to that approach as ATDD(Acceptance Test Driven Development), it involves multiple TDD cycles prior to the completion of a business valuable acceptance tests.

Here just the production implementation of the class:
 public class SomeServiceAdapter implements SomeService {  
   private final UpstreamSystem upstreamSystem;  
   private final TargetSystem targetSystem;  
   public SomeServiceAdapter(UpstreamSystem upstreamSystem, TargetSystem targetSystem) {  
     this.upstreamSystem = upstreamSystem;  
     this.targetSystem = targetSysteabout their scopepublic class UpstreamSystemStub implements UpstreamSystem { private List data = new ArrayList(); @Override public List data() { return data; } public void sends(List values) { data.addAll(values); } } a public class UpstreamSystemStub implements UpstreamSystem { private List data = new ArrayList(); @Override public List data() { return data; } public void sends(List values) { data.addAll(values); } } a m;  
   }  
   public Integer calculate() {  
     Integer result = upstreamSystem.data().stream().reduce(0, (n1, n2) -> n1 + n2);  
     targetSystem.receivesData(result);  
     return result;  
   }  
 }  

I guess each developer has its own technique when writing acceptance tests, I just want to mention that I recently show somebody who starts writing his acceptance tests from the "then" and I thought that was a very interesting approach, because he said that doing it that way can focus more in what exactly is expected from the system that is about to be developed, but as I said it is up to each to decide how you like writing your acceptance test, just remember that it is about the "what" and not about the "how" also pick your battles and build test harnesses(avoid integration testing as much as you can)

Here the link to the complete source code: git acceptance testing example

YOLO! :)

Thursday, 11 December 2014

Picture with a celebrity

Happy to have the priviledge to take a picture with one of the celebrities of the world of software development, R.C Martin. The event was held once more in central London in one of the many fabulous skyscrapers at "The sun" offices. Great atmosphere, drinks and pizza to make the huge crowd of IT professionals comfortable. Martin in a great speech counter attacked recent critics that were made to the Test Driven Development approach by David Heinemeir Hansson. Once more a great talk by one of the most important figures of modern software development.

Myself with R.C Martin(a.k.a Uncle Bob)

Monday, 8 December 2014

Tip for quicker coding in IntelliJ

Did you know that with IntelliJ you can use Live Templates, to quickly insert predefined snippets of code. This can speed up your coding...

All you have to do is create templates for anything that you want, assign an abbreviation to it and the editor will do the rest for you just by typing the abbreviation.
Here an screen that shows, where in the IDE you can find this feature:

  

This is a cool one I like very much. You just type 'test' then hit tab and you will get a cool fresh test method :)

 @Test  
 public void $END$() throws Exception {  
 }   

Fore more detailed info about live templates, you can read this nice article at JavaCodeGeeks.com

Wednesday, 29 October 2014

Indaface!



Recently my team leader gave me an S.L.A.P, hehehe...
Don't worry this is not a case of bullying, it just a way of refering to an important Object Oriented Principle known as the Single Level of Abstraction Principle.
As Usual at the office paring on an story, when we got to the refactoring bit, I was told to improve a method that had some ugly conditional logic on it, and also some duplication.
Instead of removing the duplication, I just delegated all the problematic part to another method, so the original method would look smaller and cleaner, but...
He said to me: Do you think that code you just delegated is located now at a different level of abstraction?... Indaface! Violation of S.L.A.P
Below, an example that somehow recreates todays funny situation :P Sorry, can't show you the real code(Dont wanna get in trouble :) ).

 /*  
     We have a vault that its being populated and categorized but there is a little 
     duplication issue when populating the vault with CAT-3 items. Items withGrook() 
     are definitely CAT-3, but those withTrook() are only categorized as CAT-3 if 
     klop.isHigh().   
 */  
    public Vault stuff(Klop klop, Vault vault) {  
     vault.include("CAT-1", azra());  
     vault.include("CAT-2", khy());  
     vault.include("CAT-3", things.stream().filter(withGrook()).
     map(toSponge()).map(toPlik(klop)).collect(toList()));  
     if(klop.isHigh())  
       vault.include("CAT-3", things.stream().filter(withTrook()).
       map(toSponge()).map(toPlik(klop)).collect(toList()));  
     return vault;  
     }  
 /*  
     At the beggining we may feel tempted to extract that vault.include, to its own
     method which will be specific to CAT-3 items, regardles that the stuff method looks
     shorter, the problem is that we did not remove the duplication plus we are disrespecting
     the S.L.A.P principle.  
 */      
      public Vault stuff(Klop klop, Vault vault) {  
           vault.include("CAT-1", azra());  
           vault.include("CAT-2", khy());  
           includeCat3Items(vault,klop);  
           return vault;  
        }  

   private void includeCat3Items(Vault vault, Klop klop) {  
           if(klop.isHigh())  
             vault.include("CAT-3", things.stream().filter(withGrook()).
             map(toSponge()).map(toPlik(klop)).collect(toList()));  
             vault.include("CAT-3", things.stream().filter(withTrook()).
             map(toSponge()).map(toPlik(klop)).collect(toList()));  
   }  
  
  /*If we want to respect S.L.A.P, in this case what we need to extract, is just the 
    changing part, and delegate the condition to check if klop.isHigh, to the delegate
     method.  
  */  
     public Vault stuff(Klop klop, Vault vault) {  
     vault.include("CAT-1", azra());  
     vault.include("CAT-2", khy());  
     vault.include("CAT-3", things.stream().filter(ook(klop)).
     map(toSponge()).map(toPlik(klop)).collect(toList()));  
     return vault;  
   }  

   private Predicate<String> ook(Klop klop) {  
     return klop.isHigh() ? withGrook(): withTrook();  
   }  

 /*  
     The final refactor could even go one step further by extracting implementation 
     detail into a plu(Klop klop) method  
 */      
   public Vault stuff(Klop klop, Vault vault) {  
     vault.include("CAT-1", azra());  
     vault.include("CAT-2", khy());  
     vault.include("CAT-3", plu(klop));  
     return vault;  
   }
  
   private List<Object> plu(Klop klop) {  
     return things.stream().filter(ook(klop)).map(toSponge()).map(toPlik(klop)).collect(toList());  
   }  
   private Predicate<String> ook(Klop klop) {  
     return klop.isHigh() ? withGrook(): withTrook();  
   }  


This post is dedicated to L.K, thanks for the patience :)

Tuesday, 23 September 2014

The Add-Delete ratio as a metric of code quality

Recently after reading some articles about software quality metrics, I started thinking a lot about what would be for me a metric that really could reflect the quality of the code in the system I am working in.

I am not a big fan of numbers but in the same way of an altimeter is a useful thing you want to look at when you are driving a plane, a good code quality metric could give you warning signs about system that you are building.

In page 15 of the book "Clean Code" by R.C Martin, it says that the only valid measure of code quality is, "What The Fucks per second".



Couldn't agree more, but the only problem with this metric is that is not easy to reflect in a chart so all others can see it and therefore have an idea of how bad the system really is...

Time ago while talking to a colleague here in London, we came to the conclusion that a really good metric to reflect the quality of the code we write, is the "Added-Removed code ratio".

So this is basically the ratio of added to removed lines of code in your system. 
To measure this, what you do is:

1 - At a point in time, take your system as a whole and count the amount of lines of code, of the target language, lets say Java(Just java, dont take xmls and other configuration files...).

2 - At a future point in time, take again the same system, and count the lines again.

3 - Write down the ratios and take note of the percentages the amounts of line are increasing, or decreasing and present the data in some chart.

 This metric if tracked often, such as the altimeter, will warn your department about when you need to invest more time in technical debt, and when your system is sustainable and you can keep pushing features.

I guess you wonder, how do I know this is right? 
Well, it is very simple, we just need to understand the definition of re-factoring which is:

"a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior."

Re-factoring its very important, because the more understandable the system is, the better quality it will have. So it is not about just the smaller your code is, but about tracking the amount of effort you expend in adding and removing(specially removing) stuff from it.
- In a system where we see that the remove ratio numbers across time are going in a down trend(smaller gap between added and removed lines), it means that the developers are taking care and the overall  quality is improving.

- In a system where we see that the remove ratio numbers across time are going in an up trend(big gap between added and removed lines), it means that the teams are not investing enough time in improving the quality of the software they make.

I hope you liked the post, I appreciate feedback, this is just my opinion.



Monday, 14 July 2014

Helping methods indentify their calling threads with ThreadLocal class

In multi threaded applications in occasions we want to identify the thread that called certain method.
We could pass some additional parameter for that purpose to the method we are calling, but that wouldn't be very elegant.
The class ThreadLocal allows us to manage an object within the current Thread scope.

Let's have a look at an example...

The class below called sample Thread, uses some service and that service has interest in knowing who called it.
So What we do is create some context with come kind of unique identifier(e.g ThreadName,UUID...) and then we give that context to the class ThreadLocal to handle it(We will create an special wrapper class to hold ThreadLocal).

 public class SampleThread extends Thread {  
   @Override  
   public void run() {  
     ThreadContext threadContext = new ThreadContext();  
     threadContext.setId(getName());  
     ContextManager.set(threadContext);  
     new BussinessService().bussinessMethod();  
     ContextManager.unset();  
   }  
 }  

The context could be anything we want, but in this example I will be using the Thread name.

 public class ThreadContext {  
   private String id = null;  
   public String getId() {  
     return id;  
   }  
   public void setId(String id) {  
     this.id = id;  
   }  
 }  

The class ContextManager will act as a container that will allow us to access and modify the context of the Thread.


 public class ContextManager {  
   public static final ThreadLocal threadLocal = new ThreadLocal();  
   public static void set(ThreadContext context) {  
     threadLocal.set(context);  
   }  
   public static void unset() {  
     threadLocal.remove();  
   }  
   public static ThreadContext get() {  
     return (ThreadContext) threadLocal.get();  
   }  
 }  

With this infrastructure in place, the Service can identify the calling thread.

 public class BussinessService {  
     public void bussinessMethod() {  
       ThreadContext threadContext = ContextManager.get();  
       System.out.println(threadContext.getId());  
     }  
 }  

Finally just to demo the concept we could just create a couple of Threads and run them to see how they are identified by the services via their context.

 public class Main {  
   public static void main(String args[]) {  
     SampleThread threadOne = new SampleThread();  
     threadOne.start();  
     SampleThread threadTwo = new SampleThread();  
     threadTwo.start();  
   }  
 }