Tag Archives: career

What Comes Next

In my previous post I explained why I left JavaEE behind. Now, I’m going to explain my reasoning process to come up with what I’ll learn, teach and use next. The criteria I’m using are:

  • Cloud friendly: the technology should be ready to scale horizontally, without constraints, additional products or exponential use of resources.
  • Learning curve: I should be able to learn and teach fast even if it requires me to change the way I think about programming. I recognize I have lots of new concepts to learn before I realise the advantages of other technologies.
  • Performance: everything I write I want to be faster than any interpreted language. I know that premature optimization is a bad idea, but I need a technology that even when I decide to postpone optimizations it will be fast enough.
  • Community: the community doesn’t need to be big, but it should be active and kind with newcomers. The majority of libraries they produce should be open source.
  • Reusability: I should be able to reuse the libraries I’m used to, or find equivalent ones.
  • Coverage: I should be able to write the same kind of software I’m used to and not be limited if I decide to do more.
  • Documentation: the technology should be well documented, with books, websites, blogs, wikis and teaching materials.
  • Development Stack: the stack should implement MVC for web applications, database migration, database abstraction, SSL, authentication, authorization, REST web services, etc.

The fundamental choice starts with the programming language. It must support functional programming in order to be cloud friendly, but it also has to avoid mutable state to prevent concurrency issues and make it difficult for programmers to write code with side effects. I already use Java, but at this point we have to eliminate it because mutable state is the default behaviour in the language. To avoid it, we have to write a whole bunch of additional code that needs to be tested as everything else. Java requires the use of design patterns to overcome the deficiencies of the language. Static analysis tools, such as SonarQube, are required to keep the code safe, but, unfortunately, they require a significant effort that has nothing to do with the business problem we are solving.

“Most people talk about Java the language, and this may sound odd coming from me, but I could hardly care less. At the core of the Java ecosystem is the JVM.” James Gosling (2011, TheServerSide)

Despite the Java programming language being out of the picture, the Java Virtual Machine (JVM) is still relevant. It’s a portable and mature platform capable of running multiple programming languages in several operating systems with a transparent memory and thread management, with performance peaks compared to C/C++. What Java doesn’t do for us, other programming languages do, running in the same virtual machine and reusing the existing Java ecosystem, which is huge! Therefore, languages that have their own compilers and independent virtual machines are discarded because they hardly will reach the maturity and popularity of the JVM and the Common Language Runtime (CLR). So, at this point we discard Erlang, Harskell, Go, and all other languages that don’t run in the JVM or CLR.

I have mentioned CLR, the .net runtime, but I have no experience with it so far. So, I have to narrow my choices to JVM hosted languages. I couldn’t find any official  or reliable survey about the popularity of JVM languages, but I did find several pools showing that Groovy, Scala, Clojure and JRuby are indeed the top four JVM hosted languages, in no specific order.

scala-groovy-clojure

Groovy‘s popularity is due to the fact that it looks very much like Java, but without its cumulative historical problems. Therefore, Groovy’s learning curve for Java developers is by far the lowest one compared to Scala and Clojure. Scala comes next with its richer type system and object orientation. We’re able to map our Java knowledge within Scala, but this language is so full of possibilities that it’s the hardest one to master. We have problems to read other people’s code because developers have too much freedom to express themselves. Clojure, on the other hand, is the hardest one to start programming because of its radical differences from Java, but it’s the easiest one to master because of its simplicity. We do a lot more with less code and the code is readable as long as you know functional programming principles. Since JRuby didn’t perform well in the surveys above, I’m discarding it for the moment.

The chart bellow shows job trends in the US, according to indeed.com. It actually reflects the size and influence of the community gravitating those languages. Groovy has been performing well since the beginning, but it is now threatened by Scala, although it isn’t clear yet which one will stand out. The interest for Clojure is constantly and shyly increasing, as functional programming becomes popular and the learning material available helps to reduce the introductory learning curve. In any case, it still has a long way to go.

Groovy, Scala and Clojure have at least the same coverage as Java, with the advantage of writing less to do more. There is absolutely no problem that can be solved only by using Java. Actually, concurrency problems are far more complex to solve with Java, making those alternatives much more interesting.

In order to master those programming languages, I had a look at the volume and quality of the documentation available. This is very hard to measure. For those who like numbers, I have compiled the following table:

Language Appeared Google StackOverflow
Groovy 2003 602K 11468
Scala 2003 1.510K 35207
Clojure 2007 350K 9278

The problem is that this table can be interpreted in many different ways:

  1. These numbers are far from precise. They change everyday because of the nature of the internet.
  2. Looking at the volume, we might conclude that the more entries we get the more documentation we can find, but it can also be a sign of complexity, taking a lot more documentation to explain a thing. Therefore, the fact that Clojure has less entries doesn’t mean it is less documented than Scala or Groove.
  3. Some languages are older than others, accumulating exposure to the community, thus producing more content. But in this case, older content are counted but hardly relevant nowadays.

I can say that the documentation I found was fairly good enough to address all my questions so far.

The last point is the development stack, a set of libraries and frameworks covering most of the needs of a regular enterprise developer. The following table shows a non-exhaustive list:

Feature Groovy Scala Clojure
Build Tool Gradle SBT Leiningen
Persistence Grails Slick HoneySQL
Database Migration Grails Play Framework Joplin
MVC Grails Play Framework
Lift
Compojure + Ring
Security Spring Security SecureSocial
Silhouette
Buddy
Testing Spock Scala Test Expectations
IDE Support IntelliJ
Eclipse
IntelliJ
Eclipse
IntelliJ
Eclipse
LightTable
Emacs
NightCode
Vim
RESTful Web Services Grails Spray
Play Framework
Liberator

Notice that Grails appears several times in the Groovy column. It’s a web framework offering a good deal of productivity thanks to a Convention-over-Configuration approach. The same happens in the Scala column with several occurrences of Play Framework. While the approach followed by Groovy and Scala offers more productivity and reproducibility, it also reduces the flexibility of the architecture, making it hard to replace an inefficient part by a more efficient one. Clojure is more concerned about the architecture and offers a separate library for every feature. In case a competitor library becomes more efficient than the one we are using, we can easily integrate the new library and gradually replace the inefficient one.

My strategy to use those three technologies from now on is the following:

  • Groovy: when I find a chaotic and inefficient JavaEE application, I will propose to migrate it to Groovy + Grails. It will make the project economically viable again and recover the time wasted with complexity. The team can start writing Groovy code right away in the same project, gradually replacing the Java code and JavaEE dependencies.
  • Scala: the main advantage of the Scala stack is its reactive platform, offering an unprecedented performance boost for concurrent applications. So, when performance is one of the main requirements of the application and the team is smart and organized, I will suggest Scala as the way to go.
  • Clojure: For everything else I will suggest Clojure, which is very productive, simple and has an excellent performance. That’s by far the best programming experience I ever had.

In summary, I still use JavaEE for existing well designed applications, but I will use Groovy to save chaotic JavaEE applications from complete failure. Scala and Clojure will be used for new projects, depending on their characteristics and context of use.

On The Illusion of Controlling People’s Time

Software Engineering is going through an identity crisis. The old school has failed because of too much control, moving people away from what they really have to deliver. The new school, or agile fever, is more focused on humanities, directing all the attention to human beings. This approach practically decimated the old school, showing that writing software is not an exact science. It is actually unpredictable and cannot be done without creativity, inspiration and transpiration. Maybe the old school, which insists to exist, got confused by the fact that software execution, data processing, and system and hardware design are engineering stuff, but it doesn’t mean that developing those stuff is also engineering. On the contrary… it’s human and social.

When I see managers trying to measure the time spent on people’s daily activities, I understand they are taking into consideration only the transpiration of their teams and completely ignoring the creativity and inspiration. These two last aspects are so subjective that it’s hard to compare their manifestation on different people. How to compare artists? We may find artists’ comparisons out there, but they heavily rely on who is producing the comparison. Let’s ask ourselves: why on earth would someone use quantitative criteria, such as time and money, to measure and compare subjective matters?

Measurements are only useful when we’re capable of making comparisons with them in an objective way. We compare the date of our birth with the calendar to know our age. In times of crisis, we pay attention to our water, gaz and electricity consumption and make comparisons, finding ways to save money. Climate is measured because we need to decide which clothes to wear or which precautions to take by comparing with previous experiences or common sense. Notice we put measured subjects into perspective to understand the real dimension, the scale, the direction, and then we’re able to reason about trends, behaviours, issues, patterns, etc.

When it comes to subjective matters, we might be able to make comparisons, but measurements are just useless in this case. Actually, we can come closer with a lot of statistical methods, multi-variable analysis, multi-criteria craziness, but do we really have time and resources to produce all these analyses in a daily basis, and even take the risk of being probabilistically far from what it’s expected? It sounds complicated, isn’t it? That’s because it is. 🙂

In software development, every activity is different from one another. It’s impracticable to precisely compare two activities even if they look the same and are done by the same person. It’s all subjective, otherwise it would be easy to put the project on schedule by adding more people in the team. We know that since 1975, when Frederick Brooks published “The Mythical Man-Month“. So, when a manager asks their developers to inform the time spent in every activity, he/she is in fact collecting lies (or non-truths). If he/she prepares a report with those collected times, this is going to be the spread of an even bigger lie (or useless information).

But why lies? There are two good explanations for that:

  1. most people don’t track exactly how much time they spent on their activities, thus they actually inform safer figures, which are usually higher than reality;
  2. time control gives a feeling of surveillance, making developers believe their manager wants to know whether they are really working or not, thus they smartly add some extra time to mask non-productive time.

Managers should also be aware that time control also kills creativity. When developers have new ideas, potentially useful for the project, they cannot simply turn them into activities because of scope constraints or just because those ideas are not clear enough to be described. Without activities, developers cannot report their time. Therefore, if they spend too much time working on their ideas, it will look like huge time gaps for their managers.

Eventually, managers are moved to implement time control because of visible procrastination. I remember once when a new project was starting, a big room was allocated to a team of 12 people, and the project manager asked the team to choose their places in the room. There were 15 places available, 12 with monitors facing the wall and 3 with monitors facing the door. Guess which places were left available? That’s right! The three ones facing the door 😀 I remember the symphony of mouse clicks and sudden movements when the manager walked through the room. That was so funny! 😀 But time control wouldn’t solve that problem because inputing time on activities is also a distraction. Anything that is not building deliverables is a distraction. We may never eliminate them, but we can always reduce them. The solution for procrastination, in a single word, is: Motivation!

But, is time control totally out of sense? It has some sense when the work is fully based on time, which is the case of consultants payed hourly. This is about their business model, not software engineering. That’s a good practice for consultants to inform to customers how many hours they had spent because it’s directly related to their payments. Notice that this kind of service is for a short period of time and for highly specialised skills. However, we are talking about software projects that last longer.

Finding ways to motivate your team is the best strategy to put them on track and fully productive. Every form of control that threats their feelings will put them down. Motivation doesn’t need to be monetary or entertaining. Sometimes, it’s enough to show their wins, how important is their work for society, how it changed people’s lives, and why it’s important to keep doing what they’re doing.

One reason programmers dislike meetings so much is that they’re on a different type of schedule from other people. Meetings cost them more. For someone on the maker’s schedule, having a meeting is like throwing an exception. It doesn’t merely cause you to switch from on task to another, it changes the mode in which you work. – Paul Graham

Featured on Oracle’s Java Magazine

I’m glad to see I was featured on the latest edition of Oracle’s Java Magazine (Issue January/February 2013, page 67). I would like to thank Oracle on behalf of the CEJUG Community. We have been partners for so long and this is a nice way to demonstrate that you guys, at Oracle, recognise our contribution to the Java Community.

It’s important to say that they cut part of my hair using Photoshop. I’m actually more handsome than it looks like… hehehehe 😀 Joking! Back to work!

Some Interview Questions to Hire a Java EE Developer

The Internet is full of interview questions for Java developers. The main problem of those questions is that they only prove that the candidate has a good memory, remmembering all that syntax, structures, constants, etc. There is not real evaluation of his/her logical reasoning.

I’m listing bellow some examples of interview questions that check the knowledge of the candidate based on his/her experience. The questions were formulated to verify whether the candidate is capable of fulfilling the role of a Java enterprise applications developer. I’m also putting the anwsers in case anybody want to discuss the questions.

1. Can you give some examples of improvements in the Java EE5/6 specification in comparison to the J2EE specification?

The new specification favours convention over configuration and introduces annotations to replace the use of XML for configuration. Inheritance is not used to define components anymore. They are defined, instead, as POJOs. To empower those POJOs with enterprise features, dependency injection was put in place, simplifying the use of EJBs. The persistence layer was fully replaced by the Java Persistence API (JPA).

2. Considering two enterprise systems developed in different platforms, which good options do you propose to exchange data between them?

We can see as potential options nowadays the use of web services and message queues, depending on the scenario. For example: when a system needs to send data, as soon as they are available, to another system or make data available for several systems, then a message queuing system is recommended. When a system has data to be processed by another system and needs back the result of this processing synchronously, then web service is the most indicated option.

3. What do you suggest to implement asynchronous code in Java EE?

There are several options: one can post messages to a queue to be consumed by a Message-Driven Bean (MDB); or annotate a method with @Timer to define the time to execute the code programmatically; or annotate a method with @Scheduler to define the time to execute the code declaratively.

4. Can you illustrate the use of Stateless Session Bean, Statefull Session Bean and Singleton Session Bean?

Stateless Session Beans are used when there is no need to preserve the state of objects between several business transactions. Every transaction has its own instances and instances of components can be retrieved from pools of objects. It is recommended for most cases, when several operations are performed within a transaction to keep the database consistency.

Statefull Session Beans are used when there is the need to preserve the state of objects between business transactions. Every instance of the component has its own objects. These objects are modified by different transactions and they are discarded after reaching a predefined time of inactivity. They can be used to cache those data with intensive use, such as reference data and long record sets for pagination, in order to reduce the volume of IO operations with the database.

A singleton session bean is instantiated once per application and exists for the lifecycle of the application. Singleton session beans are designed for circumstances in which a single enterprise bean instance is shared across and concurrently accessed by clients. They maintain their state between client invocations, which requires a careful implementation to avoid conflicts when accessed concurrently. This kind of component can be used, for example, to initialize the application at its start-up and share a specific object across the application.

5. What is the difference between queue and topic in a message queuing system?

In a queue there is only one producer of messages and only one consumer of these messages (1 – 1). In a topic there is a publisher of messages and several subscribers that will receive those messages (1 – N).

6. Which strategies do you consider to import and export XML content?

If the XML document is formally defined in a schema, we can use JAXB to serialize and deserialize objects into/from XML according to the schema. If the XML document does not have a schema, then there are two situations: 1) when the whole XML content should be consider: In this case, serial access to the whole document is recommended using SAX, or accessed randomly using DOM; 2) when only parts of the XML content should be considered, than XPath can be used or StAX in case operations should be executed immediately after each desired part is found in the document.

7. Can you list some differences between a relational model and an object model?

An object model can be mapped to a relational model, but there are some differences that should be taken into consideration. In the relational model a foreign key has the same type of the target’s primary key, but in the object model and attribute points to the entire related object. In the object model it is possible to have N-N relationships while in the relational model an intermediary entity is needed. There is no support for inheritance, interface, and polymorphism in the relational model.

8. What is the difference between XML Schema, XSLT, WSDL and SOAP?

A XML Schema describes the structure of an XML document and it is used to validate these documents. WSDL (Web Service Definition Language) describes the interface of SOAP-based web services. It can refer to XML schemas to define existing complex types passed by parameter or returned to the caller. SOAP (Simple Object Access Protocol) is the format of the message used to exchange data in a web service call. XSLT (eXtensible Stylesheet Language Transformation) is used to transform XML documents into other document formats.

9. How would you configure an environment to maximize productivity of a development team?

Every developer should have a personal environment capable of executing the whole application in his/her local workstation. The project should be synchronized between developers using a version control system. Integration routines must be executed periodically in order to verify the compatibility and communication between all components of the system. Unit and integration tests must be executed frequently.

You can increment this set of questions covering other subjects like unit testing, dependence injection,  version control and so on. Try to formulate the questions in a way that you don’t get a single answer, but a short analysis from the candidate. People can easily find answers on the Internet, but good analysis can be provided only with accumulated experience.

My Last Day in The Lab

In November 15th this year, Matt Welsh, a young and successful tenured professor at Harvard, decided to leave his prestigious academic position to work full time for the industry. He is now a Google employee and he claims that he has more freedom and computational power than ever to make real and substantial contributions for the computer world.

I fully understand Matt, besides not having his prestigious position neither his brilliant mind. We both love our respective universities, keeping a profound admiration for what they represent for the society. However, we both also agree that sometimes our way of thinking and acting don’t fit the way the traditional research world is moved. We love freedom of thinking with a minimal influence of external factors, but we also get upset when things happen too slow because we have a lot of formalities to deal with before having our work appropriately recognized.

Normal working day at the lab registered by Koen Cobbaert

After all, my experience of doing PhD at UCL was amazing and I do recommend it for those who ask me for an advice about the academic world. In this world I’ve learned how to solve really tough problems and to practice my favorite sports: writing and coding. It was a journey of 4 years extracting the maximum of creativity, patience and emotional control. When I finished, I had a feeling that everything else is easier and I lost the fear of facing new challenges. In summary, life gets more exciting.

After this remarkable academic period, I feel the need of changing in my heart. As my adviser said once:

“we are fundamentally educators and in order to spread knowledge people should come and go elsewhere to make useful things with their privileged knowledge”.

I do agree with his statement and that’s one of the reasons I’m leaving the lab today. Because I believe it’s time to leave my sit available for another brave student who is looking for this wonderful life experience. And I should follow my path, doing what I love to do, which is work as a Java Architect in the corporate world.