Tag Archives: security

Fluffy Encryption of Numbers Within URLs

In the REST web services world, a resource is anything that’s important enough to be referenced as a thing in itself. It is something that can be stored on a computer and represented as a stream of bits such as a document, a row in a database, or the result of running an algorithm. The book RESTful Web APIs explains very well that every resource has to have at least one URI, which is the name and address of that resource. The URL to a resource in a typical web application looks like {database-table-name}/{record-id}. For example: /blogs/myblog/entries/134 goes from the general to the specific, from a list of blogs to a particular blog, to the entries in that blog, to a particular entry.

I don’t know about you, but when I look at those URLs and see those record ids I feel very uncomfortable. I’m not sure if it has security implications, but it’s strange to let everybody know that they are dealing with the product number 14 or the order number 35. Maybe I’m being paranoid, but it is worth sharing a trick to replace those ids by fake ones.

The method is simple. We will need two secret numbers that should be configurable for each installation of an application. The secret numbers compose a formula to encrypt a number, as illustrated in Clojure:

(def secret-n1 345)
(def secret-n2 3)

(defn encrypt [id] 
  (bit-xor (* id secret-n1) 

(encrypt 35)
>> 12072

The id is multiplied by the first secret number (secret-n1) and the result is used to perform a binary XOR (bitwise exclusive or) with the second secret number (secret-n2). The result is a number that gives no clue about the id it is hiding. Here is an example of applying the function encrypt:

; Original URL
(def id 35)
(def url-entries "http://mywebsite.com/myblog/entries/%d")
(def url-entry (format url-entries id))
(format "Original URL: %s" url-entry)
>> Original URL: http://mywebsite.com/myblog/entries/35

; Modified URL
(def url-entry (format url-entries (encrypt id)))
(format "Modified URL: %s" url-entry)
>> Modified URL: http://mywebsite.com/myblog/entries/12072

Notice that 35 is not exposed anymore. In its place we have 12072, which is not a real id or it’s not related to the current record. But how can we recover the original id after processing the request with 12072? Now comes the decrypt function to recover the original id:

(defn decrypt [request-id] 
  (/ (bit-xor request-id secret-n2) 

(decrypt 12072)
>> 35

The secret numbers are used here as well but inverting the calculation. This time, we perform the binary XOR first, with the id that comes in the request and the second secret number (secret-n2), and we divide (instead of multiply) the result by the first secret number (secret-n1).

These ids are maybe meaningless to concern us so much, but what about a global user id or a social security number or even a phone number? This method can protect them all.

The only downside I have identified for this technique is that in the event of compromising the security of the secret numbers, all URLs will change when at least one secret number changes. It means that these URLs are not permanent, thus useless to bookmark them. This is not a serious concern, but you should think about the implications before adopting it.

Keeping Confidential Information Out Of Git

I’ve launched yet another open source project on GitHub a couple of months ago and now it’s time to use it in production. For that, I had to put some confidential information in a file that is currently tracked by Git. That was the database connection credentials that I keep in a property file. Obviously, I couldn’t push that file to GitHub and I did’t want to keep that file in my staging area forever, but I still wanted to keep the file in the repository, so people could use it to configure their own development/production databases as well.

The file I’m talking about is usi4biz/resources/db-config.edn. Notice that I’ve put pretty dumb credential information there just to make it work in my development environment. This file doesn’t exist anymore in the master branch. I had to do a little tweak, inspired by WordPress, to address the issue.

Before adding any confidential info, I renamed the file to db-config-example.edn:

$ git mv resources/db-config.edn resources/db-config-example.edn
$ git add resources/db-config-example.edn
$ git commit -m "Renamed the database configuration file"

Then, I copied the file and named it after the original name:

$ cp resources/db-config-example.edn resources/db-config.edn

I edited the new file and added the production credentials. Now, I just have to ask Git to ignore it:

$ echo -e "resources/db-config.edn\n" >> .gitignore

This way, db-config.edn will be always out of my staging area whenever I change it. I just have to remember that if I ever need to add a new property I have to do it in both files. For the moment, I just explained in the README file that those wiling to use the application have to copy the file db-config-example.edn with the name db-config.edn because the code refers the file db-config.edn.

New CEJUG Open Source Project

As I have mentioned yesterday, I’m working on a new project, managed by the CEJUG community, which aims to develop a web application for managing Java User Groups. We put it into production in the first day of the decade, January 1st, 2011, and we made the source code freely available on our java.net project.

The first goal we want to achieve is the definition of what is actually being a CEJUG member. Nowadays, we simply consider all those registered in our technical mailing as members. This simplicity is good for management purposes, but we lose lots of information because of that. We don’t know, for instance, for what reasons a member is leaving the group. Did we do something wrong? What can we do to get better and get members back into the boat? We also noticed that even non-technical people, as entrepreneurs, recruiters, and those who decided to unsubscribe because of too many messages, would like to keep in touch with the group, not necessarily going into technical discussions, but proposing other ways to help. Adopting a separate application to manage subscriptions would help us to collect more feedback and be more inclusive.

Developing our own solution can make data work in our favour and allow our sustained growth. Consequently, we are generating an additional source of knowledge for the community. This application is open source and everyone can run and see how CEJUG works. Beginners will have a solid source to start their studies on the development of Java web applications, experts can help with bug fixing, refactoring, and developing new features according to our issue tracking. Adopted design patterns may be subject of valuable and warming discussions in our community.

Of course the application was developed in Java 😉 We have the duty to write the software architecture document in the coming days, but we can already say in advance what we are using to develop and deploy the application. The presentation layer was developed in JSF 2.0, using the Primefaces component library; the business layer was implemented in EJB 3.1; the persistence layer was implemented in JPA 2.0; data is persisted in MySQL; and everything is running on Glassfish 3.0.1 Application Server. The current version was developed using Netbeans 6.9 due to its productivity when developing JEE applications. We rely on the container to manage security, database transactions, connection pools, and email sessions.

The next step is to document the application, add customisable features and internationalise it in order to spread its adoption by several other JUGs out there. We are looking for contributors and supporters to make this a successful open source project. We hope one day, we could promote interoperability between JUGs through this application, sharing mutual knowledge, events, effort on the growth of the Java platform and the Java community.

Yasmim Security: User Authentication using JDBC Realm

In the previous post, I introduced you to the Yasmim application. Today, I’m going into details on its implementation, starting from a very important non-functional requirement: the user authentication. Since I’m using a JEE6 Application server, I’m going to explain how we did it using the security infrastructure provided by the container.

    JDBC Realm
    This is a configuration entirely made in the application server. It may demand some time, but at least your application is going to be free from the basic security complexity. Yasmim uses JDBC Realm because we stored the user data in a database, which could be a third party database (single sign-on for multiple applications in the context of an organization) or the database of the application (users and groups embedded to be independent from the organizational context).

    The figure below proposes a data structure for users and groups. It is more complex and realistic than most examples found on the web, but it follows basically the same principle: a group contains several users and a user can be part of several groups. We created a N-N relationship between the tables account_group and user_account, using the intermediary table user_group.

    The script to create this structure is available in the repository. The beginning of the script creates the structure above in a MySQL database. For the purpose of this application, you can create the database with just these three tables. If you don’t know how to create a MySQL database, follow this post. You will need a connection poll pointing to this database. If you don’t know how to create a connection pool, I also wrote a post explaining this.

    The sql scripts below consider the model above and they are used to populate the tables for test purposes. The first one inserts 2 users in the database: John Smith and Mary Allen. Their password is secret but it is not readable because a MD5 algorithm was applied on it. The realm will need encoded passwords because we have specified the MD5 as the diggest algorithm in the realm configuration detailed later on. The column confirmation_code is not relevant now, so we defined it as null.

    insert into user_account values
    insert into user_account values

    Then, we insert the basic groups admins and users. The group users was defined as the default for registered users, but this attribute is not relevant either here. Maybe we can explore it in a future post.

    insert into access_group values
    ('admins','Administrative Group', null);
    insert into access_group values
    ('users','Default User Group', 1);

    Finally, we associate the users with the groups. Mary Allen is part of the groups admins and users and John Smith is part of the group users. So, Mary can do everything John can do, but John can’t do everything that Mary can.

    insert into user_group values
    ('admins', 'maryallen@acme.com');
    insert into user_group values
    ('users', 'maryallen@acme.com');
    insert into user_group values
    ('users', 'johnsmith@acme.com');

    Follow the steps below to configure the JDBC Realm:

    1. Go to the Glassfish admin console (http://localhost:4848/), navigate to Configuration > Security > Realms and press “New…”;
    2. Name: yasmim-realm – it will be used to refer to this realm in the application (tag realm-name in the file web.xml);
    3. Class Name: com.sun.enterprise.security
    4. Properties specific to this Class (using the model above as a reference):
      1. JAAS Context: jdbcRealm – identifier for the login module;
      2. JNDI: jdbc/yasmim – the data source JNDI name pointing to the connection pool where the tables are available;
      3. User Table: user_account – user table according to the model above;
      4. User Name Column: user_id – column where the user name is stored;
      5. Password Column: password – column where the password is stored;
      6. Group Table: user_group – notice that the group table is not the access_group, but the intermediary table user_group. access_group is used to store further details about the groups;
      7. Group Name Column: group_id – this is the foreign key to the access_group table, thus this is the id of the group;
      8. Digest Algorithm: MD5 – of course you are not going to save the password as it is. This is actually embarrassing for you to know the password of your users. So, let’s inform to the realm that the password was scrambled by the MD5 algorithm; and
      9. Encoding: Base64 –  the encoding used by the MD5 algorithm.
    That’s all for the server-side. Now, we are going to configure the application to use this realm.

    Application Configuration

    We have to configure the files web.xml and sun-web.xml. An example of web.xml configured for this purpose is available in the Yasmim repository. Here, some comments about the configuration:

    Since some pages are going to be protected, we’ll need an access denied page to alert the user:


    On the application side, the access to resources of the system is given to roles, that can be assumed by users and groups. Roles are declared this way:


    This security constraint states that only users with the role admin can access the content of the /admin folder. These users can perform GET and POST http methods only.

    <display-name>Admin Constraints</display-name>

    This second constraint states that only users with the role user can access the content of the /media folder. The tag url-pattern is flexible enough to consider several complex situations, but you may find some difficulty if the name of the files and folders are not so well organized.

    <display-name>Users Constraints</display-name>

    Now, we define which authentication method will be presented to the user. We chose the most common one, which is the FORM method. yasmim-realm is the name of the realm we have just created. /login.xhtml is the authentication page and /login_error.xhtml is shown in case of authentication failure.


    In addition to the web.xml file, the sun-web.xml file is used to associate the application roles with the user groups. It is a practical way to associate roles to several users at the same time. The file is located in the WEB-INF folder, together with web.xml, and its content is shown below:

    <sun-web-app error-url="">

    In this case, the role admin is associated with the group admins and the role user is associated with the groups admins and users.

    Implementation of the Authentication Page

    There are, at least, three pages to consider: login.xhtml, login_error.xhtml and access_denied.xhtml. The login.xhtml page was declared in the web.xml file as the page that collects the authentication data from the user though a standardized form. The form contains the username and password fields and the submit command to post those data to the server. The action of the form and the name of each field are pre-defined. Look at the code below:

    <form method="post" action="j_security_check">
    <h:panelGrid columns="2">
    <h:outputLabel for="j_username" value="E-mail"/>
    <input type="text" name="j_username"/>

    <h:outputLabel for="j_password" value="Password"/>
    <input type="password" name="j_password"/>

    <h:outputText value=" "/>
    <h:panelGrid columns="2">
    <input type="submit" name="submit" value="Sign in"/>
    <h:button outcome="index" value="Cancel"/>

    This is a JSF 2.0 page with Facelets, but it can be a simple HTML page too. The important thing is the form action, which is j_security_check, and the username and password fields, which are named respectively as j_username and j_password. When the user submit the data the web container will handle it in a special way without interfering in the logic of your application. If you don’t need any other fancy feature, the work is done here.

    According to the authentication and authorization data in the database and what was declared in the application configuration, Once authenticated, Mary Allen has full access to the content of the folders admin and media and John Smith has access only to the content of the folder media. To save your time, take the Yasmim source code as a starting point and explore other possibilities.