Spring Boot layered deployment

For a long time, when doing cloud-native development on Java, Spring Boot has been one of the go-to frameworks. Being a very popular and excellent option, it is not excluded from criticism. One of the biggest ones I have heard over the years is the enforcement of fat jars.

Despite the idea of having all dependencies packaged on a big fat jar, or uber jar, that you can just include in your container images sounds amazing, and it is, the truth is that it increases the release times, and while our application is composed of just a few megabytes, fat jars can be in de order of hundreds of megabytes.

As part of the 2.3 release, Spring Boot added the possibility of “Build layered jars for inclusion in a Docker image“. In their own words: “The layering separates the jar’s contents based on how frequently they will change. This separation allows more efficient Docker images to be built. Existing layers that have not changed can be reused with the layers that have changed being placed on top.“.

This feature allows to walk away from the fat jars long release times, allowing to be able to deploy an application, even, in milliseconds depending on the changes. Basically, it uses the Docker layering capabilities to split our application and its dependencies into a different layer, and only re-build them is required due to changes.

By default, the following layers are defined:

  • dependencies for any dependency whose version does not contain SNAPSHOT.
  • spring-boot-loader for the jar loader classes.
  • snapshot-dependencies for any dependency whose version contains SNAPSHOT.
  • application for application classes and resources.

The default order is dependencies, spring-boot-loader, snapshot-dependencies, and application. This is important as it determines how likely previous layers are to be cached when part of the application changes. If, for example, we put the application as the first layer, we will not be having any effect at all saving release time because every time we introduce a change all layers will be re-built. The layers should be sorted by having the least likely to change added first, followed by layers that are more likely to change.

The Spring Boot Gradle Plugin documentation contains a section explicitly referring to this point that can be located here. And, it should be enough to start experimenting with the feature, but as you all know, I am a hands-on person, then, let’s make a quick implementation.

The first thing you are going to need is a simple Spring Boot project, in my case is going to be based on Java 18 and Gradle. To easily get one ready to go, we can use the Spring Initializr.

To be able to build our first application using this feature, only two simple changes are required.

Modify the Gradle configuration to specify we want a layered build:

bootJar {
  layered()
}

The second modification is going to be in the Dockerfile:

FROM openjdk:18-alpine as builder
ARG JAR_FILE=build/libs/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=layertools -jar application.jar extract

FROM openjdk:18-alpine
COPY --from=builder dependencies/ ./
COPY --from=builder spring-boot-loader/ ./
COPY --from=builder snapshot-dependencies/ ./
COPY --from=builder application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]

With only this, we will see already good improvements in our release times, but any seasoned developer will be pointing out soon that not all dependencies are equal. There is usually a huge difference in the release cycles between internal and external dependencies. The former tend to change quite often, especially if they are just modules of the same project, while the latter tend to be more stable and versions are upgraded more carefully. In the previous scenario, if the dependencies layer is the first one and it contains internal dependencies the more likely scenario is that all layers are going to be re-built all the time, and that is bad.

For that reason, the layered jars feature allows the specification of custom layers. To tackle the problem we have just mentioned, we can add just a few modifications to our files.

We can extend the recently added Gradle configuration with the specification of layers we want to use:

bootJar {
    layered {
        application {
            intoLayer("spring-boot-loader") {
                include("org/springframework/boot/loader/**")
            }
            intoLayer("application")
        }
        dependencies {
            intoLayer("snapshot-dependencies") {
                include("*:*:*SNAPSHOT")
            }
            intoLayer("internal-dependencies") {
                include("dev.binarycoders:*:*")
            }
            intoLayer("dependencies")
        }
        layerOrder = ["dependencies",
                      "spring-boot-loader",
                      "internal-dependencies",
                      "snapshot-dependencies",
                      "application"
        ]
    }
}

As we can see, we have added a few more layers and ordered them at our convenience.

  • internal-dependencies will build all our internal modules if we are using proper artefact names.
  • snapshot-dependencies will build any SNAPSHOT dependencies we are using.
  • spring-boot-loader will build the loader context.

The second change is to update our Dockerfile to reflect these changes:

FROM openjdk:18-alpine as builder
ARG JAR_FILE=build/libs/*.jar
COPY ${JAR_FILE} application.jar
RUN java -Djarmode=layertools -jar application.jar extract

FROM openjdk:18-alpine
COPY --from=builder dependencies/ ./
COPY --from=builder spring-boot-loader/ ./
COPY --from=builder internal-dependencies/ ./
COPY --from=builder snapshot-dependencies/ ./
COPY --from=builder application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]

Basically, it has the same content but using the new layers.

Spring Boot layered deployment

Docker Java Client

No questions about containers been one of the latest big things. They are everywhere, everyone use them or want to use them and the truth is they are very useful and they have change the way we develop.

We have docker for example that provides us with docker and docker-compose command line tools to be able to run one or multiple containers with just a few lines or a configuration file. But, we, developers, tend to be lazy and we like to automize things. For example, what about when we are running our application in our favorite IDE the application starts the necessary containers for us? It will be nice.

The point of this article was to play a little bit the a docker java client library, the case exposed above is just an example, probably you readers can think about better cases but, for now, this is enough for my learning purposes.

Searching I have found two different docker java client libraries:

I have not analyze them or compare them, I have just found the one by GitHub before and it looks mature and usable enough. For this reason, it is the one I am going to use for the article.

Let’s start.

First, we need to add the dependency to our pom.xml file:

<dependency>
    <groupId>com.github.docker-java</groupId>
    <artifactId>docker-java</artifactId>
    <version>3.1.1</version>
</dependency>

The main class we are going to use to execute the different instructions is the DockerClient class. This is the class that establishes the communication between our application and the docker engine/daemon in our machine. The library offers us a very intuitive builder to generate the object:

DockerClient dockerClient = DockerClientBuilder.getInstance().build();

There are some options that we can configure but for a simple example is not necessary. I am just going to say there is a class called DefaultDockerClientConfig where a lot of different properties can be set. After that, we just need to call the getInstance method with our configuration object.

Image management

Listing images

List<Image> images = dockerClient.listImagesCmd().exec();

Pulling images

dockerClient.pullImageCmd("postgres")
                .withTag("11.2")
                .exec(new PullImageResultCallback())
                .awaitCompletion(30, TimeUnit.SECONDS);

Container management

Listing containers

// List running containers
dockerClient.listContainersCmd().exec();

// List existing containers
dockerClient.listContainersCmd().withShowAll(true).exec();

Creating containers

CreateContainerResponse container = dockerClient.createContainerCmd("postgres:11.2")
                .withName("postgres-11.2-db")
                .withExposedPorts(new ExposedPort(5432, InternetProtocol.TCP))
                .exec();

Starting containers

dockerClient.startContainerCmd(container.getId()).exec();

Other

There are multiple operations we can do with containers, the above is just a short list of examples but, we can extended with:

  • Image management
    • Listing images: listImagesCmd()
    • Building images: buildImageCmd()
    • Inspecting images: inspectImageCmd("id")
    • Tagging images: tagImageCmd("id", "repository", "tag")
    • Pushing images: pushImageCmd("repository")
    • Pulling images: pullImageCmd("repository")
    • Removing images: removeImageCmd("id")
    • Search in registry: searchImagesCmd("text")
  • Container management
    • Listing containers: listContainersCmd()
    • Creating containers: createContainerCmd("repository:tag")
    • Starting containers: startContainerCmd("id")
    • Stopping containers: stopContainerCmd("id")
    • Killing containers: killContainerCmd("id")
    • Inspecting containers: inspectContainerCmd("id")
    • Creating a snapshot: commitCmd("id")
  • Volume management
    • Listing volumes: listVolumesCmd()
    • Inspecting volumes: inspectVolumeCmd("id")
    • Creating volumes: createVolumeCmd()
    • Removing volumes: removeVolumeCmd("id")
  • Network management
    • Listing networks: listNetworksCmd()
    • Creating networks: createNetworkCmd()
    • Inspecting networks: inspectNetworkCmd().withNetworkId("id")
    • Removing networks: removeNetworkCmd("id")

And, that is all. There is a project suing some of the operations listed here. It is call country, it is one of my learning projects, and you can find it here.

Concretely, you can find the code using the docker java client library here, and the code using this library here, specifically in the class PopulationDevelopmentConfig.

I hope you find it useful.

Docker Java Client

Java reflection: Accessing a private field

Sometimes, when we import 3rd-party libraries, we can find cases where the information we want is has been populated in an object but, the property is a private one and there are no methods to recover it. Here, it is where the Java reflection capabilities can help us.

Note: Use reflection carefully and, usually, as a last resource.

Let’s say we have a simple class with a private property:

class A {
    private String value = "value";
}

We want to access the property “value” but, for obvious reasons, we cannot do it.

We can implement our reflection method to access the property value:

import java.lang.reflect.Field;

public class Reflection {

   public static void main(String[] args) throws NoSuchFieldException, IllegalAccessException {
      final A a = new A();
      final Field secureRenegotiationField = a.getClass().getDeclaredField("value");

      secureRenegotiationField.setAccessible(true);

      System.out.println((String) secureRenegotiationField.get(a));
   }
}

With these few lines of code, we will be able to access the value.

Java reflection: Accessing a private field

WildFly Swarm REST example

Microservices are here. They have been here for a long time now and it looks like they are going to be here for a while. Some technologies have born in recent years to help developers to implement their systems following this architectural style. One of these technologies is WildFly Swarm.

Probably, if you are familiar with the Java EE world, you know the WildFly application server. WildFly Swarm is a derivate of this application server but it has been created with microservices architectural style in mind. Taking the definition from the WildFly Sward documentation page:

WildFly Swarm is a project that has taken the WildFly Java Application Server and deconstructed it into fine-grained parts. WildFly Swarm then allows the selective reconstitution of those parts back together with your application to allow building self-contained executable “uberjars”.

It has some other components and functions but, mainly, reading the definition we can understand the concept.

As I have said this technology is here to fill the gap that the old and big application servers cannot cover (This is a huge discussion) in the microservises world. It is comparable with Spring Boot or Payara Micro. With its different flavors and features, all of them cover the same gap.

One of the things that it calls our attention in the definition is the word “uberjars“. Considered a buzzword or maybe not, it is just defining a JAR file that contains a package and all its dependencies. In this case, for example, it is going to be a package that it is going to contain our code plus all the dependencies to run a server with our application just executing the order “java -jar file.jar” in a command line prompt.

After this little explanation, let’s go to build our first WildFly Swarm application. It is going to be a very simple REST application just with a method returning some text. And it is going to look like basically as a tradicional Java EE application. One of the best things of migrating a Java EE 7 application to WildFly Swarm is that the code doesn’t change at all. We can take the old and monolithic code and run it as an uberjar just with a few modifications in the maven pom.xml file.

The first step is to define the correct pom.xml file. In this file we are going to include the proper dependencies to build our REST service and a plugin that it is going to allow us to create our uberjar and/or run our application.

...
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.wildfly.swarm</groupId>
            <artifactId>bom</artifactId>
            <version>${version.wildfly.swarm}</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>
 
<dependencies>
    ...
    <!-- Wildfly Swarm Fractions -->
    <dependency>
        <groupId>org.wildfly.swarm</groupId>
        <artifactId>jaxrs</artifactId>
    </dependency>     
</dependencies>
 
<build>
    <finalName>RestWildFlySwarm</finalName>
    <plugins>
        <plugin>
            <groupId>org.wildfly.swarm</groupId>
            <artifactId>wildfly-swarm-plugin</artifactId>
            <version>${version.wildfly.swarm}</version>
            <executions>
                <execution>
                    <goals>
                        <goal>package</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

The second thing, it is just add the necessary code to the project to implement the REST service. Like I have said before, it is simple Java EE 7 code, I don’t think it need any especial explanation, if you need some additional explanation you have in this blog some articles about REST services and how to implement them in Java EE 7. One of this examples can be found here.

package com.example.rest;
 
import javax.ws.rs.core.Application;
import javax.ws.rs.ApplicationPath;
 
@ApplicationPath("/rest")
public class RestApplication extends Application {
}
package com.example.rest;
 
import javax.ws.rs.Path;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.GET;
import javax.ws.rs.Produces;
 
@Path("/messages")
@Produces(MediaType.APPLICATION_JSON)
public class MessagesEndPoint {
    @GET
    public Response getMessage() {
        return Response.ok("This is our first WildFly Swarm App. Awesome!").build();
    }
}

And that’s all we need to add to our first project. Now, we just need to run it.

The first way to do this, it is just running a simple maven command:

mvn wildfly-swarm:run

This first time is going to download a big bunch of dependencies and stuff, and it is going to take some time. Future executions will be much faster.

The second way to execute our application is creating our uberjar file and running it from the console.

mvn wildfly-swarm:run

You can see the full code for the example here.

See you.

WildFly Swarm REST example

Java EE 7: Servlets

A servlet is a Java program that it is executed in a web server and it builds or serve web pages. This allows us to build dynamic pages.

Servlets are a standard implementation in Java EE, for this reason, we can find it under the packages javax.servlet. Here we can find all the classes and interfaces we need to use to work with servlets. The basic elements we need to know are:

  • Servlet interface: It defines the method implemented in any servlet.
  • Class GenericServlet: It is an abstract class that implements the Servlet inteface. It is a generic servlet, and it is not tied to any protocol.
  • HttpServelt: We can find this class inside the package javax.servlet.http. This class inherit from GenericServlet and it is an abstract class too. We will inherit from this class to build our servlets.
  • When a servlet receives a request from a client it receives two objects:
    • ServletRequest: This class contains all the incoming data received in the client request. This object allows us to read all the parameters like the protocol used by the client, the parrameters send by them, and so on. We can obtain an object ServletInputStream to obtain data about the client that it is doing the request. For the HTTP protocol, we can find the subclass HttpServletRequest.
    • ServletResponse: It will contain the servlet response. We can obtain an object ServletOutputStream and/or a Writer to write the response. Again, we have a concrete subclass for the HTTP protocol, HttpServletResponse.

The servlets, like some components in Java EE, have a concrete life cicle and this life cicle is common to all the servlets. The different phases of this life cicle are:

  • The server starts and initializes the servlet.
  • The server processes the clients requests. Each request starts a new thread, and all of them are executed concurrently in the same servlet object.
  • The server destroys the servlet, usually when they have periods without requests or when the server stops.

The different phases of the servlet’s life cicle have associated some specific methods that we can override and use to implement our desired behavior.

Associated to the first phase, the initialization phase, we can find two methods:

public void init() throws ServletException { ... }
public void init(ServletConfig conf) throws ServletException {
    super.init(conf);
    ...

The first method does not need external configuration parameters, but, for the second one, we can supply these parameters in an object ServletConfig. I should remark that the call to the super.init(conf) is very important due to allow the servlet to use the configuration everywhere. If we want to perform any task during the execution of these methods, we should rewrite one or both methods. It is a good practice to return an UnavailableException if there is any error during the initialization process.

During the second phase, when a request arrives from the client, it launches a new thread that calls the service() method.

public void service(HttpServletRequest request,
        HttpServletResponse response)
            throws ServletException, IOException { ... }

This method obtains the request type: GET, POST, PUT, DELETE, OPTIONS or TRACE and, depending on the type of request, it calls the appropriate method in charge to manage the request. The most important are, in general, GET and POST, the signatures of these methods are:

public void doGet(HttpServletRequest request,
        HttpServletResponse response)
            throws ServletException, IOException { ... }
public void doPost(HttpServletRequest request,
        HttpServletResponse response)
            throws ServletException, IOException { ... }

We can find similar methods for the rest of the request types: doDelete(), doPut(), doOptions(), doTrace().

The last phase, the destruction phase, calls a method to destroy the servlet and destroy any element created during the initializacion. The method signature is:

public void destroy() throws ServletException { ... }

We should pay special attention if our servlet is launching time consuming task, we should implement some way to be sure all the tasks have finished. A counter with some synchronization code to control the number of tasks, or a flag visible for the tasks to stop them if the servlet must be destroyed quickly.

Let’s see a basic code example to see everything we have seen so far.

Basic servlet:

package com.wordpress.binaycoders.examples;
 
import javax.servlet.*;
import javax.servlet.http.*;
 
public class BasicServlet extends HttpServlet {
 
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // code for GET request
    }
 
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // code for POST request
    }
}

The next step is to configure the access to the servlet. In previous versions of Java EE we should perform this configuration with XML in the web.xml file, but in the Java EE 7 version we can do this with annotations. In any case, here, we are going to see both ways.

For the XML configuration, we should declare the servlet and the URL where is going to be available:

<servlet>
    <servlet-name>basicServlet</servlet-name>
    <servlet-class>com.wordpress.binaycoders.examples.BasicServlet</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>basicServlet</servlet-name>
    <url-pattern>/basicservlet</url-pattern>
</servlet-mapping>

With this configuration the servlet should be available in the URL:

http://localhost:8080/<theapp>/basicservlet

Inside the servlet mapping we can define a pattern instead of a concrete URL, something like:

<servlet-mapping>
    <servlet-name>basicServlet</servlet-name>
    <url-pattern>/basicservlet/*</url-pattern>
</servlet-mapping>

Or we can define a JSP page:

<servlet>
    <servlet-name>basicServlet</servlet-name>
    <jsp-file>/page.jsp</jsp-file>
</servlet>

We can initialize some parameters when the servlet starts:

<servlet>
    <servlet-name>basicServlet</servlet-name>
    <servlet-class>BasicServlet</servlet-class>
    <init-param>
        <param-name>parameter</param-name>
        <param-value>value</param-value>
    </init-param>
</servlet>

And, we can load the servlet when the server starts. Even we can define the order they are going to follow when the server starts using numbers to sort properly the different servlets.

<servlet>
    <servlet-name>basicServlet</servlet-name>
    <servlet-class>BasicServlet</servlet-class>
    <load-on-startup/>
</servlet>
<servlet>
       <servlet-name>basicServlet</servlet-name>
       <servlet-class>BasicServlet</servlet-class>
       <load-on-startup>2</load-on-startup>
</servlet>

In this case, the Java EE 7 brings us the Servlet 3.0 API, and this API allows us to use annotations to configure our servlets.

Declaring the servlet and the URL where is going to be available:

package com.wordpress.binaycoders.examples;
 
import javax.servlet.*;
import javax.servlet.http.*;
 
@WebServlet(name="basicServlet", urlPatterns="/basicservlet")
public class BasicServlet extends HttpServlet {
 
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // code for GET request
    }
 
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        // code for POST request
    }
}

In the same way, we can use annotations to initializate parameters like we have done in the XML.

@WebServlet(urlPatterns = "/basicservlet",
        initParams = { @InitParam(name = "parameter", value = "value") }
)

The way to access to these initialized parameters from our java code is:

getServletConfig().getInitParameter("parameter");

And, in the same way, we have stablished the initialization order for the servlets in the XML, we can do it with annotations:

@WebServlet(name = "basicServlet",
        urlPatterns = "/basicservlet",
        loadOnStartup = "2")

And, to finish the post, let’s see some very basic examples:

Servlet to generate plain text:

package com.wordpress.binaycoders.examples;
 
import javax.servlet.*;
import javax.servlet.http.*;
 
@WebServlet(name="basicServlet", urlPatterns="/basicservlet")
public class BasicServlet extends HttpServlet {
 
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
 
        PrintWriter out = response.getWriter();
        out.println ("I am the basic servlet. Welcome!");
    }
 
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        this.doGet(request, response);
    }
}

Servlet to generate HTML:

package com.wordpress.binaycoders.examples;
 
import javax.servlet.*;
import javax.servlet.http.*;
 
@WebServlet(name="basicServlet", urlPatterns="/basicservlet")
public class BasicServlet extends HttpServlet {
 
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
 
        response.setContentType("text/html");
 
        PrintWriter out = response.getWriter();
        out.println ("<!doctype html>");
        out.println ("<html>");
        out.println ("<body>");
        out.println ("<br>I am the basic servlet (HTML). Welcome!");
        out.println ("</body>");
        out.println ("</html>");
    }
 
    public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        this.doGet(request, response);
    }
}

And, that’s all for now.

See you.

Java EE 7: Servlets

Customized date format in JAXB

Nowadays, it is very common to work with JAXB to help us to marshal and unmarshal objects into XML or XML into objects. These objects can be simple object or objects created by us if we use the correct annotations. It is clear that JAXB is very useful but, sometimes, it is not enough with the default values the API is offering us.

Recently, we had a problem with the date format. The external system where we were trying to send information was expecting a very concrete pattern for the date objects in the XML, and the default format JAXB was offering us was not correct. After test a couple of solutions, we find one that it was working very good and it was quite simple to implement.

The idea was to rewrite the adapter with the desired format and after that, using the correct annotation, tell JAXB be the correct format. Easier than it sounds.

The first step is to write our adapter object extending the object XmlAdapter. In there, we can define the exact format we want to use when the marshalling operation is performed.

package com.wordpress.binarycoders.services.jaxb.utils;
 
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.GregorianCalendar;
 
import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.datatype.DatatypeFactory;
import javax.xml.datatype.XMLGregorianCalendar;
 
public class DateAdapter extends XmlAdapter<String, XMLGregorianCalendar> {
 
    private DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
 
    @Override
    public String marshal(XMLGregorianCalendar v) throws Exception {
        Calendar calendar = v.toGregorianCalendar();
         
        return dateFormat.format(calendar.getTime());
    }
 
    @Override
    public XMLGregorianCalendar unmarshal(String v) throws Exception {
         
        GregorianCalendar c = new GregorianCalendar();
        c.setTime(dateFormat.parse(v));
         
        return  DatatypeFactory.newInstance().newXMLGregorianCalendar(c);
    }
}

This example is using a XMLGregorianCalendar object, but this can be done with any date object.

The second step, it is to indicate JAXB that this adapter should be used instead of the default one. We do this with the annotation @XmlJavaTypeAdapter. Something like this:

@XmlSchemaType(name = "date")
@XmlJavaTypeAdapter(DateAdapter.class)
protected XMLGregorianCalendar date;

That´s all. Now, when the marshaling operation is performed, we should obtain the desired date format in our XMLs.

See you.

Customized date format in JAXB

Checksum calculation

Today, we have another little code snipped. Sometimes, we need to transfer files between systems, and in these cases, it is usually interesting to have some kind of system to check the file content integrity. In these cases, it is when a checksum can be very useful. Here we have a little implementation done in Java. Simple to write, simple to use.

package com.wordpress.binarycoders.checksum;
 
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Formatter;
 
public class CheckSum {
 
    private static final String MD5 = "MD5";
    private static final String SHA1 = "SHA-1";
     
    public static String md5CheckSum(byte[] content) throws NoSuchAlgorithmException {
        MessageDigest md = MessageDigest.getInstance(MD5);
         
        return byteArray2Hex(md.digest(content));
    }
     
    public static String sha1CheckSum(byte[] content) throws NoSuchAlgorithmException {
        MessageDigest md = MessageDigest.getInstance(SHA1);
         
        return byteArray2Hex(md.digest(content));
    }
     
    private static String byteArray2Hex(final byte[] hash) {
        String result = "";
         
        try (Formatter formatter = new Formatter()) {
            for (byte b : hash) {
                formatter.format("%02x", b);
            }
             
            result = formatter.toString();
        }
         
        return result;
    }
}

See you.

Checksum calculation

Image URL to byte array

Today, I only have a little code snippet to take a URL with an image and transform it in a byte array in Java 8, the image, not the URL. It is something very simple, but sometimes, it is very useful to have this kind of snippets in a place where we can find it easily.

package com.wordpress.binarycoders.image.recovery;
 
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.net.URL;
import java.util.Arrays;
import java.util.logging.Level;
import java.util.logging.Logger;
 
public class ImageRecover {
     
    public byte[] recoverImageFromUrl(String urlText) throws Exception {
        URL url = new URL(urlText);
        ByteArrayOutputStream output = new ByteArrayOutputStream();
         
        try (InputStream inputStream = url.openStream()) {
            int n = 0;
            byte [] buffer = new byte[ 1024 ];
            while (-1 != (n = inputStream.read(buffer))) {
                output.write(buffer, 0, n);
            }
        }
     
        return output.toByteArray();
    }
}

See you.

Image URL to byte array

Java REST API + cURL + python3

During the next lines, I am going to implement a very simple REST API using JavaEE 7 and Java SE 8 technologies, maven as build tool and GlassFish as application server. I am going to test this API with cURL and, I am going to consume this API using python3 using the Requests module.

Java EE 7 REST API

First, we need to create a maven project in our favorite IDE and add our dependencies to the pom.xml file

...
<properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <failOnMissingWebXml>false</failOnMissingWebXml>
</properties>
 
<dependencies>
    <dependency>
        <groupId>org.eclipse.persistence</groupId>
        <artifactId>eclipselink</artifactId>
        <version>2.5.2</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.persistence</groupId>
        <artifactId>org.eclipse.persistence.jpa.modelgen.processor</artifactId>
        <version>2.5.2</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>javax</groupId>
        <artifactId>javaee-api</artifactId>
        <version>7.0</version>
        <scope>provided</scope>
    </dependency>
</dependencies>
 
<build>
    <finalName>apicrud</finalName>
</build>
...

Second, we are going to create a persistence.xml file. In this case, we are not interested in how to configure a DB, then, we are going to use the default datasource that GlassFish provides us as default “jdbc/__default“. With all of this in mind, the file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.1" xmlns="http://xmlns.jcp.org/xml/ns/persistence"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence
        http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd">
    <persistence-unit name="ApiCrudPU" transaction-type="JTA">
        <jta-data-source>jdbc/__default</jta-data-source>
        <exclude-unlisted-classes>false</exclude-unlisted-classes>
        <properties>
            <property name="javax.persistence.schema-generation.database.action" value="create"/>
        </properties>
    </persistence-unit>
</persistence>

Now, we need to create our Entity. In this case, I have chosen a simple one, User with three attributes:

  • id: Autogenerated id to identify the user.
  • name: User’s name.
  • email: User’s email.

The result should be something like:

package com.wordpress.binarycoders.apicrud;
 
import java.io.Serializable;
import java.util.Objects;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.NamedQueries;
import javax.persistence.NamedQuery;
import javax.persistence.Table;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlRootElement;
 
@Entity
@Table (name = "users")
@NamedQueries ({
    @NamedQuery (name = User.FIND_ALL, query = "SELECT u FROM User u")
})
@XmlRootElement
@XmlAccessorType(XmlAccessType.FIELD)
public class User implements Serializable {
     
    private static final long serialVersionUID = 1L;
     
    public static final String FIND_ALL = "User.findAll";
     
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
     
    @Column(name = "name")
    private String name;
     
    @Column(name = "email")
    private String email;
 
    /* Getters, Setters, equals and hashCode methods */   
}

The next step, it is to implement the boundary or service layer. Being a simple example, we can skip this layer, but I like to implement everything for learning purposes. This layer is going to be very simple, only the necessary object to perform the operation in the DB and the basic operations to implement a CRUD. The result looks like:

package com.wordpress.binarycoders.apicrud;
 
import java.util.List;
import javax.ejb.Stateless;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.TypedQuery;
import javax.validation.constraints.NotNull;
 
@Stateless
public class UserService {
     
    @PersistenceContext
    private EntityManager em;
     
    public void create(@NotNull User user) {
        this.em.persist(user);
    }
     
    public User findById(@NotNull Long id) {
        return this.em.find(User.class, id);
    }
     
    public List<User> findAll() {
        TypedQuery<User> typedQuery = this.em.createNamedQuery(User.FIND_ALL, User.class);
 
        return typedQuery.getResultList();
    }
     
    public void update(@NotNull User user) {
        this.em.merge(user);
    }
     
    public void delete(@NotNull Long id) {
        this.em.remove(this.findById(id));
    }
}

To do all of this available to the world, we need to implement our REST API, and for this, we need to configure the REST capabilities in our system, piece of cake with the last version of Java EE. We just need to create a class to enable these capabilities:

package com.wordpress.binarycoders.apicrud;
 
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;
 
@ApplicationPath("/rs")
public class ApplicationConfig extends Application {
     
}

And, finally, we need to implement our REST layer with the four actions that are going to allow us to perform the CRUD operations:

package com.wordpress.binarycoders.apicrud;
 
import java.net.URI;
import java.util.List;
import javax.ejb.EJB;
import javax.ejb.Stateless;
import javax.ws.rs.Consumes;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.NotFoundException;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.GenericEntity;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;
 
@Path("/users")
@Produces ({ MediaType.APPLICATION_JSON })
@Consumes ({ MediaType.APPLICATION_JSON })
@Stateless
public class UserEndPoint {
     
    @EJB
    private UserService service;
     
    @Context
    private UriInfo uriInfo;
     
    @GET
    public Response findAll() {
        List<User> users = this.service.findAll();
         
        GenericEntity<List<User>> list = new GenericEntity<List<User>>(users) {};
         
        return Response.ok(list).build();
    }
     
    @GET
    @Path("/{id}")
    public Response findById(@PathParam("id") Long id) {
        User user = this.service.findById(id);
         
        if (user == null) {
            throw new NotFoundException();
        }
         
        return Response.ok(user).build();
    }
     
    @POST
    public Response create(User user) {
        this.service.create(user);
         
        URI uri = uriInfo.getAbsolutePathBuilder().path(String.valueOf(user.getId())).build();
         
        return Response.created(uri).build();
    }
     
    @PUT
    public Response update(User user) {
        this.service.update(user);
         
        return Response.ok().build();
    }
     
    @DELETE
    @Path("/{id}")
    public Response delete(@PathParam("id") Long id) {
        this.service.delete(id);
         
        return Response.noContent().build();
    }
}

I think that in this point, assuming that everyone reading this, it is a developer and this is not an entry level post, all of you should be capable to understand everything has been written in the above lines. If there are things that you do not know or you do not understand due to a lack of information or your knowledge is not enough (all of us have been there), look at the note at the end of this post.

cURL test

Now, obviously, we need to test if our REST API is working properly. For this, we can implement some solution based on test frameworks or manual code but, due to the simplicity of the object User, it is enough with the cURL tool. To test our API we are going to perform these operations:

  • Select all the users
  • Create a user
  • Select the created user
  • Update the user
  • Delete the user

These operations can be performed with the next commands:

curl -i -H "Content-Type: application/json" http://localhost:8080/ApiCRUD/rs/users
curl -i -H "Content-Type: application/json" -X POST -d '{"name":"john","email":"john@example.org"}' http://localhost:8080/ApiCRUD/rs/users
curl -i -H "Content-Type: application/json" http://localhost:8080/ApiCRUD/rs/users/1
curl -i -H "Content-Type: application/json" -X PUT -d '{"id”:1,”name":"John Doe","email":"john.doe@example.org"}' http://localhost:8080/ApiCRUD/rs/users
curl -i -H "Content-Type: application/json" -X DELETE http://localhost:8080/ApiCRUD/rs/users/1

Everything should look well, and the appropriate HTTP codes should be received.

Python3 client

The last step today, it is to implement a very very simple python3 client to consume our API. This is going to be a basic script to invoke the CRUD operations available in the API.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
 
import requests
import json
 
URL = 'http://localhost:8080/ApiCRUD/rs/users'
 
def getUsers():
    print('Get users...\n')
 
    response = requests.get(URL)
 
    assert response.status_code == 200
 
    users = response.json()
    print(json.dumps(users, indent = 4, separators = (',', ':')))
 
def getUser(location):
    print('Get single user ...\n')
 
    response = requests.get(location)
 
    assert response.status_code == 200
 
    users = response.json()
    print(json.dumps(users, indent = 4, separators = (',', ':')))
 
def createUser():
    print('Creating user...\n')
 
    headers = {'content-type': 'application/json'}
    user = {"name": "John Doe", "email": "john.doe@example.org"}
    response = requests.post(URL, data = json.dumps(user), headers = headers)
 
    assert response.status_code == 201
 
    print('Location: ' + response.headers['location'])
 
    return response.headers['location']
 
def updateUser(location):
    print('Updating user...')
 
    loc = location.split('/')
 
    headers = {'content-type': 'application/json'}
    user = {"id": loc[-1], "name": "Jane Doe", "email": "jane.doe@example.org"}
    response = requests.put(URL, data = json.dumps(user), headers = headers)
 
    assert response.status_code == 200
 
def deleteUser(location):
    print('Deleting user...')
 
    response = requests.delete(location)
 
    assert response.status_code == 204
 
    print('Checking delete operation')
    response = requests.get(location)
 
    assert response.status_code == 404
 
def main():
    getUsers()
    location = createUser()
    getUser(location)
    updateUser(location)
    getUser(location)
    deleteUser(location)
 
if __name__ == "__main__":
    main()

And that’s all. Now, we have a little REST AP, a python3 client and a basic knowledge about how to use cURL to test it.

See you.

You can find the code for the REST API available here.

Note: All the content in this post will be splitter and explained in future post, one for each one of the three big point explained in here: Java EE 7 REST API creation, cURL test and python3 REST client.

Java REST API + cURL + python3

Spark + Freemarker basic example

Today, we are going to develop a little web page with a form using Spark and Freemarker. The purpose is simply to know how they are used together and how to manage GET and POST request. In addition, I am going to add Bootstrap to learn how to combine Freemarker with CSS.

As its web page describes “Spark – A tiny Sinatra inspired framework for creating web applications in Java 8 with minimal effort“. You can see a previous example I implemented in this blog called: Quick REST demo with Spark.

In the same way, as its web page describes “FreeMarker is a “template engine”; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates

The project has these requirements:

  • JDK 8
  • Maven
  • IDE. I have used IntelliJ IDEA CE, but you can use NetBeans, Eclipse or your favorite one.

The final structure for our project will be:

./pom.xml
./src/main/java/com/wordpress/binarycoders/SayMyName.java
./src/main/resources/css/bootstrap.css
./src/main/resources/templates/form.ftl
./src/main/resources/templates/result.ftl

The first step is to create a simple maven java project, obtaining in this way a pom.xml and a basic project structure. We should assign groupId, artifactId and version.

After that, we should edit our pom.xml file to add the dependencies that we are going to use to implement our example:

<dependency>
    <groupId>com.sparkjava</groupId>
    <artifactId>spark-core</artifactId>
    <version>2.1</version>
</dependency>
 
<dependency>
    <groupId>org.freemarker</groupId>
    <artifactId>freemarker</artifactId>
    <version>2.3.22</version>
</dependency>

As I have said at the beginning of this post, Freemarker is a template engine, then, the next step is to write the templates. We are going to write two templates:

  • form.ftl: This is going to be the first page of our application and it is going to show us a form to introduce “user” and “email“.
  • result.ftl: This page is going to show us the information introduced in the form.

The first template: form.ftl

<html>
    <head>
        <title>Welcome</title>
 
        <link href="css/bootstrap.css" rel="stylesheet" />
    </head>
    <body>
        <form class="form-inline" method="POST" action="/sait">
            <div class="form-group">
                <label for="name">Name</label>
                <input type="text"
                       class="form-control"
                       id="name"
                       name="name"
                       placeholder="John Doe">
            </div>
            <div class="form-group">
                <label for="email">Email</label>
                <input type="email"
                       class="form-control"
                       id="email"
                       name="email"
                       placeholder="john.doe@example.org">
            </div>
            <button type="submit" class="btn btn-default">Send invitation</button>
        </form>
    <body>
</html>

The second template: result.ftl

<html>
    <head>
        <title>Said</title>
    </head>
    <body>
        <h1> Your name is ${name}</h1>
        <br>
        <h1> Your email is ${email}</h1>
    </body>
</html>

It is very important to remember in this point that to be allowed to recover our form parameters in the Java code we need to add the name attribute to our inputs. It is not enough with the id attribute.

In this point, we should add Bootstrap CSS file to our project.

And finally, we need to implement our main class that it is going to process the GET and the POST requests.

package com.wordpress.binarycoders;
 
import freemarker.template.Configuration;
import freemarker.template.Template;
import freemarker.template.Version;
import spark.Spark;
 
import java.io.StringWriter;
import java.util.HashMap;
import java.util.Map;
 
public class SayMyName {
 
    public static void main(String[] args) {
        final Configuration configuration = new Configuration(new Version(2, 3, 0));
        configuration.setClassForTemplateLoading(SayMyName.class, "/");
 
        Spark.get("/", (request, response) -> {
 
            StringWriter writer = new StringWriter();
 
            try {
                Template formTemplate = configuration.getTemplate("templates/form.ftl");
 
                formTemplate.process(null, writer);
            } catch (Exception e) {
                Spark.halt(500);
            }
 
            return writer;
        });
 
        Spark.post("/sait", (request, response) -> {
            StringWriter writer = new StringWriter();
 
            try {
                String name = request.queryParams("name") != null ? request.queryParams("name") : "anonymous";
                String email = request.queryParams("email") != null ? request.queryParams("email") : "unknown";
 
                Template resultTemplate = configuration.getTemplate("templates/result.ftl");
 
                Map<String, Object> map = new HashMap<>();
                map.put("name", name);
                map.put("email", email);
 
                resultTemplate.process(map, writer);
            } catch (Exception e) {
                Spark.halt(500);
            }
 
            return writer;
        });
    }
}

Now, we only need to run this main class and… that’s all. We can go to our browser and introduce the URL:

http://localhost:4567

We should see our form with our Bootstrap style. If we introduce a username and an email, and we submit the form, we will see in the next page two messages with our form data.

Probably, as you have notice, we have not needed a server and this is because the Spark framework have an embedded Jetty server.

See you.

Spark + Freemarker basic example