design-patterns

Maintaining compatibility

fjavierm, Leave a comment

Most of the companies nowadays are implementing or want to implement architectures based on micro-services. While this can help companies to overcome multiple challenges, it can bring its own new challenges to the table.

In this article, we are going to discuss a very concrete one been maintaining compatibility when we change objects that help us to communicate the different micro-services on our systems. Sometimes, they are called API objects, Data Transfer Objects (DTO) or similar. And, more concretely, we are going to be using Jackson and JSON as a serialisation (marshalling and unmarshalling) mechanism.

There are some other methods, other technologies and other ways to achieve this but, this is just one tool to keep on our belt and be aware of to make informed decisions when faced with this challenge in the future.

Maintaining compatibility is a very broad term, to establish what we are talking about and ensure we are on the same page, let us see a few examples of real situations we are trying to mitigate:

  • To deploy breaking changes when releasing new features or services due to changes on the objects used to communicate the different services. Especially, if at deployment time, there is a small period of time where the old and the new versions are still running (almost impossible to avoid unless you stop both services, deploy them and restart them again).
  • To be forced to have a strict order of deployment for our different services. We should be able to deploy in any order and whenever it best suits the business and the different teams involved.
  • The need of, due to a change in one object in one concrete service, being forced to deploy multiple services not directly involved or affected by the change.
  • Related to the previous point, to be forced to change other services because of a small data or structural change. An example of this would be some objects that travel through different systems been, for example, enriched with extra information and finally shown to the user on the last one.

To exemplify the kind of situation we can find ourselves in, let us take a look at the image below. In this scenario, we have four different services:

  • Service A: It stores some basic user information such as the first name and last name of a user.
  • Service B: It enriches the user information with extra information about the job position of the user.
  • Service C: It adds some extra administrative information such as the number of complaints open against the user.
  • Service D: It finally uses all the information about the user to, for example, calculate some advice based on performance and area of work.

All of this is deployed and working on our production environment using the first version of our User object.

At some point, product managers decided the age field should be considered on the calculations to be able to offer users extra advice based on proximity of retirement. This added requirement is going to create a second version of our User object where the field age is present.

Just a last comment, for simplicity purposes, let us say the communication between services is asynchronous based on queues.

As we can see on the image, in this situation only services A and D should be modified and deployed. This is what we are trying to achieve and what I mean by maintaining compatibility. But, first, let us explore what are the options we have at this point:

  1. Upgrade all services to the second version of the object User before we start sending messages.
  2. Avoid sending the User from service A to service D, send just an id, and perform a call from service D to recover the User information based on the id.
  3. Keep the unknown fields on an object even, if the service processing the message at this point does not know anything about them.
  4. Fail the message, and store it for re-processing until we perform the upgrade to all services involved. This option is not valid on synchronous communications.

Option 1

As we have described, it implies the update of the dependency service-a-user in all the projects. This is possible but it brings quickly some problems to the table:

  • We not only need to update direct dependencies but indirect dependencies too what it can be hard to track, and easy to miss. In addition, a decision needs to be done about what to do when a dependency is missed, should an error be thrown? Should we fail silently?
  • We have a problem with scenarios where we need to roll back a deployment due to something going wrong. Should we roll back everything? Good luck! Should we try to fix the problem while our system is not behaving properly?
  • Heavy refactoring operations or modifications can make upgrades very hard to perform.

Option 2

Instead of sending the object information on the message, we just send an id to be able posteriorly to recover the object information using a REST call. This option while very useful in multiple cases is not exempt from problems:

  • What if, instead of just a couple of enrichers, we have a dozen of them and they need the user information? Should we consolidate all services and send ids for the enriched information crating stores on the enrichers?
  • If, instead of a queue, other mechanisms of communications are used such as RPC, do now all the services need to call service A to recover the User information and do their job? This just creates a cascade of calls.
  • And, under this scenario, we can have inconsistent data if there is any update while the different services are recovering a User.

Option 3

This is going to be the desired option and the one we are going to do a deep dive on this article using Jackson and JSON how to keep the fields even if the processing service does not know everything about them.

To add in advance that, as always, there are no silver bullets, there are problems that not even this solution can solve but it will mitigate most of the ones we have named on previous lines.

One problem we are not able to solve with this approach – especially if your company perform “all at once” releases instead of independent ones – is, if service B, once deployed tries to persist some information on service A before the new version has been deployed, or tries to perform a search using one criterion, in this case, the field age, on the service A. In this scenario, the only thing we can do is to throw an error.

Option 4

This option, especially in asynchronous situations where messages can be stored to be retried later, can be a possible solution to propagate the upgrade. It will slow down our processing capabilities temporarily, and retrying mechanism needs to be in place but, it is doable.

Using Jackson to solve versioning

Renaming a field

Plain and simple, do not do it. Especially, if it is a client-facing API and not an internal one. It will save you a lot of trouble and headaches. Unfortunately, if we are persisting JSON on our databases, this will require some migrations.

If it needs to be done, think about it again. Really, rethink it. If after rethinking it, it needs to be done a few steps need to be taken:

  1. Update the API object with the new field name using @JsonAlias.
  2. Release and update everything using the renamed field, and @JsonAlias for the old field name.
  3. Remove @JsonAlias for the old field name. This is a cleanup step, everything should work after step two.

Removing a field

Like in the previous case, do not do it, or think very hard about it before you do it. Again, if you finally must, a few steps need to be followed.

First, consider to deprecate the field:

If it must be removed:

  1. Explicitly ignore the old property with @JsonIgnoreProperties.
  2. Remove @JsonIgnoreProperties for the old field name.

Unknown fields (adding a field)

Ignoring them

The first option is the simplest one, we do not care for new fields, a rare situation but it can happen. We should just ignore them:

A note of caution in this scenario is that we need tone completely sure we want to ignore all properties. As an example, we can miss on APIs that return errors as HTTP 200 OK, and map the errors on the response if we are not aware of that, while in other circumstances it will just crash making us aware.

Ignoring enums

In a similar way, we can ignore fields, we can ignore enums, or more appropriately, we can map them to an UNKNOWN value.

Keeping them

The most common situation is that we want to keep the fields even if they do not mean anything for the service it is currently processing the object because they will be needed up or downs the stream.

Jackson offers us two interesting annotations:

  • @JsonAnySetter
  • @JsonAnyGetter

These two annotations help us to read and write fields even if I do not know what they are.

class User {
    @JsonAnySetter
    private final Map<String, Object> unknownFields = new LinkedHashMap<>();
    
    private Long id;
    private String firstname;
    private String lastname;

    @JsonAnyGetter
    public Map<String, Object> getUnknownFields() {
        return unknownFields;
    }
}

Keeping enums

In a similar way, we are keeping the fields, we can keep the enums. The best way to achieve that is to map them as strings but leave the getters and setters as the enums.

@JsonAutoDetect(
    fieldVisibility = Visibility.ANY,
    getterVisibility = Visibility.NONE,
    setterVisibility = Visibility.NONE)
class Process {
    private Long id;
    private String state;

    public void setState(State state) {
        this.state = nameOrNull(state);
    }

    public State getState() {
        return nameOrDefault(State.class, state, State.UNKNOWN);
    }

    public String getStateRaw() {
        return state;
    }
}

enum State {
    READY,
    IN_PROGRESS,
    COMPLETED,
    UNKNOWN
}

Worth pointing that the annotation @JsonAutoDetect tells Jackson to ignore the getters and setter and perform the serialisation based on the properties defined.

Unknown types

One of the things Jackson can manage is polymorphism but this implies we need to deal sometimes with unknown types. We have a few options for this:

Error when unknown type

We prepare Jackson to read an deal with known types but it will throw an error when an unknown type is given, been this the default behaviour:

@JsonTypeInfo(
    use = JsonTypeInfo.Id.NAME,
    include = JsonTypeInfo.As.PROPERTY)
@JsonSubTypes({
    @JsonSubTypes.Type(value = SelectionProcess.class, name = "SELECTION_PROCESS"),
})
interface Process {
}

Keeping the new type

In a very similar to what we have done for fields, Jackson allow as to define a default or fallback type when the given type is not found, what put together with out unknown fields previous implementation can solve our problem.

@JsonTypeInfo(
    use = JsonTypeInfo.Id.NAME,
    include = JsonTypeInfo.As.PROPERTY,
    property = "@type",
    defaultImpl = AnyProcess.class)
@JsonSubTypes({
    @JsonSubTypes.Type(value = SelectionProcess.class, name = "SELECTION_PROCESS"),
    @JsonSubTypes.Type(value = SelectionProcess.class, name = "VALIDATION_PROCESS"),
})
interface Process {
    String getType();
}

class AnyProcess implements Process {
    @JsonAnysetter
    private final Map<String, Object> unknownFields = new LinkedHashMap<>();

    @JsonProperty("@type")
    private String type;

    @Override
    public String getType() {
        return type;
    }

    @JsonAnyGetter
    public Map<String, Object> getUnknownFields() {
        return unknownFields
    }
}

And, with all of this, we have decent compatibility implemented, all provided by the Jackson serialisation.

We can go one step further and implement some basic classes with the common code e.g., unknownFields, and make our API objects extend for simplicity, to avoid boilerplate code and use some good practices. Something similar to:

class Compatibility {
    ....
}

class MyApiObject extends Compatibility {
    ...
}

With this, we have a new tool under our belt we can consider and use whenever is necessary.

Maintaining compatibility

Design patterns: Prototype

fjavierm, , Leave a comment

This is a creational pattern, the prototype pattern is designed to instantiate a class by copying, or cloning, the properties of an existing object. The new object is an exact copy of the prototype but permits modification without altering the original.

The prototype pattern usually generates deep copies of the objects, not just shallow copies of it (this depends on the implementation we decide to do and our requirements). A shallow copy duplicates all of the object’s properties. If any property contains a reference type, the reference is copied. This means that changes to the referenced object are visible in both the clone and the original object. A deep copy clones the main object and all child objects. Any properties of reference types are also cloned, giving a truly independent copy.

Elements involved:

  • Prototype: This is an abstract class or an interface if the objects does not share properties for the types of object that can be generated and cloned. The class will contain a single method named “clone” that returns a prototype object.
  • ConcretePrototype (A/B): This class inherits or implements the prototype class. In addition to copying the original object’s data to the clone, this method may also handle some edge cases of the cloning process related to cloning linked objects, untangling recursive dependencies, etc.

We should use the Prototype design pattern when:

It is generally used for complex classes or for classes that are costly to instantiate.

Let’s see some code:

public abstract class Prototype {

    private int propertyA;
    private int propertyB;
    private String propertyC;

    public Prototype() {
    }

    public Prototype(final Prototype prototype) {
        propertyA = prototype.propertyA;
        propertyB = prototype.propertyB;
        propertyC = prototype.propertyC;
    }

    public abstract Prototype clone();

   // Getters & Setters
}

public class ConcretePrototype1 extends Prototype {

    private String ownProperty1;

    public ConcretePrototype1() {
    }

    public ConcretePrototype1(final ConcretePrototype1 prototype) {
        super(prototype);

        ownProperty1 = prototype.ownProperty1;
    }

    @Override
    public Prototype clone() {
        return new ConcretePrototype1(this);
    }

   // Getters & Setters
}

public class ConcretePrototype2 extends Prototype {

    private String ownProperty2;

    public ConcretePrototype2() {
    }

    public ConcretePrototype2(final ConcretePrototype2 prototype) {
        super(prototype);

        ownProperty2 = prototype.ownProperty2;
    }

    @Override
    public Prototype clone() {
        return new ConcretePrototype2(this);
    }

   // Getters & Setters
}

public class Main {

    public static void main(String[] args) {
        final List<Prototype> prototypes = new ArrayList<>();
        final ConcretePrototype1 cp1 = new ConcretePrototype1();
        final ConcretePrototype2 cp2 = new ConcretePrototype2();

        prototypes.add(cp1);
        prototypes.add(cp2);

        // We can now copy the list without worrying about what
        // specific type is the object
        final List<Prototype> newPrototypes = new ArrayList<>();

        for (final Prototype p : prototypes) {
            newPrototypes.add(p.clone());
        }
    }
}

You can find the code in my GitHub repository design-apperns.

Design patterns: Prototype

Design patterns: Factory Method

fjavierm, , Leave a comment

This is a creational pattern, the factory method pattern is a design pattern that allows for the creation of objects without specifying the type of object that is to be created in code. A factory class contains a method that allows determination of the created type at run-time.

The factory pattern is used to replace class constructors, abstracting the process of object generation so that the type of the object instantiated can be determined at run-time.

Elements involved:

  • FactoryBase: This is an abstract base class for the concrete factory classes that will actually generate new objects.
  • ConcreteFactory: Inheriting from the FactoryBase class, the concrete factory classes inherit the actual factory method. This is overridden with the object generation code unless already implemented in full in the base class.
  • ProductBase: This abstract class is the base class for the types of object that the factory can create. It is also the return type for the factory method.
  • ConcreteProduct: Multiple subclasses of the Product class are defined, each containing specific functionality. Objects of these classes are generated by the factory method.

We should use the Factory Method design pattern when:

  • when a class can’t anticipate the type of the objects it is supposed to create.
  • when a class wants its subclasses to be the ones to specific the type of a newly created object.

Let´s see some code:

public abstract class FactoryBase {
    public abstract ProductBase build(int type);
}

public abstract class ProductBase {
    public abstract void whoIAm();
}

public class ConcreteFactory extends FactoryBase {
    @Override
    public ProductBase build(int type) {
        final ProductBase productBase;
        switch (type) {
            case 1:
                productBase = new ConcreteProduct1();
                break;
            case 2:
                productBase = new ConcreteProduct2();
                break;
            default:
                throw new IllegalArgumentException(String.format("Illegal type %s", type));        
        }
        return productBase;
    }
}

public class ConcreteProduct1 extends ProductBase {
    @Override
    public void whoIAm() {
        System.out.println("I am ConcreteProduct1");
    }
}

public class ConcreteProduct2 extends ProductBase {
    @Override
    public void whoIAm() {
        System.out.println("I am ConcreteProduct2");
    }
}

public class Main {
    public static void main(String[] args) {
        final FactoryBase factoryBase = new ConcreteFactory();
        factoryBase.build(1).whoIAm();
        factoryBase.build(2).whoIAm();
    }
}

The output after the execution should be something like that:

I am ConcreteProduct1
I am ConcreteProduct2

You can find the code in my GitHub repository “design-apperns“.

Design patterns: Factory Method

Design Patterns: Builder

fjavierm, , Leave a comment

This is a creational pattern, as it is used to control class instantiation. The builder pattern is a design pattern that allows for the step-by-step creation of complex objects using the correct sequence of actions. The construction is controlled by a director object that only needs to know the type of object it is to create.

Elements involved:

  • Product: The product class defines the type of the complex object that is to be generated by the builder pattern.
  • Builder: This abstract base class defines all of the steps that must be taken in order to correctly create a product. Each step is generally abstract as the actual functionality of the builder is carried out in the concrete subclasses. The getProduct method is used to return the final product. The builder class is often replaced with a simple interface.
  • ConcreteBuilder: There may be any number of concrete builder classes inheriting from Builder. These classes contain the functionality to create a particular complex product.
  • Director: The director class controls the algorithm that generates the final product object. A director object is instantiated and its construct method is called. The method includes a parameter to capture the specific concrete builder object that is to be used to generate the product. The director then calls methods of the concrete builder in the correct order to generate the product object. On completion of the process, the getProduct method of the builder object can be used to return the product.

We should use the Builder design pattern when:

When object creation algorithms should be decoupled from the system, and multiple representations of creation algorithms are required. This decoupling is useful as you can add new creation functionality to your system without affecting the core code. You also get control over the creation process at runtime with this approach.

Let´s see some code:

public abstract class Builder {
    public abstract void buildA();
    public abstract void buildB();
    public abstract void buildC();
    public abstract Product getProduct();
}

public class ConcreteBuilder extends Builder {
    private final Product product = new Product();
    @Override
    public void buildA() { this.product.setA("A"); }
    @Override
    public void buildB() { this.product.setB("B"); }
    @Override
    public void buildC() { this.product.setC("C"); }
    @Override
    public Product getProduct() { return this.product; }
}

public class Product {
    public String a;
    public String b;
    public String c;
    public String getA() { return a; }
    public void setA(String a) { this.a = a; }
    public String getB() { return b; }
    public void setB(String b) { this.b = b; }
    public String getC() { return c; }
    public void setC(String c) { this.c = c; }
}

public class Director {
    public Product construct(final Builder builder) {
        builder.buildA();
        builder.buildB();
        builder.buildC();
        return builder.getProduct();
    }
}

public class Main {
    public static void main(String[] args) {
        final Director director = new Director();
        final Product product = director.construct(new ConcreteBuilder());
        System.out.println(String.format("Product: A = %s, B = %s, C = %s", product.getA(), product.getB(), product.getC()));
    }
}

You can find the code in my GitHub repository “design-apperns“.

Not exactly matching the pattern described in the book wrote by the GoF, we can see a couple more definitions/implementations of the Builder pattern.

The first one is used to build immutable objects and, in addition, it makes easier to add new properties to your object without having a constructor with a huge number of parameters. It is similar to a fluent interface usually implemented by using method cascading or method chaining.

Let´s see an example.

public class Product {
    private final String a;
    private final String b;
    private final String c;
    private final String d;
    private final String e;
    private Product(ProductBuilder productBuilder) {
        this.a = productBuilder.a;
        this.b = productBuilder.b;
        this.c = productBuilder.c;
        this.d = productBuilder.d;
        this.e = productBuilder.e;
    }
    public String getA() { return a; }
    public String getB() { return b; }
    public String getC() { return c; }
    public String getD() { return d; }
    public String getE() { return e; }
    @Override
    public String toString() {
        return "Product{" + "a=" + a + ", b=" + b + ", c=" + c + ", d=" + d + ", e=" + e + '}';
    }
    public static class ProductBuilder {
        private final String a;
        private final String b;
        private String c; // Optional
        private String d; // Optional
        private String e; // Optional
        public ProductBuilder(String a, String b) {
            this.a = a;
            this.b = b;
        }
        public ProductBuilder setC(final String c) {
            this.c = c;
            return this;
        }
        public ProductBuilder setD(final String d) {
            this.d = d;
            return this;
        }
        public ProductBuilder setE(final String e) {
            this.e = e;
            return this;
        }
        public Product build() {
            return new Product(this);
        }
    }
}

public class Main {
    public static void main(String[] args) {
        System.out.print(new Product.ProductBuilder("A", "B")
                .setC("C")
                .setD("D")
                .setE("E")
                .build()
                .toString());
    }
}

You can find the example in the repository, in the builder project in the package labeled as “variant1”.

The second example, it is basically the same, but applied to POJOs to make easier to build them. It is a much more simplified version of the previous example.

Let´s see how this is:

public class Product {
    private String a;
    private String b;
    public String getA() { return this.a; }
    public Product setA(final String a) {
        this.a = a;
        return this;
    }
    public String getB() { return this.b; }
    public Product setB(final String b) {
        this.b = b;
        return this;
    }
    @Override
    public String toString() {
        return "Product{" + "a=" + a + ", b=" + b + '}';
    }
}

public class Main {
    public static void main(String[] args) {
        System.out.println(new Product()
                .setA("A")
                .setB("B")
                .toString());
    }
}

You can find the example in the repository, in the builder project in the package labeled as “variant2”.

Design Patterns: Builder

Design patterns: Abstract Factory

fjavierm, , Leave a comment

This is a creational pattern, as it is used to control class instantiation. The abstract factory pattern is used to provide a client with a set of related or dependant objects. The set of objects created by the factory is determined at run-time according to the selection of concrete factory class.

Provides a level of indirection that abstracts the creation of families of related or dependent objects without directly specifying their concrete classes. The factory object has the responsibility for providing creation services for the entire platform family. Clients never create platform objects directly, they ask the factory to do that for them.

This mechanism makes exchanging product families easy because the specific class of the factory object appears only once in the application – where it is instantiated. The application can wholesale replace the entire family of products simply by instantiating a different concrete instance of the abstract factory.

Elements involved:

  • AbstractFactory: This is an abstract base class for the concrete factory classes that will generate new sets of related objects. A method is included for each type of object that will be instantiated.
  • ConcreteFactory: Inheriting from the AbstractFactory class, the concrete factory classes override the methods that generate the suite of objects required by the client.
  • AbstractProduct: This abstract class is the base class for the types of object that a factory can create. One base type exists for each of the distinct types of product required by the client.
  • ConcreteProduct: Multiple subclasses of the ConcreteProduct classes are defined, each containing specific functionality. Objects of these classes are generated by the abstract factory to populate the client.
  • Client: This class uses the factories to generate a family of related objects. In the UML diagram, the client has two private fields that hold instances of the abstract product classes.

We should use the Abstract Factory design pattern when:

  • the system needs to be independent from the way the products it works with are created.
  • the system is or should be configured to work with multiple families of products.
  • a family of products is designed to work only all together.
  • the creation of a library of products is needed, for which is relevant only the interface, not the implementation, too.

Let´s see some code:

public abstract class AbstractFactory {
    public abstract AbstractProduct createProduct();
}

public abstract class AbstractProduct {
    public abstract void operation1();
    public abstract void operation2();
}

public class ConcreteFactory1 extends AbstractFactory {
    @Override
    public AbstractProduct createProduct() {
        return new ConcreteProduct1();
    }
}

public class ConcreteFactory2 extends AbstractFactory {
    @Override
    public AbstractProduct createProduct() {
        return new Product2();
    }
}

public class ConcreteProduct1 extends AbstractProduct {
    public ConcreteProduct1() {
        System.out.println("Creating product 1...");
    }
    @Override
    public void operation1() {
        System.out.println("Executing ConcreteProduct1::operation1...");
    }
    @Override
    public void operation2() {
        System.out.println("Executing ConcreteProduct1::operation2...");
    }
}

public class ConcreteProduct2 extends AbstractProduct {
    public ConcreteProduct2() {
        System.out.println("Creating product 2...");
    }
    @Override
    public void operation1() {
        System.out.println("Executing Product2::operation1...");
    }
    @Override
    public void operation2() {
        System.out.println("Executing Product2::operation2...");
    }
}

public enum ProductType {
    PRODUCT_1,
    PRODUCT_2
}

public class FactoryMaker {
    public static AbstractFactory getFactory(final ProductType productType) {
        final AbstractFactory abstractFactory;
        switch (productType) {
            case PRODUCT_1:
                abstractFactory = new ConcreteFactory1();
                break;
            case PRODUCT_2:
                abstractFactory = new ConcreteFactory2();
                break;
            default:
                throw new IllegalArgumentException("The product type does not exist.");
        }
        return abstractFactory;
    }
}

public class Client {
    public static void main(String[] args) {
        Arrays.stream(ProductType.values())
            .forEach(productType -> {
                AbstractFactory abstractFactory = FactoryMaker.getFactory(productType);
                AbstractProduct product = abstractFactory.createProduct();
                product.operation1();
                product.operation2();
            });
    }
}

The output after the execution should be something like that:

Creating product 1...
Executing ConcreteProduct1::operation1...
Executing ConcreteProduct1::operation2...
Creating product 2...
Executing Product2::operation1...
Executing Product2::operation2...

You can find the code in my GitHub repository “design-apperns“.

Design patterns: Abstract Factory

Design Patterns

fjavierm, , , , Leave a comment

Design patterns provide solutions to common software design problems. In the case of object-oriented programming, design patterns are generally aimed at solving the problems of object generation and interaction, rather than the larger scale problems of overall software architecture. They give generalised solutions in the form of templates that may be applied to real-world problems.

In the next series of articles we are going to be focus in the Gang of Four design patterns. They wrote a book called “Design Patterns: Elements of Reusable Object-Oriented Software”. The four authors are:

They included in their book thirty-three design patterns divided in three main categories:

  • Creational patterns: Creational patterns provide ways to instantiate single objects or groups of related objects.
  • Structural patterns: Structural patterns provide a manner to define relationships between classes or objects.
  • Behavioral patterns: Behavioural patterns define manners of communication between classes and objects.

In the next articles we will go deeper in each one of the types and in the design patterns included in each one of the categories.

Design Patterns