Today we are going to build a very simple demo code using Spring Boot and Kafka.
The application is going to contain a simple producer and consumer. In addition, we will add a simple endpoint to test our development and configuration.
Let’s start.
The project is going to be using:
A good place to start generating our project is Spring Initialzr. There we can create easily the skeleton of our project adding some basic information about our project. We will be adding two dependencies:
- Spring Web.
- Spring for Apache Kafka.
- Spring Configuration Processor (Optional).
Once we are done filling the form we only need to generate the code and open it on our favourite code editor.
As an optional dependency, I have added the “Spring Boot Configuration Processor” dependency to be able to define some extra properties that we will be using on the “application.properties” file. As I have said is optional, we are going to be able to define and use the properties without it but, it going to solve the warning of them not been defined. Up to you.
Whit the three dependencies, our “pom.xml” should look something like:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
The next step is going to be creating our kafka producer and consumer to be able to send a message using the distributed event streaming platform.
For the producer code we are just going to create a basic method to send a message making use of the “KafkaTemplate” offered by Spring.
@Service
public class KafkaProducer {
public static final String TOPIC_NAME = "example_topic";
private final KafkaTemplate<String, String> kafkaTemplate;
public KafkaProducer(KafkaTemplate<String, String> kafkaTemplate) {
this.kafkaTemplate = kafkaTemplate;
}
public void send(final String message) {
kafkaTemplate.send(TOPIC_NAME, message);
}
}
The consumer code is going to be even more simple thanks to the “KafkaListener” provided by Spring.
@Service
public class KafkaConsumer {
@KafkaListener(topics = {KafkaProducer.TOPIC_NAME}, groupId = "example_group_id")
public void read(final String message) {
System.out.println(message);
}
}
And finally, to be able to test it, we are going to define a Controller to invoke the Kafka producer.
@RestController
@RequestMapping("/kafka")
public class KafkaController {
private final KafkaProducer kafkaProducer;
public KafkaController(KafkaProducer kafkaProducer) {
this.kafkaProducer = kafkaProducer;
}
@PostMapping("/publish")
public void publish(@RequestBody String message) {
Objects.requireNonNull(message);
kafkaProducer.send(message);
}
}
With this, all the necessary code is done. Let’s now go for the configuration properties and the necessary Docker images to run all of this.
First, the “application.properties” file. It is going to contain some basic configuration properties for the producer and consumer.
server.port=8081
spring-boot-kafka.config.kafka.server=localhost
spring-boot-kafka.config.kafka.port=9092
# Kafka consumer properties
spring.kafka.consumer.bootstrap-servers=${spring-boot-kafka.config.kafka.server}:${spring-boot-kafka.config.kafka.port}
spring.kafka.consumer.group-id=example_group_id
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
# kafka producer properties
spring.kafka.producer.bootstrap-servers=${spring-boot-kafka.config.kafka.server}:${spring-boot-kafka.config.kafka.port}
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer
In the line 8, we can see the property “spring.kafka.consumer.group-id”. If we look carefully at it, we will see that it match the previous definition of the “groupId” we have done on the consumer.
Lines 10, 11 and 15, 16 define the serialization and de-serialization classes.
Finally, in the list 3 and 4 we have defined a couple of properties to avoid repetition. This are the properties that are showing us a warning message.
To fix it, if we have previously added the “Spring Configuration Processor” dependency, now, we can add the file:
spring-boot-kafka/src/main/resources/META-INF/additional-spring-configuration-metadata.json
With the definition of this properties:
{
"properties": [
{
"name": "spring-boot-kafka.config.kafka.server",
"type": "java.lang.String",
"description": "Location of the Kafka server."
},
{
"name": "spring-boot-kafka.config.kafka.port",
"type": "java.lang.String",
"description": "Port of the Kafka server."
}
]
}
We are almost there. The only thing remaining is the Apache Kafka. Because we do not want to deal with the complexity of setting an Apache Kafka server, we are going to leverage the power of Docker and create a “docker-compose” file to run it for us:
version: '3'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookepper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
environment:
KAFKA_ADVERTISED_HOST_NAME: localhost
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_CREATE_TOPIC: "example_topic:1:3"
As we can see, simple stuff, nothing out of the ordinary. Two images, one for Zookepper and one for Apache Kafka, the definition of some ports (remember to match them with the ones in the application.propeties file) and a few variables need for the Apache Kafka image.
With this, we can run the docker-compose file and obtain two containers running:
Now, we can test the end point we have built previously. In this case, to make it simple, we are going to use curl:
`curl -d '{"message":"Hello from Kafka!}' -H "Content-Type: application/json" -X POST http://localhost:8081/kafka/publish`
The result should be something like:
And, this is all. You can find the full source code here.
Enjoy it!
Binary Coders 