I recently started working with Kafka. The first thing I do when start with a tech is work out how I am going to write tests as I am a TDD/XP nut.
For HTTP I use
Wiremock, for Cassandra I wrote a test double called
Stubbed
Cassandra. The term test double comes from the awesome book
Release
It! where it recommends for each technology you integrate with having a test double that you can prime to fail in every way possible.
I couldn't find anything for Kafka but I did find a couple of
blogs and gists for people running Kafka/Zookeeper in the same JVM as tests.
That's a start, I took it one step further and wrote a version that will hide away all the details, including a JUnit rule so you don't even need to start/stop it for tests as well as convenient methods to send and receive messages. Here's an example of an integration test for the KafkaUnit class:
Let's say you have some code that sends a message to Kafka, like this:
A unit test would look something like this:
It is in Maven Central, so if you want to use it just add the following dependency:
<dependency>
<groupId>info.batey.kafka</groupId>
<artifactId>kafka-unit</artifactId>
<version>0.1.1</version>
</dependency>
If you want to contribute check it out on
github.
It is pretty limited so far, assumed String messages etc. If I keep working with Kafka I'll extend it and add support for injecting faults etc. Also for the next version I'll come up with a versioning mechanism that includes the Kafka version.