Unit testing Java Cassandra applications

I often get asked what is the best method for unit and and integration testing Java code that communicates with Cassandra.

Here are what I think the options are:
  1. Mocking libraries: Use mocking libraries such as Mockito to mock out the driver you are using
  2. Cassandra Unit
  3. Integration test the class in question with against a real Cassandra instance running locally
  4. Stubbed Cassandra (disclaimer: I made this)
Lets take these one by one and look at their respective advantages and disadvantages. The things to consider are:
  • Speed of the tests
  • Able to run the tests concurrently and how easy this is to achieve
  • Able to test everything? Even failures?
  • Are the tests brittle?
  • Readability of the tests
  • Requirements on environment e.g do you need a Cassandra instance installed?
  • Confidence it will work against a real Cassandra
  • Subjective nonsense by writer of this blog

1 Mock the library


You can use a combination of factories and mocking to stop the driver actually interacting with Cassandra. Then verify your code's interactions with these mocks.

Advantages:
  • Fast - no I/O
  • Execute tests concurrently
  • No Cassandra instance required on machine your code compile runs
  • Can test everything including the various failures as you can mock the library to throw ReadTimeout exceptions etc
Disadvantages:
  • You may mock the driver to behave in a different way to how it will behave 
  • Very hard to understand tests due to large amount of boiler plate mocking code
  • Very brittle tests. Change your driver, change your tests! Change from a query to a prepared statement, change your tests!
  • A lot of boiler plate. Take the Datstax driver for example, it returns a ResultSet, which is iterable, fancy writing the code that mocks it returning many results?
Conclusion:
  • Don't do it if you wish to remain sane

2 Cassandra Unit


Cassandra Unit is a tool for starting an embedded Cassandra in the JVM your tests are running. It also has a great API for ingesting data into Cassandra for your tests.

Advantages:
  • Pretty fast. It is all in process.
  • Can run tests concurrently if they use different keyspaces and none of the tests turn it off etc
  • Can use CQL to load data as it is a real Cassandra. I think this leads to readable tests
  • No Cassandra required on machine. Can use the it via a Maven dependency
  • High confidence your code will work against a real Cassandra 

Disadvantages:
  • Unable to test failure scenarios. What is a read time out when there is only one node running in the same JVM as your test?
  • No way to verify consistency of queries

Conclusion:
  • Very useful tool for in process happy path tests


3 Integration style tests using a real Cassandra


This is something I've done a lot of recently. Every dev machine at my current work place has a Cassandra instance running (using the awesome tool ccm). Then we test our DAOs by assuming it is there and doing testing in a dynamic keyspace.

Advantages:
  • Can run tests concurrently if they use different keyspaces and none of the tests turn it off etc
  • Tests aren't brittle, can change from queries to prepared statements, or the queries involved without changing tests
  • Readable tests - all data setup is done in CQL
  • Very high confidence it will work against a real Cassandra as the test is against a  real Cassandra :)
Disadvantages:
  • Probably the slowest option. But it is still millisecond quick.
  • Hard to test scenarios other than turning the node off. This then makes the tests slow.
  • Cassandra required to build and run tests
Conclusion:
  • Slightly slower but very useful for testing happy paths
  • Very similar to using Cassandra unit

4 Stubbed Cassandra


Stubbed Cassandra is a new open source tool that pretends to be Cassandra and can be primed to returns rows, read timeouts, write timeouts and unavailable errors. It can be used via a maven dependency or as a standalone server.

Advantages:
  • Very fast
  • Can test all types of errors and be confident in what the driver does as the driver thinks it is a real Cassandra
  • Can run many instances inside the same JVM listening on different binary ports. So can run tests concurrently with no extra effort e.g no requirement to use different keyspaces
  • Tests less brittle than mocking the driver. Can change driver without changing test but if you change queries you need to update your priming
  • No requirement to have a real Cassandra. Just brought in by a maven dependency
Disadvantages:
  • Slightly more brittle than a real Cassandra/Cassandra unit. Requires priming on the query, priming for each prepared statement
  • Slightly less confidence it will work against a real Cassandra as it isn't a real Cassandra. But more confidence than mocking
  • It is new and does not support all of Cassandra's features, so if you use a feature that Scassandra doesn't support you are stuck!

Conclusion:
  • Best solution for all error case testing
  • Best solution if you need to execute tests concurrently
← Previous Post Next Post →