Oracle Coherence 3.5 : Accessing the data grid (part 6) - Using the Coherence API - Implementing CoherenceTarget, Testing the Cache loader

7/23/2012 6:04:46 PM
Implementing CoherenceTarget

The last thing we need to do to complete the example is to create a Target interface implementation that will import items read from the CSV file into Coherence.

One of the things we will need to do is to convert the generic item representation from a Map into an instance of a class that represents the value we want to put into the cache. The naïve approach is to use Java reflection to create an instance of a class and set property values, but just as with the CSV parsing, the devil is in the details.

The property values read from the CSV files are all strings, but the properties of the target object might not be. That means that we need to perform type conversion as appropriate when setting property values on a target object.

Fortunately, we don't need to reinvent the wheel to do this. If you are familiar with the Spring Framework, you already know that property values, specified as strings within the XML configuration file, are automatically converted to appropriate type before they are injected into your objects. What you might not know is that this feature is easily accessible outside of Spring as well, in the form of the BeanWrapper interface and BeanWrapperImpl class.

Another problem we need to solve is key generation-when we put objects into a Coherence cache, we need to specify both the key and the value. The simplest option, and the one we will use for this example, is to extract the key from the target object itself. This will often be all we need, as most entities already have a field representing their identity, which is the ideal candidate for a cache key. In the case of our Country class, we will use the value of the code property as a cache key.

Finally, while we could insert every item into the cache using individual put calls, this is not the most efficient way to perform bulk loading of the data. Achieving Performance, Scalability, and Availability Objectives, each call to the put method is potentially a network call, and as such introduces some latency. A significantly better approach from a performance perspective is to batch multiple items and insert them into the cache all at once by calling the putAll method.

So, with the design considerations out of the way, let's look at the implementation of the CoherenceTarget class:

public class CoherenceTarget implements Target {
public static final int DEFAULT_BATCH_SIZE = 1000;
private NamedCache cache;
private Class itemClass;
private String idProperty;
private Map batch;
private int batchSize = DEFAULT_BATCH_SIZE;
public CoherenceTarget(String cacheName, Class itemClass,
String idProperty) {
this.cache = CacheFactory.getCache(cacheName);
this.itemClass = itemClass;
this.idProperty = idProperty;
public void setBatchSize(int batchSize) {
this.batchSize = batchSize;
public void beginImport() {
batch = new HashMap();
public void importItem(Map<String, ?> sourceItem) {
BeanWrapper targetItem = new BeanWrapperImpl(itemClass);
for (Map.Entry<String, ?> property : sourceItem.entrySet()) {
targetItem.setPropertyValue(property.getKey(), property.getValue());
Object id = targetItem.getPropertyValue(idProperty);
batch.put(id, targetItem.getWrappedInstance());
if (batch.size() % batchSize == 0) {
CoherenceTarget classimplementingbatch.clear();
public void endImport() {
if (!batch.isEmpty()) {

The constructor accepts three arguments: the name of the cache to import objects into, the class of cache items, and the name of the property of that class that should be used as a cache key. We initialize batch size to 1000 items by default, but the value can be easily overridden by calling the setBatchSize method.

The beginImport lifecycle method initializes the map representing a batch of items that need to be inserted, while the endImport method ensures that the last, potentially incomplete, batch is also inserted into the cache.

The real meat is in the importItem method, which creates a Spring BeanWrapper instance for the specified item class and sets its properties based on the entries in the sourceItem map. Once the target item is fully initialized, we use BeanWrapper again to extract the cache key from it and add the item to the batch.

Finally, whenever the batch size reaches the specified limit, we insert all items from the batch into the cache and clear the batch.

With this last piece in place, we are ready to test the loader.

Cache loader on steroids

It also contains both Source and Target implementations for CSV files, XML files, and Coherence, allowing you to import data from the existing CSV and XML files into Coherence, and to export contents of a Coherence cache into one of these formats.

Testing the Cache loader

In order to test the loader, we will use a countries.csv file containing a list of most countries in the world (I'd say all, but being from Serbia I know firsthand that a list of countries in the world is anything but static ☺).

The file header matches exactly the property names in the Country class defined earlier, which is the requirement imposed by our loader implementation:

AND,Andorra,Andorra la Vella,EUR,Euro

With the test data in place, let's write a JUnit test that will load all of the countries defined in the countries.csv file into the Coherence cache:

public class LoaderTests {
public static final NamedCache countries = CacheFactory.getCache("countries");
public void clearCache() {
public void testCsvToCoherenceLoader() {
Source source = new CsvSource("countries.csv");
CoherenceTarget target =
new CoherenceTarget("countries", Country.class, "code");
Loader loader = new Loader(source, target);
assertEquals(193, countries.size());
Country srb = (Country) countries.get("SRB");
assertEquals("Serbia", srb.getName());
assertEquals("Belgrade", srb.getCapital());
assertEquals("RSD", srb.getCurrencySymbol());
assertEquals("Dinar", srb.getCurrencyName());


As you can see, using the cache loader is quite simple-we initialize the source and target, create a Loader instance for them, and invoke the load method. We then assert that all the data from the test CSV file has been loaded, and that the properties of a single object retrieved from the cache are set correctly.

However, the real purpose of the previous test is neither to teach you how to use the loader (you could've easily figured that out yourself) nor to prove that the code in the preceding sections works (of course it does ☺). It is to lead us into the very interesting subject that follows.

  •  Oracle Coherence 3.5 : Installing Coherence, Starting up the Coherence cluster
  •  The Go-To Reference Design Map For The Cloud?
  •  Separating BPM and SOA Processes : SOA-Oriented Disputes with BEA
  •  Science! – Spintronics
  •  Linux - The Operating System With A Pure Heart (Part 2)
  •  Linux - The Operating System With A Pure Heart (Part 1)
  •  What's Your Strategy For Today’s Server Room Challenges?
  •  Truly Transparent Trinkets
  •  Gamma – Ray Lens …Made Possible
  •  Tech Chance - Revamps …Worsts Ever!
  •  Organic Computing (Part 2)
  •  Organic Computing (Part 1)
  •  The Price of Computer Components Is Going Up? (Part 3)
  •  The Price of Computer Components Is Going Up? (Part 2)
  •  The Price of Computer Components Is Going Up? (Part 1)
  •  The Best Computers You're (Probably) Never Heard Of (Part 3) - Z88, Atari Falcon
  •  The Best Computers You're (Probably) Never Heard Of (Part 2) - Tatung Einstein, Camputers Lynx
  •  The Best Computers You're (Probably) Never Heard Of (Part 1) - Xerox Star, The Grundy NewBrain
  •  Embarrassing Bugs (Part 3)
  •  Embarrassing Bugs (Part 2)
    Top 10
    Nikon 1 J2 With Stylish Design And Dependable Image And Video Quality
    Canon Powershot D20 - Super-Durable Waterproof Camera
    Fujifilm Finepix F800EXR – Another Excellent EXR
    Sony NEX-6 – The Best Compact Camera
    Teufel Cubycon 2 – An Excellent All-In-One For Films
    Dell S2740L - A Beautifully Crafted 27-inch IPS Monitor
    Philips 55PFL6007T With Fantastic Picture Quality
    Philips Gioco 278G4 – An Excellent 27-inch Screen
    Sony VPL-HW50ES – Sony’s Best Home Cinema Projector
    Windows Vista : Installing and Running Applications - Launching Applications
    Most View
    Bamboo Splash - Powerful Specs And Friendly Interface
    Powered By Windows (Part 2) - Toshiba Satellite U840 Series, Philips E248C3 MODA Lightframe Monitor & HP Envy Spectre 14
    MSI X79A-GD65 8D - Power without the Cost
    Canon EOS M With Wonderful Touchscreen Interface (Part 1)
    Windows Server 2003 : Building an Active Directory Structure (part 1) - The First Domain
    Personalize Your iPhone Case
    Speed ​​up browsing with a faster DNS
    Using and Configuring Public Folder Sharing
    Extending the Real-Time Communications Functionality of Exchange Server 2007 : Installing OCS 2007 (part 1)
    Google, privacy & you (Part 1)
    iPhone Application Development : Making Multivalue Choices with Pickers - Understanding Pickers
    Microsoft Surface With Windows RT - Truly A Unique Tablet
    Network Configuration & Troubleshooting (Part 1)
    Panasonic Lumix GH3 – The Fastest Touchscreen-Camera (Part 2)
    Programming Microsoft SQL Server 2005 : FOR XML Commands (part 3) - OPENXML Enhancements in SQL Server 2005
    Exchange Server 2010 : Track Exchange Performance (part 2) - Test the Performance Limitations in a Lab
    Extra Network Hardware Round-Up (Part 2) - NAS Drives, Media Center Extenders & Games Consoles
    Windows Server 2003 : Planning a Host Name Resolution Strategy - Understanding Name Resolution Requirements
    Google’s Data Liberation Front (Part 2)
    Datacolor SpyderLensCal (Part 1)