Writing a Topology

A topology specifies components like spouts and bolts, as well as the relation between components and proper configurations.

Spouts and Bolts discuss how to implement a spouts and bolts, respectively.

After defining the spouts and bolts, a topology can be composed using TopologyBuilder. The TopologyBuilder has two major methods to specify the components:

  • setBolt(String id, IRichBolt bolt, Number parallelismHint): id is the unique identifier that assigned to a bolt, bolt is the one previously composed, and parallelismHint is a number that specifying the number of instances of this bolt.

  • setSpout(String id, IRichSpout spout, Number parallelismHint): id is the unique identifier that assigned to a spout, spout is the one previously composed, and parallelismHint is a number that specifying the number of instances of this spout.

A simple example is as follows:


TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("word", new TestWordSpout(), 5);
builder.setBolt("exclaim", new ExclamationBolt(), 4);

In addition to the component specification, how to transmit Tuples between the components must also be specified. This is defined by different grouping strategies:

  • Fields Grouping: Tuples are transmitted to bolts based on a given field. Tuples with the same field will always go to the same bolt.
  • Global Grouping: All the Tuples are transmitted to a single instance of a bolt with the lowest task id.
  • Shuffle Grouping: Tuples are randomly transmitted to different instances of a bolt.
  • None Grouping: Currently, it equals to shuffle grouping.
  • All Grouping: All Tuples are transmitted to all instances of a bolt.
  • Custom Grouping: User-defined grouping strategy.

The following snippet is a simple example of specifying shuffle grouping between our word spout and exclaim bolt.


builder.setBolt("exclaim", new ExclamationBolt(), 4)
  .shuffleGrouping("word");

Once the components and the grouping are specified, the topology can be built.

HeronTopology topology = builder.createTopology();

See the ExclamationTopology for the complete example. More examples can be found in the examples package.