Apache Stormのインストールとサンプル実行
１．Apache Stormのインストール
# wget http://www-eu.apache.org/dist/storm/apache-storm-1.0.2/apache-storm-1.0.2.tar.gz
# tar xvzf apache-storm-1.0.2.tar.gz
# mv apache-storm-1.0.2.tar.gz /usr/local/lib/strom
# データディレクトリの作成
# cd /usr/local/lib/storm
# mkdir data
２．設定ファイルの作成
# vim conf/storm.yaml
storm.zookeeper.servers:
- "localhost"
storm.local.dir: "/usr/local/lib/storm/data"
nimbus.host: "localhost"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
３．Stormの起動(nimbus/supervisor/ui)
# bin/storm nimbus &
# bin/storm supervisor &
# bin/storm ui &
４．Zookepperの起動（インストールは割愛）
# cd /usr/local/lib/zookeeper
# ./bin/zkServer.sh start &
５．サンプルの実行
# cd /usr/local/lib/storm/examples/storm-starter/
# storm jar target/storm-starter-1.1.0.jar org.apache.storm.starter.ExclamationTopology local
Running: /usr/java/jdk1.8.0_131/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/lib/storm -Dstorm.log.dir=/usr/local/lib/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/lib/storm/lib/storm-core-1.1.0.jar:/usr/local/lib/storm/lib/kryo-3.0.3.jar:/usr/local/lib/storm/lib/reflectasm-1.10.1.jar:/usr/local/lib/storm/lib/asm-5.0.3.jar:/usr/local/lib/storm/lib/minlog-1.3.0.jar:/usr/local/lib/storm/lib/objenesis-2.1.jar:/usr/local/lib/storm/lib/clojure-1.7.0.jar:/usr/local/lib/storm/lib/ring-cors-0.1.5.jar:/usr/local/lib/storm/lib/disruptor-3.3.2.jar:/usr/local/lib/storm/lib/log4j-api-2.8.jar:/usr/local/lib/storm/lib/log4j-core-2.8.jar:/usr/local/lib/storm/lib/log4j-slf4j-impl-2.8.jar:/usr/local/lib/storm/lib/slf4j-api-1.7.21.jar:/usr/local/lib/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/lib/storm/lib/servlet-api-2.5.jar:/usr/local/lib/storm/lib/storm-rename-hack-1.1.0.jar:target/storm-starter-1.1.0.jar:/usr/local/lib/storm/conf:/usr/local/lib/storm/bin -Dstorm.jar=target/storm-starter-1.1.0.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.storm.starter.ExclamationTopology local
1890 [main] INFO o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -8548623279120140126:-5008538972653927482
2155 [main] WARN o.a.s.u.NimbusClient - Using deprecated config nimbus.host for backward compatibility. Please update your storm.yaml so it only has config nimbus.seeds
2290 [main] INFO o.a.s.u.NimbusClient - Found leader nimbus : localhost:6627
2335 [main] INFO o.a.s.s.a.AuthUtils - Got AutoCreds
2336 [main] WARN o.a.s.u.NimbusClient - Using deprecated config nimbus.host for backward compatibility. Please update your storm.yaml so it only has config nimbus.seeds
2346 [main] INFO o.a.s.u.NimbusClient - Found leader nimbus : localhost:6627
2449 [main] INFO o.a.s.StormSubmitter - Uploading dependencies - jars...
2450 [main] INFO o.a.s.StormSubmitter - Uploading dependencies - artifacts...
2450 [main] INFO o.a.s.StormSubmitter - Dependency Blob keys - jars : / artifacts : []
2476 [main] INFO o.a.s.StormSubmitter - Uploading topology jar target/storm-starter-1.1.0.jar to assigned location: /usr/local/lib/storm/data/nimbus/inbox/stormjar-e3b27253-e04f-4a0b-958b-dbccaf869716.jar
Start uploading file 'target/storm-starter-1.1.0.jar' to '/usr/local/lib/storm/data/nimbus/inbox/stormjar-e3b27253-e04f-4a0b-958b-dbccaf869716.jar' (64300713 bytes)
[==================================================] 64300713 / 64300713
６．結果確認
# ls -l /usr/local/lib/storm/logs/workers-artifacts/
…
drwxr-xr-x 5 root root 39 9月 1 20:17 local-1-1504264614
…
# less /usr/local/lib/storm/logs/workers-artifacts/local-1-1504264614/6701/worker.log
…
2017-09-01 20:18:03.475 o.a.s.d.task Thread-15-word-executor[14 14] [INFO] Emitting: word default [jackson]
2017-09-01 20:18:03.478 o.a.s.d.executor Thread-15-word-executor[14 14] [INFO] TRANSFERING tuple [dest: 5 tuple: source: word:14, stream: default, id: {}, [jackson]]
2017-09-01 20:18:03.482 o.a.s.d.executor Thread-13-exclaim1-executor[5 5] [INFO] Processing received message FOR 5 TUPLE: source: word:14, stream: default, id: {}, [jackson]
2017-09-01 20:18:03.483 o.a.s.d.task Thread-13-exclaim1-executor[5 5] [INFO] Emitting: exclaim1 default [jackson!!!]
2017-09-01 20:18:03.484 o.a.s.d.executor Thread-13-exclaim1-executor[5 5] [INFO] TRANSFERING tuple [dest: 8 tuple: source: exclaim1:5, stream: default, id: {}, [jackson!!!]]
…
とまあこんな感じ。
