Trema-edge + OVSでOpenFlow1.3テスト環境を作る


そろそろ OpenFlow 1.3 なテストをやってみたいなあと思ったので Trema-edge のビルドと KVM + Open vSwitch でのテスト環境の構築をやってみた。といっても、Trema-edge でサンプル (learning-switch) が動くってところまで。


  • Controller
    • Ubuntu Server 13.04, Linux/3.8.0-27-generic
    • ruby/2.0.0-p247
    • trema-edge/0.1.0 (8/13時点でのHead)
  • KVM Host
    • Ubuntu Server 13.04, Linux/3.8.0-26-generic
    • Open vSwitch/1.11.90

KVM Host の方は Ubuntu/KVM/OVSでOpenFlowテスト環境をつくる(7) で使った物で、OVS はソースからビルドしてあります。以下、Controller 側のセットアップ手順を主に記載。


trema/trema-edge の README を見ながらセットアップを進めるのですが、まず Ruby2.0 のインストールをします。

This repository has only been tested with ruby 2.0.0 (maybe with 1.9.3) and will not work with 1.8.x. We recommend installation of the rvm program for easy installation of ruby 2.0.0.

Ubuntu 13.04 だと ruby-1.9.3 までは apt-get してインストールできるのですが、README の推奨に従って rvm を使ってインストールしていきます。

RVM のインストール


RVM は curl を使ってインストールされるので、curl がインストールされていることを確認します。(既に入っていた)

stereocat@vnwt10ctrl-of13:~$ which curl
stereocat@vnwt10ctrl-of13:~$ curl --version
curl 7.29.0 (x86_64-pc-linux-gnu) libcurl/7.29.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.25 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

RVM: Ruby Version Manager - RVM Ruby Version Manager - Documentation に書いてある手順で rvm のインストール実行。

stereocat@vnwt10ctrl-of13:~$ curl -L | bash -s stable
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   184  100   184    0     0     73      0  0:00:02  0:00:02 --:--:--   318
100 14547  100 14547    0     0   3762      0  0:00:03  0:00:03 --:--:-- 25884
Downloading RVM from wayneeseguin branch stable
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   124  100   124    0     0    177      0 --:--:-- --:--:-- --:--:--   181
100 1083k  100 1083k    0     0  46832      0  0:00:23  0:00:23 --:--:-- 10372

Installing RVM to /home/stereocat/.rvm/
    Adding rvm PATH line to /home/stereocat/.bashrc /home/stereocat/.zshrc.
    Adding rvm loading line to /home/stereocat/.bash_profile /home/stereocat/.zprofile.
Installation of RVM in /home/stereocat/.rvm/ is almost complete:

  * To start using RVM you need to run `source /home/stereocat/.rvm/scripts/rvm`
    in all your open shell windows, in rare cases you need to reopen all shell windows.

# stereocat,
#   Thank you for using RVM!
#   I sincerely hope that RVM helps to make your life easier and more enjoyable!!!
# ~Wayne

# In case of problems:
#      run and read: rvm notes
#         read docs:
#        talk to us: (
#   read cheatsheet:
#  watch screencast:
# open a bug report:

  * WARNING: You have '~/.profile' file, you might want to load it,
    to do that add the following line to '/home/stereocat/.bash_profile':

      source ~/.profile


ユーザ別の環境を用意するために環境変数の設定とかが .bash_profile に追加されていたりします。

stereocat@vnwt10ctrl-of13:~$ cat .bash_profile

[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function*

.profile とかの設定を追加しておけ、とか出てくるので入れておきます。


stereocat@vnwt10ctrl-of13:~$ rvm list known
# MRI Rubies


ruby-2.0.0 (p247)をインストールします。

stereocat@vnwt10ctrl-of13:~$ rvm install ruby-2.0.0
ruby-2.0.0-p247 - #validate archive
ruby-2.0.0-p247 - #extract
ruby-2.0.0-p247 - #validate binary
ruby-2.0.0-p247 - #setup
Saving wrappers to '/home/stereocat/.rvm/wrappers/ruby-2.0.0-p247'........
ruby-2.0.0-p247 - #importing default gemsets, this may take time.......................
stereocat@vnwt10ctrl-of13:~$ rvm list

rvm rubies

   ruby-2.0.0-p247 [ x86_64 ]

# Default ruby not set. Try 'rvm alias create default <ruby>'.

# => - current
# =* - current && default
#  * - default



tereocat@vnwt10ctrl-of13:~$ rvm alias create default ruby-2.0.0-p247
Creating alias default for ruby-2.0.0-p247.
Recording alias default for ruby-2.0.0-p247.
Creating default links/files
Saving wrappers to '/home/stereocat/.rvm/bin'........
stereocat@vnwt10ctrl-of13:~$ rvm list

rvm rubies

=* ruby-2.0.0-p247 [ x86_64 ]

# => - current
# =* - current && default
#  * - default


ruby のバージョン確認

stereocat@vnwt10ctrl-of13:~$ which ruby
stereocat@vnwt10ctrl-of13:~$ ruby -v
ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-linux]

trema-edge ビルドのためのパッケージ類をインストール

trema-edge README に従ってその他必要なパッケージ類を入れます。

stereocat@vnwt10ctrl-of13:~$ sudo apt-get install gcc make libsqlite3-dev libpcap-dev libssl-dev

trema-edge のビルド

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ gem install bundler

bundle install でコケる

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ bundle install
ERROR: RVM Ruby not used, run `rvm use ruby` first.
stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ rvm use ruby ruby-2.0.0-p247

RVM is not a function, selecting rubies with 'rvm use ...' will not work.

You need to change your terminal emulator preferences to allow login shell.
Sometimes it is required to use `/bin/bash --login` as the command.
Please visit for a example.


RVM の環境変数設定の都合のようだ。(screen つかって shell 起動してるから)

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ source ~/.bash_profile
You are using '.rvmrc', it requires trusting, it is slower and it is not compatible with other ruby managers,
you can switch to '.ruby-version' using 'rvm rvmrc to [.]ruby-version'
or ignore this warning with 'rvm rvmrc warning ignore /home/stereocat/trema-orig/trema-edge/.rvmrc',
'.rvmrc' will continue to be the default project file in RVM 1 and RVM 2,
to ignore the warning for all files run 'rvm rvmrc warning ignore all.rvmrcs'.

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ bundle install
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
Post-install message from reek:
Thank you for downloading Reek. For info see the reek wiki


stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ rake | tee rake.log 2>&1


learning-switch を起動。

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ ./trema run src/examples/learning_switch/learning-switch.rb -c src/examples/learning_switch/sample.conf


stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ ./trema send_packets --source=host1 --dest=host2
stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ ./trema show_stats host1
Sent packets:
Received packets:

stereocat@vnwt10ctrl-of13:~/trema-orig/trema-edge$ ./trema show_stats host2
Sent packets:

Received packets:

Open vSwitch 設定

KVM Guest 用にブリッジ(br1)を作ります。

  • Guest
    • ofvm01,, tap1
    • ofvm02,, tap2
root@vnwt11:~# sudo ovs-vsctl add-br br1
root@vnwt11:~# sudo ovs-vsctl set bridge br1 other-config:datapath-id=0000000000000011
root@vnwt11:~# sudo ovs-vsctl set bridge br1 protocols=OpenFlow13
root@vnwt11:~# sudo ovs-vsctl set-controller br1 tcp:
root@vnwt11:~# sudo ovs-vsctl show
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
        Port "eth0"
            Interface "eth0"
    Bridge "br1"
        Controller "tcp:"
        Port "tap2"
            Interface "tap2"
        Port "tap1"
            Interface "tap1"
        Port "br1"
            Interface "br1"
                type: internal

ovs-vsctl show では出てきていませんが、OVSDB 上にプロトコル指定が入ります。

root@vnwt11:~# ovs-vsctl list bridge
_uuid               : 65445e5a-ddc6-488f-ac56-1cac61c21b25
controller          : [9f711a8e-d1e4-410d-8dce-841850cb4dba]
datapath_id         : "0000000000000011"
datapath_type       : ""
external_ids        : {}
fail_mode           : []
flood_vlans         : []
flow_tables         : {}
ipfix               : []
mirrors             : []
name                : "br1"
netflow             : []
other_config        : {datapath-id="0000000000000011"}
ports               : [02c5f677-efbd-4392-8e88-225e11e3de21, 80250ddc-15ef-4512-a760-d1390c00b491, c37cb2f9-35a8-49a8-af13-1a5bf258a7c0]
protocols           : ["OpenFlow13"]
sflow               : []
status              : {}
stp_enable          : false

Controller ( と接続してフロー情報を見てみます。

root@vnwt11:~# sudo ovs-ofctl dump-flows br1
2013-08-18T07:43:15Z|00001|vconn|WARN|unix:/usr/local/var/run/openvswitch/br1.mgmt: version negotiation failed (we support version 0x01, peer supports version 0x04)
ovs-ofctl: br1: failed to connect to socket (Broken pipe)

…と、OpenFlow バージョンが違うって言われてしもた。プロトコル指定が必要。(これは Trema-edge/learning-switch Controller でホスト間通信 (ofvm01-ofvm02 間) を行ったときの状態。)

root@vnwt11:~# sudo ovs-ofctl dump-flows br1 --protocol=OpenFlow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x1, duration=182.155s, table=0, n_packets=18, n_bytes=1540, send_flow_rem priority=0 actions=CONTROLLER:65535
cookie=0x11, duration=69.399s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=0,icmp,in_port=1,dl_src=52:54:00:01:00:01,dl_dst=52:54:00:01:00:02,nw_src=,nw_dst=,nw_tos=0,nw_ecn=0,icmp_type=8,icmp_code=0 actions=output:2
cookie=0x12, duration=69.387s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=0,icmp,in_port=2,dl_src=52:54:00:01:00:02,dl_dst=52:54:00:01:00:01,nw_src=,nw_dst=,nw_tos=0,nw_ecn=0,icmp_type=0,icmp_code=0 actions=output:1
cookie=0xb, duration=87.5s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=0,arp,in_port=2,dl_src=52:54:00:01:00:02,dl_dst=52:54:00:01:00:01,arp_spa=,arp_tpa=,arp_op=1,arp_sha=52:54:00:01:00:02,arp_tha=00:00:00:00:00:00 actions=output:1
cookie=0x2, duration=92.576s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=0,arp,in_port=2,dl_src=52:54:00:01:00:02,dl_dst=52:54:00:01:00:01,arp_spa=,arp_tpa=,arp_op=2,arp_sha=52:54:00:01:00:02,arp_tha=52:54:00:01:00:01 actions=output:1
cookie=0xc, duration=87.488s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=0,arp,in_port=1,dl_src=52:54:00:01:00:01,dl_dst=52:54:00:01:00:02,arp_spa=,arp_tpa=,arp_op=2,arp_sha=52:54:00:01:00:01,arp_tha=52:54:00:01:00:02 actions=output:2

OpenFlow Dissector をインストールする

Wireshark OpenFlow Dissector and TremaShark とかに書いていますが、これまでは

を使っていました。が、これ、OpenFlow/1.0 にしか対応してないんだよね。こんな感じになる。("Warning: Dissector written for OpenFlow v0x1)

で、探すと CPqD で OpenFlow 1.0-1.3 対応の Dissector が公開されているのでそっちを使います。


stereocat@vnwt10ctrl-of13:~/trema-orig$ sudo apt-get install wireshark-dev libgtk2.0-dev scons

Dissector のソース取得とビルド

stereocat@vnwt10ctrl-of13:~/trema-orig$ git clone
stereocat@vnwt10ctrl-of13:~/trema-orig$ cd ofdissector/src/
stereocat@vnwt10ctrl-of13:~/trema-orig/ofdissector/src$ ls
of10  of11  of12  of13  openflow  openflow-common.cpp  openflow-common.hpp  plugin.cpp  Sconstruct  util
stereocat@vnwt10ctrl-of13:~/trema-orig/ofdissector/src$ export WIRESHARK=/usr/include/wireshark/
stereocat@vnwt10ctrl-of13:~/trema-orig/ofdissector/src$ scons install | tee scons-install.log 2>&1
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o openflow-common.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include openflow-common.cpp
In file included from openflow-common.cpp:9:0:
./of13/openflow-130.hpp:12:0: warning: "PROTO_TAG_OPENFLOW_VER" redefined [enabled by default]
In file included from openflow-common.cpp:8:0:
./of12/openflow-120.hpp:13:0: note: this is the location of the previous definition
g++ -o plugin.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include plugin.cpp
g++ -o of10/openflow-100.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include of10/openflow-100.cpp
g++ -o of11/openflow-110.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include of11/openflow-110.cpp
g++ -o of12/openflow-120.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include of12/openflow-120.cpp
g++ -o of13/openflow-130.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include of13/openflow-130.cpp
g++ -o util/FieldManager.os -c -fPIC -I. -I/usr/include/wireshark -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include util/FieldManager.cpp
g++ -o -shared openflow-common.os plugin.os of10/openflow-100.os of11/openflow-110.os of12/openflow-120.os of13/openflow-130.os util/FieldManager.os -L/usr/lib -L/usr/local/lib
Install file: "" as "/home/stereocat/.wireshark/plugins/"
scons: done building targets.

Plugin の確認。 が残っているけど、これは OF1.0 のみ対応のものなので消しておきます。

stereocat@vnwt10ctrl-of13:~/.wireshark/plugins$ ls -l
total 604
-rwxrwxr-x 1 stereocat stereocat 422852 Aug 18 18:58
-rw-r--r-- 1 stereocat stereocat 189961 Aug 18 17:03

OVS-Controller(trema-edge/learning-switch) 間のキャプチャを取ってみるとこんな感じ。


とりあえず Trema-edge が動くってところまで。動くと言ってもまだ learning-switch しか動かしてないけど。動くってことと、OpenFlow 通信の観測ってところまではできたので、あとは具体的なアプリ作ってみるってところに行けるかな−。そのまえに OF1.3 そのものの知識を入れておかねば…。

マスタリングTCP/IP OpenFlow編

マスタリングTCP/IP OpenFlow編