Merge pull request #5 from bestephe/axon-dev

Adding in a P4 example that implements the Axon source-routed Ethernet Protocol

examples/axon/README.md

@@ -0,0 +1,102 @@
+# Axon source routing protocol
+
+## Description
+
+This program implements the Axon protocol for source-routed Ethernet
+described in the ANCS '10 paper "Axon: A Flexible Substrate for
+Source-routed Ethernet".  The most notable aspect of the Axon protocol
+is that, in addition to maintaining a list of forward hops, it also
+builds a list of the input ports the packet was received on.  Together,
+these hops define a reverse path to the source that uses the same links
+as the forward path.
+
+More specifically, the Axon header format is as follows:
+
+|   |   |   |   |   |   |
+|---|---|---|---|---|---|
+| AxonType : 8 | AxonHdrLength : 16 | FwdHopCount : 8 | RevHopCount : 8 | [FwdHops : 8] | [RevHops : 8] |
+|   |   |   |   |   |   |
+
+Note that the number of bits per field both shown above and used in this
+program are different from those described in the Axon ANCS '10 paper.
+Specifically, the width of the AxonType, FwdHopCount, and RevHopCount
+fields have been rounded up to the nearest multiple of 8.
+
+Upon receiving an Axon packet, an Axon switch performs three operations:
+
+1. It validates the header length matches the described number of
+   forward hops and reverse hops (and is less than a maximum length).
+2. It pushes the input port of the packet onto the list of reverse hops
+   and increments the reverse hop count.
+3. It pops the head off of the list of forward hops, decrements the
+   forward hop count, and then uses this port as an output port for the
+   packet.
+
+This program implements a switch that only performs these three operations.
+
+As an example, this program builds upon the P4 concepts introduced by
+the EasyRoute protocol, adding in simple TLV processing.  Similar to
+EasyRoute, the Axon protocol pops the next hop it should follow off of a
+list and decrements a header field.  However, the EasyRoute program can
+avoid TLV parsing by only parsing up to the first hop of the source
+route and then removing it.   On the other hand, the Axon protocol also
+requires that the input port of a packet is pushed onto a list.  This
+means that at the list of forward hops must be parsed. Because this list
+may be variable in length, this program must perform simple TLV parsing.
+However, unlike parsing an IP header, this TLV example is not
+complicated by issues related to ordering packet headers.  Additionally,
+this program parses the reverse hops of the Axon header, even though
+they do not strictly need to be if only Axon forwarding is performed,
+i.e., subsequent packet headers do not need to be parsed.
+
+Note that the header stacks parsed in this program (`axon_fwdHop` and
+`axon_revHop`) can only hold 64 entries, even though the parser could
+try to parse up to 256 entries (8 bits).  This is because of a
+limitation in `p4c-bmv2/p4c_bm/gen_json.py`.  If stack sizes of 256 are
+used, the script stalls for an extended period of time then generates
+the following error: `RuntimeError: maximum recursion depth exceeded`.
+
+Because of this error and because *parser exceptions* are not yet
+supported by bmv2, improperly formatted packets can cause simple\_switch
+to crash.  In practice, this occurs when IPv6 discovery packets are
+received.  In order to avoid this problem, like EasyRoute, this program
+also adds a 64bit preamble to the start of packets and requires that
+this preamble equals 0.  However, this only mitigates the problem.  A
+carefully crafted packet could still exceed the header stack of 64
+entries.
+
+### Running the demo
+
+We provide a small demo to let you test the program. It consists of the
+following scripts:
+- [run_demo.sh] (run_demo.sh): compiles the P4 program, starts the switch,
+  configures the data plane by running the CLI [commands]
+  (commands.txt), and starts the mininet console.
+- [receive.py] (receive.py): listens for Axon formatted packets.  This
+  command is intended to be run by a mininet host.
+- [send.py] (send.py): sends Axon formatted packets from one host to
+  another.  This command is intended to be run by a mininet host.
+
+To run the demo:
+./run_demo.sh will compile your code and create the Mininet network described
+above. It will also use commands.txt to configure each one of the switches.
+Once the network is up and running, you should type the following in the Mininet
+CLI:
+
+- `xterm h1`
+- `xterm h3`
+
+This will open a terminal for you on h1 and h3.
+
+On h3 run: `./receive.py`.
+
+On h1 run: `./send.py h1 h3`.
+
+You should then be able to type messages on h1 and receive them on h3. The
+`send.py` program finds the shortest path between h1 and h3 using Dijkstra, then
+send correctly-formatted packets to h3 through s1 and s3.  Once you are
+done testing, quit mininet.  .pcap files will be generated for every
+interface (9 files: 3 for each of the 3 switches). You can look at the
+appropriate files and check that packets are being processed correctly,
+e.g., the forward hops and reverse hops are updated appropriately and
+the correct output and input ports are used.

examples/axon/cleanup

@@ -0,0 +1,2 @@
+sudo killall simple_switch
+redis-cli FLUSHDB

examples/axon/commands.txt

@@ -0,0 +1,3 @@
+table_set_default route_pkt route
+table_add route_pkt _drop 0 0 =>
+table_set_default drop_pkt _drop

examples/axon/p4src/axon.p4

@@ -0,0 +1,196 @@
+/*
+Copyright 2013-present Barefoot Networks, Inc. 
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+/*
+ * Author: Brent Stephens (brentstephens "at" cs.wisc.edu)
+ *
+ * Notes and Future Directions:
+ *
+ * 1) In order to avoid problems caused by IPv6 packets, this program follows
+ *    the EasyRoute conventiion and requires that all packets start with a 64b
+ *    preamble that must be all zeros.  
+ *
+ * 2) Currently this program assumes that hosts are sending Axon packets.  In
+ *    some scenarios, it could also be desirable to transparently encapsulate
+ *    and decapsulate standard Ethernet packets  based on whether the switch
+ *    port is configured as an Ethernet port or an Axon port as is done on the
+ *    NetFPGA implementation of Axons.
+ */
+
+/*
+ * The sizes of these fields differ from those published in the Axon
+ * paper, but they seem like reasonable values.
+ */
+header_type axon_head_t {
+    fields {
+        /*
+         * If compatibility with EasyRoute is desirable, then a 64-bit preamble
+         * should also be included as a field of the packet header.
+         *
+         * While not strictly necessary and would waste header space on a real
+         * network, because p4-validate crashes with a stack of fwd/rev hops of
+         * size 256, if IPv6 discovery packets are not disabled, then we get
+         * the following error without this: "simple_switch:
+         * ./include/bm_sim/header_stacks.h:128: Header&
+         * HeaderStack::get_next(): Assertion `next < headers.size() && "header
+         * stack full"' failed."
+         */
+        preamble : 64;
+
+        axonType : 8;
+        axonLength : 16;
+        fwdHopCount : 8;
+        revHopCount : 8;
+    }
+}
+
+header_type axon_hop_t {
+    fields {
+        port : 8;
+    }
+}
+
+header_type my_metadata_t {
+    fields {
+        fwdHopCount : 8;
+        revHopCount : 8;
+        headerLen : 16;
+    }
+}
+
+header axon_head_t axon_head;
+//header axon_hop_t axon_fwdHop[256];
+//header axon_hop_t axon_revHop[256];
+//XXX: Workaround to avoid a "RuntimeError: maximum recursion depth exceeded" error from p4-validate
+/* 
+ * Specifically, using a stack size of 256 causes the following error:
+ * File "p4c-bmv2/p4c_bm/gen_json.py", line 370, in walk_rec
+ *   if hdr not in header_graph:
+ * RuntimeError: maximum recursion depth exceeded
+ */
+header axon_hop_t axon_fwdHop[64];
+header axon_hop_t axon_revHop[64];
+metadata my_metadata_t my_metadata;
+
+parser start {
+    /* Enable if compatibility with EasyRoute is desired. */
+    return select(current(0, 64)) {
+        0: parse_head;
+        default: ingress;
+    }
+
+    /* Enable if EasyRoute is being ignored */
+    //return parse_head;
+}
+
+parser parse_head {
+    extract(axon_head);
+    set_metadata(my_metadata.fwdHopCount, latest.fwdHopCount);
+    set_metadata(my_metadata.revHopCount, latest.revHopCount);
+    set_metadata(my_metadata.headerLen, 2 + axon_head.fwdHopCount + axon_head.revHopCount);
+    return select(latest.fwdHopCount) {
+        0: ingress;             // Drop packets with no forward hop
+        default: parse_next_fwdHop;
+    }
+}
+
+parser parse_next_fwdHop {
+    // Parse fwdHops until we have parsed them all
+    return select(my_metadata.fwdHopCount) {
+        0x0 : parse_next_revHop;
+        default : parse_fwdHop;
+    }
+}
+
+parser parse_fwdHop {
+    extract(axon_fwdHop[next]);
+    set_metadata(my_metadata.fwdHopCount,
+                 my_metadata.fwdHopCount - 1);
+    return parse_next_fwdHop;
+}
+
+parser parse_next_revHop {
+    // Parse revHops until we have parsed them all
+    return select(my_metadata.revHopCount) {
+        0x0 : ingress;
+        default : parse_revHop;
+    }
+}
+
+parser parse_revHop {
+    extract(axon_revHop[next]);
+    set_metadata(my_metadata.revHopCount,
+                 my_metadata.revHopCount - 1);
+    return parse_next_revHop;
+}
+
+action _drop() {
+    drop();
+}
+
+action route() {
+    // Set the output port
+    modify_field(standard_metadata.egress_spec, axon_fwdHop[0].port);
+
+    // Pop the fwdHop
+    modify_field(axon_head.fwdHopCount, axon_head.fwdHopCount -  1);
+    pop(axon_fwdHop, 1);
+
+    // Push the revHop
+    modify_field(axon_head.revHopCount, axon_head.revHopCount + 1);
+    push(axon_revHop, 1);
+    modify_field(axon_revHop[0].port, standard_metadata.ingress_port);
+
+    // Because we push and pop one port, the total length of the header does
+    // not change and thus does not need to be updated.
+}
+
+table drop_pkt {
+    actions {
+        _drop;
+    }
+    size: 1;
+}
+
+/* Question: will this drop packets that did not have a forward hop? */
+table route_pkt {
+    reads {
+        /* Technically axon_head is only written, not read.  Is this still
+         * correct then? */
+        axon_head: valid;
+
+        axon_fwdHop[0]: valid;  // Is using axon_fwdHop[0] correct?
+    }
+    actions {
+        _drop;
+        route;
+    }
+    size: 1;
+}
+
+control ingress {
+    // Drop packets whose length does not equal the total length of the header
+    if (axon_head.axonLength != my_metadata.headerLen) {
+        apply(drop_pkt);
+    }
+    else {
+        apply(route_pkt);
+    }
+}
+
+control egress {
+    // leave empty
+}

examples/axon/receive.py

@@ -0,0 +1,60 @@
+#!/usr/bin/python
+
+# Copyright 2013-present Barefoot Networks, Inc. 
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Author: Brent Stephens 
+#
+
+from scapy.all import sniff, sendp
+from scapy.all import Packet
+from scapy.all import ShortField, IntField, LongField, BitField
+
+import sys
+import struct
+
+def handle_pkt(pkt):
+    pkt = str(pkt)
+    if len(pkt) < 13: return
+    preamble = pkt[:8]
+    preamble_exp = "\x00" * 8
+    if preamble != preamble_exp: return
+    axonType = pkt[8]
+    if axonType != "\x00": return
+    axonLength = struct.unpack("!H", pkt[9:11])[0]
+    fwdHopCount = struct.unpack("B", pkt[11])[0]
+    revHopCount = struct.unpack("B", pkt[12])[0]
+    if fwdHopCount != 0:
+        print 'received a packet that has not been fully forwarded'
+    if revHopCount <= 0:
+        print 'received a packet that has no reverse hops'
+    if axonLength != 2 + fwdHopCount + revHopCount:
+        print 'received a packet with either an incorrect axonLength, fwdHopCount, or revHopCount'
+    msg = pkt[11 + axonLength:]
+    print msg
+    sys.stdout.flush()
+
+    # Optional debugging
+    #print 'axonLength:', axonLength
+    #print 'fwdHopCount:', fwdHopCount
+    #print 'revHopCount:', revHopCount
+    #print pkt
+
+def main():
+    sniff(iface = "eth0",
+          prn = lambda x: handle_pkt(x))
+
+if __name__ == '__main__':
+    main()

examples/axon/run_demo.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+# Copyright 2013-present Barefoot Networks, Inc. 
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+THIS_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
+
+source $THIS_DIR/../env.sh
+
+P4C_BM_SCRIPT=$P4C_BM_PATH/p4c_bm/__main__.py
+
+SWITCH_PATH=$BMV2_PATH/targets/simple_switch/simple_switch
+
+CLI_PATH=$BMV2_PATH/tools/runtime_CLI.py
+
+$P4C_BM_SCRIPT p4src/axon.p4 --json axon.json
+sudo PYTHONPATH=$PYTHONPATH:$BMV2_PATH/mininet/ python topo.py \
+    --behavioral-exe $BMV2_PATH/targets/simple_switch/simple_switch \
+    --json axon.json \
+    --cli $CLI_PATH

examples/axon/send.py

@@ -0,0 +1,109 @@
+#!/usr/bin/python
+
+# Copyright 2013-present Barefoot Networks, Inc. 
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from scapy.all import sniff, sendp
+from scapy.all import Packet
+from scapy.all import ByteField, ShortField, IntField, LongField, BitField
+
+import networkx as nx
+
+import sys
+
+class AxonRoute(Packet):
+    name = "AxonRoute"
+    fields_desc = [
+        LongField("preamble", 0),
+        ByteField("axonType", 0),
+        ShortField("axonLength", 0),
+        ByteField("fwdHopCount", 0),
+        ByteField("revHopCount", 0)
+    ]
+
+def read_topo():
+    nb_hosts = 0
+    nb_switches = 0
+    links = []
+    with open("topo.txt", "r") as f:
+        line = f.readline()[:-1]
+        w, nb_switches = line.split()
+        assert(w == "switches")
+        line = f.readline()[:-1]
+        w, nb_hosts = line.split()
+        assert(w == "hosts")
+        for line in f:
+            if not f: break
+            a, b = line.split()
+            links.append( (a, b) )
+    return int(nb_hosts), int(nb_switches), links
+
+def main():
+    if len(sys.argv) != 3:
+        print "Usage: send.py [this_host] [target_host]"
+        print "For example: send.py h1 h2"
+        sys.exit(1)
+
+    src, dst = sys.argv[1:]
+
+    nb_hosts, nb_switches, links = read_topo()
+
+    port_map = {}
+
+    for a, b in links:
+        if a not in port_map:
+            port_map[a] = {}
+        if b not in port_map:
+            port_map[b] = {}
+
+        assert(b not in port_map[a])
+        assert(a not in port_map[b])
+        port_map[a][b] = len(port_map[a]) + 1
+        port_map[b][a] = len(port_map[b]) + 1
+
+
+    G = nx.Graph()
+    for a, b in links:
+        G.add_edge(a, b)
+
+    shortest_paths = nx.shortest_path(G)
+    shortest_path = shortest_paths[src][dst]
+
+    print "path is:", shortest_path
+
+    port_list = []
+    first = shortest_path[1]
+    for h in shortest_path[2:]:
+        port_list.append(port_map[first][h])
+        first = h
+
+    print "port list is:", port_list
+
+    port_str = ""
+    for p in port_list:
+        port_str += chr(p)
+
+    while(1):
+        msg = raw_input("What do you want to send: ")
+
+        # finding the route
+        first = None
+
+        p = AxonRoute(axonLength = (2 + len(port_list)), fwdHopCount = len(port_list)) / port_str / msg
+        print p.show()
+        sendp(p, iface = "eth0")
+        # print msg
+
+if __name__ == '__main__':
+    main()

examples/axon/topo.py

@@ -0,0 +1,129 @@
+#!/usr/bin/python
+
+# Copyright 2013-present Barefoot Networks, Inc. 
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from mininet.net import Mininet
+from mininet.topo import Topo
+from mininet.log import setLogLevel, info
+from mininet.cli import CLI
+from mininet.link import TCLink
+
+from p4_mininet import P4Switch, P4Host
+
+import argparse
+from time import sleep
+import os
+import subprocess
+
+_THIS_DIR = os.path.dirname(os.path.realpath(__file__))
+_THRIFT_BASE_PORT = 22222
+
+parser = argparse.ArgumentParser(description='Mininet demo')
+parser.add_argument('--behavioral-exe', help='Path to behavioral executable',
+                    type=str, action="store", required=True)
+parser.add_argument('--json', help='Path to JSON config file',
+                    type=str, action="store", required=True)
+parser.add_argument('--cli', help='Path to BM CLI',
+                    type=str, action="store", required=True)
+
+args = parser.parse_args()
+
+class MyTopo(Topo):
+    def __init__(self, sw_path, json_path, nb_hosts, nb_switches, links, **opts):
+        # Initialize topology and default options
+        Topo.__init__(self, **opts)
+
+        for i in xrange(nb_switches):
+            switch = self.addSwitch('s%d' % (i + 1),
+                                    sw_path = sw_path,
+                                    json_path = json_path,
+                                    thrift_port = _THRIFT_BASE_PORT + i,
+                                    pcap_dump = True,
+                                    device_id = i)
+        
+        for h in xrange(nb_hosts):
+            host = self.addHost('h%d' % (h + 1))
+
+        for a, b in links:
+            self.addLink(a, b)
+
+def read_topo():
+    nb_hosts = 0
+    nb_switches = 0
+    links = []
+    with open("topo.txt", "r") as f:
+        line = f.readline()[:-1]
+        w, nb_switches = line.split()
+        assert(w == "switches")
+        line = f.readline()[:-1]
+        w, nb_hosts = line.split()
+        assert(w == "hosts")
+        for line in f:
+            if not f: break
+            a, b = line.split()
+            links.append( (a, b) )
+    return int(nb_hosts), int(nb_switches), links
+            
+
+def main():
+    nb_hosts, nb_switches, links = read_topo()
+
+    topo = MyTopo(args.behavioral_exe,
+                  args.json,
+                  nb_hosts, nb_switches, links)
+
+    net = Mininet(topo = topo,
+                  host = P4Host,
+                  switch = P4Switch,
+                  controller = None )
+    net.start()
+
+    for n in xrange(nb_hosts):
+        h = net.get('h%d' % (n + 1))
+        for off in ["rx", "tx", "sg"]:
+            cmd = "/sbin/ethtool --offload eth0 %s off" % off
+            print cmd
+            h.cmd(cmd)
+        print "disable ipv6"
+        h.cmd("sysctl -w net.ipv6.conf.all.disable_ipv6=1")
+        h.cmd("sysctl -w net.ipv6.conf.default.disable_ipv6=1")
+        h.cmd("sysctl -w net.ipv6.conf.lo.disable_ipv6=1")
+        h.cmd("sysctl -w net.ipv4.tcp_congestion_control=reno")
+        h.cmd("iptables -I OUTPUT -p icmp --icmp-type destination-unreachable -j DROP")
+
+    sleep(1)
+
+    for i in xrange(nb_switches):
+        cmd = [args.cli, "--json", args.json,
+               "--thrift-port", str(_THRIFT_BASE_PORT + i)]
+        with open("commands.txt", "r") as f:
+            print " ".join(cmd)
+            try:
+                output = subprocess.check_output(cmd, stdin = f)
+                print output
+            except subprocess.CalledProcessError as e:
+                print e
+                print e.output
+
+    sleep(1)
+
+    print "Ready !"
+
+    CLI( net )
+    net.stop()
+
+if __name__ == '__main__':
+    setLogLevel( 'info' )
+    main()

examples/axon/topo.txt

@@ -0,0 +1,8 @@
+switches 3
+hosts 3
+h1 s1
+h2 s2
+h3 s3
+s1 s2
+s1 s3
+s2 s3