Adding starter code and solution for p4runtime exercise (#81)

Summary of changes:
- Adding the p4runtime starter code and solution.
- Adding NO_P4, BMV2_SWITCH_EXE and P4C_ARGS to utils/Makefile
- Updated p4runtime/Makefile to use variables
- Adding conversion functions for match and action param values
- Separating P4Info and P4Runtime libraries
- Updating global README and adding p4runtime/README.md
- Disabling screen saver on VM GUI
- Adding desktop icons for Terminal, Wireshare and Sublime Text
- Updating topo.pdf -> png for Markdown viewing in basic_tunnel and
  p4runtime READMEs

.gitignore

@@ -17,6 +17,9 @@ solution*/
 # Build folders
 build*/
 
+# Logs folders
+logs/
+
 # Vagrant
 .vagrant/
 

P4D2_2017_Fall/exercises/README.md

@@ -7,24 +7,27 @@ Welcome to the P4 Tutorial!
 We've prepared a set of exercises to help you get started with P4
 programming, organized into four modules:
 
-1. Introduction
+1. Introduction and Language Basics
 * [Basic Forwarding](./basic)
 * [Basic Tunneling](./basic_tunnel)
 
-2. Monitoring and Debugging
+2. P4 Runtime and the Control Plane
+* [P4 Runtime](./p4runtime)
+
+3. Monitoring and Debugging
 * [Explicit Congestion Notification](./ecn)
 * [Multi-Hop Route Inspection](./mri)
 
-3. Advanced Data Structures
+4. Advanced Data Structures
 * [Source Routing](./source_routing)
 * [Calculator](./calc)
 
-4. Dynamic Behavior
+5. Dynamic Behavior
 * [Load Balancing](./load_balance)
 
 ## Obtaining required software
 
-If you are starting this tutorial at SIGCOMM 2017, then we've already
+If you are starting this tutorial at the Fall 2017 P4 Developer Day, then we've already
 provided you with a virtual machine that has all of the required
 software installed.
 
@@ -36,25 +39,11 @@ To build the virtual machine:
 - `cd vm`
 - `vagrant up`
 - Log in with username `p4` and password `p4` and issue the command `sudo shutdown -r now`
-- When the machine reboots, you should have a graphical desktop machine with the required software pre-installed.
+- When the machine reboots, you should have a graphical desktop machine with the required
+software pre-installed.
 
-To install dependences by hand:
+To install dependencies by hand, please reference the [vm](../vm) installation scripts.
-- `git clone https://github.com/p4lang/behavioral-model.git`
+They contain the dependencies, versions, and installation procedure.
-- `git clone https://github.com/p4lang/p4c`
+You can run them directly on an Ubuntu 16.04 machine:
-- `git clone https://github.com/p4lang/tutorials`
+- `sudo ./root-bootstrap.sh`
-Then follow the instructions for how to build each package. Each of
+- `sudo ./user-bootstrap.sh`
-these repositories come with dependencies, which can be installed
-using the supplied instructions. The first repository
-([behavioral-model](https://github.com/p4lang/behavioral-model))
-contains the P4 behavioral model. It is a C++ software switch that
-will implement the functionality specified in your P4 program. The
-second repository ([p4c](https://github.com/p4lang/p4c-bm)) is the
-compiler for the behavioral model. It takes P4 program and produces a
-JSON file which can be loaded by the behavioral model. The third
-repository ([tutorial](https://github.com/p4lang/tutorial)) is the P4
-Tutorial itself. You will also need to install `mininet`. On Ubuntu,
-it would look like this:
-
-```
-$ sudo apt-get install mininet
-```

P4D2_2017_Fall/exercises/basic_tunnel/README.md

@@ -98,7 +98,7 @@ Your job will be to do the following:
 6. **TODO:** Update the deparser to emit the `ethernet`, then `myTunnel`, then `ipv4` headers. Remember that the deparser will only emit a header if it is valid. A header's implicit validity bit is set by the parser upon extraction. So there is no need to check header validity here.
 7. **TODO:** Add static rules for your newly defined table so that the switches will forward correctly for each possible value of `dst_id`. See the diagram below for the topology's port configuration as well as how we will assign IDs to hosts. For this step you will need to add your forwarding rules to the `sX-commands.txt` files.
 
-![topology](./topo.pdf)
+![topology](./topo.png)
 
 ## Step 3: Run your solution
 

P4D2_2017_Fall/exercises/p4runtime/Makefile

@@ -1,10 +1,5 @@
+BMV2_SWITCH_EXE = simple_switch_grpc
+NO_P4 = true
+P4C_ARGS = --p4runtime-file $(basename $@).p4info --p4runtime-format text
+
 include ../../utils/Makefile
-
-# Override build method to use simple_switch_grpc target
-run: build
-	sudo python $(RUN_SCRIPT) -t $(TOPO) -b simple_switch_grpc
-
-# Override p4c step to also produce p4info file
-P4INFO_ARGS = --p4runtime-file $(basename $@).p4info --p4runtime-format text
-$(BUILD_DIR)/%.json: %.p4
-	$(P4C) --p4v 16 $(P4INFO_ARGS) -o $@ $<

P4D2_2017_Fall/exercises/p4runtime/README.md

@@ -0,0 +1,146 @@
+# Implementing a Control Plane using P4 Runtime
+
+## Introduction
+
+In this exercise, we will be using P4 Runtime to send flow entries to the 
+switch, instead of using the switch's CLI. We will be using the same P4
+program that you used in the previous in the basic_tunnel exercise. The
+P4 program has be renamed to `advanced_tunnel.py` and has been augmented
+with a counter, `tunnelCount`, and two new actions, `myTunnel_ingress`
+and `myTunnel_egress`.
+ 
+You will use the starter program, `mycontroller.py`, and a few helper
+libraries in the `p4runtime_lib` directory to create the table entries
+necessary to tunnel traffic between host 1 and 2.
+
+> **Spoiler alert:** There is a reference solution in the `solution`
+> sub-directory. Feel free to compare your implementation to the
+> reference.
+
+## Step 1: Run the (incomplete) starter code
+
+The starter code for this assignment is in a file called `mycontroller.py` 
+and it will install only some of the rules that you need tunnel traffic between
+two hosts.
+
+Let's first compile the new P4 program, start the network, use `mycontroller.py`
+to install a few rules, and look at the tunnel ingress counter to see that things
+are working as expected.
+
+1. In your shell, run:
+   ```bash
+   make
+   ```
+   This will:
+   * compile `advanced_tunnel.p4`, and
+   * start a Mininet instance with three switches (`s1`, `s2`, `s3`)
+     configured in a triangle, each connected to one host (`h1`, `h2`,
+     and `h3`).
+   * The hosts are assigned IPs of `10.0.1.1`, `10.0.2.2`, etc.
+
+2. You should now see a Mininet command prompt. Start a ping between h1 and h2:
+   ```bash
+   mininet> h1 ping h2
+   ```
+   Because there are no rules on the switches, you should **not** receive any
+   replies yet.
+   
+3. Open another shell and run the starter code:
+   ```bash
+   cd ~/tutorials/P4D2_2017_Fall/exercises/p4runtime
+   ./mycontroller.py
+   ```
+   This will install the `advanced_tunnel.p4` program on the switches and push the
+   tunnel ingress rules.
+   The program prints the tunnel ingress and egress counters every 2 seconds.
+   You should see the ingress tunnel counter for s1 increasing:
+   ```
+    s1 ingressTunnelCounter 100: 2 packets
+   ```
+   The other counters should remain at zero. 
+
+4. Press `Ctrl-C` to the second shell to stop `mycontroller.py`
+
+Each switch is currently mapping traffic into tunnels based on the destination IP
+address. Your job is to write the rules that forward the traffic between the switches
+based on the tunnel ID.
+
+### A note about the control plane
+
+A P4 program defines a packet-processing pipeline, but the rules
+within each table are inserted by the control plane. In this case,
+`mycontroller.py` implements our control plane, instead of installing static
+table entries like we have in the previous exercises.
+
+**Important:** A P4 program also defines the interface between the
+switch pipeline and control plane. This interface is defined in the
+`advanced_tunnel.p4info` file. The table entries that you build in `mycontroller.py`
+refer to specific tables, keys, and actions by name, and we use a P4Info helper
+to convert the names into the IDs that are required for P4 Runtime. Any changes
+in the P4 program that add or rename tables, keys, or actions will need to be
+reflected in your table entries.
+
+## Step 2: Implement Tunnel Forwarding
+
+The `mycontroller.py` file is a basic controller plane that does the following:
+1. Establishes a gRPC connection to the switches for the P4 Runtime service.
+2. Pushes the P4 program to each switch.
+3. Writes tunnel ingress and tunnel egress rules for two tunnels between h1 and h2.
+4. Reads tunnel ingress and egress counters every 2 seconds.
+
+It also contains comments marked with `TODO` which indicate the functionality
+that you need to implement.
+
+Your job will be to write the tunnel transit rule in the `writeTunnelRules` function
+that will match on tunnel ID and forward packets to the next hop.
+
+![topology](../basic_tunnel/topo.png)
+
+## Step 3: Run your solution
+
+Follow the instructions from Step 1. If your Mininet network is still running,
+you will just need to run the following in your second shell:
+```bash
+./my_controller.py
+```
+
+You should start to see ICMP replies in your Mininet prompt, and you should start to
+see the values for all counters start to increment.
+
+### Extra Credit and Food for Thought 
+
+You might notice that the rules that are printed by `mycontroller.py` contain the entity
+IDs rather than the table names. You can use the P4Info helper to translate these IDs
+into entry names. 
+
+Also, you may want to think about the following:
+- What assumptions about the topology are baked into your implementation? How would you
+need to change it for a more realistic network?
+
+- Why are the byte counters different between the ingress and egress counters?
+
+- What is the TTL in the ICMP replies? Why is it the value that it is?
+Hint: The default TTL is 64 for packets sent by the hosts.
+
+#### Cleaning up Mininet
+
+If the Mininet shell crashes, it may leave a Mininet instance
+running in the background. Use the following command to clean up:
+```bash
+make clean
+```
+
+#### Running the reference solution
+
+To run the reference solution, you should run the following command from the
+`~/tutorials/P4D2_2017_Fall/exercises/p4runtime` directory:
+```bash
+solution/my_controller.py
+```
+
+
+## Next Steps
+
+Congratulations, your implementation works! Move onto the next assignment
+[ecn](../ecn)!
+

P4D2_2017_Fall/exercises/p4runtime/advanced_tunnel.p4

@@ -4,6 +4,7 @@
 
 const bit<16> TYPE_MYTUNNEL = 0x1212;
 const bit<16> TYPE_IPV4 = 0x800;
+const bit<32> MAX_TUNNEL_ID = 1 << 16;
 
 /*************************************************************************
 *********************** H E A D E R S  ***********************************
@@ -103,6 +104,9 @@ control MyIngress(inout headers hdr,
                   inout metadata meta,
                   inout standard_metadata_t standard_metadata) {
 
+    counter(MAX_TUNNEL_ID, CounterType.packets_and_bytes) ingressTunnelCounter;
+    counter(MAX_TUNNEL_ID, CounterType.packets_and_bytes) egressTunnelCounter;
+
     action drop() {
         mark_to_drop();
     }
@@ -119,6 +123,19 @@ control MyIngress(inout headers hdr,
         hdr.myTunnel.dst_id = dst_id;
         hdr.myTunnel.proto_id = hdr.ethernet.etherType;
         hdr.ethernet.etherType = TYPE_MYTUNNEL;
+        ingressTunnelCounter.count((bit<32>) hdr.myTunnel.dst_id);
+    }
+
+    action myTunnel_forward(egressSpec_t port) {
+        standard_metadata.egress_spec = port;
+    }
+
+    action myTunnel_egress(macAddr_t dstAddr, egressSpec_t port) {
+        standard_metadata.egress_spec = port;
+        hdr.ethernet.dstAddr = dstAddr;
+        hdr.ethernet.etherType = hdr.myTunnel.proto_id;
+        hdr.myTunnel.setInvalid();
+        egressTunnelCounter.count((bit<32>) hdr.myTunnel.dst_id);
     }
 
     table ipv4_lpm {
@@ -135,19 +152,6 @@ control MyIngress(inout headers hdr,
         default_action = NoAction();
     }
 
-    direct_counter(CounterType.packets_and_bytes) tunnelCount;
-
-    action myTunnel_forward(egressSpec_t port) {
-        standard_metadata.egress_spec = port;
-        tunnelCount.count();
-    }
-
-    action myTunnel_egress(egressSpec_t port) {
-        standard_metadata.egress_spec = port;
-        hdr.myTunnel.setInvalid();
-        tunnelCount.count();
-    }
-
     table myTunnel_exact {
         key = {
             hdr.myTunnel.dst_id: exact;
@@ -158,12 +162,10 @@ control MyIngress(inout headers hdr,
             drop;
         }
         size = 1024;
-        counters = tunnelCount;
         default_action = drop();
     }
 
     apply {
-
         if (hdr.ipv4.isValid() && !hdr.myTunnel.isValid()) {
             // Process only non-tunneled IPv4 packets.
             ipv4_lpm.apply();

P4D2_2017_Fall/exercises/p4runtime/mycontroller.py

@@ -0,0 +1,159 @@
+#!/usr/bin/env python2
+import argparse
+import os
+from time import sleep
+
+import p4runtime_lib.bmv2
+import p4runtime_lib.helper
+
+SWITCH_TO_HOST_PORT = 1
+SWITCH_TO_SWITCH_PORT = 2
+
+def writeTunnelRules(p4info_helper, ingressSw, egressSw, tunnelId, dstEthAddr, dstIpAddr):
+    '''
+    Installs three rules:
+    1) An tunnel ingress rule on the ingress switch in the ipv4_lpm table that encapsulates traffic
+       into a tunnel with the specified ID
+    2) A transit rule on the ingress switch that forwards traffic based on the specified ID
+    3) An tunnel egress rule on the egress switch that decapsulates traffic with the specified ID
+       and sends it to the host
+
+    :param p4info_helper: the P4Info helper
+    :param ingressSw: the ingress switch connection
+    :param egressSw: the egress switch connection
+    :param tunnelId: the specified tunnel ID
+    :param dstEthAddr: the destination IP to match in the ingress rule
+    :param dstIpAddr: the destination Ethernet address to write in the egress rule
+    '''
+    # 1) Tunnel Ingress Rule
+    table_entry = p4info_helper.buildTableEntry(
+        table_name="ipv4_lpm",
+        match_fields={
+            "hdr.ipv4.dstAddr": (dstIpAddr, 32)
+        },
+        action_name="myTunnel_ingress",
+        action_params={
+            "dst_id": tunnelId,
+        })
+    ingressSw.WriteTableEntry(table_entry)
+    print "Installed ingress tunnel rule on %s" % ingressSw.name
+
+    # 2) Tunnel Transit Rule
+    # TODO you will need to implement this rule
+    # The rule will need to be added to the myTunnel_exact table and match on the tunnel ID (hdr.myTunnel.dst_id).
+    # For our simple topology, transit traffic will need to be forwarded using the myTunnel_egress action to
+    # the SWITCH_TO_SWITCH_PORT (port 2).
+    # We will only need on transit rule on the ingress switch because we are using a simple topology.
+    # In general, you'll need on transit rule for each switch in the path (except the last one)
+    #
+    # If you are stuck, start by copying the tunnel ingress rule from above. Then, try to make the suggested
+    # modifications.
+    print "TODO Install transit tunnel rule"
+
+    # 3) Tunnel Egress Rule
+    # For our simple topology, the host will always be located on the SWITCH_TO_HOST_PORT (port 1).
+    # In general, you will need to keep track of which port the host is connected to.
+    table_entry = p4info_helper.buildTableEntry(
+        table_name="myTunnel_exact",
+        match_fields={
+            "hdr.myTunnel.dst_id": tunnelId
+        },
+        action_name="myTunnel_egress",
+        action_params={
+            "dstAddr": dstEthAddr,
+            "port": SWITCH_TO_HOST_PORT
+        })
+    egressSw.WriteTableEntry(table_entry)
+    print "Installed egress tunnel rule on %s" % egressSw.name
+
+def readTableRules(p4info_helper, sw):
+    '''
+    Reads the table entries from all tables on the switch.
+
+    :param p4info_helper: the P4Info helper
+    :param sw: the switch connection
+    '''
+    print '\n----- Reading tables rules for %s -----' % sw.name
+    for response in sw.ReadTableEntries():
+        for entity in response.entities:
+            entry = entity.table_entry
+            # TODO For extra credit, you can use the p4info_helper to translate the IDs the entry to names
+            print entry
+            print '-----'
+
+def printCounter(p4info_helper, sw, counter_name, index):
+    '''
+    Reads the specified counter at the specified index from the switch. In our program, the index
+    is the tunnel ID. If the index is 0, it will return all values from the counter.
+
+    :param p4info_helper: the P4Info helper
+    :param sw:  the switch connection
+    :param counter_name: the name of the counter from the P4 program
+    :param index: the counter index (in our case, the tunnel ID)
+    '''
+    for response in sw.ReadCounters(p4info_helper.get_counters_id(counter_name), index):
+        for entity in response.entities:
+            counter = entity.counter_entry
+            print "%s %s %d: %d packets (%d bytes)" % (
+                sw.name, counter_name, index,
+                counter.data.packet_count, counter.data.byte_count
+            )
+
+
+def main(p4info_file_path, bmv2_file_path):
+    # Instantiate a P4 Runtime helper from the p4info file
+    p4info_helper = p4runtime_lib.helper.P4InfoHelper(p4info_file_path)
+
+    # Create a switch connection object for s1 and s2; this is backed by a P4 Runtime gRPC connection
+    s1 = p4runtime_lib.bmv2.Bmv2SwitchConnection('s1', address='127.0.0.1:50051')
+    s2 = p4runtime_lib.bmv2.Bmv2SwitchConnection('s2', address='127.0.0.1:50052')
+
+    # Install the P4 program on the switches
+    s1.SetForwardingPipelineConfig(p4info=p4info_helper.p4info, bmv2_json_file_path=bmv2_file_path)
+    print "Installed P4 Program using SetForwardingPipelineConfig on %s" % s1.name
+    s2.SetForwardingPipelineConfig(p4info=p4info_helper.p4info, bmv2_json_file_path=bmv2_file_path)
+    print "Installed P4 Program using SetForwardingPipelineConfig on %s" % s2.name
+
+    # Write the rules that tunnel traffic from h1 to h2
+    writeTunnelRules(p4info_helper, ingressSw=s1, egressSw=s2, tunnelId=100,
+                     dstEthAddr="00:00:00:00:02:02", dstIpAddr="10.0.2.2")
+
+    # Write the rules that tunnel traffic from h2 to h1
+    writeTunnelRules(p4info_helper, ingressSw=s2, egressSw=s1, tunnelId=200,
+                     dstEthAddr="00:00:00:00:01:01", dstIpAddr="10.0.1.1")
+
+    # TODO Uncomment the following two lines to read table entries from s1 and s2
+    #readTableRules(p4info_helper, s1)
+    #readTableRules(p4info_helper, s2)
+
+    # Print the tunnel counters every 2 seconds
+    try:
+        while True:
+            sleep(2)
+            print '\n----- Reading tunnel counters -----'
+            printCounter(p4info_helper, s1, "ingressTunnelCounter", 100)
+            printCounter(p4info_helper, s2, "egressTunnelCounter", 100)
+            printCounter(p4info_helper, s2, "ingressTunnelCounter", 200)
+            printCounter(p4info_helper, s1, "egressTunnelCounter", 200)
+    except KeyboardInterrupt:
+        print " Shutting down."
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description='P4Runtime Controller')
+    parser.add_argument('--p4info', help='p4info proto in text format from p4c',
+                        type=str, action="store", required=False, default='./build/advanced_tunnel.p4info')
+    parser.add_argument('--bmv2-json', help='BMv2 JSON file from p4c',
+                        type=str, action="store", required=False, default='./build/advanced_tunnel.json')
+    args = parser.parse_args()
+
+    if not os.path.exists(args.p4info):
+        parser.print_help()
+        print "\np4info file not found: %s\nHave you run 'make'?" % args.p4info
+        parser.exit(1)
+    if not os.path.exists(args.bmv2_json):
+        parser.print_help()
+        print "\nBMv2 JSON file not found: %s\nHave you run 'make'?" % args.bmv2_json
+        parser.exit(1)
+
+    main(args.p4info, args.bmv2_json)

P4D2_2017_Fall/exercises/p4runtime/p4info/p4browser.py

@@ -1,135 +0,0 @@
-# Copyright 2017-present Open Networking Foundation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-import re
-
-import google.protobuf.text_format
-from p4 import p4runtime_pb2
-from p4.config import p4info_pb2
-
-
-class P4InfoBrowser(object):
-    def __init__(self, p4_info_filepath):
-        p4info = p4info_pb2.P4Info()
-        # Load the p4info file into a skeleton P4Info object
-        with open(p4_info_filepath) as p4info_f:
-            google.protobuf.text_format.Merge(p4info_f.read(), p4info)
-        self.p4info = p4info
-
-    def get(self, entity_type, name=None, id=None):
-        if name is not None and id is not None:
-            raise AssertionError("name or id must be None")
-
-        for o in getattr(self.p4info, entity_type):
-            pre = o.preamble
-            if name:
-                if (pre.name == name or pre.alias == name):
-                    return o
-            else:
-                if pre.id == id:
-                    return o
-
-        if name:
-            raise AttributeError("Could not find %r of type %s" % (name, entity_type))
-        else:
-            raise AttributeError("Could not find id %r of type %s" % (id, entity_type))
-
-    def get_id(self, entity_type, name):
-        return self.get(entity_type, name=name).preamble.id
-
-    def get_name(self, entity_type, id):
-        return self.get(entity_type, id=id).preamble.name
-
-    def get_alias(self, entity_type, id):
-        return self.get(entity_type, id=id).preamble.alias
-
-    def __getattr__(self, attr):
-        # Synthesize convenience functions for name to id lookups for top-level entities
-        # e.g. get_table_id() or get_action_id()
-        m = re.search("^get_(\w+)_id$", attr)
-        if m:
-            primitive = m.group(1)
-            return lambda name: self.get_id(primitive, name)
-
-        # Synthesize convenience functions for id to name lookups
-        m = re.search("^get_(\w+)_name$", attr)
-        if m:
-            primitive = m.group(1)
-            return lambda id: self.get_name(primitive, id)
-
-        raise AttributeError("%r object has no attribute %r" % (self.__class__, attr))
-
-    # TODO remove
-    def get_table_entry(self, table_name):
-        t = self.get(table_name, "table")
-        entry = p4runtime_pb2.TableEntry()
-        entry.table_id = t.preamble.id
-        entry
-        pass
-
-    def get_match_field(self, table_name, match_field_name):
-        for t in self.p4info.tables:
-            pre = t.preamble
-            if pre.name == table_name:
-                for mf in t.match_fields:
-                    if mf.name == match_field_name:
-                        return mf
-
-    def get_match_field_id(self, table_name, match_field_name):
-        return self.get_match_field(table_name,match_field_name).id
-
-    def get_match_field_pb(self, table_name, match_field_name, value):
-        p4info_match = self.get_match_field(table_name, match_field_name)
-        bw = p4info_match.bitwidth
-        p4runtime_match = p4runtime_pb2.FieldMatch()
-        p4runtime_match.field_id = p4info_match.id
-        # TODO switch on match type and map the value into the appropriate message type
-        match_type = p4info_pb2._MATCHFIELD_MATCHTYPE.values_by_number[
-            p4info_match.match_type].name
-        if match_type == 'EXACT':
-            exact = p4runtime_match.exact
-            exact.value = value
-        elif match_type == 'LPM':
-            lpm = p4runtime_match.lpm
-            lpm.value = value[0]
-            lpm.prefix_len = value[1]
-        # TODO finish cases and validate types and bitwidth
-        # VALID = 1;
-        # EXACT = 2;
-        # LPM = 3;
-        # TERNARY = 4;
-        # RANGE = 5;
-        # and raise exception
-        return p4runtime_match
-
-    def get_action_param(self, action_name, param_name):
-        for a in self.p4info.actions:
-            pre = a.preamble
-            if pre.name == action_name:
-                for p in a.params:
-                    if p.name == param_name:
-                        return p
-        raise AttributeError("%r has no attribute %r" % (action_name, param_name))
-
-
-    def get_action_param_id(self, action_name, param_name):
-        return self.get_action_param(action_name, param_name).id
-
-    def get_action_param_pb(self, action_name, param_name, value):
-        p4info_param = self.get_action_param(action_name, param_name)
-        #bw = p4info_param.bitwidth
-        p4runtime_param = p4runtime_pb2.Action.Param()
-        p4runtime_param.param_id = p4info_param.id
-        p4runtime_param.value = value # TODO make sure it's the correct bitwidth
-        return p4runtime_param

P4D2_2017_Fall/exercises/p4runtime/p4runtime_lib/bmv2.py

@@ -20,7 +20,6 @@ def buildDeviceConfig(bmv2_json_file_path=None):
     "Builds the device config for BMv2"
     device_config = p4config_pb2.P4DeviceConfig()
     device_config.reassign = True
-    # set device_config.extra to default instance
     with open(bmv2_json_file_path) as f:
         device_config.device_data = f.read()
     return device_config

P4D2_2017_Fall/exercises/p4runtime/p4runtime_lib/convert.py

@@ -0,0 +1,119 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import re
+import socket
+
+import math
+
+'''
+This package contains several helper functions for encoding to and decoding from byte strings:
+- integers
+- IPv4 address strings
+- Ethernet address strings
+'''
+
+mac_pattern = re.compile('^([\da-fA-F]{2}:){5}([\da-fA-F]{2})$')
+def matchesMac(mac_addr_string):
+    return mac_pattern.match(mac_addr_string) is not None
+
+def encodeMac(mac_addr_string):
+    return mac_addr_string.replace(':', '').decode('hex')
+
+def decodeMac(encoded_mac_addr):
+    return ':'.join(s.encode('hex') for s in encoded_mac_addr)
+
+ip_pattern = re.compile('^(\d{1,3}\.){3}(\d{1,3})$')
+def matchesIPv4(ip_addr_string):
+    return ip_pattern.match(ip_addr_string) is not None
+
+def encodeIPv4(ip_addr_string):
+    return socket.inet_aton(ip_addr_string)
+
+def decodeIPv4(encoded_ip_addr):
+    return socket.inet_ntoa(encoded_ip_addr)
+
+def bitwidthToBytes(bitwidth):
+    return int(math.ceil(bitwidth / 8.0))
+
+def encodeNum(number, bitwidth):
+    byte_len = bitwidthToBytes(bitwidth)
+    num_str = '%x' % number
+    if number >= 2 ** bitwidth:
+        raise Exception("Number, %d, does not fit in %d bits" % (number, bitwidth))
+    return ('0' * (byte_len * 2 - len(num_str)) + num_str).decode('hex')
+
+def decodeNum(encoded_number):
+    return int(encoded_number.encode('hex'), 16)
+
+def encode(x, bitwidth):
+    'Tries to infer the type of `x` and encode it'
+    byte_len = bitwidthToBytes(bitwidth)
+    if (type(x) == list or type(x) == tuple) and len(x) == 1:
+        x = x[0]
+    encoded_bytes = None
+    if type(x) == str:
+        if matchesMac(x):
+            encoded_bytes = encodeMac(x)
+        elif matchesIPv4(x):
+            encoded_bytes = encodeIPv4(x)
+        else:
+            # Assume that the string is already encoded
+            encoded_bytes = x
+    elif type(x) == int:
+        encoded_bytes = encodeNum(x, bitwidth)
+    else:
+        raise Exception("Encoding objects of %r is not supported" % type(x))
+    assert(len(encoded_bytes) == byte_len)
+    return encoded_bytes
+
+if __name__ == '__main__':
+    # TODO These tests should be moved out of main eventually
+    mac = "aa:bb:cc:dd:ee:ff"
+    enc_mac = encodeMac(mac)
+    assert(enc_mac == '\xaa\xbb\xcc\xdd\xee\xff')
+    dec_mac = decodeMac(enc_mac)
+    assert(mac == dec_mac)
+
+    ip = "10.0.0.1"
+    enc_ip = encodeIPv4(ip)
+    assert(enc_ip == '\x0a\x00\x00\x01')
+    dec_ip = decodeIPv4(enc_ip)
+    assert(ip == dec_ip)
+
+    num = 1337
+    byte_len = 5
+    enc_num = encodeNum(num, byte_len * 8)
+    assert(enc_num == '\x00\x00\x00\x05\x39')
+    dec_num = decodeNum(enc_num)
+    assert(num == dec_num)
+
+    assert(matchesIPv4('10.0.0.1'))
+    assert(not matchesIPv4('10.0.0.1.5'))
+    assert(not matchesIPv4('1000.0.0.1'))
+    assert(not matchesIPv4('10001'))
+
+    assert(encode(mac, 6 * 8) == enc_mac)
+    assert(encode(ip, 4 * 8) == enc_ip)
+    assert(encode(num, 5 * 8) == enc_num)
+    assert(encode((num,), 5 * 8) == enc_num)
+    assert(encode([num], 5 * 8) == enc_num)
+
+    num = 256
+    byte_len = 2
+    try:
+        enc_num = encodeNum(num, 8)
+        raise Exception("expected exception")
+    except Exception as e:
+        print e

P4D2_2017_Fall/exercises/p4runtime/p4runtime_lib/helper.py

@@ -0,0 +1,183 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import re
+
+import google.protobuf.text_format
+from p4 import p4runtime_pb2
+from p4.config import p4info_pb2
+
+from p4runtime_lib.convert import encode
+
+class P4InfoHelper(object):
+    def __init__(self, p4_info_filepath):
+        p4info = p4info_pb2.P4Info()
+        # Load the p4info file into a skeleton P4Info object
+        with open(p4_info_filepath) as p4info_f:
+            google.protobuf.text_format.Merge(p4info_f.read(), p4info)
+        self.p4info = p4info
+
+    def get(self, entity_type, name=None, id=None):
+        if name is not None and id is not None:
+            raise AssertionError("name or id must be None")
+
+        for o in getattr(self.p4info, entity_type):
+            pre = o.preamble
+            if name:
+                if (pre.name == name or pre.alias == name):
+                    return o
+            else:
+                if pre.id == id:
+                    return o
+
+        if name:
+            raise AttributeError("Could not find %r of type %s" % (name, entity_type))
+        else:
+            raise AttributeError("Could not find id %r of type %s" % (id, entity_type))
+
+    def get_id(self, entity_type, name):
+        return self.get(entity_type, name=name).preamble.id
+
+    def get_name(self, entity_type, id):
+        return self.get(entity_type, id=id).preamble.name
+
+    def get_alias(self, entity_type, id):
+        return self.get(entity_type, id=id).preamble.alias
+
+    def __getattr__(self, attr):
+        # Synthesize convenience functions for name to id lookups for top-level entities
+        # e.g. get_tables_id(name_string) or get_actions_id(name_string)
+        m = re.search("^get_(\w+)_id$", attr)
+        if m:
+            primitive = m.group(1)
+            return lambda name: self.get_id(primitive, name)
+
+        # Synthesize convenience functions for id to name lookups
+        # e.g. get_tables_name(id) or get_actions_name(id)
+        m = re.search("^get_(\w+)_name$", attr)
+        if m:
+            primitive = m.group(1)
+            return lambda id: self.get_name(primitive, id)
+
+        raise AttributeError("%r object has no attribute %r" % (self.__class__, attr))
+
+    def get_match_field(self, table_name, name=None, id=None):
+        for t in self.p4info.tables:
+            pre = t.preamble
+            if pre.name == table_name:
+                for mf in t.match_fields:
+                    if name is not None:
+                        if mf.name == name:
+                            return mf
+                    elif id is not None:
+                        if mf.id == id:
+                            return mf
+        raise AttributeError("%r has no attribute %r" % (table_name, name if name is not None else id))
+
+    def get_match_field_id(self, table_name, match_field_name):
+        return self.get_match_field(table_name, name=match_field_name).id
+
+    def get_match_field_name(self, table_name, match_field_id):
+        return self.get_match_field(table_name, id=match_field_id).name
+
+    def get_match_field_pb(self, table_name, match_field_name, value):
+        p4info_match = self.get_match_field(table_name, match_field_name)
+        bitwidth = p4info_match.bitwidth
+        p4runtime_match = p4runtime_pb2.FieldMatch()
+        p4runtime_match.field_id = p4info_match.id
+        match_type = p4info_match.match_type
+        if match_type == p4info_pb2.MatchField.VALID:
+            valid = p4runtime_match.valid
+            valid.value = bool(value)
+        elif match_type == p4info_pb2.MatchField.EXACT:
+            exact = p4runtime_match.exact
+            exact.value = encode(value, bitwidth)
+        elif match_type == p4info_pb2.MatchField.LPM:
+            lpm = p4runtime_match.lpm
+            lpm.value = encode(value[0], bitwidth)
+            lpm.prefix_len = value[1]
+        elif match_type == p4info_pb2.MatchField.TERNARY:
+            lpm = p4runtime_match.ternary
+            lpm.value = encode(value[0], bitwidth)
+            lpm.mask = encode(value[1], bitwidth)
+        elif match_type == p4info_pb2.MatchField.RANGE:
+            lpm = p4runtime_match.range
+            lpm.low = encode(value[0], bitwidth)
+            lpm.high = encode(value[1], bitwidth)
+        else:
+            raise Exception("Unsupported match type with type %r" % match_type)
+        return p4runtime_match
+
+    def get_match_field_value(self, match_field):
+        match_type = match_field.WhichOneof("field_match_type")
+        if match_type == 'valid':
+            return match_field.valid.value
+        elif match_type == 'exact':
+            return match_field.exact.value
+        elif match_type == 'lpm':
+            return (match_field.lpm.value, match_field.lpm.prefix_len)
+        elif match_type == 'ternary':
+            return (match_field.ternary.value, match_field.ternary.mask)
+        elif match_type == 'range':
+            return (match_field.range.low, match_field.range.high)
+        else:
+            raise Exception("Unsupported match type with type %r" % match_type)
+
+    def get_action_param(self, action_name, name=None, id=None):
+        for a in self.p4info.actions:
+            pre = a.preamble
+            if pre.name == action_name:
+                for p in a.params:
+                    if name is not None:
+                        if p.name == name:
+                            return p
+                    elif id is not None:
+                        if p.id == id:
+                            return p
+        raise AttributeError("action %r has no param %r" % (action_name, name if name is not None else id))
+
+    def get_action_param_id(self, action_name, param_name):
+        return self.get_action_param(action_name, name=param_name).id
+
+    def get_action_param_name(self, action_name, param_id):
+        return self.get_action_param(action_name, id=param_id).name
+
+    def get_action_param_pb(self, action_name, param_name, value):
+        p4info_param = self.get_action_param(action_name, param_name)
+        p4runtime_param = p4runtime_pb2.Action.Param()
+        p4runtime_param.param_id = p4info_param.id
+        p4runtime_param.value = encode(value, p4info_param.bitwidth)
+        return p4runtime_param
+
+    def buildTableEntry(self,
+                        table_name,
+                        match_fields={},
+                        action_name=None,
+                        action_params={}):
+        table_entry = p4runtime_pb2.TableEntry()
+        table_entry.table_id = self.get_tables_id(table_name)
+        if match_fields:
+            table_entry.match.extend([
+                self.get_match_field_pb(table_name, match_field_name, value)
+                for match_field_name, value in match_fields.iteritems()
+            ])
+        if action_name:
+            action = table_entry.action.action
+            action.action_id = self.get_actions_id(action_name)
+            if action_params:
+                action.params.extend([
+                    self.get_action_param_pb(action_name, field_name, value)
+                    for field_name, value in action_params.iteritems()
+                ])
+        return table_entry

P4D2_2017_Fall/exercises/p4runtime/p4runtime_lib/switch.py

@@ -0,0 +1,88 @@
+# Copyright 2017-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+from abc import abstractmethod
+
+import grpc
+from p4 import p4runtime_pb2
+from p4.tmp import p4config_pb2
+
+class SwitchConnection(object):
+    def __init__(self, name, address='127.0.0.1:50051', device_id=0):
+        self.name = name
+        self.address = address
+        self.device_id = device_id
+        self.p4info = None
+        self.channel = grpc.insecure_channel(self.address)
+        self.client_stub = p4runtime_pb2.P4RuntimeStub(self.channel)
+
+    @abstractmethod
+    def buildDeviceConfig(self, **kwargs):
+        return p4config_pb2.P4DeviceConfig()
+
+    def SetForwardingPipelineConfig(self, p4info, dry_run=False, **kwargs):
+        device_config = self.buildDeviceConfig(**kwargs)
+        request = p4runtime_pb2.SetForwardingPipelineConfigRequest()
+        config = request.configs.add()
+        config.device_id = self.device_id
+        config.p4info.CopyFrom(p4info)
+        config.p4_device_config = device_config.SerializeToString()
+        request.action = p4runtime_pb2.SetForwardingPipelineConfigRequest.VERIFY_AND_COMMIT
+        if dry_run:
+            print "P4 Runtime SetForwardingPipelineConfig:", request
+        else:
+            self.client_stub.SetForwardingPipelineConfig(request)
+
+    def WriteTableEntry(self, table_entry, dry_run=False):
+        request = p4runtime_pb2.WriteRequest()
+        request.device_id = self.device_id
+        update = request.updates.add()
+        update.type = p4runtime_pb2.Update.INSERT
+        update.entity.table_entry.CopyFrom(table_entry)
+        if dry_run:
+            print "P4 Runtime Write:", request
+        else:
+            self.client_stub.Write(request)
+
+    def ReadTableEntries(self, table_id=None, dry_run=False):
+        request = p4runtime_pb2.ReadRequest()
+        request.device_id = self.device_id
+        entity = request.entities.add()
+        table_entry = entity.table_entry
+        if table_id is not None:
+            table_entry.table_id = table_id
+        else:
+            table_entry.table_id = 0
+        if dry_run:
+            print "P4 Runtime Read:", request
+        else:
+            for response in self.client_stub.Read(request):
+                yield response
+
+    def ReadCounters(self, counter_id=None, index=None, dry_run=False):
+        request = p4runtime_pb2.ReadRequest()
+        request.device_id = self.device_id
+        entity = request.entities.add()
+        counter_entry = entity.counter_entry
+        if counter_id is not None:
+            counter_entry.counter_id = counter_id
+        else:
+            counter_entry.counter_id = 0
+        if index is not None:
+            counter_entry.index = index
+        if dry_run:
+            print "P4 Runtime Read:", request
+        else:
+            for response in self.client_stub.Read(request):
+                yield response

P4D2_2017_Fall/exercises/p4runtime/switches/switch.py

@@ -1,130 +0,0 @@
-# Copyright 2017-present Open Networking Foundation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-from abc import abstractmethod
-
-import grpc
-from p4 import p4runtime_pb2
-from p4.tmp import p4config_pb2
-
-from p4info import p4browser
-
-
-def buildSetPipelineRequest(p4info, device_config, device_id):
-    request = p4runtime_pb2.SetForwardingPipelineConfigRequest()
-    config = request.configs.add()
-    config.device_id = device_id
-    config.p4info.CopyFrom(p4info)
-    config.p4_device_config = device_config.SerializeToString()
-    request.action = p4runtime_pb2.SetForwardingPipelineConfigRequest.VERIFY_AND_COMMIT
-    return request
-
-
-def buildTableEntry(p4info_browser,
-                    table_name,
-                    match_fields={},
-                    action_name=None,
-                    action_params={}):
-    table_entry = p4runtime_pb2.TableEntry()
-    table_entry.table_id = p4info_browser.get_tables_id(table_name)
-    if match_fields:
-        table_entry.match.extend([
-            p4info_browser.get_match_field_pb(table_name, match_field_name, value)
-            for match_field_name, value in match_fields.iteritems()
-        ])
-    if action_name:
-        action = table_entry.action.action
-        action.action_id = p4info_browser.get_actions_id(action_name)
-        if action_params:
-            action.params.extend([
-                p4info_browser.get_action_param_pb(action_name, field_name, value)
-                for field_name, value in action_params.iteritems()
-            ])
-    return table_entry
-
-
-class SwitchConnection(object):
-    def __init__(self, name, address='127.0.0.1:50051', device_id=0):
-        self.name = name
-        self.address = address
-        self.device_id = device_id
-        self.p4info = None
-        self.channel = grpc.insecure_channel(self.address)
-        # TODO Do want to do a better job managing stub?
-        self.client_stub = p4runtime_pb2.P4RuntimeStub(self.channel)
-
-    @abstractmethod
-    def buildDeviceConfig(self, **kwargs):
-        return p4config_pb2.P4DeviceConfig()
-
-    def SetForwardingPipelineConfig(self, p4info_file_path, dry_run=False, **kwargs):
-        p4info_broswer = p4browser.P4InfoBrowser(p4info_file_path)
-        device_config = self.buildDeviceConfig(**kwargs)
-        request = buildSetPipelineRequest(p4info_broswer.p4info, device_config, self.device_id)
-        if dry_run:
-            print "P4 Runtime SetForwardingPipelineConfig:", request
-        else:
-            self.client_stub.SetForwardingPipelineConfig(request)
-        # Update the local P4 Info reference
-        self.p4info_broswer = p4info_broswer
-
-    def buildTableEntry(self,
-                        table_name,
-                        match_fields={},
-                        action_name=None,
-                        action_params={}):
-        return buildTableEntry(self.p4info_broswer, table_name, match_fields, action_name, action_params)
-
-    def WriteTableEntry(self, table_entry, dry_run=False):
-        request = p4runtime_pb2.WriteRequest()
-        request.device_id = self.device_id
-        update = request.updates.add()
-        update.type = p4runtime_pb2.Update.INSERT
-        update.entity.table_entry.CopyFrom(table_entry)
-        if dry_run:
-            print "P4 Runtime Write:", request
-        else:
-            print self.client_stub.Write(request)
-
-    def ReadTableEntries(self, table_name, dry_run=False):
-        request = p4runtime_pb2.ReadRequest()
-        request.device_id = self.device_id
-        entity = request.entities.add()
-        table_entry = entity.table_entry
-        table_entry.table_id = self.p4info_broswer.get_tables_id(table_name)
-        if dry_run:
-            print "P4 Runtime Read:", request
-        else:
-            for response in self.client_stub.Read(request):
-                yield response
-
-    def ReadDirectCounters(self, table_name=None, counter_name=None, table_entry=None, dry_run=False):
-        request = p4runtime_pb2.ReadRequest()
-        request.device_id = self.device_id
-        entity = request.entities.add()
-        counter_entry = entity.direct_counter_entry
-        if counter_name:
-            counter_entry.counter_id = self.p4info_broswer.get_direct_counters_id(counter_name)
-        else:
-            counter_entry.counter_id = 0
-        # TODO we may not need this table entry
-        if table_name:
-            table_entry.table_id = self.p4info_broswer.get_tables_id(table_name)
-            counter_entry.table_entry.CopyFrom(table_entry)
-        counter_entry.data.packet_count = 0
-        if dry_run:
-            print "P4 Runtime Read:", request
-        else:
-            for response in self.client_stub.Read(request):
-                print response

P4D2_2017_Fall/utils/Makefile

@@ -11,10 +11,20 @@ outfile := $(source:.p4=.json)
 
 compiled_json := $(BUILD_DIR)/$(outfile)
 
+# Define NO_P4 to start BMv2 without a program
+ifndef NO_P4
+run_args += -j $(compiled_json)
+endif
+
+# Set BMV2_SWITCH_EXE to override the BMv2 target
+ifdef BMV2_SWITCH_EXE
+run_args += -b $(BMV2_SWITCH_EXE)
+endif
+
 all: run
 
 run: build
-	sudo python $(RUN_SCRIPT) -t $(TOPO) -j $(compiled_json)
+	sudo python $(RUN_SCRIPT) -t $(TOPO) $(run_args)
 
 stop:
 	sudo mn -c
@@ -22,7 +32,7 @@ stop:
 build: dirs $(compiled_json)
 
 $(BUILD_DIR)/%.json: %.p4
-	$(P4C) --p4v 16 -o $@ $<
+	$(P4C) --p4v 16 $(P4C_ARGS) -o $@ $<
 
 dirs:
 	mkdir -p $(BUILD_DIR) $(PCAP_DIR) $(LOG_DIR)

P4D2_2017_Fall/utils/run_exercise.py

@@ -325,11 +325,11 @@ class ExerciseRunner:
                 - A mininet instance is stored as self.net and self.net.start() has
                   been called.
         """
-        self.logger("Starting mininet CLI")
         for s in self.net.switches:
             s.describe()
         for h in self.net.hosts:
             h.describe()
+        self.logger("Starting mininet CLI")
         # Generate a message that will be printed by the Mininet CLI to make
         # interacting with the simple switch a little easier.
         print('')

P4D2_2017_Fall/vm/root-bootstrap.sh

@@ -76,3 +76,6 @@ ln -s p4-logo.png lubuntu-default-wallpaper.png
 rm /home/vagrant/p4-logo.png
 cd /home/vagrant
 sed -i s@#background=@background=/usr/share/lubuntu/wallpapers/1604-lubuntu-default-wallpaper.png@ /etc/lightdm/lightdm-gtk-greeter.conf
+
+# Disable screensaver
+apt-get -y remove light-locker

P4D2_2017_Fall/vm/user-bootstrap.sh

@@ -131,3 +131,39 @@ cd /home/vagrant
 sudo mv .vim /home/p4/.vim
 sudo chown -R p4:p4 /home/p4/.vim
 
+# Adding Desktop icons
+DESKTOP=/home/${USER}/Desktop
+mkdir -p ${DESKTOP}
+
+cat > ${DESKTOP}/Terminal << EOF
+[Desktop Entry]
+Encoding=UTF-8
+Type=Application
+Name=Terminal
+Name[en_US]=Terminal
+Icon=konsole
+Exec=/usr/bin/x-terminal-emulator
+Comment[en_US]=
+EOF
+
+cat > ${DESKTOP}/Wireshark << EOF
+[Desktop Entry]
+Encoding=UTF-8
+Type=Application
+Name=Wireshark
+Name[en_US]=Wireshark
+Icon=wireshark
+Exec=/usr/bin/wireshark
+Comment[en_US]=
+EOF
+
+cat > ${DESKTOP}/Sublime\ Text << EOF
+[Desktop Entry]
+Encoding=UTF-8
+Type=Application
+Name=Sublime Text
+Name[en_US]=Sublime Text
+Icon=sublime-text
+Exec=/opt/sublime_text/sublime_text
+Comment[en_US]=
+EOF