Manpages

NAME

ovn-northd and ovn-northd-ddlog - Open Virtual Network central control daemon

SYNOPSIS

ovn-northd [options]

DESCRIPTION

ovn-northd is a centralized daemon responsible for translating the high-level OVN configuration into logical configuration consumable by daemons such as ovn-controller. It translates the logical network configuration in terms of conventional network concepts, taken from the OVN Northbound Database (see ovn-nb(5)), into logical datapath flows in the OVN Southbound Database (see ovn-sb(5)) below it.

ovn-northd is implemented in C. ovn-northd-ddlog is a compatible implementation written in DDlog, a language for incremental database processing. This documentation applies to both implementations, with differences indicated where relevant.

OPTIONS

--ovnnb-db=database

The OVSDB database containing the OVN Northbound Database. If the OVN_NB_DB environment variable is set, its value is used as the default. Otherwise, the default is unix:/ovnnb_db.sock.

--ovnsb-db=database

The OVSDB database containing the OVN Southbound Database. If the OVN_SB_DB environment variable is set, its value is used as the default. Otherwise, the default is unix:/ovnsb_db.sock.

--ddlog-record=file

This option is for ovn-north-ddlog only. It causes the daemon to record the initial database state and later changes to file in the text-based DDlog command format. The ovn_northd_cli program can later replay these changes for debugging purposes. This option has a performance impact. See debugging-ddlog.rst in the OVN documentation for more details.

--dry-run

Causes ovn-northd to start paused. In the paused state, ovn-northd does not apply any changes to the databases, although it continues to monitor them. For more information, see the pause command, under Runtime Management Commands below.

For ovn-northd-ddlog, one could use this option with --ddlog-record to generate a replay log without restarting a process or disturbing a running system.

n-threads N

In certain situations, it may be desirable to enable parallelization on a system to decrease latency (at the potential cost of increasing CPU usage).

This option will cause ovn-northd to use N threads when building logical flows, when N is within [2-256]. If N is 1, parallelization is disabled (default behavior). If N is less than 1, then N is set to 1, parallelization is disabled and a warning is logged. If N is more than 256, then N is set to 256, parallelization is enabled (with 256 threads) and a warning is logged.

ovn-northd-ddlog does not support this option.

database in the above options must be an OVSDB active or passive connection method, as described in ovsdb(7).

Daemon Options
--pidfile[=pidfile]

Causes a file (by default, program.pid) to be created indicating the PID of the running process. If the pidfile argument is not specified, or if it does not begin with /, then it is created in .

If --pidfile is not specified, no pidfile is created.

--overwrite-pidfile

By default, when --pidfile is specified and the specified pidfile already exists and is locked by a running process, the daemon refuses to start. Specify --overwrite-pidfile to cause it to instead overwrite the pidfile.

When --pidfile is not specified, this option has no effect.

--detach

Runs this program as a background process. The process forks, and in the child it starts a new session, closes the standard file descriptors (which has the side effect of disabling logging to the console), and changes its current directory to the root (unless --no-chdir is specified). After the child completes its initialization, the parent exits.

--monitor

Creates an additional process to monitor this program. If it dies due to a signal that indicates a programming error (SIGABRT, SIGALRM, SIGBUS, SIGFPE, SIGILL, SIGPIPE, SIGSEGV, SIGXCPU, or SIGXFSZ) then the monitor process starts a new copy of it. If the daemon dies or exits for another reason, the monitor process exits.

This option is normally used with --detach, but it also functions without it.

--no-chdir

By default, when --detach is specified, the daemon changes its current working directory to the root directory after it detaches. Otherwise, invoking the daemon from a carelessly chosen directory would prevent the administrator from unmounting the file system that holds that directory.

Specifying --no-chdir suppresses this behavior, preventing the daemon from changing its current working directory. This may be useful for collecting core files, since it is common behavior to write core dumps into the current working directory and the root directory is not a good directory to use.

This option has no effect when --detach is not specified.

--no-self-confinement

By default this daemon will try to self-confine itself to work with files under well-known directories determined at build time. It is better to stick with this default behavior and not to use this flag unless some other Access Control is used to confine daemon. Note that in contrast to other access control implementations that are typically enforced from kernel-space (e.g. DAC or MAC), self-confinement is imposed from the user-space daemon itself and hence should not be considered as a full confinement strategy, but instead should be viewed as an additional layer of security.

--user=user:group

Causes this program to run as a different user specified in user:group, thus dropping most of the root privileges. Short forms user and :group are also allowed, with current user or group assumed, respectively. Only daemons started by the root user accepts this argument.

On Linux, daemons will be granted CAP_IPC_LOCK and CAP_NET_BIND_SERVICES before dropping root privileges. Daemons that interact with a datapath, such as ovs-vswitchd, will be granted three additional capabilities, namely CAP_NET_ADMIN, CAP_NET_BROADCAST and CAP_NET_RAW. The capability change will apply even if the new user is root.

On Windows, this option is not currently supported. For security reasons, specifying this option will cause the daemon process not to start.

Logging Options
-v[spec]
--verbose=[spec]

Sets logging levels. Without any spec, sets the log level for every module and destination to dbg. Otherwise, spec is a list of words separated by spaces or commas or colons, up to one from each category below:

On Windows platform, syslog is accepted as a word and is only useful along with the --syslog-target option (the word has no effect otherwise).

Case is not significant within spec.

Regardless of the log levels set for file, logging to a file will not take place unless --log-file is also specified (see below).

For compatibility with older versions of OVS, any is accepted as a word but has no effect.

--verbose

Sets the maximum logging verbosity level, equivalent to --verbose=dbg.

-vPATTERN:destination:pattern
--verbose=PATTERN:destination:pattern

Sets the log pattern for destination to pattern. Refer to ovs-appctl(8) for a description of the valid syntax for pattern.

-vFACILITY:facility
--verbose=FACILITY:facility

Sets the RFC5424 facility of the log message. facility can be one of kern, user, mail, daemon, auth, syslog, lpr, news, uucp, clock, ftp, ntp, audit, alert, clock2, local0, local1, local2, local3, local4, local5, local6 or local7. If this option is not specified, daemon is used as the default for the local system syslog and local0 is used while sending a message to the target provided via the --syslog-target option.

--log-file[=file]

Enables logging to a file. If file is specified, then it is used as the exact name for the log file. The default log file name used if file is omitted is /var/log/ovn/program.log.

--syslog-target=host:port

Send syslog messages to UDP port on host, in addition to the system syslog. The host must be a numerical IP address, not a hostname.

--syslog-method=method

Specify method as how syslog messages should be sent to syslog daemon. The following forms are supported:

The default is taken from the OVS_SYSLOG_METHOD environment variable; if it is unset, the default is libc.

PKI Options
PKI configuration is required in order to use SSL for the connections to the Northbound and Southbound databases.

-p privkey.pem
--private-key=privkey.pem

Specifies a PEM file containing the private key used as identity for outgoing SSL connections.

-c cert.pem
--certificate=cert.pem

Specifies a PEM file containing a certificate that certifies the private key specified on -p or --private-key to be trustworthy. The certificate must be signed by the certificate authority (CA) that the peer in SSL connections will use to verify it.

-C cacert.pem
--ca-cert=cacert.pem

Specifies a PEM file containing the CA certificate for verifying certificates presented to this program by SSL peers. (This may be the same certificate that SSL peers use to verify the certificate specified on -c or --certificate, or it may be a different one, depending on the PKI design in use.)

-C none
--ca-cert=none

Disables verification of certificates presented by SSL peers. This introduces a security risk, because it means that certificates cannot be verified to be those of known trusted hosts.

Other Options
--unixctl=socket

Sets the name of the control socket on which program listens for runtime management commands (see RUNTIME MANAGEMENT COMMANDS, below). If socket does not begin with /, it is interpreted as relative to . If --unixctl is not used at all, the default socket is /program.pid.ctl, where pid is program’s process ID.

On Windows a local named pipe is used to listen for runtime management commands. A file is created in the absolute path as pointed by socket or if --unixctl is not used at all, a file is created as program in the configured OVS_RUNDIR directory. The file exists just to mimic the behavior of a Unix domain socket.

Specifying none for socket disables the control socket feature.

--help

Prints a brief help message to the console.

--version

Prints version information to the console.

RUNTIME MANAGEMENT COMMANDS

ovs-appctl can send commands to a running ovn-northd process. The currently supported commands are described below.

is-paused

Returns "true" if ovn-northd is currently paused, "false" otherwise.

sb-cluster-state-reset

Reset southbound database cluster status when databases are destroyed and rebuilt.

If all databases in a clustered southbound database are removed from disk, then the stored index of all databases will be reset to zero. This will cause ovn-northd to be unable to read or write to the southbound database, because it will always detect the data as stale. In such a case, run this command so that ovn-northd will reset its local index so that it can interact with the southbound database again.

nb-cluster-state-reset

Reset northbound database cluster status when databases are destroyed and rebuilt.

This performs the same task as sb-cluster-state-reset except for the northbound database client.

set-n-threads N

Set the number of threads used for building logical flows. When N is within [2-256], parallelization is enabled. When N is 1 parallelization is disabled. When N is less than 1 or more than 256, an error is returned. If ovn-northd fails to start parallelization (e.g. fails to setup semaphores, parallelization is disabled and an error is returned.

get-n-threads

Return the number of threads used for building logical flows.

inc-engine/show-stats

Display ovn-northd engine counters. For each engine node the following counters have been added:

inc-engine/show-stats engine_node_name counter_name

Display the ovn-northd engine counter(s) for the specified engine_node_name. counter_name is optional and can be one of recompute, compute or abort.

inc-engine/clear-stats

Reset ovn-northd engine counters.

Only ovn-northd-ddlog supports the following commands:

enable-cpu-profiling
disable-cpu-profiling

Enables or disables profiling of CPU time used by the DDlog engine. When CPU profiling is enabled, the profile command (see below) will include DDlog CPU usage statistics in its output. Enabling CPU profiling will slow ovn-northd-ddlog. Disabling CPU profiling does not clear any previously recorded statistics.

profile

Outputs a profile of the current and peak sizes of arrangements inside DDlog. This profiling data can be useful for optimizing DDlog code. If CPU profiling was previously enabled (even if it was later disabled), the output also includes a CPU time profile. See Profiling inside the tutorial in the DDlog repository for an introduction to profiling DDlog.

ACTIVE-STANDBY FOR HIGH AVAILABILITY

You may run ovn-northd more than once in an OVN deployment. When connected to a standalone or clustered DB setup, OVN will automatically ensure that only one of them is active at a time. If multiple instances of ovn-northd are running and the active ovn-northd fails, one of the hot standby instances of ovn-northd will automatically take over.

Active-Standby with multiple OVN DB servers
You may run multiple OVN DB servers in an OVN deployment with:

In such deployments, the ovn-northds on the passive nodes will process the DB changes and compute logical flows to be thrown out later, because write transactions are not allowed by the passive ovsdb-servers. It results in unnecessary CPU usage.

With the help of runtime management command pause, you can pause ovn-northd on these nodes. When a passive node becomes master, you can use the runtime management command resume to resume the ovn-northd to process the DB changes.

LOGICAL FLOW TABLE STRUCTURE

One of the main purposes of ovn-northd is to populate the Logical_Flow table in the OVN_Southbound database. This section describes how ovn-northd does this for switch and router logical datapaths.

Logical Switch Datapaths

Ingress Table 0: Admission Control and Ingress Port Security check

Ingress table 0 contains these logical flows:

Ingress Table 1: Ingress Port Security - Apply

This table drops the packets if the port security check failed in the previous stage i.e the register bit REGBIT_PORT_SEC_DROP is set to 1.

Ingress table 1 contains these logical flows:

Ingress Table 2: Lookup MAC address learning table

This table looks up the MAC learning table of the logical switch datapath to check if the port-mac pair is present or not. MAC is learnt only for logical switch VIF ports whose port security is disabled and ’unknown’ address set.

•

Ingress Table 3: Learn MAC of ’unknown’ ports.

This table learns the MAC addresses seen on the logical ports whose port security is disabled and ’unknown’ address set if the lookup_fdb action returned false in the previous table.

•

Ingress Table 4: from-lport Pre-ACLs

This table prepares flows for possible stateful ACL processing in ingress table ACLs. It contains a priority-0 flow that simply moves traffic to the next table. If stateful ACLs are used in the logical datapath, a priority-100 flow is added that sets a hint (with reg0[0] = 1; next;) for table Pre-stateful to send IP packets to the connection tracker before eventually advancing to ingress table ACLs. If special ports such as route ports or localnet ports can’t use ct(), a priority-110 flow is added to skip over stateful ACLs. Multicast, IPv6 Neighbor Discovery and MLD traffic also skips stateful ACLs. For "allow-stateless" ACLs, a flow is added to bypass setting the hint for connection tracker processing when there are stateful ACLs or LB rules; REGBIT_ACL_STATELESS is set for traffic matching stateless ACL flows.

This table also has a priority-110 flow with the match eth.dst == E for all logical switch datapaths to move traffic to the next table. Where E is the service monitor mac defined in the options:svc_monitor_mac column of NB_Global table.

Ingress Table 5: Pre-LB

This table prepares flows for possible stateful load balancing processing in ingress table LB and Stateful. It contains a priority-0 flow that simply moves traffic to the next table. Moreover it contains two priority-110 flows to move multicast, IPv6 Neighbor Discovery and MLD traffic to the next table. It also contains two priority-110 flows to move stateless traffic, i.e traffic for which REGBIT_ACL_STATELESS is set, to the next table. If load balancing rules with virtual IP addresses (and ports) are configured in OVN_Northbound database for a logical switch datapath, a priority-100 flow is added with the match ip to match on IP packets and sets the action reg0[2] = 1; next; to act as a hint for table Pre-stateful to send IP packets to the connection tracker for packet de-fragmentation (and to possibly do DNAT for already established load balanced traffic) before eventually advancing to ingress table Stateful. If controller_event has been enabled and load balancing rules with empty backends have been added in OVN_Northbound, a 130 flow is added to trigger ovn-controller events whenever the chassis receives a packet for that particular VIP. If event-elb meter has been previously created, it will be associated to the empty_lb logical flow

Prior to OVN 20.09 we were setting the reg0[0] = 1 only if the IP destination matches the load balancer VIP. However this had few issues cases where a logical switch doesn’t have any ACLs with allow-related action. To understand the issue lets a take a TCP load balancer - 10.0.0.10:80=10.0.0.3:80. If a logical port - p1 with IP - 10.0.0.5 opens a TCP connection with the VIP - 10.0.0.10, then the packet in the ingress pipeline of ’p1’ is sent to the p1’s conntrack zone id and the packet is load balanced to the backend - 10.0.0.3. For the reply packet from the backend lport, it is not sent to the conntrack of backend lport’s zone id. This is fine as long as the packet is valid. Suppose the backend lport sends an invalid TCP packet (like incorrect sequence number), the packet gets delivered to the lport ’p1’ without unDNATing the packet to the VIP - 10.0.0.10. And this causes the connection to be reset by the lport p1’s VIF.

We can’t fix this issue by adding a logical flow to drop ct.inv packets in the egress pipeline since it will drop all other connections not destined to the load balancers. To fix this issue, we send all the packets to the conntrack in the ingress pipeline if a load balancer is configured. We can now add a lflow to drop ct.inv packets.

This table also has priority-120 flows that punt all IGMP/MLD packets to ovn-controller if the switch is an interconnect switch with multicast snooping enabled.

This table also has a priority-110 flow with the match eth.dst == E for all logical switch datapaths to move traffic to the next table. Where E is the service monitor mac defined in the options:svc_monitor_mac column of NB_Global table.

This table also has a priority-110 flow with the match inport == I for all logical switch datapaths to move traffic to the next table. Where I is the peer of a logical router port. This flow is added to skip the connection tracking of packets which enter from logical router datapath to logical switch datapath.

Ingress Table 6: Pre-stateful

This table prepares flows for all possible stateful processing in next tables. It contains a priority-0 flow that simply moves traffic to the next table.

Ingress Table 7: from-lport ACL hints

This table consists of logical flows that set hints (reg0 bits) to be used in the next stage, in the ACL processing table, if stateful ACLs or load balancers are configured. Multiple hints can be set for the same packet. The possible hints are:

The table contains the following flows:

Ingress table 8: from-lport ACLs before LB

Logical flows in this table closely reproduce those in the ACL table in the OVN_Northbound database for the from-lport direction without the option apply-after-lb set or set to false. The priority values from the ACL table have a limited range and have 1000 added to them to leave room for OVN default flows at both higher and lower priorities.

This table contains a priority-65535 flow to advance to the next table if the logical switch has no ACLs configured, otherwise a priority-0 flow to advance to the next table so that ACLs allow packets by default if options:default_acl_drop column of NB_Global is false or not set. Otherwise the flow action is set to drop; to implement a default drop behavior.

If the logical datapath has a stateful ACL or a load balancer with VIP configured, the following flows will also be added:

If the logical datapath has any ACL or a load balancer with VIP configured, the following flow will also be added:

Ingress Table 9: from-lport QoS Marking

Logical flows in this table closely reproduce those in the QoS table with the action column set in the OVN_Northbound database for the from-lport direction.

Ingress Table 10: from-lport QoS Meter

Logical flows in this table closely reproduce those in the QoS table with the bandwidth column set in the OVN_Northbound database for the from-lport direction.

Ingress Table 11: Load balancing affinity check

Load balancing affinity check table contains the following logical flows:

Ingress Table 12: LB

Ingress Table 13: Load balancing affinity learn

Load balancing affinity learn table contains the following logical flows:

Ingress Table 14: Pre-Hairpin

Ingress Table 15: Nat-Hairpin

Ingress Table 16: Hairpin

and action next; to pass the traffic to the next table to respond to the ARP requests for the router port IPs.

reg0[14] register bit is set in the ingress L2 port security check table for traffic received from HW VTEP (ramp) ports.

Ingress table 17: from-lport ACLs after LB

Logical flows in this table closely reproduce those in the ACL table in the OVN_Northbound database for the from-lport direction with the option apply-after-lb set to true. The priority values from the ACL table have a limited range and have 1000 added to them to leave room for OVN default flows at both higher and lower priorities.

Ingress Table 18: Stateful

Ingress Table 19: ARP/ND responder

This table implements ARP/ND responder in a logical switch for known IPs. The advantage of the ARP responder flow is to limit ARP broadcasts by locally responding to ARP requests without the need to send to other hypervisors. One common case is when the inport is a logical port associated with a VIF and the broadcast is responded to on the local hypervisor rather than broadcast across the whole network and responded to by the destination VM. This behavior is proxy ARP.

ARP requests arrive from VMs from a logical switch inport of type default. For this case, the logical switch proxy ARP rules can be for other VMs or logical router ports. Logical switch proxy ARP rules may be programmed both for mac binding of IP addresses on other logical switch VIF ports (which are of the default logical switch port type, representing connectivity to VMs or containers), and for mac binding of IP addresses on logical switch router type ports, representing their logical router port peers. In order to support proxy ARP for logical router ports, an IP address must be configured on the logical switch router type port, with the same value as the peer logical router port. The configured MAC addresses must match as well. When a VM sends an ARP request for a distributed logical router port and if the peer router type port of the attached logical switch does not have an IP address configured, the ARP request will be broadcast on the logical switch. One of the copies of the ARP request will go through the logical switch router type port to the logical router datapath, where the logical router ARP responder will generate a reply. The MAC binding of a distributed logical router, once learned by an associated VM, is used for all that VM’s communication needing routing. Hence, the action of a VM re-arping for the mac binding of the logical router port should be rare.

Logical switch ARP responder proxy ARP rules can also be hit when receiving ARP requests externally on a L2 gateway port. In this case, the hypervisor acting as an L2 gateway, responds to the ARP request on behalf of a destination VM.

Note that ARP requests received from localnet logical inports can either go directly to VMs, in which case the VM responds or can hit an ARP responder for a logical router port if the packet is used to resolve a logical router port next hop address. In either case, logical switch ARP responder rules will not be hit. It contains these logical flows:

inport == P && && ((arp.op == 1 && arp.spa == VIP && arp.tpa == VIP) || (arp.op == 2 && arp.spa == VIP))
inport == P && && ((nd_ns && ip6.dst == {VIP, NS_MULTICAST_ADDR} && nd.target == VIP) || (nd_na && nd.target == VIP))

and applies the action

bind_vport(V, inport);

and advances the packet to the next table.

Where VIP is the virtual ip configured in the column options:virtual-ip and NS_MULTICAST_ADDR is solicited-node multicast address corresponding to the VIP.

eth.dst = eth.src;
eth.src = E;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = E;
arp.tpa = arp.spa;
arp.spa = A;
outport = inport;
flags.loopback = 1;
output;

These flows are omitted for logical ports (other than router ports or localport ports) that are down (unless ignore_lsp_down is configured as true in options column of NB_Global table of the Northbound database), for logical ports of type virtual, for logical ports with ’unknown’ address set and for logical ports of a logical switch configured with other_config:vlan-passthru=true.

The above ARP responder flows are added for the list of IPv4 addresses if defined in options:arp_proxy column of Logical_Switch_Port table for logical switch ports of type router.

nd_na {
eth.src = E;
ip6.src = A;
nd.target = A;
nd.tll = E;
outport = inport;
flags.loopback = 1;
output;
};

Priority-50 flows that match IPv6 ND neighbor solicitations to each known IP address A (and A’s solicited node address) of logical switch port of type router, and respond with neighbor advertisements directly with corresponding Ethernet address E:

nd_na_router {
eth.src = E;
ip6.src = A;
nd.target = A;
nd.tll = E;
outport = inport;
flags.loopback = 1;
output;
};

These flows are omitted for logical ports (other than router ports or localport ports) that are down (unless ignore_lsp_down is configured as true in options column of NB_Global table of the Northbound database), for logical ports of type virtual and for logical ports with ’unknown’ address set.

In place of next;, it would be reasonable to use drop; for the flows’ actions. If everything is working as it is configured, then this would produce equivalent results, since no host should reply to the request. But ARPing for one’s own IP address is intended to detect situations where the network is not working as configured, so dropping the request would frustrate that intent.

eth.dst = eth.src;
eth.src = E;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = E;
arp.tpa = arp.spa;
arp.spa = A;
outport = inport;
flags.loopback = 1;
output;

where E is the service monitor source mac defined in the options:svc_monitor_mac column in the NB_Global table. This mac is used as the source mac in the service monitor packets for the load balancer endpoint IP health checks.

SVC_MON_SRC_IP is used as the source ip in the service monitor IPv4 packets for the load balancer endpoint IP health checks.

These flows are required if an ARP request is sent for the IP SVC_MON_SRC_IP.

For IPv6 the similar flow is added with the following action

nd_na {
eth.dst = eth.src;
eth.src = E;
ip6.src = A;
nd.target = A;
nd.tll = E;
outport = inport;
flags.loopback = 1;
output;
};

and applies the action

eth.dst = eth.src;
eth.src = E;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = E;
arp.tpa = arp.spa;
arp.spa = A;
outport = inport;
flags.loopback = 1;
output;

where E is the forwarding group’s mac defined in the vmac.

A is used as either the destination ip for load balancing traffic to child ports or as nexthop to hosts behind the child ports.

These flows are required to respond to an ARP request if an ARP request is sent for the IP vip.

Ingress Table 20: DHCP option processing

This table adds the DHCPv4 options to a DHCPv4 packet from the logical ports configured with IPv4 address(es) and DHCPv4 options, and similarly for DHCPv6 options. This table also adds flows for the logical ports of type external.

reg0[3] = put_dhcp_opts(offer_ip = ip, options...);
next;

For DHCPDISCOVER and DHCPREQUEST, this transforms the packet into a DHCP reply, adds the DHCP offer IP ip and options to the packet, and stores 1 into reg0[3]. For other kinds of packets, it just stores 0 into reg0[3]. Either way, it continues to the next table.

reg0[3] = put_dhcpv6_opts(ia_addr = ip, options...);
next;

For DHCPv6 Solicit/Request/Confirm packets, this transforms the packet into a DHCPv6 Advertise/Reply, adds the DHCPv6 offer IP ip and options to the packet, and stores 1 into reg0[3]. For other kinds of packets, it just stores 0 into reg0[3]. Either way, it continues to the next table.

Ingress Table 21: DHCP responses

This table implements DHCP responder for the DHCP replies generated by the previous table.

eth.dst = eth.src;
eth.src = E;
ip4.src = S;
udp.src = 67;
udp.dst = 68;
outport = P;
flags.loopback = 1;
output;

where E is the server MAC address and S is the server IPv4 address defined in the DHCPv4 options. Note that ip4.dst field is handled by put_dhcp_opts.

(This terminates ingress packet processing; the packet does not go to the next ingress table.)

eth.dst = eth.src;
eth.src = E;
ip6.dst = A;
ip6.src = S;
udp.src = 547;
udp.dst = 546;
outport = P;
flags.loopback = 1;
output;

where E is the server MAC address and S is the server IPv6 LLA address generated from the server_id defined in the DHCPv6 options and A is the IPv6 address defined in the logical port’s addresses column.

(This terminates packet processing; the packet does not go on the next ingress table.)

Ingress Table 22 DNS Lookup

This table looks up and resolves the DNS names to the corresponding configured IP address(es).

reg0[4] = dns_lookup(); next;

For valid DNS packets, this transforms the packet into a DNS reply if the DNS name can be resolved, and stores 1 into reg0[4]. For failed DNS resolution or other kinds of packets, it just stores 0 into reg0[4]. Either way, it continues to the next table.

Ingress Table 23 DNS Responses

This table implements DNS responder for the DNS replies generated by the previous table.

eth.dst <-> eth.src;
ip4.src <-> ip4.dst;
udp.dst = udp.src;
udp.src = 53;
outport = P;
flags.loopback = 1;
output;

(This terminates ingress packet processing; the packet does not go to the next ingress table.)

Ingress table 24 External ports

Traffic from the external logical ports enter the ingress datapath pipeline via the localnet port. This table adds the below logical flows to handle the traffic from these ports.

This flow guarantees that the ARP/NS request to the router IP address from the external ports is responded by only the chassis which has claimed these external ports. All the other chassis, drops these packets.

A priority-100 flow is added for each external logical port which doesn’t reside on a chassis to drop any packet destined to the router mac - with the match inport == external && eth.src == E && eth.dst == R && !is_chassis_resident("external") where E is the external port mac and R is the router port mac.

Ingress Table 25 Destination Lookup

This table implements switching behavior. It contains these logical flows:

For the Ethernet address on a logical switch port of type router, when that logical switch port’s addresses column is set to router and the connected logical router port has a gateway chassis:

For the Ethernet address on a logical switch port of type router, when that logical switch port’s addresses column is set to router and the connected logical router port specifies a reside-on-redirect-chassis and the logical router to which the connected logical router port belongs to has a distributed gateway LRP:

For each forwarding group configured on the logical switch datapath, a priority-50 flow that matches on eth.dst == VIP
with an action of fwd_group(childports=args ), where args contains comma separated logical switch child ports to load balance to. If liveness is enabled, then action also includes liveness=true.

Ingress Table 26 Destination unknown

This table handles the packets whose destination was not found or and looked up in the MAC learning table of the logical switch datapath. It contains the following flows.

•

If the logical switch has no logical ports with ’unknown’ address set, then the below logical flow is added

One priority-0 fallback flow that outputs the packet to the egress stage with the outport learnt from get_fdb action.

Egress Table 0: to-lport Pre-ACLs

This is similar to ingress table Pre-ACLs except for to-lport traffic.

This table also has a priority-110 flow with the match eth.src == E for all logical switch datapaths to move traffic to the next table. Where E is the service monitor mac defined in the options:svc_monitor_mac column of NB_Global table.

This table also has a priority-110 flow with the match outport == I for all logical switch datapaths to move traffic to the next table. Where I is the peer of a logical router port. This flow is added to skip the connection tracking of packets which will be entering logical router datapath from logical switch datapath for routing.

Egress Table 1: Pre-LB

This table is similar to ingress table Pre-LB. It contains a priority-0 flow that simply moves traffic to the next table. Moreover it contains two priority-110 flows to move multicast, IPv6 Neighbor Discovery and MLD traffic to the next table. If any load balancing rules exist for the datapath, a priority-100 flow is added with a match of ip and action of reg0[2] = 1; next; to act as a hint for table Pre-stateful to send IP packets to the connection tracker for packet de-fragmentation and possibly DNAT the destination VIP to one of the selected backend for already committed load balanced traffic.

This table also has a priority-110 flow with the match eth.src == E for all logical switch datapaths to move traffic to the next table. Where E is the service monitor mac defined in the options:svc_monitor_mac column of NB_Global table.

Egress Table 2: Pre-stateful

This is similar to ingress table Pre-stateful. This table adds the below 3 logical flows.

Egress Table 3: from-lport ACL hints

This is similar to ingress table ACL hints.

Egress Table 4: to-lport ACLs

This is similar to ingress table ACLs except for to-lport ACLs.

In addition, the following flows are added.

Egress Table 5: to-lport QoS Marking

This is similar to ingress table QoS marking except they apply to to-lport QoS rules.

Egress Table 6: to-lport QoS Meter

This is similar to ingress table QoS meter except they apply to to-lport QoS rules.

Egress Table 7: Stateful

This is similar to ingress table Stateful except that there are no rules added for load balancing new connections.

Egress Table 8: Egress Port Security - check

This is similar to the port security logic in table Ingress Port Security check except that action check_out_port_sec is used to check the port security rules. This table adds the below logical flows.

Egress Table 9: Egress Port Security - Apply

This is similar to the ingress port security logic in ingress table A Ingress Port Security - Apply. This table drops the packets if the port security check failed in the previous stage i.e the register bit REGBIT_PORT_SEC_DROP is set to 1.

The following flows are added.

Please remember to mark the corresponding physical interface with ovn-egress-iface set to true in external_ids.

Logical Router Datapaths
Logical router datapaths will only exist for Logical_Router rows in the OVN_Northbound database that do not have enabled set to false

Ingress Table 0: L2 Admission Control

This table drops packets that the router shouldn’t see at all based on their Ethernet headers. It contains the following flows:

For the gateway port on a distributed logical router (where one of the logical router ports specifies a gateway chassis), the above flow matching eth.dst == E is only programmed on the gateway port instance on the gateway chassis. If LRP’s logical switch has attached LSP of vtep type, the is_chassis_resident() part is not added to lflow to allow traffic originated from logical switch to reach LR services (LBs, NAT).

For a distributed logical router or for gateway router where the port is configured with options:gateway_mtu the action of the above flow is modified adding check_pkt_larger in order to mark the packet setting REGBIT_PKT_LARGER if the size is greater than the MTU. If the port is also configured with options:gateway_mtu_bypass then another flow is added, with priority-55, to bypass the check_pkt_larger flow. This is useful for traffic that normally doesn’t need to be fragmented and for which check_pkt_larger, which might not be offloadable, is not really needed. One such example is TCP traffic.

This flow is only programmed on the gateway port instance on the chassis where the logical_port specified in the NAT rule resides.

Other packets are implicitly dropped.

Ingress Table 1: Neighbor lookup

For ARP and IPv6 Neighbor Discovery packets, this table looks into the MAC_Binding records to determine if OVN needs to learn the mac bindings. Following flows are added:

reg9[2] = lookup_arp(inport, arp.spa, arp.sha);
next;

If the option always_learn_from_arp_request is false, the following two flows are added.

A priority-110 flow is added which matches inport == P && arp.spa == S/L && arp.tpa == A && arp.op == 1 (ARP request) with the following actions:

reg9[2] = lookup_arp(inport, arp.spa, arp.sha);
reg9[3] = 1;
next;

A priority-100 flow is added which matches inport == P && arp.spa == S/L && arp.op == 1 (ARP request) with the following actions:

reg9[2] = lookup_arp(inport, arp.spa, arp.sha);
reg9[3] = lookup_arp_ip(inport, arp.spa);
next;

If the logical router port P is a distributed gateway router port, additional match is_chassis_resident(cr-P) is added for all these flows.

reg9[2] = lookup_arp(inport, arp.spa, arp.sha);
next;

If the option always_learn_from_arp_request is false, the above actions will be:

reg9[2] = lookup_arp(inport, arp.spa, arp.sha);
reg9[3] = 1;
next;

reg9[2] = lookup_nd(inport, nd.target, nd.tll);
next;

If the option always_learn_from_arp_request is false, the above actions will be:

reg9[2] = lookup_nd(inport, nd.target, nd.tll);
reg9[3] = 1;
next;

reg9[2] = lookup_nd(inport, ip6.src, nd.sll);
next;

If the option always_learn_from_arp_request is false, the above actions will be:

reg9[2] = lookup_nd(inport, ip6.src, nd.sll);
reg9[3] = lookup_nd_ip(inport, ip6.src);
next;

Ingress Table 2: Neighbor learning

This table adds flows to learn the mac bindings from the ARP and IPv6 Neighbor Solicitation/Advertisement packets if it is needed according to the lookup results from the previous stage.

reg9[2] will be 1 if the lookup_arp/lookup_nd in the previous table was successful or skipped, meaning no need to learn mac binding from the packet.

reg9[3] will be 1 if the lookup_arp_ip/lookup_nd_ip in the previous table was successful or skipped, meaning it is ok to learn mac binding from the packet (if reg9[2] is 0).

Ingress Table 3: IP Input

This table is the core of the logical router datapath functionality. It contains the following flows to implement very basic IP host functionality.

icmp4_error {
icmp4.type = 3; /* Destination Unreachable. */
icmp4.code = 4; /* Frag Needed and DF was Set. */
icmp4.frag_mtu = M;
eth.dst = eth.src;
eth.src = E;
ip4.dst = ip4.src;
ip4.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER 0;
outport = LRP;
flags.loopback = 1;
output;
};
icmp6_error {
icmp6.type = 2;
icmp6.code = 0;
icmp6.frag_mtu = M;
eth.dst = eth.src;
eth.src = E;
ip6.dst = ip6.src;
ip6.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER 0;
outport = LRP;
flags.loopback = 1;
output;
};

where E and I are the NAT rule external mac and IP respectively.

icmp4_error {
icmp4.type = 3; /* Destination Unreachable. */
icmp4.code = 4; /* Frag Needed and DF was Set. */
icmp4.frag_mtu = M;
eth.dst = E;
ip4.dst = ip4.src;
ip4.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER 0;
next(pipeline=ingress, table=0);
};
icmp6_error {
icmp6.type = 2;
icmp6.code = 0;
icmp6.frag_mtu = M;
eth.dst = E;
ip6.dst = ip6.src;
ip6.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER 0;
next(pipeline=ingress, table=0);
};

The above flow is required to handle the routing of the East/west NAT traffic.

•

L3 admission control: Priority-120 flows allows IGMP and MLD packets if the router has logical ports that have options :mcast_flood=’true’.

•

A priority-100 flow parses DHCPv6 replies from IPv6 prefix delegation routers (udp.src == 547 && udp.dst == 546). The handle_dhcpv6_reply is used to send IPv6 prefix delegation messages to the delegation router.

ip4.dst <-> ip4.src;
ip.ttl = 255;
icmp4.type = 0;
flags.loopback = 1;
next;

Flows for ICMPv6 echo requests use the following actions:

ip6.dst <-> ip6.src;
ip.ttl = 255;
icmp6.type = 129;
flags.loopback = 1;
next;

These flows reply to ARP requests for the router’s own IP address. The ARP requests are handled only if the requestor’s IP belongs to the same subnets of the logical router port. For each router port P that owns IP address A, which belongs to subnet S with prefix length L, and Ethernet address E, a priority-90 flow matches inport == P && arp.spa == S/L && arp.op == 1 && arp.tpa == A (ARP request) with the following actions:

eth.dst = eth.src;
eth.src = xreg0[0..47];
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = xreg0[0..47];
arp.tpa = arp.spa;
arp.spa = A;
outport = inport;
flags.loopback = 1;
output;

For the gateway port on a distributed logical router (where one of the logical router ports specifies a gateway chassis), the above flows are only programmed on the gateway port instance on the gateway chassis. This behavior avoids generation of multiple ARP responses from different chassis, and allows upstream MAC learning to point to the gateway chassis.

For the logical router port with the option reside-on-redirect-chassis set (which is centralized), the above flows are only programmed on the gateway port instance on the gateway chassis (if the logical router has a distributed gateway port). This behavior avoids generation of multiple ARP responses from different chassis, and allows upstream MAC learning to point to the gateway chassis.

For each router port P that owns IPv6 address A, solicited node address S, and Ethernet address E, a priority-90 flow matches inport == P && nd_ns && ip6.dst == {A, E} && nd.target == A with the following actions:

nd_na_router {
eth.src = xreg0[0..47];
ip6.src = A;
nd.target = A;
nd.tll = xreg0[0..47];
outport = inport;
flags.loopback = 1;
output;
};

For the gateway port on a distributed logical router (where one of the logical router ports specifies a gateway chassis), the above flows replying to IPv6 Neighbor Solicitations are only programmed on the gateway port instance on the gateway chassis. This behavior avoids generation of multiple replies from different chassis, and allows upstream MAC learning to point to the gateway chassis.

IPv4: For a configured NAT (both DNAT and SNAT) IP address or a load balancer IPv4 VIP A, for each router port P with Ethernet address E, a priority-90 flow matches arp.op == 1 && arp.tpa == A (ARP request) with the following actions:

eth.dst = eth.src;
eth.src = xreg0[0..47];
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = xreg0[0..47];
arp.tpa <-> arp.spa;
outport = inport;
flags.loopback = 1;
output;

IPv4: For a configured load balancer IPv4 VIP, a similar flow is added with the additional match inport == P if the VIP is reachable from any logical router port of the logical router.

If the router port P is a distributed gateway router port, then the is_chassis_resident(P) is also added in the match condition for the load balancer IPv4 VIP A.

IPv6: For a configured NAT (both DNAT and SNAT) IP address or a load balancer IPv6 VIP A (if the VIP is reachable from any logical router port of the logical router), solicited node address S, for each router port P with Ethernet address E, a priority-90 flow matches inport == P && nd_ns && ip6.dst == {A, S} && nd.target == A with the following actions:

eth.dst = eth.src;
nd_na {
eth.src = xreg0[0..47];
nd.tll = xreg0[0..47];
ip6.src = A;
nd.target = A;
outport = inport;
flags.loopback = 1;
output;
}

If the router port P is a distributed gateway router port, then the is_chassis_resident(P) is also added in the match condition for the load balancer IPv6 VIP A.

For the gateway port on a distributed logical router with NAT (where one of the logical router ports specifies a gateway chassis):

Some of the actions are different for this case, using the external_mac specified in the NAT rule rather than the gateway port’s Ethernet address E:

eth.src = external_mac;
arp.sha = external_mac;

or in the case of IPv6 neighbor solicition:

eth.src = external_mac;
nd.tll = external_mac;

This behavior avoids generation of multiple ARP responses from different chassis, and allows upstream MAC learning to point to the correct chassis.

These flows should not match IP fragments with nonzero offset.

These flows should not match IP fragments with nonzero offset.

These flows should not match IP fragments with nonzero offset.

The flows above handle all of the traffic that might be directed to the router itself. The following flows (with lower priorities) handle the remaining traffic, potentially for forwarding:

icmp4 {
icmp4.type = 11; /* Time exceeded. */
icmp4.code = 0; /* TTL exceeded in transit. */
ip4.dst = ip4.src;
ip4.src = A;
ip.ttl = 254;
next;
};
icmp6 {
icmp6.type = 3; /* Time exceeded. */
icmp6.code = 0; /* TTL exceeded in transit. */
ip6.dst = ip6.src;
ip6.src = A;
ip.ttl = 254;
next;
};

Ingress Table 4: UNSNAT

This is for already established connections’ reverse traffic. i.e., SNAT has already been done in egress pipeline and now the packet has entered the ingress pipeline as part of a reply. It is unSNATted here.

Ingress Table 4: UNSNAT on Gateway and Distributed Routers

For each IPv4 address A defined as load balancer VIP with the protocol P (and the protocol port T if defined) is also present as an external_ip in the NAT table, a priority-120 logical flow is added with the match ip4 && ip4.dst == A && P with the action next; to advance the packet to the next table. If the load balancer has protocol port B defined, then the match also has P.dst == B.

The above flows are also added for IPv6 load balancers.

Ingress Table 4: UNSNAT on Gateway Routers

If the Gateway router is configured with lb_force_snat_ip=router_ip then for every logical router port P attached to the Gateway router with the router ip B, a priority-110 flow is added with the match inport == P && ip4.dst == B or inport == P && ip6.dst == B with an action ct_snat; .

If the Gateway router has been configured to force SNAT any previously load-balanced packets to B, a priority-100 flow matches ip && ip4.dst == B or ip && ip6.dst == B with an action ct_snat; .

For each NAT configuration in the OVN Northbound database, that asks to change the source IP address of a packet from A to B, a priority-90 flow matches ip && ip4.dst == B or ip && ip6.dst == B with an action ct_snat; . If the NAT rule is of type dnat_and_snat and has stateless=true in the options, then the action would be next;.

A priority-0 logical flow with match 1 has actions next;.

Ingress Table 4: UNSNAT on Distributed Routers

If the NAT rule cannot be handled in a distributed manner, then the below priority-100 flows are only programmed on the gateway chassis.

If the NAT entry is of type snat, then there is an additional match is_chassis_resident(cr-GW)
where cr-GW is the chassis resident port of GW.

If the NAT entry is of type snat, then there is an additional match is_chassis_resident(cr-GW)
where cr-GW is the chassis resident port of GW.

A priority-0 logical flow with match 1 has actions next;.

Ingress Table 5: DEFRAG

This is to send packets to connection tracker for tracking and defragmentation. It contains a priority-0 flow that simply moves traffic to the next table.

If load balancing rules with only virtual IP addresses are configured in OVN_Northbound database for a Gateway router, a priority-100 flow is added for each configured virtual IP address VIP. For IPv4 VIPs the flow matches ip && ip4.dst == VIP. For IPv6 VIPs, the flow matches ip && ip6.dst == VIP. The flow applies the action reg0 = VIP; ct_dnat; (or xxreg0 for IPv6) to send IP packets to the connection tracker for packet de-fragmentation and to dnat the destination IP for the committed connection before sending it to the next table.

If load balancing rules with virtual IP addresses and ports are configured in OVN_Northbound database for a Gateway router, a priority-110 flow is added for each configured virtual IP address VIP, protocol PROTO and port PORT. For IPv4 VIPs the flow matches ip && ip4.dst == VIP && PROTO && PROTO.dst == PORT. For IPv6 VIPs, the flow matches ip && ip6.dst == VIP && PROTO && PROTO.dst == PORT. The flow applies the action reg0 = VIP; reg9[16..31] = PROTO.dst; ct_dnat; (or xxreg0 for IPv6) to send IP packets to the connection tracker for packet de-fragmentation and to dnat the destination IP for the committed connection before sending it to the next table.

If ECMP routes with symmetric reply are configured in the OVN_Northbound database for a gateway router, a priority-100 flow is added for each router port on which symmetric replies are configured. The matching logic for these ports essentially reverses the configured logic of the ECMP route. So for instance, a route with a destination routing policy will instead match if the source IP address matches the static route’s prefix. The flow uses the actions chk_ecmp_nh_mac(); ct_next or chk_ecmp_nh(); ct_next to send IP packets to table 76 or to table 77 in order to check if source info are already stored by OVN and then to the connection tracker for packet de-fragmentation and tracking before sending it to the next table.

If load balancing rules are configured in OVN_Northbound database for a Gateway router, a priority 50 flow that matches icmp || icmp6 with an action of ct_dnat;, this allows potentially related ICMP traffic to pass through CT.

Ingress Table 6: Load balancing affinity check

Load balancing affinity check table contains the following logical flows:

in the IPv6 case) with an action of reg9[6] = chk_lb_aff(); next;

Ingress Table 7: DNAT

Packets enter the pipeline with destination IP address that needs to be DNATted from a virtual IP address to a real IP address. Packets in the reverse direction needs to be unDNATed.

Ingress Table 7: Load balancing DNAT rules

Following load balancing DNAT flows are added for Gateway router or Router with gateway port. These flows are programmed only on the gateway chassis. These flows do not get programmed for load balancers with IPv6 VIPs.

PORT (xxreg0 == VIP
in the IPv6 case) with an action of ct_lb_mark(args), where args contains comma separated IPv4 or IPv6 addresses (and optional port numbers) to load balance to. If the router is configured to force SNAT any load-balanced packets, the above action will be replaced by flags.force_snat_for_lb = 1; ct_lb_mark(args);. If the load balancing rule is configured with skip_snat set to true, the above action will be replaced by flags.skip_snat_for_lb = 1; ct_lb_mark(args);. If health check is enabled, then args will only contain those endpoints whose service monitor status entry in OVN_Southbound db is either online or empty.

The previous table lr_in_defrag sets the register reg0 (or xxreg0 for IPv6) and does ct_dnat. Hence for established traffic, this table just advances the packet to the next stage.

PORT (ip6 and xxreg0 == VIP in the IPv6 case) with an action of next;. If the router is configured to force SNAT any load-balanced packets, the above action will be replaced by flags.force_snat_for_lb = 1; next;. If the load balancing rule is configured with skip_snat set to true, the above action will be replaced by flags.skip_snat_for_lb = 1; next;.

The previous table lr_in_defrag sets the register reg0 (or xxreg0 for IPv6) and does ct_dnat. Hence for established traffic, this table just advances the packet to the next stage.

The previous table lr_in_defrag sets the register reg0 (or xxreg0 for IPv6) and does ct_dnat. Hence for established traffic, this table just advances the packet to the next stage.

The previous table lr_in_defrag sets the register reg0 (or xxreg0 for IPv6) and does ct_dnat. Hence for established traffic, this table just advances the packet to the next stage.

Ingress Table 7: DNAT on Gateway Routers

If the NAT rule has allowed_ext_ips configured, then there is an additional match ip4.src == allowed_ext_ips . Similarly, for IPV6, match would be ip6.src == allowed_ext_ips.

If the NAT rule has exempted_ext_ips set, then there is an additional flow configured at priority 101. The flow matches if source ip is an exempted_ext_ip and the action is next; . This flow is used to bypass the ct_dnat action for a packet originating from exempted_ext_ips.

Ingress Table 7: DNAT on Distributed Routers

On distributed routers, the DNAT table only handles packets with destination IP address that needs to be DNATted from a virtual IP address to a real IP address. The unDNAT processing in the reverse direction is handled in a separate table in the egress pipeline.

If the NAT rule cannot be handled in a distributed manner, then the priority-100 flow above is only programmed on the gateway chassis.

If the NAT rule has allowed_ext_ips configured, then there is an additional match ip4.src == allowed_ext_ips . Similarly, for IPV6, match would be ip6.src == allowed_ext_ips.

If the NAT rule has exempted_ext_ips set, then there is an additional flow configured at priority 101. The flow matches if source ip is an exempted_ext_ip and the action is next; . This flow is used to bypass the ct_dnat action for a packet originating from exempted_ext_ips.

A priority-0 logical flow with match 1 has actions next;.

Ingress Table 8: Load balancing affinity learn

Load balancing affinity learn table contains the following logical flows:

column, that includes a L4 port PORT of protocol P and IPv4 or IPv6 address VIP, a priority-100 flow that matches on reg9[6] == 0 && ct.new && ip && reg0 == VIP && P && reg9[16..31] == PORT (xxreg0 == VIP in the IPv6 case) with an action of commit_lb_aff(vip = VIP:PORT, backend = backend ip: backend port, proto = P, timeout = T);.

Ingress Table 9: ECMP symmetric reply processing

to commit the connection and storing eth.src and the ECMP reply port binding tunnel key K in the ct_label and the traffic pattern to table 76 or 77.

Ingress Table 10: IPv6 ND RA option processing

reg0[5] = put_nd_ra_opts(options);next;

For a valid IPv6 ND RS packet, this transforms the packet into an IPv6 ND RA reply and sets the RA options to the packet and stores 1 into reg0[5]. For other kinds of packets, it just stores 0 into reg0[5]. Either way, it continues to the next table.

Ingress Table 11: IPv6 ND RA responder

This table implements IPv6 ND RA responder for the IPv6 ND RA replies generated by the previous table.

eth.dst = eth.src;
eth.src = E;
ip6.dst = ip6.src;
ip6.src = I;
outport = P;
flags.loopback = 1;
output;

where E is the MAC address and I is the IPv6 link local address of the logical router port.

(This terminates packet processing in ingress pipeline; the packet does not go to the next ingress table.)

Ingress Table 12: IP Routing Pre

If a packet arrived at this table from Logical Router Port P which has options:route_table value set, a logical flow with match inport == "P" with priority 100 and action setting unique-generated per-datapath 32-bit value (non-zero) in OVS register 7. This register’s value is checked in next table. If packet didn’t match any configured inport (<main> route table), register 7 value is set to 0.

This table contains the following logical flows:

A priority-0 logical flow with match 1 has actions reg7 = 0; next;.

Ingress Table 13: IP Routing

A packet that arrives at this table is an IP packet that should be routed to the address in ip4.dst or ip6.dst. This table implements IP routing, setting reg0 (or xxreg0 for IPv6) to the next-hop IP address (leaving ip4.dst or ip6.dst, the packet’s final destination, unchanged) and advances to the next table for ARP resolution. It also sets reg1 (or xxreg1) to the IP address owned by the selected router port (ingress table ARP Request will generate an ARP request, if needed, with reg0 as the target protocol address and reg1 as the source protocol address).

For ECMP routes, i.e. multiple static routes with same policy and prefix but different nexthops, the above actions are deferred to next table. This table, instead, is responsible for determine the ECMP group id and select a member id within the group based on 5-tuple hashing. It stores group id in reg8[0..15] and member id in reg8[16..31]. This step is skipped with a priority-10300 rule if the traffic going out the ECMP route is reply traffic, and the ECMP route was configured to use symmetric replies. Instead, the stored values in conntrack is used to choose the destination. The ct_label.ecmp_reply_eth tells the destination MAC address to which the packet should be sent. The ct_mark.ecmp_reply_port tells the logical router port on which the packet should be sent. These values saved to the conntrack fields when the initial ingress traffic is received over the ECMP route and committed to conntrack. If REGBIT_KNOWN_ECMP_NH is set, the priority-10300 flows in this stage set the outport, while the eth.dst is set by flows at the ARP/ND Resolution stage.

This table contains the following logical flows:

ip.ttl--;
reg8[0..15] = 0;
reg0 = G;
reg1 = A;
eth.src = E;
outport = P;
flags.loopback = 1;
next;

(Ingress table 1 already verified that ip.ttl--; will not yield a TTL exceeded error.)

If the route has a gateway, G is the gateway IP address. Instead, if the route is from a configured static route, G is the next hop IP address. Else it is ip4.dst.

ip.ttl--;
reg8[0..15] = 0;
xxreg0 = G;
xxreg1 = A;
eth.src = E;
outport = inport;
flags.loopback = 1;
next;

(Ingress table 1 already verified that ip.ttl--; will not yield a TTL exceeded error.)

If the route has a gateway, G is the gateway IP address. Instead, if the route is from a configured static route, G is the next hop IP address. Else it is ip6.dst.

If the address A is in the link-local scope, the route will be limited to sending on the ingress port.

For each static route the reg7 == id && is prefixed in logical flow match portion. For routes with route_table value set a unique non-zero id is used. For routes within <main> route table (no route table set), this id value is 0.

For each connected route (route to the LRP’s subnet CIDR) the logical flow match portion has no reg7 == id && prefix to have route to LRP’s subnets in all routing tables.

For each IPv4/IPv6 ECMP group with group id GID and member ids MID1, MID2, ..., a logical flow with match in CIDR notation ip4.dst == N/M, or ip6.dst == N/M, whose priority is the integer value of M, has the following actions:

ip.ttl--;
flags.loopback = 1;
reg8[0..15] = GID;
select(reg8[16..31], MID1, MID2, ...);

Ingress Table 14: IP_ROUTING_ECMP

This table implements the second part of IP routing for ECMP routes following the previous table. If a packet matched a ECMP group in the previous table, this table matches the group id and member id stored from the previous table, setting reg0 (or xxreg0 for IPv6) to the next-hop IP address (leaving ip4.dst or ip6.dst, the packet’s final destination, unchanged) and advances to the next table for ARP resolution. It also sets reg1 (or xxreg1) to the IP address owned by the selected router port (ingress table ARP Request will generate an ARP request, if needed, with reg0 as the target protocol address and reg1 as the source protocol address).

This processing is skipped for reply traffic being sent out of an ECMP route if the route was configured to use symmetric replies.

This table contains the following logical flows:

[xx]reg0 = G;
[xx]reg1 = A;
eth.src = E;
outport = P;

Ingress Table 15: Router policies

This table adds flows for the logical router policies configured on the logical router. Please see the OVN_Northbound database Logical_Router_Policy table documentation in ovn-nb for supported actions.

reg8[0..15] = GID;
reg8[16..31] = select(1,..n);

where GID is the ECMP group id generated by ovn-northd for this policy and n is the number of nexthops. select action selects one of the nexthop member id, stores it in the register reg8[16..31] and advances the packet to the next stage.

[xx]reg0 = H;
eth.src = E;
outport = P;
reg8[0..15] = 0;
flags.loopback = 1;
next;

where H is the nexthop defined in the router policy, E is the ethernet address of the logical router port from which the nexthop is reachable and P is the logical router port from which the nexthop is reachable.

Ingress Table 16: ECMP handling for router policies

This table handles the ECMP for the router policies configured with multiple nexthops.

[xx]reg0 = H;
eth.src = E;
outport = P
"flags.loopback = 1; "
"next;"

where H is the nexthop defined in the router policy, E is the ethernet address of the logical router port from which the nexthop is reachable and P is the logical router port from which the nexthop is reachable.

Ingress Table 17: ARP/ND Resolution

Any packet that reaches this table is an IP packet whose next-hop IPv4 address is in reg0 or IPv6 address is in xxreg0. (ip4.dst or ip6.dst contains the final destination.) This table resolves the IP address in reg0 (or xxreg0) into an output port in outport and an Ethernet address in eth.dst, using the following flows:

For each IPv4 address A whose host is known to have Ethernet address E on router port P, a priority-100 flow with match outport === P && reg0 == A has actions eth.dst = E; next;.

For each virtual ip A configured on a logical port of type virtual and its virtual parent set in its corresponding Port_Binding record and the virtual parent with the Ethernet address E and the virtual ip is reachable via the router port P, a priority-100 flow with match outport === P && xxreg0/reg0 == A has actions eth.dst = E; next;.

For each virtual ip A configured on a logical port of type virtual and its virtual parent not set in its corresponding Port_Binding record and the virtual ip A is reachable via the router port P, a priority-100 flow with match outport === P && xxreg0/reg0 == A has actions eth.dst = 00:00:00:00:00:00; next;. This flow is added so that the ARP is always resolved for the virtual ip A by generating ARP request and not consulting the MAC_Binding table as it can have incorrect value for the virtual ip A.

For each IPv6 address A whose host is known to have Ethernet address E on router port P, a priority-100 flow with match outport === P && xxreg0 == A has actions eth.dst = E; next;.

For each logical router port with an IPv4 address A and a mac address of E that is reachable via a different logical router port P, a priority-100 flow with match outport === P && reg0 == A has actions eth.dst = E; next;.

For each logical router port with an IPv6 address A and a mac address of E that is reachable via a different logical router port P, a priority-100 flow with match outport === P && xxreg0 == A has actions eth.dst = E; next;.

For each row in the NAT table with IPv4 address A in the external_ip column of NAT table, below two flows are programmed:

A priority-100 flow with the match outport == P && reg0 == A has actions eth.dst = E; next;, where P is the distributed logical router port, E is the Ethernet address if set in the external_mac column of NAT table for of type dnat_and_snat, otherwise the Ethernet address of the distributed logical router port. Note that if the external_ip is not within a subnet on the owning logical router, then OVN will only create ARP resolution flows if the options:add_route is set to true. Otherwise, no ARP resolution flows will be added.

Corresponding to the above flow, a priority-150 flow with the match inport == P && outport == P && ip4.dst == A has actions drop; to exclude packets that have gone through DNAT/unSNAT stage but failed to convert the destination, to avoid loop.

For IPv6 NAT entries, same flows are added, but using the register xxreg0 and field ip6 for the match.

A priority-2 logical flow with match ip4.dst = {..} matches on traffic destined to router owned IPv4 addresses which are also SNAT IPs. This flow has action drop;.

A priority-2 logical flow with match ip6.dst = {..} matches on traffic destined to router owned IPv6 addresses which are also SNAT IPs. This flow has action drop;.

A priority-0 logical that flow matches all packets not already handled (match 1) and drops them (action drop;).

A priority-0 logical flow with match ip4 has actions get_arp(outport, reg0); next;.

A priority-0 logical flow with match ip6 has actions get_nd(outport, xxreg0); next;.

Ingress Table 18: Check packet length

For distributed logical routers or gateway routers with gateway port configured with options:gateway_mtu to a valid integer value, this table adds a priority-50 logical flow with the match outport == GW_PORT where GW_PORT is the gateway router port and applies the action check_pkt_larger and advances the packet to the next table.

REGBIT_PKT_LARGER = check_pkt_larger(L); next;

where L is the packet length to check for. If the packet is larger than L, it stores 1 in the register bit REGBIT_PKT_LARGER. The value of L is taken from options:gateway_mtu column of Logical_Router_Port row.

If the port is also configured with options:gateway_mtu_bypass then another flow is added, with priority-55, to bypass the check_pkt_larger flow.

This table adds one priority-0 fallback flow that matches all packets and advances to the next table.

Ingress Table 19: Handle larger packets

For distributed logical routers or gateway routers with gateway port configured with options:gateway_mtu to a valid integer value, this table adds the following priority-150 logical flow for each logical router port with the match inport == LRP && outport == GW_PORT && REGBIT_PKT_LARGER && !REGBIT_EGRESS_LOOPBACK, where LRP is the logical router port and GW_PORT is the gateway port and applies the following action for ipv4 and ipv6 respectively:

icmp4 {
icmp4.type = 3; /* Destination Unreachable. */
icmp4.code = 4; /* Frag Needed and DF was Set. */
icmp4.frag_mtu = M;
eth.dst = E;
ip4.dst = ip4.src;
ip4.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER = 0;
next(pipeline=ingress, table=0);
};
icmp6 {
icmp6.type = 2;
icmp6.code = 0;
icmp6.frag_mtu = M;
eth.dst = E;
ip6.dst = ip6.src;
ip6.src = I;
ip.ttl = 255;
REGBIT_EGRESS_LOOPBACK = 1;
REGBIT_PKT_LARGER = 0;
next(pipeline=ingress, table=0);
};

This table adds one priority-0 fallback flow that matches all packets and advances to the next table.

Ingress Table 20: Gateway Redirect

For distributed logical routers where one or more of the logical router ports specifies a gateway chassis, this table redirects certain packets to the distributed gateway port instances on the gateway chassises. This table has the following flows:

Ingress Table 21: ARP Request

In the common case where the Ethernet destination has been resolved, this table outputs the packet. Otherwise, it composes and sends an ARP or IPv6 Neighbor Solicitation request. It holds the following flows:

arp {
eth.dst = ff:ff:ff:ff:ff:ff;
arp.spa = reg1;
arp.tpa = reg0;
arp.op = 1; /* ARP request. */
output;
};

Unknown MAC address. For each IPv6 static route associated with the router with the nexthop IP: G, a priority-200 flow for IPv6 packets with match eth.dst == 00:00:00:00:00:00 && xxreg0 == G with the following actions is added:

nd_ns {
eth.dst = E;
ip6.dst = I
nd.target = G;
output;
};

Where E is the multicast mac derived from the Gateway IP, I is the solicited-node multicast address corresponding to the target address G.

Unknown MAC address. A priority-100 flow for IPv6 packets with match eth.dst == 00:00:00:00:00:00 has the following actions:

nd_ns {
nd.target = xxreg0;
output;
};

(Ingress table IP Routing initialized reg1 with the IP address owned by outport and (xx)reg0 with the next-hop IP address)

The IP packet that triggers the ARP/IPv6 NS request is dropped.

Egress Table 0: Check DNAT local

This table checks if the packet needs to be DNATed in the router ingress table lr_in_dnat after it is SNATed and looped back to the ingress pipeline. This check is done only for routers configured with distributed gateway ports and NAT entries. This check is done so that SNAT and DNAT is done in different zones instead of a common zone.

This table also installs a priority-50 logical flow for each logical router that has NATs configured on it. The flow has match ip && ct_label.natted == 1 and action REGBIT_DST_NAT_IP_LOCAL = 1; next;. This is intended to ensure that traffic that was DNATted locally will use a separate conntrack zone for SNAT if SNAT is required later in the egress pipeline. Note that this flow checks the value of ct_label.natted, which is set in the ingress pipeline. This means that ovn-northd assumes that this value is carried over from the ingress pipeline to the egress pipeline and is not altered or cleared. If conntrack label values are ever changed to be cleared between the ingress and egress pipelines, then the match conditions of this flow will be updated accordingly.

Egress Table 1: UNDNAT

This is for already established connections’ reverse traffic. i.e., DNAT has already been done in ingress pipeline and now the packet has entered the egress pipeline as part of a reply. This traffic is unDNATed here.

Egress Table 1: UNDNAT on Gateway Routers

Egress Table 1: UNDNAT on Distributed Routers

If the router is configured to force SNAT any load-balanced packets, above action will be replaced by flags.force_snat_for_lb = 1; ct_dnat;.

If the NAT rule cannot be handled in a distributed manner, then the priority-100 flow above is only programmed on the gateway chassis with the action ct_dnat_in_czone.

If the NAT rule can be handled in a distributed manner, then there is an additional action eth.src = EA;, where EA is the ethernet address associated with the IP address A in the NAT rule. This allows upstream MAC learning to point to the correct chassis.

Egress Table 2: Post UNDNAT

Egress Table 3: SNAT

Packets that are configured to be SNATed get their source IP address changed based on the configuration in the OVN Northbound database.

Egress Table 3: SNAT on Gateway Routers

with an action ct_snat(R); where R is the IP configured on the router port. If R is an IPv4 address then the match will also include ip4 and if it is an IPv6 address, then the match will also include ip6.

If the logical router port P is configured with multiple IPv4 and multiple IPv6 addresses, only the first IPv4 and first IPv6 address is considered.

Egress Table 3: SNAT on Distributed Routers

If the NAT rule cannot be handled in a distributed manner, then the below flows are only programmed on the gateway chassis increasing flow priority by 128 in order to be run first.

If the NAT rule can be handled in a distributed manner, then there is an additional action (for both the flows) eth.src = EA;, where EA is the ethernet address associated with the IP address A in the NAT rule. This allows upstream MAC learning to point to the correct chassis.

If the NAT rule has allowed_ext_ips configured, then there is an additional match ip4.dst == allowed_ext_ips . Similarly, for IPV6, match would be ip6.dst == allowed_ext_ips.

If the NAT rule has exempted_ext_ips set, then there is an additional flow configured at the priority P + 2 of corresponding NAT rule. The flow matches if destination ip is an exempted_ext_ip and the action is next; . This flow is used to bypass the ct_snat action for a flow which is destinted to exempted_ext_ips.

Egress Table 4: Egress Loopback

For distributed logical routers where one of the logical router ports specifies a gateway chassis.

While UNDNAT and SNAT processing have already occurred by this point, this traffic needs to be forced through egress loopback on this distributed gateway port instance, in order for UNSNAT and DNAT processing to be applied, and also for IP routing and ARP resolution after all of the NAT processing, so that the packet can be forwarded to the destination.

This table has the following flows:

clone {
ct_clear;
inport = outport;
outport = "";
flags = 0;
flags.loopback = 1;
flags.use_snat_zone = REGBIT_DST_NAT_IP_LOCAL;
reg0 = 0;
reg1 = 0;
...
reg9 = 0;
REGBIT_EGRESS_LOOPBACK = 1;
next(pipeline=ingress, table=0);
};

flags.loopback is set since in_port is unchanged and the packet may return back to that port after NAT processing. REGBIT_EGRESS_LOOPBACK is set to indicate that egress loopback has occurred, in order to skip the source IP address check against the router address.

Egress Table 5: Delivery

Packets that reach this table are ready for delivery. It contains:

DROP SAMPLING

As described in the previous section, there are several places where ovn-northd might decided to drop a packet by explicitly creating a Logical_Flow with the drop; action.

When debug drop-sampling has been cofigured in the OVN Northbound database, the ovn-northd will replace all the drop; actions with a sample(priority=65535, collector_set=id, obs_domain=obs_id, obs_point=@cookie) action, where: