Junos: L2Circuit over GRE with fragmentation configuration

8 03 2011

A customer has asked me to work out how to get layer-2 traffic from a Juniper SRX, across the public internet and back into another SRX so that they can conserve IP address space. I worked the followign out in the lab, using a pair of SRX240s, and a pair of Cisco routers to simulate the public internet. Crucially, fragmentation of large packets is important, since MTU sizes across the internet could be variable. GRE tunnels don’t fragment by default, but this configuration permits that.

There might be scalability issues with the configuration as shown below, so it needs more work doing to it, but it at least proves that this is possible.

What we’re doing here is running MPLS over a GRE tunnel across the Internet, and putting a layer-2 circuit (Circuit Cross-Connect as Junos calls it) over that.

GRE does not fragment packets by default, but in this scenario, packet fragmentation is required. Originally the intention was to use IPSec to do this, but it seems that the tested Junos version (10.0R3.1), GRE is able to fragment packets.

An example configuration from Juniper’s Application Note entitled “Branch SRX Series and J Series  Selective Packet Services” was used to achieve this.  It should be noted that the IDP configuration is not included below – the solution was tested with and without the IDP, and it seemed to work in both situations.  Also in the Application Note, the “policy-options policy-statement export-ldp” is defined, but appears to be unused.

What appears key to this working is the use of a firewall filters that turn off flow-based processing of packets received in the master routing-instance. This is required so that MPLS can run. There is then a logical tunnel (lt) interface between it and a virtual router, which is operating in flow mode.  The GRE tunnel is built from an IP address on the logical tunnel.  Conceivably, an IPSec tunnel could be built from the flow-mode virtual router, or security policy could be applied without affecting the operation of the GRE tunnel.

Note that for the purposes of this lab test, no security measures are in place – all host-inbound-traffic is being permitted. As such, this is not a configuration recommended for use on the public internet – it is posted here to show a working example. Further testing is advisable.

The test-bed network diagram looks as follows:

Packets can be pinged between the client devices shown in green at MTUs higher than 1500 bytes (tested up to 9000 bytes) with no issues.  The only discernible difference is the round-trip time achieved – with a standard 100 byte ping, the average RTT is 2ms.  With 9000 byte pings, this increases to 20ms – presumably because of the fragmentation overhead involved.

SRX1 Configuration

This configuration has been colour-coded to indicate which interfaces/policies/protocols etc. apply to which virtual router

Red indicates the Flow-VR

Green indicates the packet-mode master routing instance.

## Last changed: 2011-02-17 22:46:57 UTC
version 10.0R3.10;
system {
    host-name SRX1;
# Standard, unrelated configuration removed #
interfaces {
    gr-0/0/0 {
        description "MPLS interface";
        unit 0 {
            clear-dont-fragment-bit;  
            tunnel {
                source 10.1.1.3;      #Source is lt-0/0/0.0 locally
                destination 10.1.1.4; #Dest is lt-0/0/0.0 on SRX2
                allow-fragmentation;  #Permit fragmentation on the tunnel
            }
            family inet {
                mtu 1500;
                filter {
                    input PACKETMODE; #Turns on packet mode
                }
            }
            family mpls {
                filter {
                    input PACKETMODEMPLS; #Packet mdoe for MPLS too
                }
            }
        }
    }
    lt-0/0/0 {
        unit 0 {
            description "Tunnel interface bound to packet-mode VR";
            encapsulation frame-relay;
            dlci 100;                 #DLCI must agree
            peer-unit 1;              #Unit ID of other i/f
            family inet {
                address 10.1.1.3/32;  #Address used to source tunnel
            }
        }
        unit 1 {
            description "Tunnel interface bound to flow-mode VR";
            encapsulation frame-relay;
            dlci 100;
            peer-unit 0;
            family inet;
        }
    }
    ge-0/0/14 {                #This interface faces client device
        vlan-tagging;          #Tagged here, but could be plain ethernet
        encapsulation vlan-ccc;
        unit 600 {
            description "Tagged interface for L2Circuit";
            encapsulation vlan-ccc;
            vlan-id 600;
            family ccc {
                filter {
                    input PACKETMODECCC;  #Packet mode for received L2
                }                         #packets
            }
        }
    }
    ge-0/0/15 {                 #Interface facing ‘the internet’
        description "outside interface";
        unit 0 {
            family inet {
                address 192.168.4.1/30;
            }
        }
    }
    lo0 {
        unit 0 {

            family inet {
                address 1.1.1.1/32;
            }
        }
    }
}
routing-options {
    static {
        route 0.0.0.0/32 {
            next-hop 192.168.4.2;
            install;
        }
        route 192.168.3.0/24 next-hop 192.168.4.2;
        route 192.168.2.0/24 next-hop 192.168.4.2;
        route 10.1.1.0/24 next-hop lt-0/0/0.0; #Tells the master instance
    }                                          #how to find the other end
}                                              #of tunnel: via lt-0/0/0.0
protocols {
    mpls {
        interface all;
    }
    ospf {
        area 0.0.0.0 {
            interface gr-0/0/0.0;
            interface lo0.0 {
                passive;
            }
        }
    }
    ldp {
        interface gr-0/0/0.0;
        interface lo0.0;
    }
    l2circuit {
        neighbor 2.2.2.2 {
            interface ge-0/0/14.600 {            #Source of circuit
                virtual-circuit-id 2;              #VCID must agree
                encapsulation-type ethernet-vlan;
            }
        }
    }
}
security {
    zones {
        security-zone untrust {
            host-inbound-traffic {
                system-services {
                    all;
                }
            }
            interfaces {
                ge-0/0/15.0;
            }
        }
        security-zone signalling {
            host-inbound-traffic {
                system-services {
                    all;
                }
                protocols {
                    all;
                }
            }
            interfaces {
                gr-0/0/0.0;
                lt-0/0/0.0;
                lo0.0;
                ge-0/0/14.600;
            }
        }
        security-zone trust {
            host-inbound-traffic {
                system-services {
                    all;
                }
            }
            interfaces {
                lt-0/0/0.1;
            }
        }
    }
    policies {
        from-zone trust to-zone untrust {
            policy p1 {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        from-zone untrust to-zone trust {
            policy p1 {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        default-policy {
            permit-all;
        }
        policy-rematch;
    }
}
firewall {                    #Three firewall filters to turn on packet
    family inet {             #mode for different families
        filter PACKETMODE {
            term t1 {
                then {
                    packet-mode;
                    accept;
                }
            }
        }
    }
    family mpls {
        filter PACKETMODEMPLS {
            term t1 {
                then {
                    packet-mode;
                    accept;
                }
            }
        }
    }
    family ccc {
        filter PACKETMODECCC {
            term t1 {
                then {
                    packet-mode;
                    accept;
                }
            }
        }
    }
}
routing-instances {     #Flow-mode VR definition
    FlowVR {
        instance-type virtual-router;
        interface lt-0/0/0.1;
        interface ge-0/0/15.0;
        routing-options {
            static {     #Static routes to find each end of tunnel
                route 10.1.1.4/32 next-hop 192.168.4.2;
                route 10.1.1.3/32 next-hop lt-0/0/0.1;
            }
        }
    }
}
[edit]
lab@SRX1#

 

The configuration for SRX2 is not shown here, but is essentially the same, with different IP addressing.

Advertisements

Actions

Information

12 responses

14 04 2011
Leigh

You see, that pesky flow mode is just a pain. I really do wish they would let you tuen it off and still have everything work (like IPsec !)

I have this working on some MX80s at the moment but there are some issues with reassembly. Juniper recommend you turn on tunnel keys it seems.

Some other vendors implement over-size-MTU tunnels using TCP so they see a stream they can easily reassemble without holding fragments in buffers. Of course this is really inefficient and subject to stalling when packets are dropped.

3 02 2012
DataPlumber

Hey – only just noticed your reply, Leigh! Hope things are going well…

15 02 2012
Dean

I assume I can substitute st interfaces instead of the lt interfaces to enable this over IPSec? Also can a hub and spoke be built using multipoint vpn?

20 04 2012
Per Westerlund (@PerWesterlund)

Nice writeup, I was just going to lab something similar up myself, although for a different reason: being able to use packet-mode routing with SRXs using IPsec and possibly having asymmetric routing. (MPLS is in the back of my mind as a future enhancement (path separation) for our setup).

I had just read the same paper as you, and had in the margin made notes:
– Why use IDP, doesn’t GRE fragment/reassemble in flow-mode?
Now with that verified, the next margin note is:
– What use is the Flow-VR in this setup, it is only connecting ge-3/0/1.0 with lt-0/0/0.1? (In your setup: ge-0/0/15.0 with lt-0/0/0.1)

By the way, the RTT values with large ping, were they the same w/wo IDP?
Was there a sharp increase when fragmentation kicked in?

20 10 2012
Mark G

Been tackling this problem as well recently, I tried your configs but had trouble getting them to work with 12.1.

1) GRE would NOT reassemble without using IDP rule.

2) I could not get this static route to work:
routing-options {
static {
route 0.0.0.0/32 {
next-hop 192.168.4.2;
install;

Instead I used “static route 0.0.0.0/0 next-hop lt-0/0/0.0” and that fixed the problem.

2 01 2013
DataPlumber

Interesting – not tried it with 12.1 myself – I’ll bear this in mind for when I need it! Thanks…

Andrew

11 02 2013
Justin

you sir are a legend…..i wasnt trying to achieve a GRE tunnel but just a plain l2circuit in selective packet mode and your guide helped me fix my issues in minutes

14 03 2013
DataPlumber

Nice of you to say – thanks!

7 02 2014
expo

how you are going to reach this 10.1.1.3 when you have a static def route to 192.168.4.2 and srx -2 far end would have no idea of the 10.1.1.3 ( unless you keep the static route in all the routers ///

ftp://80.252.223.236/upload/Bengan/BranchSRXForwardingArchitecture_v6a.pdf

and

25 09 2014
Patrik Olsson

And you have this workign with no issues? Normal symptom of MTU problem is lost sessions and lack of ability to transfer packets with a payload not being able to get through.

28 10 2014
DataPlumber

Well to be honest this was a few years ago now – I think there may have been an issue with MTU but nothing that couldn’t be sorted out. As it happened this was the last thing I did for a previous employer, so I don’t know if the customer went ahead.

But actually, OpenContrail works in a similar way – creating GRE tunnels with MPLS running over them. Quite cool.

1 11 2014
ix explorer

srx does fragment the packet mode , when you have the fe interface in the packet mode . keep the mtu for the IP packet more than 1500 and mpls mtu need to be set to default (1460) . MX fragment he packet in 13.3 onwards …

I am runniing mpls between srx 110 and mx5 / 80 on gre and does fragment the packets .

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: