Studying for my JNCIP-SP, I did a practice exam on the Juniper website and it seems that CoS is something I am weak in, so I am writing up some notes here for my own reference.

Overview

Packets that need different levels of treatment when passing through a Juniper device have to be classified as they enter the device, and queued for transmission before exiting the device. How packets are taken from the queues for transmission is called ‘scheduling’ – some queues are treated as higher priority than other queues so that more important packets such as voice or video get sent first.  Within that queue, other things can happen such as shaping the traffic down to a particular rate, re-marking packets, or dropping packets.  These things are done to proactively manage the queues in the device so that they don’t fill up. If they did, all packets would be dropped (tail-drop), so it is better to drop less critical packets early using Random Early Detection.

Classification of Packets

Packets can be classified in two ways:

1) By observing QoS markings on the packet (such as EXP bits, DSCP, IP ToS or 802.1P bits)

2) By observing other fields such as source/destination address, port numbers etc.

The first method is called Behaviour Aggregate classification. It implies that your whole network has been set up from end to end to classify (for example) voice packets using the DSCP bits.

The second method is called a Multi-Field Classifier, and is used when there are no markings on the packets that can be observed. The MFC might be used at the edge of the network to classify voice packets which are then marked with DSCP values.  Once marked, classification can be done through the core of the network using a BA classifier instead.

Packet Flow through Junos CoS

As a packet enters the router, the classifier assigns each packet a ‘forwarding class’ and a ‘loss priority’.  (Although I prefer to talk about a ‘loss probability’ because it makes sense to me – higher LP = higher chance of the packet being dropped).

Forwarding classes are assigned to queues.

Each queue has a scheduler applied to it.  The scheduler determines how packets are treated in the queue (i.e. dropped, re-marked, shaped) and how the packets are transmitted (i.e. which queue to serve first).

The packet is transmitted.

Classifiers

There is a default classifier applied to all interfaces on a Juniper router, which is based on IP Type of Service – it is called ‘ipprec-compatibility’ and you can see it below:

root@LAB_SRX2> show class-of-service classifier name ipprec-compatibility
Classifier: ipprec-compatibility, Code point type: inet-precedence, Index: 13
Code point         Forwarding class                    Loss priority
000                best-effort                         low
001                best-effort                         high
010                best-effort                         low
011                best-effort                         high
100                best-effort                         low
101                best-effort                         high
110                network-control                     low
111                network-control                     high

The code-points are 3-bit patterns in the left column, and these each map to a forwarding class and loss priority in the middle and right columns.  Out of the box, everything except network control traffic hits the best-effort class.

You don’t see any config statements associating the classifier with interfaces within Junos – that all happens under the hood.  However, when you associate a classifier with an interface yourself, the ‘ipprec-default’. You assign a classifier to an interface by typing “set class-of-service interface <name> unit <number> classifiers inet-precedence default”.

The default classifier for inet-precedence looks a bit different to the compatibility one.

Alternatively, you can make your own classifier – based on precedence or DSCP or whatever.  You can import the default one, and simply make the necessary changes you need – in the snippet below, I import the default classifier, change the expedited forwarding for packets marked with 111 and assign it to an interface:

root@LAB_SRX2# show class-of-service
classifiers {
  inet-precedence CLASS1 {
    import default;
    forwarding-class expedited-forwarding {
      loss-priority high code-points 111;
    }
  }
}
interfaces {
  ge-0/0/0 {
    unit 0 {
      classifiers {
        inet-precedence CLASS1;
      }
    }
  }
}

You can see that the new classifier has taken effect by issuing the operational command “show class-of-service interface”:

Physical interface: ge-0/0/0, Index: 134
Queues supported: 8, Queues in use: 4
  Scheduler map: , Index: 2
  Congestion-notification: Disabled

  Logical interface: ge-0/0/0.0, Index: 78
    Object                  Name                   Type                    Index
    Classifier              CLASS1                 ip                       4804

So that’s all good. Now I’m classifying, what to do next?

Schedulers

As I’ve said, a scheduler governs how the queue is treated.  There are four properties of a scheduler that you can define:

1) The bandwidth assigned to the queue
2) The buffer size for storing packets in the queue
3) The priority of the queue
4) The drop profile

By default, only two schedulers are used – queue 0 for best effort traffic (allocated 95% of interface bandwidth), and queue 3 for network control stuff (getting the remaining 5% bandwidth).

So here’s a scheduler and a drop profile:

drop-profiles {
    DP1 {
        fill-level 50 drop-probability 50;
        fill-level 80 drop-probability 80;
    }
}
schedulers {
    SCHED1 {
        transmit-rate percent 10;
        buffer-size percent 10;
        drop-profile-map loss-priority low protocol any drop-profile DP1;
    }
}

This sets up a drop profile that randomly discards 50% of packets when the queue is 50% full, and 80% when it is 80% full. The scheduler SCHED1 is created with 10% of line rate and 10% of the interface’s buffers.  You might be wondering how I came up with those numbers.  To be honest, I just made them up.  You won’t find a right answer for creating these values anywhere – you need to set something up that looks OK and then monitor the results closely in most situations.  If you get it wrong, by default a queue can exceed the defined bandwidth if there is room in other queues.  (Quite what happens if those queues then fill up, I don’t know…)

Now that the scheduler has been created, we need to associate it to a forwarding class – this is done through a scheduler map.  Using the examples above:  “set class-of-service scheduler-map MAP1 forwarding-class expedited-forwarding scheduler SCHED1″

Finally, assign the scheduler map to the interface using “set class-of-service interface ge-0/0/0 scheduler-map MAP1″.

Testing

To see if this works there are a couple of things you can do.  Junos allows you to set the TOS bits in ping packets, so you can do that to create the right markings and then see if the packets are hitting the queues correctly.

Pinging with the right TOS value requires a calculation.  If your classifier is looking for the pattern 101 (as per the example), you have to remember that the TOS field is eight bits in length, so there are five zeros in the remainder of the field (10100000).  The two 1s occupy the 128 and 32 bit positions – add these together and you get a value of 160.  So the command from the neighbouring device to ping the one you’ve configured your CoS on is:

ping tos 160 10.0.0.2

This creates the marked packets – now go to the device where you configured your CoS, and look at the interface statistics in detail. On the second page of the output you should see packets hitting the expedited-forwarding queue:

  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    0 best-effort                61155                61155                    0
    1 expedited-fo                  11                   11                    0
    2 assured-forw                   0                    0                    0
    3 network-cont               22795                22795                    0
  Queue number:         Mapped forwarding classes
    0                   best-effort
    1                   expedited-forwarding
    2                   assured-forwarding
    3                   network-control

So that’s some basic CoS configuration based on IP ToS. I’m off for a coffee.

Browse to the web interface of the switch, log in and click the “Troubleshoot” tab along the top:

Hit the “Troubleshoot port” button and a wizard will start.  Select the port you are interested in:

The tests run are a link test, a cable test (the TDR one we are talking about here) and a configuration test:

And here’s our result – cable pair is open at 17 metres from the switch:

Here are two sides of a call from the log file – we had to get the exact time of the call and the numbers being called from and to in order to determine the call GUID.  Once the GUID was obtained, we could use a find in Notepad to find the related messages:

G-MST: 4000001E "00040000-0fbf-4e89-e048-001049194446" ("192.168.3.2","192.168.0.144"),2(ULaw),rsn:1,09:23:05.602 (UTC),pl:20,(s:86, r:73, l:0),(j:2,u:0,o:0) flgs:0x00000000 "sip:TGrp_1,p30@192.168.3.2:5441",vpn:0
G-MST: 2000003E "00040000-0fbf-4e89-e048-001049194446" ("192.168.0.144","192.168.3.2"),2(ULaw),rsn:1,09:23:05.568 (UTC),pl:20,(s:73, r:86, l:0),(j:3,u:0,o:0) flgs:0x00000000 "sip:100@192.168.3.4:5441",vpn:0

In the above, the long number is the call GUID we were searching on, and the “sip:TGrp_1″ line (scroll to the right) is the incoming leg of the call, coming in off the ISDN and to the user’s softphone. You can see the IP address of the Shoregear E1K (192.168.3.2) and also of the user’s PC.

The second line with “sip:100″ is extension 100 leg of the call, out via the trunk.

The statistics are a bit cryptic for each of these lines, so let’s just look at one:

(s:86, r:73, l:0),(j:2,u:0,o:0)

This section of the logged message is saying that there were 86 packets sent, 73 packets received and 0 packets lost. The second set of figures in brackets shows values for jitter (in this case 2), underrun and overrun.

The values in the second section should all be zero in an ideal world. The j:2 part in this particular example indicates that the jitter buffer had to be adjusted twice during the call, and with a soft client this is not very well tolerated.  In some instances the jitter figure was up as high as 14, which the TAC engineer told me was a killer for softphones.  We’ve been finding that hard phones actually deal with this a lot better.

So, I guess we need to review the QoS on the network then…   

First you switch on the debugging, then wait until the problem recurs. When it does, note the exact time the problem happened and generate a zip file containing the logs for your support people to analyse.

The steps are described in detail below.

1. Open up Communicator and make sure your cursor is blinking in the “quickdial” bar at the top.  Then press CTRL-F12:

2. A diagnostic pane will appear.  Select the checkboxes as per the following two screenshots, and ensure that debug level is set to 7 in the second:

3. Apply the changes and wait until the problem happens again.  When it does, note the time as precisely as you can.

4. Click the Start button in Windows, then Shoretel | Support | Send Client Logs.

5. The following windows will appear enabling you to collect the logs, zip them up and send them on an email to your support representative:

There are actually two events, both ID 246, but subtly different:

The Telephony Management Server has logged an informational debug message. The debug message contents are:
.\NccMsgZin.cpp(1470) : Assertion failed: !”SIPReg: Error sending SIP Register”

The Telephony Management Server has logged an informational debug message. The debug message contents are:
.\NccMsgZin.cpp(1460) : Assertion failed: !”SIPReg: Error sending SIP Register”

I managed to repoduce this in two ways. We have Shoretel 12 CR3 (build 17.10.1730.0) here and a Shoretel Mobility Router running 4.5.  What was causing these messages to appear every two minutes was a user configured on the SMR that didn’t exist in the Director user database.  So the SMR must have been trying to register on the user’s behalf, but there was no such account.  This was happening every two minutes.  Disable the user, and the error messages stop appearing.

I also noticed this when configuring a Pancode SIP doorphone onto the system.  I hadn’t set the user’s SIP password in Director correctly – as a result, I was getting about 10 of these messages every minute as the doorphone tried to register with the SIP proxy.  Once I got the password set correctly, all the errors stopped occurring.

It does seem a shame that this is not being reported in more plain english – I may well log it as an improvement request on the Shoretel suggestions site!

The system booted up but said “waiting for network” or something like that in the Ubuntu splash screen.  It then said “Waiting another 60 seconds for network” and never came to a login prompt.  If you press escape at that point, you can see that there’s some kind of DBUS problem.

After some experimentation and finally this thread, I did the following:

Press CTRL-ALT-F1 to get a login prompt.

Log in as myself.

sudo mv /var/run /run
sudo mv /var/lock /run/lock
sudo rmdir /var/run
sudo rmdir /var/lock
sudo ln -s /run /var/run
sudo ln -s /run/lock /var/lock
rm /run/dbus/*

reboot

What a pain in the backside.  I won’t be upgrading too quickly in the future.  I think Ubuntu has some way to go if a distribution upgrade can cause such a headache for the average user.

-rw-r--r-- 1 andrew andrew 210M 2011-02-28 16:01 junos-srxsme-10.4R2.7-domestic.tgz
-rw-r--r-- 1 andrew andrew 211M 2011-05-06 11:02 junos-srxsme-10.4R3.4-domestic.tgz
-rw-r--r-- 1 andrew andrew 211M 2011-09-01 10:27 junos-srxsme-10.4R6.5-domestic.tgz
-rw-r--r-- 1 andrew andrew 137M 2011-05-05 09:55 junos-srxsme-11.1R1.10-domestic.tgz
-rw-r--r-- 1 andrew andrew 137M 2011-06-06 17:00 junos-srxsme-11.1R2.3-domestic.tgz

The command I issued was “dd if=/dev/sda of=/media/Iomega\ HDD/laptop-image-30th-july-2011.img” – which by default will take 512Kb blocks at a time and write them to the image file.

Instead, I stuck the options “bs=100M conv=notrunc” on the end of it and the process has speeded up massively.  This tells dd to copy 100MB blocks instead.  After about 20 minutes, I’m already 10% of the way through making the image.

Another helpful thing I discovered this morning is the -h switch on the ls command. -h makes the output ‘human-readable’ – i.e. it uses K, M and G to indicate kilobytes, megabytes and gigabytes. See below:

ubuntu@ubuntu:/media/Iomega HDD$ ls -lh
total 29G
drwx------ 1 ubuntu ubuntu    0 2011-03-26 21:03 Old drive images
drwx------ 1 ubuntu ubuntu 4.0K 2011-02-20 15:15 Study resources
-rw------- 1 ubuntu ubuntu  29G 2011-07-31 11:37 laptop-image-30th-jul-2011.img
drwx------ 1 ubuntu ubuntu    0 2008-06-11 14:19 System Volume Information
ubuntu@ubuntu:/media/Iomega HDD$ 

All this is probably no news to some people, but I’ve been using Linux for about 10 years and didn’t know all this stuff for some reason.  You learn something new every day…

There are a couple of things to watch out for though.  Although the wizard is very good, it doesn’t do a couple of important things for you that are going to stop things working.  (This is based on the assumption that your SRX has a fairly default config, and your VPN clients are coming in on the ‘untrust’ zone):

• You will need to enable the HTTP and HTTPS on the untrust interface (e.g. ‘set sys services web-management http interface ge-0/0/0′ and the same again for https)
• Be sure to permit HTTP and HTTPS inbound on the untrust zone (e.g. ‘set security zone security-zone untrust host-inbound-traffic system-services http’ and the same again for https)

The resulting config from the wizard looks like this (ge-0/0/14 is my outside/untrust interface in this instance):

[edit]
root# show security
ike {
    policy ike_pol_wizard_dyn_vpn {
        mode aggressive;        # Must be aggressive, not main mode
        proposal-set compatible;
        pre-shared-key ascii-text "$9$RkbElM7Nb2oGVw.P"; ## SECRET-DATA
    }
    gateway gw_wizard_dyn_vpn {
        ike-policy ike_pol_wizard_dyn_vpn;
        dynamic {
            hostname NoName;
            connections-limit 50;
            ike-user-type group-ike-id;
        }
        external-interface ge-0/0/14.0;
        xauth access-profile remote_access_profile;
    }
}
ipsec {
    policy ipsec_pol_wizard_dyn_vpn {
        perfect-forward-secrecy {
            keys group2;
        }
        proposal-set compatible;
    }                                   
    vpn wizard_dyn_vpn {
        ike {
            gateway gw_wizard_dyn_vpn;
            ipsec-policy ipsec_pol_wizard_dyn_vpn;
        }
    }
}
zones {
    security-zone trust {
        host-inbound-traffic {
            system-services {
                all;
            }
            protocols {
                all;
            }
        }
        interfaces {
            vlan.0;
            ge-0/0/0.0;
        }
    }
    security-zone untrust {
        screen untrust-screen;
        host-inbound-traffic {
            system-services {
                ike;
                http;
                https;
            }                           
        }
        interfaces {
            ge-0/0/14.0;
        }
    }
}
policies {
    from-zone trust to-zone untrust {
        policy trust-to-untrust {
            match {
                source-address any;
                destination-address any;
                application any;
            }
            then {
                permit;
            }
        }
    }
    from-zone untrust to-zone trust {
        policy policy_in_wizard_dyn_vpn {
            match {
                source-address any;     
                destination-address any;
                application any;
            }
            then {
                permit {
                    tunnel {
                        ipsec-vpn wizard_dyn_vpn;
                    }
                }
            }
        }
    }
}
dynamic-vpn {
    access-profile remote_access_profile;
    clients {
        wizard-dyn-group {
            remote-protected-resources {
                192.168.5.0/24;
            }
            ipsec-vpn wizard_dyn_vpn;
            user {
                andy;                   
            }
        }
    }
}

And the other bit the wizard didn’t do:

root# show system services
ssh;
telnet;
xnm-clear-text;
web-management {
    http {
        interface [ vlan.0 ge-0/0/0.0 ge-0/0/14.0 ];
    }
    https {
        system-generated-certificate;
        interface [ vlan.0 ge-0/0/14.0 ];
    }
}