Configuring the New Cisco ACE Card in Layer 3 Mode

I’ve recently been configuring up a pair of Cisco ACE (Application Control Engine) blades for a customer to install into a Cat 6509. These things are pretty new and constitute the latest generation of their content-switching products. They’re so new in fact that there doesn’t appear to be a sample configuration to be had anywhere on Cisco’s website.

If you want some basic product overview stuff, have a look at this page.

What I wanted to do was to configure basic layer-3 load-balancing, with a public Virtual IP address (VIP) and a pair of servers at the back-end. If you’ve not used a service module in a Catalyst 6500 before, it is a bit odd to get your head around.

Basically, you create a VLAN that runs between the MSFC and the service module, put an IP address on the MSFC (an SVI), and hand control of the VLAN to the ACE module. The MSFC part of the config looks like this:

vlan 10
vlan 11
vlan 12

interface Vlan10
    description **Client side**
    ip address 172.16.1.1 255.255.255.0

svclc module 2 vlan-group 10
svclc vlan-group 10  10-12

So we’re making a VLAN for the side of the box where the clients are and giving it an IP address. After that, we create a VLAN-group (10) for the ACE module in slot 2, and assign VLANs 10, 11 and 12 to it.

Next, we get a session going to the ACE module. (Default login is admin with a password of admin). The first thing we need to do is the IP addressing on the VLAN interfaces:

interface vlan 10
    description **Client side**
    ip address 172.16.1.2 255.255.255.0
    no shutdown
interface vlan 12
    description **SERVER VLAN**
    ip address 192.168.1.1 255.255.255.0
    no shutdown

At this point, you should be able to ping the MSFC from the ACE. If you have your servers up and running in VLAN 12, you should be able to ping them too. They should be configured with the 192.168.1.1 as their default gateway.

Next, create the “rservers” or real servers. This is where we define each individual server and bring it into service:

rserver host SERVER1
    description **SERVER1**
    ip address 192.168.1.10
    inservice
rserver host SERVER2
    description **SERVER2**
    ip address 192.168.1.11
    inservice

And put the rservers into a server-farm:

serverfarm host SFARM1
    rserver SERVER1
        inservice
    rserver SERVER2
        inservice

Now we need to write some policy to match on traffic to a VIP (in this case, 172.16.1.100), and balance it across the two servers in our farm. We’re just doing the default round-robin method here for the time being. Once we’ve got this working, then we can get a bit more clever.

If you have experience using Cisco’s Modular QoS CLI (MQC) you will find some of the statements below familiar, but with some extra bits added to support the load-balancing.

There is a four-step process going on here – matching traffic at layers 3/4, matching traffic at layer 7, performing an action and applying the policy to an interface:

1. In the class-map below, we’re looking to match any protocol that hits our VIP.

class-map match-all MATCH_VIP
    match virtual-address 172.16.1.100 any

2. In our layer 7 policy map (L7_POLICY) we’re not actually doing any L7 load-balancing at all – we’re just stating which serverfarm to use (SFARM1) in the default class (class-default).
Class-default is a catch-all and it’s configured action is performed on everything not already acted upon in some other class. In this case, since there are no other classes referenced, that means everything. When we get a bit more adventurous, this is where our layer 7 actions (cookies etc.) would be done.

policy-map type loadbalance first-match L7_POLICY
    class class-default
       serverfarm SFARM1

3. In the L4_POLICY, we are binding together the VIP and the L7 load-balancing policy (which tells us which serverfarm to use)

policy-map multi-match L4_POLICY
    class MATCH_VIP
        loadbalance vip inservice
        loadbalance policy L7_POLICY

4. And we apply the policy to the incoming VLAN interface:

interface vlan 55
    service-policy input L4_POLICY

Finally – the ACE denies all traffic to the VIP by default (a security measure), so we need to write a standard ACL to permit the traffic on the VLAN interface:

access-list ALL line 10 extended permit ip any any

interface vlan 55
    access-group input ALL

And you should be good to go. Yes – I know the ACL above is very liberal. Since we were testing in the lab, it wasn’t really all that important to be secure – and anyway, if you’re running a datacentre with web-servers in it, you wouldn’t protect them with ACLs would you?

Useful commands to check this out are:

show rservers – to see if the servers are up
show serverfarm – to see if they are in the farm and getting hits
ping – do I really need to explain?
show service-policy vlan55 – to see the policy as applied to the VLAN
show access-list ALL – to see if the ACL is getting hits