Managing Junos Commit Time

17 09 2015

I’ve been working with an ISP that is going to be using a large amount of configuration in the ‘groups’ section.  The statements there will be inherited into the main configuration using the ‘apply-groups’ statement.

This is a clever way of writing commands once and having them apply to multiple parts of the configuration.  At a basic level you could match on interfaces beginning with ‘ge-‘ or ‘xe-‘ and set an MTU on them all using one group statement. This MTU setting would not appear in the main configuration unless the configuration was displayed using “show | display inheritance”. There’s a nice explanation of how groups work over at this Packetpushers blog.

The downside is that if large amounts of configuration work is done in groups, applying the config can become slow during the ‘commit’ process.  

What happens under the hood when the user issues a commit in Junos?  You can see what happens if you issue a ‘commit | display detail’.  There is an example in this KB article.   As you can see there is a lot of parsing for commit-scripts, interface ranges and apply-groups at the start.  The config in these needs to be expanded and incorporated into the configuration at commit time.  A new config database is then created, and the various daemons are notified that they need to read in their config.  Finally the rollback configs are rotated, and of course if you have two REs in your router, this all has to happen again on the backup RE!

So when the groups section becomes large, this results in a lot of extra work for the mgd daemon to perform at commit time.

The ISP I’m working with has several config elements that I had not come across until now in their system stanza that aims to improve commit time:

system {
    commit {
        fast-synchronize;
        synchronize;
        persist-groups-inheritance;
    }
    configuration-database {
        extend-size;
    }
  }
}
fast-synchronize: causes the commits to happen on both REs in parallel, rather than doing the master first, then the backup. This may help a bit.  Arrived with Junos 12.2 it seems.
 
synchronize: this is a standard command to make sure both REs get the changes.  It isn’t relevant to improving commit time, but is necessary here.
persist-groups-inheritance:  According to this, the command increases performance during commits for groups that use wildcards.  For example matching on ge-* on a device that has a lot of gigabit ports.
configuration-database extend-size:  Valid since Junos 13.2, this command allows the config database to use more memory on an as-needed basis.  It might help in situations where processes (such as mgd which does the commit) temporarily have high memory usage requirements, e.g, parsing of a config with a lot of group inheritance during commit time. More info on this is available  here and here.  If you want to you can get clever and specify which process and how much memory as well if you want.
Caveat – I’ve not tested these myself yet, but thought it worth recording some little-known configuration knobs for future reference.   If I get a chance I will try enabling each in turn and and then record the time it takes for each commit.
Update on 2nd October 2015:
I’ve just had a chance to test the ‘fast synchronize’ command on an MX480 with two RE1800s installed.  The config isn’t particularly big, but the difference is significant.  Commit time without this command is approximately 15 second, and with this command it drops to 3 or 4 seconds.
Advertisements

Actions

Information

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: