Continuing Education Program

I had the pleasure of re-certifying my CCIE this year using the Continuing Education Program. Cisco rewarded me by renewing my status for 2 more years and unlocking the 10 year badge.

10 years? That has to be a mistake right? It feels like I just passed the lab in San Jose a couple of years ago. What do you mean there isn't a CCIE lab in San Jose anymore? Hmm, maybe it really was 10 year ago. Wow, time flies when you're doing what you love!

Marrying Cloud Video with On-Premises Calling 2/2

This blog is working under the assumption that the following on-premises pieces have already been built out and are working independently of the WebEx cloud piece:

• Cisco Unified Communications Manager solution on-premises
• On-premises PSTN connectivity via a Cisco Voice Gateway or CUBE

This blog also assumes that the WebEx piece has already been provisioned and that Video endpoints are already registered to the cloud.

Part 2 of this blog adds some details around the pieces that added to what’s listed above to pull this together. Here’s the high-level list of technologies mentioned again.

• WebEx Control Hub
• Cisco CUCM
• Cisco Expressways
• Firewalls
• NAT
• DNS
• Certificates
• Cisco Directory Connector software

Please read all of the Cisco guides thoroughly before attempting design or installation. With that said here’s a little bit of information about each technology with my tech notes.

The curve balls I ran into in the process are in orange text. 

WebEx Control Hub

The WebEx Control Hub is the partner/customer configuration touchpoint for the WebEx cloud. Below is a high-level list of most of the things configuring in the WebEx Control Hub for Hybrid Calling.

• Created a custom SIP subdomain (for example joshualearn.com)

• “Verify” the subdomain via a DNS text record with a special hash value provided in the WebEx Control Hub
• Optionally “claim” the subdomain with a TAC case. (If the subdomain is claimed then any new WebEx Teams account with that email domain has to be created in the official Company organization)
• Call Connect
• It gets enabled pointed to the SRV record for the MTLS connection to the Expressway-E For example, _sips._tcp.sipmtls.joshualearn.com (It’s also possible to use an IP and port in the case of a quick POC)
• Upload the expressway certs to WebEx (if they’re self-signed)
• Active Hybrid Call Service
• Add a default SIP destination for Hybrid Call Connect Service
• Enable Hybrid Calling for the user in the WebEx cloud. (The directory URI of the Spark Remote Destination should match the user's mail id in CUCM which should also match the user-id in the WebEx control hub

Cisco CUCM

The Cisco CUCM should already be in place but there a few things that to be validated and or remediated to start and there a few things that need to be configured specifically to the integration build-out.

General Prerequisites

• The CUCM database replication needs to be built out using FQDNs. The Expressway will connect to the CUCMs using that same value and need a FQDN that can be resolved via DNS. This caught me off guard the first time and required additional planning to update the system to FQDN entries and to re-establish the database replication between CUCMs.

• Cluster Fully Qualified Domain Name in the Enterprise Parameters needs to be configured to something unique. For example cluster1.joshualearn.com

New build out for Hybrid Calling

• The CUCM needs a service account with AXL permissions (which is pretty standard for most applications)

• Cisco CUCM needs a SIP trunk built out to the Expressway C server
• Cisco CUCM needs SIP route patterns to send calls to WebEx.
• *.calls.webex.com

• *.rooms.webex.com

• Configure the Spark remote device
• Configure a Spark Remote Destination for the user

Cisco Expressway

The Cisco Expressways is a bit of Swiss Army Knife for Cisco when it comes to additional software required for Hybrid Calling. Most of the specific connector software runs on the Expressway-C Connector host. There is also a transit pair required to create the secure transport between CUCM on the WebEx cloud.

Cisco Expressway-C Services Host

• Cisco recommends that the Expressway-C Services Host is updated to the latest version of code. This will allow for interoperability with the WebEx Control Hub. This is also the reason why it’s helpful to have the Services host on its own dedicated Expressway. (Some customers have very small deployments and are tempted to use the same Expressway-C as the Connector host and the transit host. However, these can cause additional challenges with keeping up the latest Expressway code on the C and E. If the Expressway-C Connector host is separate than the transit pair for the Expressways can be updated on a different interval.

• The Management Connector is installed to establish connectivity to the WebEx Control Hub from the Expressway

• The Call Connector is installed to create the data hook between the CUCM and the WebEx control hub

• Linking the Call Connector to CUCM pulls the IP address or the FQDN of the CUCM. If the system was defined using a hostname in CUCM the connection will fail. This is an issue I ran once. For whatever reason, the folks that deployed the initial CUCM decided to use hostnames. The Expressway-C Connector host could not resolve the hostname and therefore could not connect the CUCM.

Cisco Expressway Traversal Pair

The traversal pair is basically a way for the SIP traffic to traverse from the Cisco CUCM on-premises environment to the WebEx Control Hub and cloud registered video endpoints. The expressway traversal hosts can be dedicated or shared with existing MRA and or B2B expressways. Refer to the Expressway guides for details on how to spin up and license them.

Cisco Expressway-C Traversal Host

• The Expressway-C host sits on the internal voice server network.

Cisco Expressway-E Traversal Host (with 2 NICs)

• The Expressway-E ideally sits in a firewall environment with a transit DMZ and a service DMZ. However, I have also seen it deployed with one NIC in a service DMZ and the other NIC on the inside network.

• Create an MTLS zone between the Expressway-E and the WebEx cloud.

• Follow the guide to configure zones and search rules on the Expressways.

• One caveat I ran into once was with using the automatic WebEx DNS zone. It ended up causing an error message to pop up. I don’t remember any more specifically what the message was. It may have been that is was looking for additional SIP ports to be available instead of just the MTLS SIP port 5062. In any case, I manually built this rule and I didn’t receive any errors.
• Another caveat I ran into was the tool to validate the connection to the WebEx control hub was bugged out and didn't support MTLS. That warning can be ignored. (At least until Cisco updates that portion of the code). Bug ID: 

Firewall

• Two firewall DMZs are recommended by Cisco however, some customers have opted to implement one and put the “transport DMZ NIC” on the inside network.

• Firewall policies
• Open up port 5062 on the firewall for traffic towards the Expressway-E (for the WebEx Control Hub)

NAT

• NAT the “Service DMZ” interface of the Expressway-E to a public IP address

DNS

• A record – create an A record for the Expressways. For example expe.us1.joshualearn.com.

• SRV record – create an MTLS SIP service record. For example _sips._tcp.sipmtls.joshualearn.com. 300 IN SRV 5 10 5062 expe.us1.joshualearn.com.

Certificates

• Internal certificates are allowable if the Expressway-E is only hosting the Hybrid Calling, however, the WebEx Control Hub needs to trust the issuing CA chain. If the Expressway-E is also hosting MRA and or B2B then a publicly signed certificate is required on the Expressway-E

Have you implemented this before, have thoughts about trying it out, or do you just like to leave comments? If so, please leave one below.

Marrying Cloud Video with On-Premises Calling 1/2

Cisco's current line of Video endpoints (Room Kit, Room Kit Plus, SX, DX, MX, and WebEx Board) can be registered to the WebEx Calling Cloud for signaling and SIP URI dialing. This allows a video endpoint to call another endpoint or call into a WebEx meeting using a URI.

(A URI looks like an email address. For example, I might set up a URI as dx80@joshualearn.com for folks to reach me on a Cisco DX-80. The URIs on WebEx are actually a bit longer. One might look more like dx80@joshualearn.rooms.webex.com on WebEx. Most of the time folks don't type out the URIs so that's not usually a concern.)

So what happens when you want to dial a PSTN telephone number?  How do we associate a PSTN telephone number to a video device?  Previously, the only option was to turn up a SIP trunk with a telco provider within the WebEx Control Hub.  Then one would order new DIDs, assign a DID, and assign an internal number to a video endpoint.   That process involves starting a service contract with a telco provider that has an established relationship with Cisco WebEx. Cisco has termed this type of company as a "Cisco Preferred Media Provider".

While that is still an option, some folks have asked how they can bring their own PSTN by integrating with their existing on-premises Cisco Collaboration investment.  Before there was not an option.  Now those folks are in luck.  Utilizing what's known as Cisco WebEx Hybrid Call Service the Cloud Video environment can be married to On-Premises Calling environment.

What's involved to make that work? I'll write a follow-up post with more detail.  In the meantime, here is a high-level list of what's under the hood.

• Cisco WebEx Cloud registered video endpoints
• Cisco Unified Communications Manager solution on-premises
• On-premises PSTN connectivity via a Cisco Voice Gateway or CUBE
• Cisco Expressways
• Cisco Expressway-C Services Host
• Cisco Expressway-C Traversal Host
• Cisco Expressway-E Traversal Host (with 2 NICs)
• Firewalls
• 2 firewalls are recommended, however, I have also seen it designed with 1.
• Firewall policies
• NAT
• DNS
• A records
• SRV records
• Certificates
• Internal certificates are allowable however the WebEx Control Hub needs to trust the issuing CA chain.

Part 2 of this blog series will include more of the detail on some of the specifics related to the "how-to" design and deploy the solution.

If you have tried this already, have thoughts about trying this, or just like to leave comments, please leave one below.

SIP Bindings on CME with an authenticatied SIP Trunk

Let's say there is a Communications Manager Express and a PSTN SIP trunk to the telco that requires authentication.  How does CME bind SIP messaging to the telco from a Northbound interface (E.G. GigibitEthernet 0/0/0) while binding it's local SIP traffic to a loopback address?

I'll start with the second part of that.  If I want all SIP traffic bound to an interface I bind it globally under the "voice service voip" portion of the configuration.  The section below shows how one might bind SIP traffic to a loopback interface.

!
voice service voip
 allow-connections sip to sip
 sip
  bind control source-interface Loopback0
  bind media source-interface Loopback0
!

The example above works well for local SIP traffic that should be bound to the loopback address.  However, the registration to the Telco Provider would likely fail assuming that they are expecting the IP address of the Northbound interface of the CUBE.  (e.g. GigabitEthernet 0/0/0).

It's seems to me that the tenant feature in CUBE is helpful for sourcing the registration message to the telco from an interface and in fact overrides the global SIP binding.  Here is an example of what a tenant configuration might look like with the traffic bound to Gi0/0/0.

!        
voice class tenant 1
  registrar 1 dns:example.telcosbc.com expires 3600
  credentials username 5551212 password 0 5551212 realm example.telcosbc.com
  timers buffer-invite 5000
  bind control source-interface GigabitEthernet0/0/0
  no pass-thru content custom-sdp
  no outbound-proxy
!

The example above calls out the interface to bind the registration messages,  the registrar destination, the credentials and the realm.  In order for this work in production I had to duplicate the registrar configuration and add an authentication statement (that matched the credentials in the tenant) under the sip-ua section.  The following is an example of what that sip-ua section might look like.

!
sip-ua
 authentication username 5551212 password 0 5551212 realm example.telcosbc.com retry invite 2
 retry bye 2
 retry cancel 2
 registrar 1 dns:example.telcosbc.com expires 3600
!

After entering that configuration we typically find that the "show sip register status" returns back a yes for the username.  In this case it would look something like.

cme-cube.example.com#show sip register status
--------------------- Registrar-Index  1 ---------------------

Line                             peer       expires(sec) reg survival P-Associ-URI
================================ ========== ============ === ======== ============
5551212                         -1         1663         yes normal

For whatever reason we have run into scenarios where we had to reboot the CME-CUBE before we received back a response from the telco SBC.

I won't go into the dial-peers in detail in this blog.  However, we did also have dial-peers with bindings on them.  Inbound and outbound calls use the bindings on the dial-peers as apposed to the global SIP binding or the tenant SIP binding.

(The following configuration example was from a Cisco ISR 4300 series ISR running Cisco IOS XE Software, Version 16.05.02)


Has anyone else tried this method or another method to bind the traffic to the telco SIP SBC from a specific interface?

B-channel selection for PRIs in a trunk group.

Why doesn't the ISDN B-channel selection configuration work on my Cisco Voice Gateway?

I had the PRIs in a trunk group and had the IOS configuration setup to perform channel selection on the serial interfaces corresponding with our PRIs.  However, the PRIs ignored the channel selection commands.  How strange, I would have thought that the serial interface would be the most specific configuration option to define the hunting scheme.  Well wouldn't you know, all of the hunt scheme magic is performed at the trunk group level.  (I didn't know)  The commands bellow illustrate how to configure the PRIs to hunt in a descending order which is typically my preference as carriers usually hunt in an ascending order. 

router(config-trunk-group)#trunk group PRI
router(config-trunk-group)#hunt-scheme ?
  sequential    The interface with highest preference is selected

router(config-trunk-group)#hunt-scheme sequential ?
  both  Select from all available timeslots

router(config-trunk-group)#hunt-scheme sequential both ?
  down  Timeslots are selected in the descending order 

router(config-trunk-group)#hunt-scheme sequential both down

Why can't I enter any voice commands?

So you RMA'd your Cisco 4300 or 4400 ISR Voice Gateway and now it won't take any voice commands.  Here's how to get the voice licensing back on the replacement VG so you can load up your voice config and get back in production.

Update the boot licensing level to uck9 and reboot.
 license boot level uck9

Accept the EULA for SRST 
 license accept end user agreement

Update the srst license to right to use licensing.
 license right-to-use move cme-srst

Update the uck9 license to right to use licensing.
 license right-to-use move uck9

That's it.  Now you can apply you existing configuration back to the RMA replacement voice gateway and get it back in business.