Why's the mainline forwarded to a cell phone!?! (Cellphone Carrier plays error after 60 seconds)

Here's a "fun" one.

Symptom:
I received intermittent reports/questions about the main line, "being forwarded to a cell phone".  I know there's no reason why I would have forwarded the mainline to a cell phone.

Troubleshooting:
I checked Cisco Communications Manager and confirmed the main line wasn't forwarded to another number.  Then I began to call the mainline from a land line & a cell phone but could not to reproduce the reported problem.  During the tests when an operator wasn't available after 90 seconds my call was routed to the auto attendant as designed.  I checked the logs for the reported problem call and found a normal call clearing from the PSTN.  Sadly, I wasn't any closer to finding where the end user's disconnect was.

Luckily, one of the reporting persons then sent me a video of an example problem call to the main line on her mobile phone.  Her  mobile phone rang for right around 60 seconds and then played the following message.

"
Welcome to Verizon wireless.  The wireless customer you called is not available at this time  Please try your call again later.  Announcement 1 switch [redacted]
"

This helped explain the confusion around why users were reporting the main line was being forwarded to a cell phone.  The Verizon error states that the "wireless customer" is not available.

I validated that the reporting user was indeed using a Verizon mobile phone.  Armed with that information I repeated the test from a couple of other Verizon mobile phones and was able to consistently recreate the issue.

Issue:
Verizon Wireless pulled back the call at 60 seconds and played the following error message:

"
Welcome to Verizon wireless.  The wireless customer you called is not available at this time  Please try your call again later.  Announcement 1 switch [redacted]
"

Work Around:
Modified the timers on the Cisco Communications Manager.  Previously the system was configured to route calls to the auto attendant after 90 seconds.  I discussed the issue with the customer and we decided to change the value to 30 seconds.  Now, when an operator doesn't pick up the call within 30 seconds, the call is routed to the auto-attendant instead of being disconnected by our friends at Verizon.

Where's my Caller-ID Calling Name Man? (PRI Inbound Calling Name in subsequent FACILITY)

It's been a long minute since I've run into this particular scenario.  Unfortunately, I didn't have the solution in my notes so I reached out to my old pal google for some inspiration.

We just had a new telco company install a voice PRI with the intention of replacing our existing telco company's PRI at one of our remote sites.  After installing and testing the new voice PRI we realized that the inbound calling name was not displayed on the phones.

In this particular scenario the setup looked something like this:

PRI -----ISDN-----> Cisco ISR-G2 VG -----SIP-----> CUCM -----SCCP-----> Cisco IP Phone

We ran a "debug isdn q931" to troubleshoot the problem.  Upon inspection we found that the calling party name was not being sent in the initial ISDN setup message.  In fact, it indicated that the calling name would be send in a "subsequent FACILITY message".

First, the voice gateway won't process the calling name in a subsequent Facility messages by default. The fix for that is to apply the following to the d channel of the PRI:

interface Serial X/X/X:23
 isdn supp-service name calling

Secondly, there is a problem in this scenario with the voice gateway sending a SIP invite to Communications Manager right after it gets the ISDN setup message.  Since the initial ISDN setup message doesn't have the calling name the SIP invite will also be lacking the calling name. When we tested at this point the phones were showing "From pending" as the calling name. The work around for this issue is to introduce a delay before the voice gateway sends the SIP invite to CUCM. Each scenario may require a different amount of time. One can look at the debugs to find out how long it takes between the time the initial ISDN setup message is received and when the ISDN facility message with the name is received. In this particular case we tuned the delay to 500 milliseconds and it seems to work well. The command to delay the setup for 500 milliseconds is.

sip-ua
 timers buffer-invite 500

For more information, Cisco also has a bug id CSCup91440 that references using a buffer-invite on individual dial peers.

Below are some samples of what the debugs looked like on a typical inbound telco call and the telco that sends the calling name in a later message.

Standard Telco "Before Debug"
(The actual telephone numbers have been changed to protect the innocent.)

507963: Jan 22 09:53:13.269: ISDN Se0/2/0:23 Q931: RX <- SETUP pd = 8  callref = 0x14D4
        Bearer Capability i = 0x8090A2
                Standard = CCITT
                Transfer Capability = Speech 
                Transfer Mode = Circuit
                Transfer Rate = 64 kbit/s
        Channel ID i = 0xA98382
                Exclusive, Channel 2
        Facility i = 0x9F8B0100A11702010D020100800F4368696361676F202020202020494C
                Protocol Profile =  Networking Extensions
                0xA11702010D020100800F4368696361676F202020202020494C
                Component = Invoke component
                        Invoke Id = 13
                        Operation = CallingName
                                Name Presentation Allowed Extended
                                Name = Joshua Learn
        Display i = 'Joshua Learn'
        Calling Party Number i = 0x2180, '8005551212'
                Plan:ISDN, Type:National
        Called Party Number i = 0x80, '8885551212'
                Plan:ISDN, Type:National
507964: Jan 22 09:53:13.269: ISDN Se0/2/0:23 Q931: Received SETUP  callref = 0x94D4 callID = 0x13E4 switch = primary-ni interface = User 


Subsequent Facility Message Telco "After"
(The actual telephone numbers have been changed to protect the innocent.)

040306: Jan 22 07:49:27.801 PST: ISDN Se0/2/0:23 Q931: RX <- SETUP pd = 8  callref = 0x1A22
        Bearer Capability i = 0x8090A2
                Standard = CCITT
                Transfer Capability = Speech
                Transfer Mode = Circuit
                Transfer Rate = 64 kbit/s
        Channel ID i = 0xA98381
                Exclusive, Channel 1
        Facility i = 0x9F8B0100A10F02016A06072A8648CE1500040A0100
                Protocol Profile =  Networking Extensions
                0xA10F02016A06072A8648CE1500040A0100
                Component = Invoke component
                        Invoke Id = 106
                        Operation = InformationFollowing (calling_name)
                                Name information in subsequent FACILITY message
        Progress Ind i = 0x8281 - Call not end-to-end ISDN, may have in-band info
        Calling Party Number i = 0x2183, '8005551212'
                Plan:ISDN, Type:National
        Called Party Number i = 0xA1, '8885551212'
                Plan:ISDN, Type:National
040307: Jan 22 07:49:27.801 PST: ISDN Se0/2/0:23 Q931: Received SETUP  callref = 0x9A22 callID = 0x1D81 switch = primary-ni interface = User
040308: Jan 22 07:49:27.805 PST: ISDN Se0/2/0:23 Q931: TX -> CALL_PROC pd = 8  callref = 0x9A22
        Channel ID i = 0xA98381
                Exclusive, Channel 1
040309: Jan 22 07:49:27.848 PST: ISDN Se0/2/0:23 Q931: RX <- FACILITY pd = 8  callref = 0x1A22
        Facility i = 0x9F8B0100A11702016B020100800F332054524143452020202020202020
                Protocol Profile =  Networking Extensions
                0xA11702016B020100800F332054524143452020202020202020
                Component = Invoke component
                        Invoke Id = 107
                        Operation = CallingName
                                Name Presentation Allowed Extended
                                Name = Joshua Learn
                              

How do I enable the line selection features for the Cisco 8800 phones on Communications Manager Express (CME)?

Many of us who have deployed the Cisco 88XX phones have run across 2 features that are life savers when working with multi-line phones connected to CUCM. Those features are "Show ALL Calls On Primary Line" and "Revert All Calls". For those who haven't used those features before I recommend checking out Amy Engineer's blog on why they are helpful and how they work with CUCM.

https://amyengineer.com/tag/8841/

While these 88XX line selection options have existed in the full Unified Communications Manager offering for a while. These options are not officially available in CME at the time of this writing. Luckily, since the phones have the functionality in firmware, there is a bit of a dirty hack to invoke these features in CME.

The following configuration will enable the "Show ALL Calls On Primary Line" feature on ALL phones.

telephony-service
service phone allCallsOnPrimary 1

The following configuration will enabled the "Revert All Calls" feature on ALL phones.

telephony-service
service phone revertToAllCalls 1

After those features have been enabled the phone profiles (config files) need to be re-created.  The following commands can be used to recreate the profiles.

voice register global
 no create profile
 create profile

Then we can validate the configs or skip this part and just reboot the phones and test.  Here's a sample scenario to check the config file.  Assuming CME is writing the config files to a directory called "its" and there is a phone with a name of "SEP101010101010", the following cli commands would show if the new configuration options are written to the config file.

more bootflash:/its/SEP101010101010.cnf.xml | i allCallsOnPrimary
and
more bootflash:/its/SEP101010101010.cnf.xml | i revertToAllCalls

After validating the the new addition to the configuration the phones will need to be rebooted and then voilà the new features are active!