Cisco IPSec Mutual Group Authentication with Apple racoon

In OS X, all certificates and passwords are stored in the Keychain. If your VPN works in Apple iPhone, it might not work in OS X, because the VPN client racoon is not linked to Apple Keychain correctly. If you look up the source code of the IPsec-Tools, you see the reason: This combination was not tested; there are two little typos[1][2] and OpenSSL is used instead.

1. Configure the Service

Click on your university and the installation starts, tested with OS X v10.7.5 and v10.8.4. v10.6.8 requires the settings to be done manually.
If you have iPhone, iPod touch, or iPad, visit this page and touch the name of your university; tested with iOS 3.1.3, 4.2.1, 5.1.1, 6.1.3, and 7.0.

RWTH  Aachen (MoPS)  moved to eduroam
Uni  Augsburg  does not work in v10.6 (certificate-order wrong); instead of non-hybrid I would use AnyConnect
FU  Berlin  as future-proof alternative consider AnyConnect
HTW  Berlin  as future-proof alternative consider AnyConnect
Uni  Chicago  moved to AnyConnect
Duke University  moved to AnyConnect
Uni  Erlangen  as future-proof alternative consider AnyConnect
Uni  Hannover  as future-proof alternative consider AnyConnect
Uni  Hannover (UHWLAN)  as future-proof alternative consider eduroam: OS X or iOS
TU  Harburg  as future-proof alternative consider AnyConnect
Uni  Heidelberg  does not work since iOS 4.4 (certificate is not SHA); instead of non-hybrid I would use AnyConnect
Uni  Heidelberg (MedMA)  does not work since iOS 4.4 (certificate is not SHA) nor in v10.6 (certificate-order wrong)
Uni  Jena 
TU  Kaiserslautern  moved to AnyConnect
Uni  Karlsruhe  changed to Juniper Junos Pulse (SSL)
Uni  Konstanz 
Uni  Mannheim  does not work since iOS 4.4 (certificate is not SHA); instead of non-hybrid I would use AnyConnect
Uni  Mannheim (Doniluma)  does not work since iOS 4.4 (certificate is not SHA); instead of non-hybrid I would use Adonis
North Carolina State University  moved to AnyConnect
Northern British Columbia  moved to AnyConnect, Microsoft RDP, and WPA-Enterprise
Monash University  moved to AnyConnect
Uni  Regensburg  changed to Juniper Junos Pulse (SSL)
Uni  South Florida  changed to Juniper Junos Pulse (SSL)
Uni  Stuttgart  as future-proof alternative consider AnyConnect
Uni  Ulm  as future-proof alternative consider AnyConnect
more on request …

2. Download the Patch (optional)

Since OS X v10.8 (Mountain Lion), you do not need this patch, because Apple has fixed those typos. However, they are neither fixed in v10.6.8 (Snow Leopard) nor in v10.7.5 (Lion). Only use this patch if you append [hybrid] to your group name. If you have a .pcf file, this is AuthType=5. This authentication type is named like IPSec Hybrid RSA (Android 4), Auth type Hybrid (Samsung Bada 2), Hybrid-Legitimierung (Ubuntu 11.04), HybridInitRSA (Wireshark), Hybrid Authentication (IETF), Hybrid GRP + XAuth (Shrew) and Mutual Group Authentication (Cisco). Some abbreviate mutual group-authentication with MGA. That term is used in this webpage. Other authentication types (for phase 1) are not discussed on this page. In the second phase, extended authenication (XAUTH; user@realm plus password) takes place. This webpage is not about problems of phase 2, just about phase 1.[3]

Common Error Messages

If your error message is not mentioned, contact me. If the reason does not apply in your case, contact me. I will have a look and append the cause to this list. In the download, there is a script to enable logging. After that, go to Applications → Utilities → Terminal → enter tail -f /var/log/debug.log

Certificate on your Computer

English:  Could not validate the server certificate. Verify your settings and try reconnecting.
German:  Das Serverzertifikat konnte nicht überprüft werden. Überprüfen Sie die Einstellungen und versuchen Sie erneut, eine Verbindung herzustellen.
 
Console Log:  racoon […] ERROR: unable to get local issuer certificate(20) at depth:0 SubjectName: […]
racoon […] ERROR: the peer's certificate is not verified.
configd […] IPSec Controller: connection failed <IKE Error 22 (0x16) Invalid cert authority>
Solution:  If you are not in OS X v10.8: The certificate was not found on your computer. Try my patch or upgrade to OS X v10.8.
 
Console Log:  racoon […] ERROR: Error evaluating certificate.
racoon […] DEBUG: eval result = kSecTrustResultRecoverableTrustFailure.
racoon […] ERROR: the peer's certificate is not verified.
Solution:  If you are in OS X v10.8: Do not use my patch, but go to Applications → Utilities → Keychain Access → Keychains = ‘login’ (German: Anmeldung) and Category = ‘certificates’ → open your certificate → expand the triangle ‘Trust’ and set drop-down menu at ‘IPSec’ to ‘Always Trust’ → then drag and drop your certificate to the category ‘System’.

Pre-Shared Key

Indication:  The button changes between Connect and Disconnect very fast.
Console Log: 
Solution:  Your pre-shared key (PSK) was not entered correctly into the ‘shared secret’ field: System Preferences → Network → your VPN (icon: padlock) → Authentication Settings…
That input field in OS X allows invisible characters. If you have copied and pasted the server address from somewhere else, those invisibles might create trouble. Delete the whole key and rather type than copy it.
English:  No VPN shared secret was provided. Verify your settings and try reconnecting.
German:  Es wurde kein VPN-Schlüssel (Shared Secret) angegeben. Überprüfen Sie die Einstellungen und versuchen Sie erneut, eine Verbindung herzustellen.
Console Log: 
Solution:  Applications → Utilities → Keychain Access → Keychains = ‘login’ (German: Anmeldung) and Category = ‘Passwords’ → in the upper right corner, search for: IPSec → double click the shared secret → enable ‘Show password’ → delete everything → hit ‘Save Changes’.

Server Address

English:  The VPN server did not respond. Verify the server address and try reconnecting.
German:  Der VPN-Server antwortet nicht. Überprüfen Sie die Serveradresse und versuchen Sie erneut, eine Verbindung herzustellen.
Console Log: 
Solution A:  Your server address might have changed. That input field in OS X allows invisible characters. If you have copied and pasted the server address from somewhere else, those invisibles might create trouble. Delete the whole address and rather type than copy it.
Solution B:  Your VPN does not use MGA. Tried without [hybrid] in the group name.
Solution C:  Your group was not entered into the ‘group name’ field correctly: System Preferences → Network → your VPN (icon: padlock) → Authentication Settings…
That input field in OS X allows invisible characters. If you have copied and pasted the name from somewhere else, those invisibles might create trouble. Delete the whole address and rather type than copy it.
Solution D:  If you got a personal certificate = a certificate authenticating you (not the remote server), then you have to click on ‘certificate’ rather than on ‘shared secret’.

Configuration File on your Computer

English:  A configuration error occurred. Verify your settings and try reconnecting.
German:  Ein Konfigurationsfehler ist aufgetreten. Überprüfen Sie die Einstellungen und versuchen Sie erneut, eine Verbindung herzustellen.
Console Log:  racoon […] Configuration Parse Error. (cfparse: yyparse erred, filename /etc/racoon/racoon.conf). (failure: fatal parse failure)
com.apple.launchd[1]: (com.apple.racoon[…]) Exited with code: 1
com.apple.launchd[1]: (com.apple.racoon) Throttling respawn: Will start in 10 seconds
Solution:  One of the configuration files got corrupted. This should not happen at all. You have to contact me.

Load Balancing on Server

English:  VPN Connection The negotiation with the VPN server failed. Verify the server address and try reconnecting.
German:  Die Kommunikation mit dem VPN-Server ist fehlgeschlagen. Überprüfen Sie die Serveradresse und versuchen Sie erneut, eine Verbindung herzustellen.
Console Log:  unusal IKE error, I try to reproduce this
Solution:  Apple racoon of OS X v10.6.8 does not like Cisco Load Balancing. Have a look at Applications → Utilities → Keychain Access → System → Certificates. The certificate there might give you more specific server addresses like asa1.rz.exmaple.edu or vpn1.example.edu. Try that as server address instead. If that does not help, please, contact me and I will have a look at it.

Certificate Chain on Server

English:  Could not validate the server certificate. Verify your settings and try reconnecting.
German:  Das Serverzertifikat konnte nicht überprüft werden. Überprüfen Sie die Einstellungen und versuchen Sie erneut, eine Verbindung herzustellen.
Console Log:  racoon […] ERROR: unable to get local issuer certificate(20) at depth:0
racoon […] ERROR: the peer's certificate is not verified.
configd[…] IPSec Controller: connection failed <IKE Error 50008 (0xc358) Server certificate subjectName invalid>
Solution:  Only if you are in OS X v10.6.8 and only if you face IKE error 50008 (0xc358): Your server sends several certificates, it sends its certificates as a chain. That chain does not start but ends with the identity certificate. This happens with all Cisco 3000 Series Concentrators as server. Ask your adminstrator not to send the whole chain but only the identity certificate; that is just a radio button to switch. If your adminstrator does not dare this, ask him to create a new group (name) with this switched off. Alternatively, upgrade to OS X v10.7 or newer because it copes with a wrong order of the certificate-chain.

Identity Certificate on Server

English:  The server certificate's identity is incorrect. Contact your network administrator.
German:  Die Identität des Serverzertifikats ist inkorrekt. Wenden Sie sich an Ihren Netzwerkadministrator.
 
Console Log:  racoon […] ERROR: failed to get subjectAltName
configd […] IPSec Controller: connection failed <IKE Error 50009 (0xc359) Server certificate subjectAltName invalid>
Solution:  Since OS X v10.8.5, a subject alternative name (SAN) is required in the certificate of your VPN server. Contact the administrator of your server and request him to add a SAN. Currently, only University of Erlangen and University of Stuttgart offer a SAN. As alternative consider AnyConnect.
 
Console Log:  racoon […] ERROR: ID mismatched with subjectName.
configd […] IPSec Controller: connection failed <IKE Error 18 (0x12) Invalid id information>
Solution:  This is a common bug if you deal with personal certificates which is another authentication type (AuthType=3). This error is very uncommon to MGA. This error only can happen after several months of usage of my patch, because your server administrator changed the certificate. Simply, start my patch again and it will fix it again.

Support on Mobile Platforms

‘No’ is the same as: I could not find a way. If you know a solution, please, tell me and I update this page for everyone!

VPN MGA AnyConnect
Android 4 built-in requires TUN broken yes
Android 2.2 no IPSec no Juniper Junos Pulse
iOS 4.2 built-in built-in yes
Juniper Junos Pulse
iOS 2.0 built-in built-in no
Windows Phone 8 no no no
RIM BlackBerry OS 10 built-in no yes
RIM BlackBerry PlayBook built-in no Juniper Junos Pulse
Nokia Series 40 no no no
Nokia Symbian/S60 yes (since 3rd Edition) CRACK well (2.4)
Nokia Series 80 built-in CRACK no
HP webOS 2 built-in no built-in
Samsung Bada 2 built-in broken[4] no
Windows Mobile yes no well (2.5)

With SSL VPN (Juniper or Cisco), you gain Android and BlackBerry support. Without MGA, you loose the Apple iPhone (original), Apple iPod touch (1st Generation), and Mac OS X v10.3 and lower. Mac OS X v10.4 requires AnyConnect 2.3. What about a current Cisco ASA with AnyConnect Mobile, MGA (no groups), plus CRACK (for Symbian)?

References

  1. ^ File oakley.c Function oakley_validate_auth Case ISAKMP_CERT_X509SIGN Line 1760 is not VERIFICATION_MODULE_SEC_FRAMEWORK.
  2. ^ File oakley.c Function oakley_skeyid Line 3294 HAVE_KEYCHAIN is undefined.
  3. ^ To be more precise, MGA is the Cisco extension to Check Point Hybrid Authentication, limiting it to RSA and requiring (optionally) a PSK and (optionally) a group.
  4. ^ Entire certificate-chains are rejected. Even with that in mind, not one of the known services worked.

Version History

24. Sep. 2013  since OS X v10.8.5, a subject alternative name (SAN) is required
successfully tested with iOS 7.0, SAN not required
21. Feb. 2013  successfully tested with iOS 4.2.1 and iOS 5.1.1
added a few more universities, all used hybrid in the past
04. Dec. 2012  no certificate issues in OS X v10.8 anymore, because University of Konstanz notified me about changing the .mobileconfigs from user to device profiles: PayloadScope = System.
25. Nov. 2012  Alexander Traud