Reason: The web service call is client certificate protected and although the certificate is installed on the test box, the web site, which runs as "App Pool Identity" did not have access to the certificate.
Hint: Use logging where possible and attempt other ways to access the same URL, for instance in a browser. This indicated that the logged-in user had no problems but the site did which made me realise that it was a permissions issue.
Fix: The certificate must be installed in the Local Machine store in mmc.exe. If you double-click it in Explorer, it will be installed in the Current User store, where you cannot manage permissions. In Windows 7 and Server 2012 (and probably 2003/8 as well):
- run up mmc.exe and choose File->Add/Remove Snap-in, select certificates on the left-hand list and when it asks you, choose Computer Account and Local Computer. Click OK to exit the selection page.
- Open up the Personal->Certificates folder under the snap-in you just enabled. It is possible, that the Certificates folder doesn't exist (if it is empty).
- Right-click in the contents pane of Personal or Certificates and choose All Tasks->Import
- Go through the wizard to import your certificate and select the option to "mark it exportable" which is usually needed for SSL usage (I think it includes the private key only when this is ticked).
- With the certificate imported, select it, choose All Tasks->Manage Private Keys
- You will get a familiar security dialog where you can add users who can access the private key. Add the account for the user that is running the application pool for your web site.
- If you are using "App Pool Identity", then the users are found with IIS Apppool\app.pool.name Note that if you are running some versions of Windows Server, you will need to change the "location" parameter to point to the local machine rather than the domain which is selected by default, otherwise the user won't be found.