Posted August 17, 200717 yr Hi, May I ask a question about an NTE_BAD_DATA error returned by CryptDecrypt()? I am working on an application of SSL client/server connection using Microsoft Certificates and Crypto API. I'm running Windows XP. Currently the SSL connection between my client and server works, but when I use IE 6.0 (SSL 3.0/TLS 1.0) to connect to the server, the server failed with an NTE_BAD_DATA error at CryptDecrypt(). This is the step where the server tries to decrypt the pre-master secret encrypted by IE. In my own client code, I use CryptEncrypt() to encrypt the pre-master secret with the server's public key. Not sure if IE uses the same functions.. Below I've included all the functions I called to decrypt the encrypted message in the server side: (1) CryptAcquireContext(&hProv, mcskp->szKeyContainerName, mcskp->szProvName, PROV_RSA_FULL, 0) (2) CryptGetUserKey (hProv, AT_KEYEXCHANGE, &hKey) (3) CryptDecrypt(hKey, 0, TRUE, 0, (BYTE *)tempbuffer, &tempbufferlength) <--- FALSE was returned here, and NTE_BAD_DATA (80090005) was reported by GetLastError(). * I've tried four CSPs, but I got the same error in all of them: "Microsoft Enhanced Cryptographic Provider v1.0", "Microsoft Strong Cryptographic Provider", "Microsoft RSA Schannel Cryptographic Provider", and "Microsoft Base Cryptographic Provider v1.0". * The pre-master secret is supposed to be of size 48 bytes, so setting the Final flag as TRUE should be correct. Because NTE_BAD_DATA indicates that the data to be decrypted is invalid, I logged the value in 'tempbuffer' as below. ('tempbufferlength'==128) 5F EB 8C 0B D7 C2 7C C0 94 8F F9 0F 85 33 30 86 73 B6 DA BA 11 E3 93 AE 3C 02 28 56 E5 06 B4 49 83 2F 44 1B C5 CF C0 21 95 BC 53 0B 03 1C 49 C9 49 0F A6 69 B8 E1 22 9F 04 A2 08 68 43 63 9D 69 53 57 79 C9 FE 5C 02 72 BB 4F 1B 7D EF EB 16 53 E3 62 1A C0 83 D4 71 C5 89 52 E9 1D FE 65 DB C4 CE 33 75 F1 D0 54 E0 87 59 14 43 E5 5E 1C C6 49 62 A7 D4 EC 67 D4 73 7C BB 5F F5 FB 08 EA AC 62 I intended to check why it's invalid using the server's private key, but I don't know how to print the value of a HCRYPTKEY variable or how to learn it from the certificate. I created the user certificate and its trust point, a self-signed certificate, by the following commands: * makecert -pe -n "CN=Trusted" -r -b 08/01/2007 -e 12/31/2010 -sp "Microsoft Strong Cryptographic Provider" -sky exchange -ss ca * makecert -pe -is ca -in "Trusted" -n "CN=User" -b 08/01/2007 -e 12/31/2010 -sp "Microsoft Strong Cryptographic Provider" -sky exchange -eku 1.3.6.1.5.5.7.3.1,1.3.6.1.5.5.7.3.2 -ss my May someone kindly advise what could be the problem in my case? Your help would be much appreciated. Thanks, Sean
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.