Jump to content

Featured Replies

Posted

Hi,

 

May I ask a question about an NTE_BAD_DATA error returned by CryptDecrypt()?

 

I am working on an application of SSL client/server connection using

Microsoft Certificates and Crypto API. I'm running Windows XP.

 

Currently the SSL connection between my client and server works, but when I

use IE 6.0 (SSL 3.0/TLS 1.0) to connect to the server, the server failed with

an NTE_BAD_DATA error at CryptDecrypt().

 

This is the step where the server tries to decrypt the pre-master secret

encrypted by IE. In my own client code, I use CryptEncrypt() to encrypt the

pre-master secret with the server's public key. Not sure if IE uses the same

functions..

 

Below I've included all the functions I called to decrypt the encrypted

message in the server side:

 

(1) CryptAcquireContext(&hProv, mcskp->szKeyContainerName,

mcskp->szProvName, PROV_RSA_FULL, 0)

(2) CryptGetUserKey (hProv, AT_KEYEXCHANGE, &hKey)

(3) CryptDecrypt(hKey, 0, TRUE, 0, (BYTE *)tempbuffer,

&tempbufferlength) <--- FALSE was returned here, and NTE_BAD_DATA

(80090005) was reported by GetLastError().

 

* I've tried four CSPs, but I got the same error in all of them:

"Microsoft Enhanced Cryptographic Provider v1.0", "Microsoft Strong

Cryptographic Provider", "Microsoft RSA Schannel Cryptographic

Provider", and "Microsoft Base Cryptographic Provider v1.0".

 

* The pre-master secret is supposed to be of size 48 bytes, so setting

the Final flag as TRUE should be correct.

 

Because NTE_BAD_DATA indicates that the data to be decrypted is invalid, I

logged the value in 'tempbuffer' as below. ('tempbufferlength'==128)

 

5F EB 8C 0B D7 C2 7C C0 94 8F F9 0F 85 33 30 86

73 B6 DA BA 11 E3 93 AE 3C 02 28 56 E5 06 B4 49

83 2F 44 1B C5 CF C0 21 95 BC 53 0B 03 1C 49 C9

49 0F A6 69 B8 E1 22 9F 04 A2 08 68 43 63 9D 69

53 57 79 C9 FE 5C 02 72 BB 4F 1B 7D EF EB 16 53

E3 62 1A C0 83 D4 71 C5 89 52 E9 1D FE 65 DB C4

CE 33 75 F1 D0 54 E0 87 59 14 43 E5 5E 1C C6 49

62 A7 D4 EC 67 D4 73 7C BB 5F F5 FB 08 EA AC 62

 

I intended to check why it's invalid using the server's private key, but I

don't know how to print the value of a HCRYPTKEY variable or how to learn it

from the certificate.

 

I created the user certificate and its trust point, a self-signed

certificate, by the following commands:

 

* makecert -pe -n "CN=Trusted" -r -b 08/01/2007 -e 12/31/2010 -sp

"Microsoft Strong Cryptographic Provider" -sky exchange -ss ca

* makecert -pe -is ca -in "Trusted" -n "CN=User" -b 08/01/2007 -e

12/31/2010 -sp "Microsoft Strong Cryptographic Provider" -sky exchange

-eku 1.3.6.1.5.5.7.3.1,1.3.6.1.5.5.7.3.2 -ss my

 

May someone kindly advise what could be the problem in my case? Your help

would be much appreciated.

 

Thanks,

Sean

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...