Are BER encoded PEM private keys supported for reading by PemParser?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Are BER encoded PEM private keys supported for reading by PemParser?

Jesse Schulman
The private key shared below seems to be BER encoded but not DER encoded, and therefore I get an exception when calling PemParser.readObject against it.  I was able however to check it using openssl, which results in a valid DER encoded version being written out which does not give me an exception.

This private key was generated by a 3rd party tool, and we have only seen this issue once out of dozens of other keys it has generated, all other keys it has generated seem to be valid DER encoded.  I have reported this issue to the 3rd party tool, since it seems wrong and inconsistent.

Questions:
Is there any way to use PemParser or another BouncyCastle API that would handle keys encoded in this way, similar to how openssl was able to handle it?  If so, how?

If the answer to my first question is yes, is there a strong reason to not allow such BER encoded keys in our application?


Thanks!
Jesse

This private key is not being used anywhere and is now only being used for my current troubleshooting, so I am not concerned about distributing:

$cat pkcs1_private_key.pem
-----BEGIN RSA PRIVATE KEY-----
MIIJKQIBAAKCAgEAqKcgKIC0oYCywWIJI3VL56shKN5gXA6m4BhSQ5Q1PNDKmbS+
WhUDELRkDGSDzufNUIUvfPCk/5AY/jhFmIcCUQRXrVCOnDlhwGLhSFnqn5+c2T2F
IDrqlCslevAO4es0yCjwb6ACYvMTUTtXu8+cwi8F7keMYVVByaNVG6Zm/D6vp7fB
V42lPWaGVmCIgkhzps1m74cl7NtPZ6Rm25tQ8VLjGl2SP0+SRjcVfKHnF/1anskv
QiEE8Ii7sYe1BnlYhrCPFqSSuDf2e3o14Wtq52VKuWT6c9Hec99hKFY91uHjDIXb
FRSq+xRC7B/met5gF9MziK0wM7fA6v+ZpalOnbpBVL14kRbGV/DYk0sileh9kJO4
e80SqsI3lYShXwFWqx2rcyU2daqHrc+T4FsoIU0abtsWkJ+vOaZqZODLBA8oTLZc
wuql20If2AithxY6uaZjI0p0jb0S5iTczbJ9pUnyZ3PsgMbw2SNE3z80zjqW3Pq4
h7e+SjHfM+y6iR57DPkhf+UpT6S7hQzVAAOTvNE3JS+Z0CeYQ6DYj7KIEKQyQTs5
NC+Z2Vt9YvrqPqR++nma6P6fOX+Q1hL/kf2NYbovSDvNR8c/XAsjMlq3IZ/UJufd
ItvAD3snE4NPOv+/kszwuYb6azvYkilYSLXhDYG0n7j+fMcAe4MW3FwSdLcCAwEA
AQKCAgBbnESkm50CqVTwJJ+QKjPsCJH/kCPJfY/55FwzJztv6H5U1dAcQJOkrEg4
TxUl+9YT6fSsqk8GHzDkTo7HVnbUqUYy97IvfB/0EAINegIvaNUbUAaTRk2L5I/8
Yo/C+i92ln5cNA93zk/s7Q4wD4s4E7DmCOlhKZ2NuPIPDemdFTP3J6KXO3LJknpg
Sxl7o31m2uYSMFhSe6J7StswaQKjw4FnHfLF/yQWje9W/t73sgBahi/sfOEhAzTp
Ly0O8jU7HIbtJsIn3QY/dlf0NZQhU20unYJKDXMHgAxXi9kVcygQ8ljZ75scvZnI
h1972HVcG7Y/hMvNHABn5iWAziHMOYPArNCANkQXyTu56mnu2PDsGhhF22jLqiXk
LcJTQMh1kJ6gOyLmBZDEp+N+4YFyzU9zpfp42NzCOiBT8644U6ytPgzA7rzrC88O
bZxz+k7XWu/cRiZf5mSqhPuxM0MiYJNitxL4l3WJzg7fWXVKCM3Fr403PohMJQF1
QKtiH4SxO+iKC9sAGmhTrZtZPqP6nKNvGIMrcBQRCxpITHH1JBgXBAAYr2Rh89Aw
+gSewWM95VexStI91w8kofONlQxBlEf8yZYu6/8IWouxTn17Z3y7H2U+D1YpPwbS
MFANgPDUYrHFbH/L5ZWNhdFJcNjno8ilnVKBhwco10p71CbTwQKCAQEA7836W3ZI
pudWxzj7mc3B+u3K0kqyDRR2GlEPyGrCaTbY2wXlSzRBx4W2SBnRlC+5G4LobJig
/j6kYLwTmMyw+rDAncN/JgWGmYmgZoAWlSRpqMDrtUzT0AlXvgLjxlSTYrJ3aRVM
OeuMAB+XZvaplqW9MxtvoTH8UlwOcrVPuxRQqYIoftO9dtdfuWTQn7uGZqnbZnOv
rYGI2LkvTlXjIcZ+cNNBYgFjXxH8geSr1zktQydAwP6A55wotOU84zeI6o5Q3qwB
5yFiE8PJdT4tIkeA61VVGjsqRk4Kv7q1lISnOEkIu7qlUraXEWen9OEbfwuUw4uU
kFFMSnyeMuOoFwKCAQEAtAr+KJ1VH7y7GKQqQ1QV1EO7dv9CbZHemVlBgBOntgjd
TjOcQ7YY0NC/p+wl4fE01feBrNTqP8sKbNGW//5pKGEiAY7dyTNzOSEirkJ0hMcN
p0XYLxXro/TGNx1LSrcdBeSqKXo7hBwf1A8ZmlJiAnFpb9/WU4S6Gxn+nyEzEWMI
rWlvt7PXRtWcm2hejNGJjKUmjJb+ScDFrNnkY1h5VU8Ng4yybIGpidVt1cUONc+0
maSld+p72OR99IqtSqWmhraGaEMeSzHQgTpY6LPuUhIDIiF2WYovEr8fbCtSfaFM
l/Rl/wY2No6rBh1DDwHw5eLfJm89mv1W6dIICQTcYQKCAQEAgdEJ1PWFgwT9T0Aj
xDFE76hHAex50ubewIRdfOt+byLH5lmctUeGzJIwCXwgno3vMVt+oztE/B3BMrTe
DvvQWwXQYBdy/4xCP1/nu1Vf5EEgCcDWWQFMMPS90RkTYUxrtSRTzElBSKGg2ng/
p/ej+u+iKsaRebvrklJPZ/2LOgVXXUwey/moOWuCYISZA7+qlG0jXqUF13GmOVyW
pVuNZ71iuYVDgeCqgM7l2ROHiy1iuVXIkjG0aq14w+TNP1vS/NcqJEENjqfdxkFZ
D+Um3cXDnqJFPbwFuKWK4BwdLFdbhvbSiX4S/WHnCf8JR2GNVwtX7zviovW/zRsF
MRtY5QKCAQACfSxqT1J/79H+UzaqW505o/4RdRHsBdo38H3xUVQ+Nf0pOZltbjUF
nf0oSyFy8Cy00IkgZIkgfKQWQQd/XnsBreYUc28peuaa4ZRjKFQICeBscZC9heYO
DDI5TNRfogxqqnCXxjLQbrsZ6WOHOKBWPE8i1stVuD4CZeeZN5JHKUFTdGOw/HF7
bzSNCXJVLAhkw1u3EddOGbYh35lCrlRvE3qyWN2WlmxLlYHBNMovgEGU5ivLbphr
tSzwloIyx/t3XUqaYmMm0vd48d+MyAABbnJQpHnIXxqrfaznh+nKBtLSPvSyA9n4
AxzmJr1olbRI2UwQcfU+EUaEUPz/cuvBAoIBAQAOxrB6CWqcgPIYP/WwTNQu88Zd
Ua2wgyRD5Rko4gAuWMnaSDwFz5pYDJmvY8Ra1u7mE3bPG0R2CxkYtWh+7aRz5TYW
SlL2iIRlvE8mLGk+uxZObjofQ0Lx1bLNjDXF0lzqxA93ppRVWGyLD3SqXLXV8l3K
hRtZBRkgKKMUndPWzBCI4cl1k8AkdacC6Znd73NyJy3Em4jmzZ2oMbpKGfjAmmoO
CWlF2V+WID9W8efZGQqorMM2IJPa1cbcENrFt4e2v+MYp8ZtGqabhTZRjKoS/HE0
iJnhRPGQeDfMZ6AhrH/extMahmtEM2SAarjFlCDZMwOoYxoBvrxVBLee/0c+
-----END RSA PRIVATE KEY-----

$openssl rsa -in pkcs1_private_key.pem -check
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
MIIJKAIBAAKCAgEAqKcgKIC0oYCywWIJI3VL56shKN5gXA6m4BhSQ5Q1PNDKmbS+
WhUDELRkDGSDzufNUIUvfPCk/5AY/jhFmIcCUQRXrVCOnDlhwGLhSFnqn5+c2T2F
IDrqlCslevAO4es0yCjwb6ACYvMTUTtXu8+cwi8F7keMYVVByaNVG6Zm/D6vp7fB
V42lPWaGVmCIgkhzps1m74cl7NtPZ6Rm25tQ8VLjGl2SP0+SRjcVfKHnF/1anskv
QiEE8Ii7sYe1BnlYhrCPFqSSuDf2e3o14Wtq52VKuWT6c9Hec99hKFY91uHjDIXb
FRSq+xRC7B/met5gF9MziK0wM7fA6v+ZpalOnbpBVL14kRbGV/DYk0sileh9kJO4
e80SqsI3lYShXwFWqx2rcyU2daqHrc+T4FsoIU0abtsWkJ+vOaZqZODLBA8oTLZc
wuql20If2AithxY6uaZjI0p0jb0S5iTczbJ9pUnyZ3PsgMbw2SNE3z80zjqW3Pq4
h7e+SjHfM+y6iR57DPkhf+UpT6S7hQzVAAOTvNE3JS+Z0CeYQ6DYj7KIEKQyQTs5
NC+Z2Vt9YvrqPqR++nma6P6fOX+Q1hL/kf2NYbovSDvNR8c/XAsjMlq3IZ/UJufd
ItvAD3snE4NPOv+/kszwuYb6azvYkilYSLXhDYG0n7j+fMcAe4MW3FwSdLcCAwEA
AQKCAgBbnESkm50CqVTwJJ+QKjPsCJH/kCPJfY/55FwzJztv6H5U1dAcQJOkrEg4
TxUl+9YT6fSsqk8GHzDkTo7HVnbUqUYy97IvfB/0EAINegIvaNUbUAaTRk2L5I/8
Yo/C+i92ln5cNA93zk/s7Q4wD4s4E7DmCOlhKZ2NuPIPDemdFTP3J6KXO3LJknpg
Sxl7o31m2uYSMFhSe6J7StswaQKjw4FnHfLF/yQWje9W/t73sgBahi/sfOEhAzTp
Ly0O8jU7HIbtJsIn3QY/dlf0NZQhU20unYJKDXMHgAxXi9kVcygQ8ljZ75scvZnI
h1972HVcG7Y/hMvNHABn5iWAziHMOYPArNCANkQXyTu56mnu2PDsGhhF22jLqiXk
LcJTQMh1kJ6gOyLmBZDEp+N+4YFyzU9zpfp42NzCOiBT8644U6ytPgzA7rzrC88O
bZxz+k7XWu/cRiZf5mSqhPuxM0MiYJNitxL4l3WJzg7fWXVKCM3Fr403PohMJQF1
QKtiH4SxO+iKC9sAGmhTrZtZPqP6nKNvGIMrcBQRCxpITHH1JBgXBAAYr2Rh89Aw
+gSewWM95VexStI91w8kofONlQxBlEf8yZYu6/8IWouxTn17Z3y7H2U+D1YpPwbS
MFANgPDUYrHFbH/L5ZWNhdFJcNjno8ilnVKBhwco10p71CbTwQKCAQEA7836W3ZI
pudWxzj7mc3B+u3K0kqyDRR2GlEPyGrCaTbY2wXlSzRBx4W2SBnRlC+5G4LobJig
/j6kYLwTmMyw+rDAncN/JgWGmYmgZoAWlSRpqMDrtUzT0AlXvgLjxlSTYrJ3aRVM
OeuMAB+XZvaplqW9MxtvoTH8UlwOcrVPuxRQqYIoftO9dtdfuWTQn7uGZqnbZnOv
rYGI2LkvTlXjIcZ+cNNBYgFjXxH8geSr1zktQydAwP6A55wotOU84zeI6o5Q3qwB
5yFiE8PJdT4tIkeA61VVGjsqRk4Kv7q1lISnOEkIu7qlUraXEWen9OEbfwuUw4uU
kFFMSnyeMuOoFwKCAQEAtAr+KJ1VH7y7GKQqQ1QV1EO7dv9CbZHemVlBgBOntgjd
TjOcQ7YY0NC/p+wl4fE01feBrNTqP8sKbNGW//5pKGEiAY7dyTNzOSEirkJ0hMcN
p0XYLxXro/TGNx1LSrcdBeSqKXo7hBwf1A8ZmlJiAnFpb9/WU4S6Gxn+nyEzEWMI
rWlvt7PXRtWcm2hejNGJjKUmjJb+ScDFrNnkY1h5VU8Ng4yybIGpidVt1cUONc+0
maSld+p72OR99IqtSqWmhraGaEMeSzHQgTpY6LPuUhIDIiF2WYovEr8fbCtSfaFM
l/Rl/wY2No6rBh1DDwHw5eLfJm89mv1W6dIICQTcYQKCAQEAgdEJ1PWFgwT9T0Aj
xDFE76hHAex50ubewIRdfOt+byLH5lmctUeGzJIwCXwgno3vMVt+oztE/B3BMrTe
DvvQWwXQYBdy/4xCP1/nu1Vf5EEgCcDWWQFMMPS90RkTYUxrtSRTzElBSKGg2ng/
p/ej+u+iKsaRebvrklJPZ/2LOgVXXUwey/moOWuCYISZA7+qlG0jXqUF13GmOVyW
pVuNZ71iuYVDgeCqgM7l2ROHiy1iuVXIkjG0aq14w+TNP1vS/NcqJEENjqfdxkFZ
D+Um3cXDnqJFPbwFuKWK4BwdLFdbhvbSiX4S/WHnCf8JR2GNVwtX7zviovW/zRsF
MRtY5QKCAQACfSxqT1J/79H+UzaqW505o/4RdRHsBdo38H3xUVQ+Nf0pOZltbjUF
nf0oSyFy8Cy00IkgZIkgfKQWQQd/XnsBreYUc28peuaa4ZRjKFQICeBscZC9heYO
DDI5TNRfogxqqnCXxjLQbrsZ6WOHOKBWPE8i1stVuD4CZeeZN5JHKUFTdGOw/HF7
bzSNCXJVLAhkw1u3EddOGbYh35lCrlRvE3qyWN2WlmxLlYHBNMovgEGU5ivLbphr
tSzwloIyx/t3XUqaYmMm0vd48d+MyAABbnJQpHnIXxqrfaznh+nKBtLSPvSyA9n4
AxzmJr1olbRI2UwQcfU+EUaEUPz/cuvBAoIBAA7GsHoJapyA8hg/9bBM1C7zxl1R
rbCDJEPlGSjiAC5YydpIPAXPmlgMma9jxFrW7uYTds8bRHYLGRi1aH7tpHPlNhZK
UvaIhGW8TyYsaT67Fk5uOh9DQvHVss2MNcXSXOrED3emlFVYbIsPdKpctdXyXcqF
G1kFGSAooxSd09bMEIjhyXWTwCR1pwLpmd3vc3InLcSbiObNnagxukoZ+MCaag4J
aUXZX5YgP1bx59kZCqiswzYgk9rVxtwQ2sW3h7a/4xinxm0appuFNlGMqhL8cTSI
meFE8ZB4N8xnoCGsf97G0xqGa0QzZIBquMWUINkzA6hjGgG+vFUEt57/Rz4=
-----END RSA PRIVATE KEY-----

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Are BER encoded PEM private keys supported for reading by PemParser?

Peter Dettman-3
Hi Jesse,
The example PEM file you give contains an invalid ASN.1 encoding.
Specifically, the final Integer value (of 9 in the Sequence, the
"Coefficient" value of the key) is encoded with 257 content octets
beginning with the following:

    00 0E C6 ...

This violates section 8.3 "Encoding of an integer value" from the X.690
specification (describes ASN.1 encoding rules for BER/CER/DER):

    "8.3.2 If the contents octets of an integer value encoding consist
of more than one octet, then the bits of the first octet
and bit 8 of the second octet:
a) shall not all be ones; and
b) shall not all be zero.
NOTE – These rules ensure that an integer value is always encoded in the
smallest possible number of octets."

Note that this applies equally to BER encodings and is not only for the
canonical encodings. I didn't really check whether this file uses DER or
not, but it shouldn't matter either way to PEMParser or to this issue.

This sort of error tends to occur somewhat at random as you have seen,
because it will be correct/incorrect depending on what the high bits of
the value happen to be. Nevertheless the bug is that the tool is not
handling the case where it needs to shorten the length to eliminate
redundant leading zeros (a single leading zero byte is needed if the
high bit of the first byte would otherwise be 1).

Of course it's possible to ignore this restriction and interpret the
value in the obvious way, so yes openssl is able to read it (and then
presumably writes it out correctly).

I recommend you notify the authors of the 3rd party tool of the
specifics. Feel free to put me in touch if further explanation is needed.

Regards,
Pete Dettman


On 7/02/2017 2:38 AM, Jesse Schulman wrote:

> The private key shared below seems to be BER encoded but not DER
> encoded, and therefore I get an exception when calling
> PemParser.readObject against it.  I was able however to check it using
> openssl, which results in a valid DER encoded version being written out
> which does not give me an exception.
>
> This private key was generated by a 3rd party tool, and we have only
> seen this issue once out of dozens of other keys it has generated, all
> other keys it has generated seem to be valid DER encoded.  I have
> reported this issue to the 3rd party tool, since it seems wrong and
> inconsistent.
>
> Questions:
> Is there any way to use PemParser or another BouncyCastle API that would
> handle keys encoded in this way, similar to how openssl was able to
> handle it?  If so, how?
>
> If the answer to my first question is yes, is there a strong reason to
> not allow such BER encoded keys in our application?


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Are BER encoded PEM private keys supported for reading by PemParser?

Jesse Schulman
Thank you for the explanation. I have already notified the source of the key, and provided it to them as well.

Thanks!
Jesse
On Sat, Feb 18, 2017 at 1:41 AM Peter Dettman <[hidden email]> wrote:
Hi Jesse,
The example PEM file you give contains an invalid ASN.1 encoding.
Specifically, the final Integer value (of 9 in the Sequence, the
"Coefficient" value of the key) is encoded with 257 content octets
beginning with the following:

    00 0E C6 ...

This violates section 8.3 "Encoding of an integer value" from the X.690
specification (describes ASN.1 encoding rules for BER/CER/DER):

    "8.3.2 If the contents octets of an integer value encoding consist
of more than one octet, then the bits of the first octet
and bit 8 of the second octet:
a) shall not all be ones; and
b) shall not all be zero.
NOTE – These rules ensure that an integer value is always encoded in the
smallest possible number of octets."

Note that this applies equally to BER encodings and is not only for the
canonical encodings. I didn't really check whether this file uses DER or
not, but it shouldn't matter either way to PEMParser or to this issue.

This sort of error tends to occur somewhat at random as you have seen,
because it will be correct/incorrect depending on what the high bits of
the value happen to be. Nevertheless the bug is that the tool is not
handling the case where it needs to shorten the length to eliminate
redundant leading zeros (a single leading zero byte is needed if the
high bit of the first byte would otherwise be 1).

Of course it's possible to ignore this restriction and interpret the
value in the obvious way, so yes openssl is able to read it (and then
presumably writes it out correctly).

I recommend you notify the authors of the 3rd party tool of the
specifics. Feel free to put me in touch if further explanation is needed.

Regards,
Pete Dettman


On 7/02/2017 2:38 AM, Jesse Schulman wrote:
> The private key shared below seems to be BER encoded but not DER
> encoded, and therefore I get an exception when calling
> PemParser.readObject against it.  I was able however to check it using
> openssl, which results in a valid DER encoded version being written out
> which does not give me an exception.
>
> This private key was generated by a 3rd party tool, and we have only
> seen this issue once out of dozens of other keys it has generated, all
> other keys it has generated seem to be valid DER encoded.  I have
> reported this issue to the 3rd party tool, since it seems wrong and
> inconsistent.
>
> Questions:
> Is there any way to use PemParser or another BouncyCastle API that would
> handle keys encoded in this way, similar to how openssl was able to
> handle it?  If so, how?
>
> If the answer to my first question is yes, is there a strong reason to
> not allow such BER encoded keys in our application?


Loading...