TLS Session Reuse with BC TLS Sockets

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
Hi,

when doing FTPS many FTP-servers require the data connection to use the
same TLS session that is used for the control channel. The way,
session reuse is done in Java this didn't work and you have to
do quite a hack to get it working as it is described e.g. at [1].
It's even worse if you have to implement the server side but I don't
want to bore people. ;-)

That issue has been raised to OpenJDK already a couple of times but
it is always ignored, so hoping for a change seems like a waste of time.

Above hack stops working with Java 11 and I have to either find a
new hack which will ultimately stop working when accesses to
internal java classes end up as error and no longer as warning
on STDOUT. The other option is to use something else than the
SunJSSE that allows me to get and set the session-data of a TLS-session
and I wonder if BC JSSE can provide that and if there are examples
how to do that. The sources tell me that it should be possible but
some example showing me the how would make my life much easier ;-)


Thanks and best regards,

Lothar Kimmeringer

[1] https://eng.wealthfront.com/2016/06/10/connecting-to-an-ftps-server-with-ssl-session-reuse-in-java-7-and-8/

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Peter Dettman-3
Hi Lothar,
Current beta jars have new BCJSSE extension methods to support the "same
session" requirement:

  BCExtendedSSLSession sessionToResume =
((BCSSLSocket)controlSocket).getBCSession();
  // ...
  if (null != sessionToResume) {
    ((BCSSLSocket)dataSocket).setBCSessionToResume(sessionToResume);
  }


Of course this will only work for BCJSSE, but we couldn't see any way to
do this within the standard JSSE API.

Regards,
Pete Dettman


On 26/2/20 11:13 pm, Lothar Kimmeringer wrote:

> Hi,
>
> when doing FTPS many FTP-servers require the data connection to use the
> same TLS session that is used for the control channel. The way,
> session reuse is done in Java this didn't work and you have to
> do quite a hack to get it working as it is described e.g. at [1].
> It's even worse if you have to implement the server side but I don't
> want to bore people. ;-)
>
> That issue has been raised to OpenJDK already a couple of times but
> it is always ignored, so hoping for a change seems like a waste of time.
>
> Above hack stops working with Java 11 and I have to either find a
> new hack which will ultimately stop working when accesses to
> internal java classes end up as error and no longer as warning
> on STDOUT. The other option is to use something else than the
> SunJSSE that allows me to get and set the session-data of a TLS-session
> and I wonder if BC JSSE can provide that and if there are examples
> how to do that. The sources tell me that it should be possible but
> some example showing me the how would make my life much easier ;-)
>
>
> Thanks and best regards,
>
> Lothar Kimmeringer
>
> [1]
> https://eng.wealthfront.com/2016/06/10/connecting-to-an-ftps-server-with-ssl-session-reuse-in-java-7-and-8/
>
>


Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4


Am 27.02.2020 um 04:53 schrieb Peter Dettman:

> Current beta jars have new BCJSSE extension methods to support the "same
> session" requirement:
>
>    BCExtendedSSLSession sessionToResume =
> ((BCSSLSocket)controlSocket).getBCSession();
>    // ...
>    if (null != sessionToResume) {
>      ((BCSSLSocket)dataSocket).setBCSessionToResume(sessionToResume);
>    }
>
>
> Of course this will only work for BCJSSE, but we couldn't see any way to
> do this within the standard JSSE API.

I've got not problem with that. Using the JSSE-API has the benefit that
I don't need to rewrite too much of the already existing code.

I'll try it out but found out that with the introduction of DLExternal
there is now a bug in BC I need to get around and fixed first. I'm going
to open an issue about that as soon as the test case is finished.


Thanks and best regards,

Lothar Kimmeringer

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
In reply to this post by Peter Dettman-3
Hi,

Am 27.02.2020 um 04:53 schrieb Peter Dettman:

> Current beta jars have new BCJSSE extension methods to support the "same
> session" requirement:
>
>    BCExtendedSSLSession sessionToResume =
> ((BCSSLSocket)controlSocket).getBCSession();
>    // ...
>    if (null != sessionToResume) {
>      ((BCSSLSocket)dataSocket).setBCSessionToResume(sessionToResume);
>    }

I've waited for 1.65 to try this and it seems to work concerning the
session resumption but my test case falls with a NullPointerException
in ProvSSLSessionBase when I check the certificate of the session.
Here is some excerpt from my test:

     bw.write("auth tls\r\n");
     bw.flush();
     line = br.readLine();
     assertTrue("check that service is sending TLS switch message (" + line + ")",
         line.equals("234 Switch over to TLS-based communication."));
             
     s = switchToTLS(s, ftp);
     checkCertificate(s, ftp);

switchToTLS looks like this:

     private Socket switchToTLS(Socket s, FtpService ftp) throws Exception {
         boolean[] monitor = new boolean[1];
         SSLSocketFactory factory = (SSLSocketFactory) getSocketFactory(ftp);
         SSLSocket ret = (SSLSocket) factory.createSocket(s, s.getInetAddress().getHostName(), s.getPort(), true);
         ret.addHandshakeCompletedListener((event) -> {
             System.out.println("completed handshake: " + event);
             synchronized(monitor) {
                 monitor[0] = true;
                 monitor.notifyAll();
             }
         });
         ret.setSoTimeout(10000);
         synchronized(monitor) {
             ret.startHandshake();
             if (!monitor[0]) {
                 monitor.wait(10000);
             }
             System.out.println("After start handshake");
         }
         return ret;
     }

The wait/notified and the System.out.printlns are already attempts to
find the reason for the problem. The call gives out the two messages:

   completed handshake: javax.net.ssl.HandshakeCompletedEvent[
             source=Socket[addr=locahost/127.0.0.1,port=47078,localport=47080]]
   After start handshake

so the subsequent part of the test definitely took place after the handshake.


The exception happens in checkCertificate that is called after switchToTLS:
     
     private void checkCertificate(Socket s, FtpService ftp) throws Exception{
         assertTrue("check type of created socket (" + s.getClass() + ")", s instanceof SSLSocket);
         SSLSocket ssl = (SSLSocket) s;
         SSLSession session = ssl.getSession();
         assertNotNull("check existence of ssl-session", session);

         X509Certificate certchain[] = session.getPeerCertificateChain();
         assertNotNull("check existence of server certificate chain", certchain);
         [...]
     }

The exception happens when calling getPeerCertificateChain, here is the
complete stacktrace:

java.lang.NullPointerException
        at org.bouncycastle.jsse.provider.ProvSSLSessionBase.getPeerCertificateChain(Unknown Source)
        at org.bouncycastle.jsse.provider.ExportSSLSession_7.getPeerCertificateChain(Unknown Source)
        at c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.checkCertificate(__Test_FtpService_Ftps_Explicit.java:215)
        at c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.performFileUploadTest(__Test_FtpService_Ftps_Explicit.java:1895)
        at c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.testFileUpload(__Test_FtpService_Ftps_Explicit.java:1855)
         [...]

I'm not sure if this is something I've done wrong on my side (on the other
hand, this worked quite well with JSSE for the last 13 years) or if I
found a bug.


Thanks and cheers, Lothar

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Peter Dettman-3
Hi Lothar,
I can't immediately reproduce this, although I can maybe guess at what
is null. Out of curiosity, what do you get if you call
getPeerCertificates() instead)?

What do session.getProtocol() and session.toString() return?

Do you get the same problem if you don't use the layered socket?

If you could send me a minimal reproducer I will get to the bottom of
it. I may try again tomorrow anyway.

BTW, startHandshake blocks for the handshake, so I doubt any of the
listener stuff is needed, even for testing.

Regards,
Pete Dettman


On 1/4/20 8:46 pm, Lothar Kimmeringer wrote:

> Hi,
>
> Am 27.02.2020 um 04:53 schrieb Peter Dettman:
>
>> Current beta jars have new BCJSSE extension methods to support the "same
>> session" requirement:
>>
>>    BCExtendedSSLSession sessionToResume =
>> ((BCSSLSocket)controlSocket).getBCSession();
>>    // ...
>>    if (null != sessionToResume) {
>>      ((BCSSLSocket)dataSocket).setBCSessionToResume(sessionToResume);
>>    }
>
> I've waited for 1.65 to try this and it seems to work concerning the
> session resumption but my test case falls with a NullPointerException
> in ProvSSLSessionBase when I check the certificate of the session.
> Here is some excerpt from my test:
>
>     bw.write("auth tls\r\n");
>     bw.flush();
>     line = br.readLine();
>     assertTrue("check that service is sending TLS switch message (" +
> line + ")",
>         line.equals("234 Switch over to TLS-based communication."));
>                 s = switchToTLS(s, ftp);
>     checkCertificate(s, ftp);
>
> switchToTLS looks like this:
>
>     private Socket switchToTLS(Socket s, FtpService ftp) throws Exception {
>         boolean[] monitor = new boolean[1];
>         SSLSocketFactory factory = (SSLSocketFactory)
> getSocketFactory(ftp);
>         SSLSocket ret = (SSLSocket) factory.createSocket(s,
> s.getInetAddress().getHostName(), s.getPort(), true);
>         ret.addHandshakeCompletedListener((event) -> {
>             System.out.println("completed handshake: " + event);
>             synchronized(monitor) {
>                 monitor[0] = true;
>                 monitor.notifyAll();
>             }
>         });
>         ret.setSoTimeout(10000);
>         synchronized(monitor) {
>             ret.startHandshake();
>             if (!monitor[0]) {
>                 monitor.wait(10000);
>             }
>             System.out.println("After start handshake");
>         }
>         return ret;
>     }
>
> The wait/notified and the System.out.printlns are already attempts to
> find the reason for the problem. The call gives out the two messages:
>
>   completed handshake: javax.net.ssl.HandshakeCompletedEvent[
>            
> source=Socket[addr=locahost/127.0.0.1,port=47078,localport=47080]]
>   After start handshake
>
> so the subsequent part of the test definitely took place after the
> handshake.
>
>
> The exception happens in checkCertificate that is called after switchToTLS:
>         private void checkCertificate(Socket s, FtpService ftp) throws
> Exception{
>         assertTrue("check type of created socket (" + s.getClass() +
> ")", s instanceof SSLSocket);
>         SSLSocket ssl = (SSLSocket) s;
>         SSLSession session = ssl.getSession();
>         assertNotNull("check existence of ssl-session", session);
>
>         X509Certificate certchain[] = session.getPeerCertificateChain();
>         assertNotNull("check existence of server certificate chain",
> certchain);
>         [...]
>     }
>
> The exception happens when calling getPeerCertificateChain, here is the
> complete stacktrace:
>
> java.lang.NullPointerException
>     at
> org.bouncycastle.jsse.provider.ProvSSLSessionBase.getPeerCertificateChain(Unknown
> Source)
>     at
> org.bouncycastle.jsse.provider.ExportSSLSession_7.getPeerCertificateChain(Unknown
> Source)
>     at
> c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.checkCertificate(__Test_FtpService_Ftps_Explicit.java:215)
>
>     at
> c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.performFileUploadTest(__Test_FtpService_Ftps_Explicit.java:1895)
>
>     at
> c.e.h.s.ftp.__Test_FtpService_Ftps_Explicit.testFileUpload(__Test_FtpService_Ftps_Explicit.java:1855)
>
>         [...]
>
> I'm not sure if this is something I've done wrong on my side (on the other
> hand, this worked quite well with JSSE for the last 13 years) or if I
> found a bug.
>
>
> Thanks and cheers, Lothar
>


Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
Hi Peter,

Am 01.04.2020 um 21:32 schrieb Peter Dettman:

> I can't immediately reproduce this, although I can maybe guess at what
> is null. Out of curiosity, what do you get if you call
> getPeerCertificates() instead)?

That returns without a NullPointerException returning an array of Certificates.
I assume that there is something happening during the creation of the
X509Certificate instances, e.g. because it assumes elements of the certificate
to be present.

> What do session.getProtocol() and session.toString() return?

TLSv1.2

> Do you get the same problem if you don't use the layered socket?

You mean using the JSSE instead of BCJSSE? That testcase is -
as I said - 13 years old and hasn't changed when I changed the
implementation to use BCJSSE so it worked without problem.

> If you could send me a minimal reproducer I will get to the bottom of
> it. I may try again tomorrow anyway.

I'll do that but for starters I'll extended the debug output:

         System.out.println("After start handshake");
         try {
             Certificate[] certs = ret.getSession().getPeerCertificates();
             for (int i = 0; i < certs.length; i++) {
                 System.out.println("peer certificate " + (i+1) + ": " + certs[i].toString());
             }
         }
         catch(Exception e) {
             e.printStackTrace();
         }
         System.out.println("Protocol: " + ret.getSession().getProtocol());
         System.out.println("toString: " + ret.getSession().toString());

outputs

After start handshake
completed handshake: javax.net.ssl.HandshakeCompletedEvent[source=Socket[addr=locahost/127.0.0.1,port=63607,localport=63609]]
peer certificates: [
[
   Version: V3
   Subject: OU=Testsender Certificate Default, O=Test GmbH, C=DE
   Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

   Key:  Sun RSA public key, 1024 bits
   modulus: 100718279042695294057402168531977398771683651530399831657471843895056350324530251001215107855115737312027743769089980817436466439733373337167605988246364871905673279525164969037423397940567765983016245195433130155824536342219431503604980853602826731699599865511600314319016075723055322600981914042215792150981
   public exponent: 65537
   Validity: [From: Tue Mar 03 08:59:43 CET 2020,
                To: Sat May 02 09:59:43 CEST 2020]
   Issuer: OU=Testsender Certificate Default, O=Test GmbH, C=DE
   SerialNumber: [    12345678 90abcdef]

Certificate Extensions: 3
[1]: ObjectId: 2.5.29.37 Criticality=true
ExtendedKeyUsages [
   clientAuth
   serverAuth
]

[2]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
   DigitalSignature
   Key_Encipherment
   Data_Encipherment
   Key_CertSign
]

[3]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 0D 5D 46 5F E7 3D B9 A2   07 33 F1 C7 62 39 D0 54  .]F_.=...3..b9.T
0010: B2 C4 CD 3E D5 AF C0 8F   32 72 5C 81 E5 C6 C8 0D  ...>....2r\.....
]
]

]
   Algorithm: [SHA256withRSA]
   Signature:
0000: 73 B4 DE A8 8E B0 3D C1   4E 66 5A F8 75 90 32 BE  s.....=.NfZ.u.2.
0010: 2D 13 CE 92 9C A2 9B 84   6D 28 1C 33 A6 88 9E 98  -.......m(.3....
0020: CC 38 34 8A 24 19 68 1B   73 BC 7C 58 47 5A D2 89  .84.$.h.s..XGZ..
0030: F9 A0 70 EC 54 9F 31 80   1B C9 15 61 86 25 6A 8E  ..p.T.1....a.%j.
0040: 01 DE DA E3 E3 01 CF BD   51 11 27 2D CF 7A 8C 5C  ........Q.'-.z.\
0050: D3 14 54 77 1F 58 7B 4D   C7 3C AB 32 31 F8 77 0C  ..Tw.X.M.<.21.w.
0060: 53 4F 7A 6B 7F A2 CB BD   7A F0 E8 B3 60 61 99 6F  SOzk....z...`a.o
0070: 4F 56 7A A5 FC DC 5D D4   01 AC DD 35 C0 D1 B6 A1  OVz...]....5....

]
TLSv1.2
Session(1585814384302|TLS_RSA_WITH_AES_256_GCM_SHA384)

> BTW, startHandshake blocks for the handshake, so I doubt any of the
> listener stuff is needed, even for testing.

I've added that stuff after having the theory that there might be threading
issues and actually seeing the output of the two System.out.printlns
the other way around.

   After start handshake
   completed handshake: javax.net.ssl.HandshakeCompletedEvent[source=Socket[addr=locahost/127.0.0.1,port=63529,localport=63531]]

Haven't checked the sources but because of the non-effect of the outcome
of my test I assume that the listener notification happens deferred which
is why you see the handshake complete message after the call of starting
the handshake.


Thanks and cheers, Lothar

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Peter Dettman-3
Hi Lothar,
I think it's clear now that it must be the SSLSessionContext in the
first line of the method that is null. We've patched the code and will
put up a new beta version shortly.

Regards,
Pete Dettman

On 2/4/20 3:08 pm, Lothar Kimmeringer wrote:

> Hi Peter,
>
> Am 01.04.2020 um 21:32 schrieb Peter Dettman:
>
>> I can't immediately reproduce this, although I can maybe guess at what
>> is null. Out of curiosity, what do you get if you call
>> getPeerCertificates() instead)?
>
> That returns without a NullPointerException returning an array of
> Certificates.
> I assume that there is something happening during the creation of the
> X509Certificate instances, e.g. because it assumes elements of the
> certificate
> to be present.
>
>> What do session.getProtocol() and session.toString() return?
>
> TLSv1.2

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4


Am 02.04.2020 um 14:03 schrieb Peter Dettman:

> I think it's clear now that it must be the SSLSessionContext in the
> first line of the method that is null. We've patched the code and will
> put up a new beta version shortly.

OK, I've found out the same after updating my local copy of the BC-sources
and were able to have a look at the changes. The reason became quite
obvious, especially because the constructor shows that SSLSessionContext
can be null.


Thanks for fixing it and cheers, Lothar

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
In reply to this post by Peter Dettman-3
Hi,

Am 27.02.2020 um 04:53 schrieb Peter Dettman:
>
> Of course this will only work for BCJSSE, but we couldn't see any way to
> do this within the standard JSSE API.

More tests, more problems ;-) I've had with the new JSSE in Java 11 as well.
It seems that the session ID you get when calling SSLSession.getId() is
empty. My understanding is that it's a security measure but that brings
up a problem when implementing an FTPS-server where TLS session resumption
is used to prevent data connections being hijacked. So as a server I
need a way to check if the data's TLS connection is a resumed sesssion
and with getId returning an empty array my question is how that can now
been done.

A similar question came up by somebody else on dev-sec but never got
a usable answer other than checking the creation time (which would in
my eyes allow timing attacks).


Thanks and best regards,

Lothar Kimmeringer

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
In reply to this post by Peter Dettman-3

Hi,

Am 27.02.2020 um 04:53 schrieb Peter Dettman:
>
> Of course this will only work for BCJSSE, but we couldn't see any way to
> do this within the standard JSSE API.

More tests, more problems ;-) I've had with the new JSSE in Java 11 as well.
It seems that the session ID you get when calling SSLSession.getId() is
empty. My understanding is that it's a security measure but that brings
up a problem when implementing an FTPS-server where TLS session resumption
is used to prevent data connections being hijacked. So as a server I
need a way to check if the data's TLS connection is a resumed sesssion
and with getId returning an empty array my question is how that can now
been done.

A similar question came up by somebody else on secdev but never got
a usable answer other than checking the creation time (which would in
my eyes allow timing attacks).


Thanks and best regards,

Lothar Kimmeringer

Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Peter Dettman-3
In reply to this post by Lothar Kimmeringer-4
Hi Lothar,
Not sure I understand this. AFAIK, SSLSession.getId() returns an empty
value only when the server didn't allocate a session (including the
cases of a failed connection, or rejected session resumption).

Servers are not obliged to create a session (although presumably in your
case it's supposed to). A client should be checking SSLSession.isValid()
for the control connection before trying to open the data connection.

Or are you implementing a server? Both?

Regards,
Pete Dettman

On 6/4/20 5:08 pm, Lothar Kimmeringer wrote:

> Hi,
>
> Am 27.02.2020 um 04:53 schrieb Peter Dettman:
>>
>> Of course this will only work for BCJSSE, but we couldn't see any way to
>> do this within the standard JSSE API.
>
> More tests, more problems ;-) I've had with the new JSSE in Java 11 as
> well.
> It seems that the session ID you get when calling SSLSession.getId() is
> empty. My understanding is that it's a security measure but that brings
> up a problem when implementing an FTPS-server where TLS session resumption
> is used to prevent data connections being hijacked. So as a server I
> need a way to check if the data's TLS connection is a resumed sesssion
> and with getId returning an empty array my question is how that can now
> been done.
>
> A similar question came up by somebody else on dev-sec but never got
> a usable answer other than checking the creation time (which would in
> my eyes allow timing attacks).
>
>
> Thanks and best regards,
>
> Lothar Kimmeringer
>


Reply | Threaded
Open this post in threaded view
|

Re: TLS Session Reuse with BC TLS Sockets

Lothar Kimmeringer-4
Hi,

Am 07.04.2020 um 09:21 schrieb Peter Dettman:

> Servers are not obliged to create a session (although presumably in your
> case it's supposed to). A client should be checking SSLSession.isValid()
> for the control connection before trying to open the data connection.
>
> Or are you implementing a server? Both?

I'm implementing both sides and with FTPS, TLS session resumption is
used as a security feature when opening data connections. The client
is supposed to open the data connection and does a session resumption
with the session that has been created for the control channel. That's
been quite a feat with pre-Java-11 where you needed to fiddle with the
internal structures of JSSE-classes resumption was only taking place
if host and port stayed the same which is clearly not the case with
a FTP data connection.

Java 11 changed the internals of JSSE more or less completely so
client and server needed a new way of doing things (which is why
I now try to switch from JSSE to BCJSSE to get around reflection-hell).

So as a client I need a way to specify the session to be used for
resumption (done) and as a server I need a way to check that the
data connection being opened has a TLS session that is "equivalent"
to the session used for the still active control connection. The
latter was done in the past by checking the session id (getId returned
a non-empty value in Java 8) which doesn't seem to be possible that
way anymore.


Thanks and best regards,

Lothar Kimmeringer