read throws EOFException at the end of TLS decoded stream

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

read throws EOFException at the end of TLS decoded stream

Alexander Farber
Good evening,

I have prepared a simple TLS PSK client test case based on MockPSKTlsClient at
https://github.com/afarber/jetty-newbie/tree/master/TlsPskClient2/src/main/java/de/afarber/tlspskclient2

In the main method I call:

public static void main(String[] args) throws IOException {
    SecureRandom random      = new SecureRandom();
    TlsPSKIdentity identity  = new BasicTlsPSKIdentity("Client_identity", Hex.decode("1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A"));
    Socket socket            = new Socket(InetAddress.getLocalHost(), 12345);
    TlsClientProtocol proto  = new TlsClientProtocol(socket.getInputStream(), socket.getOutputStream(), random);
    MockPSKTlsClient client  = new MockPSKTlsClient(null, identity);
    proto.connect(client);

    OutputStream clearOs = proto.getOutputStream();
    InputStream clearIs = proto.getInputStream();
    clearOs.write("GET / HTTP/1.1\r\n\r\n".getBytes("UTF-8"));
    Streams.pipeAll(clearIs, System.out);   // why is java.io.EOFException thrown?
}

As you can see, I send a GET / HTTP/1.1 string to the openssl server, which is started as:

# openssl s_server \
        -psk 1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A \
        -psk_hint Client_identity\
        -cipher PSK-AES256-CBC-SHA \
        -debug -state -nocert -accept 12345 -tls1_2 -www

After that I call pipeAll method, which is merely:

public static void pipeAll(InputStream inStr, OutputStream outStr)
    throws IOException
{
    byte[] bs = new byte[BUFFER_SIZE];
    int numRead;
    while ((numRead = inStr.read(bs, 0, bs.length)) > 0) // Why is EOFException thrown?
    {
        outStr.write(bs, 0, numRead);
    }
}

This copies "openssl s_server" answer to the screen and also surprisingly throws an EOFException at the end:

TLS-PSK client negotiated TLS 1.2
Established session: 68e647e3276f345e82effdb7cc04649f6872d245ae01489c08ed109c5906dd16
HTTP/1.0 200 ok
Content-type: text/html
.............
TLS-PSK client raised alert: fatal(2), internal_error(80)
> Failed to read record
java.io.EOFException
    at org.bouncycastle.crypto.tls.TlsProtocol.safeReadRecord(Unknown Source)
    at org.bouncycastle.crypto.tls.TlsProtocol.readApplicationData(Unknown Source)
    at org.bouncycastle.crypto.tls.TlsInputStream.read(Unknown Source)
    at de.afarber.tlspskclient2.Main.pipeAll(Main.java:52)
    at de.afarber.tlspskclient2.Main.main(Main.java:44)

My (maybe stupid) question is: why is EOFEception thrown?

Usually InputStream.read() is supposed to return 0 at the end of stream and not throw an exception.

How to detect an end of stream properly, when TLS encryption is used?

Regards
Alex

PS: You can view the full output at
http://stackoverflow.com/questions/40636982/read-throws-eofexception-at-the-end-of-tls-decoded-stream



Reply | Threaded
Open this post in threaded view
|

Re: read throws EOFException at the end of TLS decoded stream

Peter Dettman-3
Hi Alex,

The meaning of this EOFException is simply that the peer closed its
output stream without sending a close_notify alert (this is quite common
behaviour in practice). In TLS 1.0, this required the invalidation of
the session, and we ended up classifying it as EOFException ("Signals
that an end of file or end of stream has been reached unexpectedly
during input.").

I think from my own testing "openssl s_server" never sends a
close_notify, so you'll see this every connection. You should see a
"normal" clean exit if you test with e.g. gnutls_serv.

We will probably change this behaviour in BC quite soon, as it is
obsolete behaviour as of TLS 1.1 (the session invalidation part), and
appears to mostly cause problems/confusion for users anyway. The
alternate code is in an inline comment in TlsProtocol.safeReadRecord()
(https://github.com/bcgit/bc-java/blob/master/core/src/main/java/org/bouncycastle/crypto/tls/TlsProtocol.java#L514),
but I can't immediately see a way for you to patch that easily (due to
various access specifiers).

Perhaps for the moment you could ignore EOFException and rely on parsing
at the application level to determine if a valid/complete response was
received? If necessary, check the first element of the stack trace to
filter only an EOFException from TlsProtocol.safeReadRecord.

Regards,
Pete Dettman

On 16/11/2016 11:20 PM, Alexander Farber wrote:

> Good evening,
>
> I have prepared a simple TLS PSK client test case based on
> MockPSKTlsClient at
> https://github.com/afarber/jetty-newbie/tree/master/TlsPskClient2/src/main/java/de/afarber/tlspskclient2
>
> In the main method I call:
>
> public static void main(String[] args) throws IOException {
>     SecureRandom random      = new SecureRandom();
>     TlsPSKIdentity identity  = new
> BasicTlsPSKIdentity("Client_identity",
> Hex.decode("1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A"));
>     Socket socket            = new Socket(InetAddress.getLocalHost(),
> 12345);
>     TlsClientProtocol proto  = new
> TlsClientProtocol(socket.getInputStream(), socket.getOutputStream(),
> random);
>     MockPSKTlsClient client  = new MockPSKTlsClient(null, identity);
>     proto.connect(client);
>
>     OutputStream clearOs = proto.getOutputStream();
>     InputStream clearIs = proto.getInputStream();
>     clearOs.write("GET / HTTP/1.1\r\n\r\n".getBytes("UTF-8"));
>     Streams.pipeAll(clearIs, System.out);   // why is
> java.io.EOFException thrown?
> }
>
> As you can see, I send a GET / HTTP/1.1 string to the openssl server,
> which is started as:
>
> # openssl s_server \
>         -psk 1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A \
>         -psk_hint Client_identity\
>         -cipher PSK-AES256-CBC-SHA \
>         -debug -state -nocert -accept 12345 -tls1_2 -www
>
> After that I call pipeAll method, which is merely:
>
> public static void pipeAll(InputStream inStr, OutputStream outStr)
>     throws IOException
> {
>     byte[] bs = new byte[BUFFER_SIZE];
>     int numRead;
>     while ((numRead = inStr.read(bs, 0, bs.length)) > 0) // Why is
> EOFException thrown?
>     {
>         outStr.write(bs, 0, numRead);
>     }
> }
>
> This copies "openssl s_server" answer to the screen and also
> surprisingly throws an EOFException at the end:
>
> TLS-PSK client negotiated TLS 1.2
> Established session:
> 68e647e3276f345e82effdb7cc04649f6872d245ae01489c08ed109c5906dd16
> HTTP/1.0 200 ok
> Content-type: text/html
> .............
> TLS-PSK client raised alert: fatal(2), internal_error(80)
>> Failed to read record
> java.io.EOFException
>     at org.bouncycastle.crypto.tls.TlsProtocol.safeReadRecord(Unknown
> Source)
>     at
> org.bouncycastle.crypto.tls.TlsProtocol.readApplicationData(Unknown Source)
>     at org.bouncycastle.crypto.tls.TlsInputStream.read(Unknown Source)
>     at de.afarber.tlspskclient2.Main.pipeAll(Main.java:52)
>     at de.afarber.tlspskclient2.Main.main(Main.java:44)
>
> My (maybe stupid) question is: why is EOFEception thrown?
>
> Usually InputStream.read() is supposed to return 0 at the end of stream
> and not throw an exception.
>
> How to detect an end of stream properly, when TLS encryption is used?
>
> Regards
> Alex
>
> PS: You can view the full output at
> http://stackoverflow.com/questions/40636982/read-throws-eofexception-at-the-end-of-tls-decoded-stream
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: read throws EOFException at the end of TLS decoded stream

Alexander Farber
Hello Pete,

On Fri, Nov 18, 2016 at 8:08 AM, Peter Dettman <[hidden email]> wrote:

The meaning of this EOFException is simply that the peer closed its
output stream without sending a close_notify alert (this is quite common
behaviour in practice). In TLS 1.0, this required the invalidation of
the session, and we ended up classifying it as EOFException ("Signals
that an end of file or end of stream has been reached unexpectedly
during input.").

I think from my own testing "openssl s_server" never sends a
close_notify, so you'll see this every connection. You should see a
"normal" clean exit if you test with e.g. gnutls_serv.

We will probably change this behaviour in BC quite soon, as it is
obsolete behaviour as of TLS 1.1 (the session invalidation part), and
appears to mostly cause problems/confusion for users anyway. The
alternate code is in an inline comment in TlsProtocol.safeReadRecord()
(https://github.com/bcgit/bc-java/blob/master/core/src/main/java/org/bouncycastle/crypto/tls/TlsProtocol.java#L514),
but I can't immediately see a way for you to patch that easily (due to
various access specifiers).

Perhaps for the moment you could ignore EOFException and rely on parsing
at the application level to determine if a valid/complete response was
received? If necessary, check the first element of the stack trace to
filter only an EOFException from TlsProtocol.safeReadRecord.



what do you please think about the "truncation attack" mentioned in the comment at

http://stackoverflow.com/questions/40636982/how-to-detect-an-end-of-stream-properly-when-tls-psk-encryption-is-used

Do you think it is a valid concern and will it be addressed too?

Thank you
Alex

Reply | Threaded
Open this post in threaded view
|

Re: read throws EOFException at the end of TLS decoded stream

Peter Dettman-3
For those interested, I finally posted an answer there:
http://stackoverflow.com/a/42324209/264294.

Of note is that we are introducing a specific EOFException subclass -
TlsNoCloseNotifyException - for this situation, as of v1.57.

Regards,
Pete Dettman

On 18/11/2016 2:47 PM, Alexander Farber wrote:

> what do you please think about the "truncation attack" mentioned in the
> comment at
>
> http://stackoverflow.com/questions/40636982/how-to-detect-an-end-of-stream-properly-when-tls-psk-encryption-is-used
>
>
> Do you think it is a valid concern and will it be addressed too?
>
> Thank you
> Alex
>