Invalid packet types in generated RSA PGP key

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Invalid packet types in generated RSA PGP key

Paul Schaub

Hi!

I'm currently participating in the Google Summer of Code. My project is about implementing XEP-0373[1] (OpenPGP for XMPP) and XEP-0374[2] (OpenPGP for XMPP: Instant Messaging) for the XMPP client library Smack[3]. For that reason I'm using Bouncycastle 1.59.

I was able to successfully create PGP keypairs which I could use to encrypt and decrypt messages. Next I wanted to add recommended signature sub-packets to the key. I'd like to have keys generated using bc to follow the same scheme as keys generated using gnupg. So I used pgpdump on keys generated by gnupg to exermine, which subpackets were added.

The issue is, that when I try to recreate the same scheme using bc, the keys seem to contain unknown packet types.


Here is the code I use to generate my PGP key pair:


    static Provider PROVIDER = new BouncycastleProvider();

    public PGPKeyRingGenerator build() throws NoSuchAlgorithmException, PGPException {
        KeyPairGenerator bareGenerator = KeyPairGenerator.getInstance("RSA", PROVIDER);
        bareGenerator.initialize(2048);
        KeyPair rsaPair = bareGenerator.generateKeyPair();

        PGPDigestCalculator calculator = new JcaPGPDigestCalculatorProviderBuilder()
                    .setProvider(PROVIDER)
                    .build()
                    .get(HashAlgorithmTags.SHA256);

        PGPKeyPair pgpPair = new JcaPGPKeyPair(PGPPublicKey.RSA_GENERAL, rsaPair, new Date());

        PGPSignatureSubpacketGenerator subpackets = new PGPSignatureSubpacketGenerator();

        // Key flags
        subpackets.setKeyFlags(true,
                KeyFlags.CERTIFY_OTHER
                            | KeyFlags.SIGN_DATA
                            | KeyFlags.ENCRYPT_COMMS
                            | KeyFlags.ENCRYPT_STORAGE
                            | KeyFlags.AUTHENTICATION);

       // Encryption Algorithms
       subpackets.setPreferredSymmetricAlgorithms(true, new int[]{
                    PGPSymmetricEncryptionAlgorithms.AES_256.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.AES_192.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.AES_128.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.TRIPLE_DES.getAlgorithmId()
        });

        // Hash Algorithms
        subpackets.setPreferredHashAlgorithms(true, new int[] {
                    PGPHashAlgorithms.SHA_512.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_384.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_256.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_224.getAlgorithmId(),
                    PGPHashAlgorithms.SHA1.getAlgorithmId()
        });

        // Compression Algorithms
        subpackets.setPreferredCompressionAlgorithms(true, new int[] {
                    PGPCompressionAlgorithms.ZLIB.getAlgorithmId(),
                    PGPCompressionAlgorithms.BZIP2.getAlgorithmId(),
                    PGPCompressionAlgorithms.ZIP.getAlgorithmId()
        });

        // Modification Detection
        subpackets.setFeature(true, Features.FEATURE_MODIFICATION_DETECTION);

        PGPContentSignerBuilder signer = new JcaPGPContentSignerBuilder(
                    pgpPair.getPublicKey().getAlgorithm(), HashAlgorithmTags.SHA256);

        PBESecretKeyEncryptor encryptor;
        if (passPhrase == null) {
            // unencrypted key
            encryptor = null;
        } else {
            encryptor = new JcePBESecretKeyEncryptorBuilder(PGPEncryptedData.AES_256, calculator)
                        .setProvider(PROVIDER).build(passPhrase);
        }

        PGPKeyRingGenerator ringGenerator = new PGPKeyRingGenerator(PGPSignature.POSITIVE_CERTIFICATION, pgpPair,
                    BuildPGPKeyGeneratorAPI.this.identity, calculator, subpackets.generate(), null, signer, encryptor);

        return ringGenerator;
    }

    public void test() {
        PGPKeyRingGenerator g1 = build(); // I use a builder structure to provide values like password, so this is simplified.
        PGPSecretKey s1 = g1.generateSecretKeyRing().getSecretKey();
        ArmoredOutputStream outputStream = new ArmoredOutputStream(
                new BufferedOutputStream(
                        new FileOutputStream(
                                new File("/path/to/key.out"))));
        s1.encode(outputStream);
        outputStream.close();
    }

The output of pgpdump on a secret key exported from gnupg looks like this:

$ pgpdump gnupg.sec
Old: Secret Key Packet(tag 5)(920 bytes)
    Ver 4 - new
    Public key creation time - Tue May  8 15:15:42 CEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    RSA d(2046 bits) - ...
    RSA p(1024 bits) - ...
    RSA q(1024 bits) - ...
    RSA u(1024 bits) - ...
    Checksum - 3b 8c
Old: User ID Packet(tag 13)(23 bytes)
    User ID - <a class="moz-txt-link-freetext" href="xmpp:juliet@capulet.lit">xmpp:juliet@...
Old: Signature Packet(tag 2)(334 bytes)
    Ver 4 - new
    Sig type - Positive certification of a User ID and Public Key packet(0x13).
    Pub alg - RSA Encrypt or Sign(pub 1)
    Hash alg - SHA256(hash 8)
    Hashed Sub: issuer fingerprint(sub 33)(21 bytes)
     v4 -    Fingerprint - 1d 01 8c 77 2d f8 c5 ef 86 a1 dc c9 b4 b5 09 cb 59 36 e0 3e
    Hashed Sub: signature creation time(sub 2)(4 bytes)
        Time - Tue May  8 15:15:42 CEST 2018
    Hashed Sub: key flags(sub 27)(1 bytes)
        Flag - This key may be used to certify other keys
        Flag - This key may be used to sign data
        Flag - This key may be used to encrypt communications
        Flag - This key may be used to encrypt storage
        Flag - This key may be used for authentication
    Hashed Sub: preferred symmetric algorithms(sub 11)(4 bytes)
        Sym alg - AES with 256-bit key(sym 9)
        Sym alg - AES with 192-bit key(sym 8)
        Sym alg - AES with 128-bit key(sym 7)
        Sym alg - Triple-DES(sym 2)
    Hashed Sub: preferred hash algorithms(sub 21)(5 bytes)
        Hash alg - SHA512(hash 10)
        Hash alg - SHA384(hash 9)
        Hash alg - SHA256(hash 8)
        Hash alg - SHA224(hash 11)
        Hash alg - SHA1(hash 2)
    Hashed Sub: preferred compression algorithms(sub 22)(3 bytes)
        Comp alg - ZLIB <RFC1950>(comp 2)
        Comp alg - BZip2(comp 3)
        Comp alg - ZIP <RFC1951>(comp 1)
    Hashed Sub: features(sub 30)(1 bytes)
        Flag - Modification detection (packets 18 and 19)
    Hashed Sub: key server preferences(sub 23)(1 bytes)
        Flag - No-modify
    Sub: issuer key ID(sub 16)(8 bytes)
        Key ID - 0xB4B509CB5936E03E
    Hash left 2 bytes - 87 ec
    RSA m^d mod n(2048 bits) - ...
        -> PKCS-1


The output of pgpdump for a key created using bc looks like this:

Old: Secret Key Packet(tag 5)(950 bytes)
    Ver 4 - new
    Public key creation time - Mon May 14 13:40:11 CEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    RSA d(2047 bits) - ...
    RSA p(1024 bits) - ...
    RSA q(1024 bits) - ...
    RSA u(1020 bits) - ...
    Checksum - ca 3d
Old: Literal Data Packet(tag 11)(16486 bytes)
    Format - unknown
    Filename - O�3�mL�H0 0i��L!n�T��<a class="moz-txt-link-freetext" href="xmpp:alice@wonderland.lit�4Z�u��/�">xmpp:alice@...�4Z�u��/�   �
   
       ��
    �Y���t�Xo|�.&ڙ�a_3��ީ9�]S3���C`=�<.-`�s����c�%����2p�
}�;����ݠu9���
    File modified time - Wed Mar 15 21:19:39 CET 1995
pgpdump: unexpected end of file.
    Literal - ...


I noticed, that the output gets a little more "readable" (or less wrong) when I change the "isCritical" value of some of the subpackets values to "false". Is this a bug in BC?


Do you have any hint for me? Unfortunately most examples I found on the topic of key creation are more or less outdated :/

Kind regards
Paul Schaub

[1]: https://xmpp.org/extensions/xep-0373.html
[2]: https://xmpp.org/extensions/xep-0374.html
[3]: https://github.com/igniterealtime/Smack/


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

David Hook-3

With the original dump, it could not have been generated using the posted code. Looking at the dump it appears someone tried to role their own certification by following an example for adding a signature. Nothing else would explain the presence of the literal data object.

One other thing, the posted code was using SHA-256 for the checksum calculator - this is not allowed either, RFC 4880 specifies SHA-1 which BC's PGP API enforces. Please make sure you are using an actual version of the BC APIs.

Try the following.  It works with BC 1.59.

Regards,

David
public static PGPKeyRingGenerator build() throws NoSuchAlgorithmException, PGPException {
    char[] passPhrase = "fred".toCharArray();
    KeyPairGenerator bareGenerator = KeyPairGenerator.getInstance("RSA", new BouncyCastleProvider());
    bareGenerator.initialize(2048);
    KeyPair rsaPair = bareGenerator.generateKeyPair();

    PGPDigestCalculator calculator = new JcaPGPDigestCalculatorProviderBuilder()
                .setProvider(new BouncyCastleProvider())
                .build()
                .get(HashAlgorithmTags.SHA1);

    PGPKeyPair pgpPair = new JcaPGPKeyPair(PGPPublicKey.RSA_GENERAL, rsaPair, new Date());

    PGPSignatureSubpacketGenerator subpackets = new PGPSignatureSubpacketGenerator();

    // Key flags
    subpackets.setKeyFlags(true,
            KeyFlags.CERTIFY_OTHER
                        | KeyFlags.SIGN_DATA
                        | KeyFlags.ENCRYPT_COMMS
                        | KeyFlags.ENCRYPT_STORAGE
                        | KeyFlags.AUTHENTICATION);

   // Encryption Algorithms
   subpackets.setPreferredSymmetricAlgorithms(true, new int[]{
                SymmetricKeyAlgorithmTags.AES_256,
       SymmetricKeyAlgorithmTags.AES_192,
       SymmetricKeyAlgorithmTags.AES_128,
       SymmetricKeyAlgorithmTags.TRIPLE_DES
    });

    // Hash Algorithms
    subpackets.setPreferredHashAlgorithms(true, new int[] {
                HashAlgorithmTags.SHA512,
        HashAlgorithmTags.SHA384,
        HashAlgorithmTags.SHA256,
        HashAlgorithmTags.SHA224,
        HashAlgorithmTags.SHA1
    });

    // Compression Algorithms
    subpackets.setPreferredCompressionAlgorithms(true, new int[] {
                CompressionAlgorithmTags.ZLIB,
        CompressionAlgorithmTags.BZIP2,
        CompressionAlgorithmTags.ZIP
    });

    // Modification Detection
    subpackets.setFeature(true, Features.FEATURE_MODIFICATION_DETECTION);

    PGPContentSignerBuilder signer = new JcaPGPContentSignerBuilder(
                pgpPair.getPublicKey().getAlgorithm(), HashAlgorithmTags.SHA256);

    PBESecretKeyEncryptor encryptor;
    if (passPhrase == null) {
        // unencrypted key
        encryptor = null;
    } else {
        encryptor = new JcePBESecretKeyEncryptorBuilder(PGPEncryptedData.AES_256, calculator)
                    .setProvider(new BouncyCastleProvider()).build(passPhrase);
    }

    PGPKeyRingGenerator ringGenerator = new PGPKeyRingGenerator(PGPSignature.POSITIVE_CERTIFICATION, pgpPair,
                [hidden email], calculator, subpackets.generate(), null, signer, encryptor);

    return ringGenerator;
}

public static void test()
throws Exception {
    PGPKeyRingGenerator g1 = build(); // I use a builder structure to provide values like password, so this is simplified.
    PGPSecretKey s1 = g1.generateSecretKeyRing().getSecretKey();
    ArmoredOutputStream outputStream = new ArmoredOutputStream(
            new BufferedOutputStream(
                    new FileOutputStream(
                            new File("/tmp/key.out"))));
    s1.encode(outputStream);
    outputStream.close();
}

On 14/05/18 22:19, Paul Schaub wrote:

Hi!

I'm currently participating in the Google Summer of Code. My project is about implementing XEP-0373[1] (OpenPGP for XMPP) and XEP-0374[2] (OpenPGP for XMPP: Instant Messaging) for the XMPP client library Smack[3]. For that reason I'm using Bouncycastle 1.59.

I was able to successfully create PGP keypairs which I could use to encrypt and decrypt messages. Next I wanted to add recommended signature sub-packets to the key. I'd like to have keys generated using bc to follow the same scheme as keys generated using gnupg. So I used pgpdump on keys generated by gnupg to exermine, which subpackets were added.

The issue is, that when I try to recreate the same scheme using bc, the keys seem to contain unknown packet types.


Here is the code I use to generate my PGP key pair:


    static Provider PROVIDER = new BouncycastleProvider();

    public PGPKeyRingGenerator build() throws NoSuchAlgorithmException, PGPException {
        KeyPairGenerator bareGenerator = KeyPairGenerator.getInstance("RSA", PROVIDER);
        bareGenerator.initialize(2048);
        KeyPair rsaPair = bareGenerator.generateKeyPair();

        PGPDigestCalculator calculator = new JcaPGPDigestCalculatorProviderBuilder()
                    .setProvider(PROVIDER)
                    .build()
                    .get(HashAlgorithmTags.SHA256);

        PGPKeyPair pgpPair = new JcaPGPKeyPair(PGPPublicKey.RSA_GENERAL, rsaPair, new Date());

        PGPSignatureSubpacketGenerator subpackets = new PGPSignatureSubpacketGenerator();

        // Key flags
        subpackets.setKeyFlags(true,
                KeyFlags.CERTIFY_OTHER
                            | KeyFlags.SIGN_DATA
                            | KeyFlags.ENCRYPT_COMMS
                            | KeyFlags.ENCRYPT_STORAGE
                            | KeyFlags.AUTHENTICATION);

       // Encryption Algorithms
       subpackets.setPreferredSymmetricAlgorithms(true, new int[]{
                    PGPSymmetricEncryptionAlgorithms.AES_256.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.AES_192.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.AES_128.getAlgorithmId(),
                    PGPSymmetricEncryptionAlgorithms.TRIPLE_DES.getAlgorithmId()
        });

        // Hash Algorithms
        subpackets.setPreferredHashAlgorithms(true, new int[] {
                    PGPHashAlgorithms.SHA_512.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_384.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_256.getAlgorithmId(),
                    PGPHashAlgorithms.SHA_224.getAlgorithmId(),
                    PGPHashAlgorithms.SHA1.getAlgorithmId()
        });

        // Compression Algorithms
        subpackets.setPreferredCompressionAlgorithms(true, new int[] {
                    PGPCompressionAlgorithms.ZLIB.getAlgorithmId(),
                    PGPCompressionAlgorithms.BZIP2.getAlgorithmId(),
                    PGPCompressionAlgorithms.ZIP.getAlgorithmId()
        });

        // Modification Detection
        subpackets.setFeature(true, Features.FEATURE_MODIFICATION_DETECTION);

        PGPContentSignerBuilder signer = new JcaPGPContentSignerBuilder(
                    pgpPair.getPublicKey().getAlgorithm(), HashAlgorithmTags.SHA256);

        PBESecretKeyEncryptor encryptor;
        if (passPhrase == null) {
            // unencrypted key
            encryptor = null;
        } else {
            encryptor = new JcePBESecretKeyEncryptorBuilder(PGPEncryptedData.AES_256, calculator)
                        .setProvider(PROVIDER).build(passPhrase);
        }

        PGPKeyRingGenerator ringGenerator = new PGPKeyRingGenerator(PGPSignature.POSITIVE_CERTIFICATION, pgpPair,
                    BuildPGPKeyGeneratorAPI.this.identity, calculator, subpackets.generate(), null, signer, encryptor);

        return ringGenerator;
    }

    public void test() {
        PGPKeyRingGenerator g1 = build(); // I use a builder structure to provide values like password, so this is simplified.
        PGPSecretKey s1 = g1.generateSecretKeyRing().getSecretKey();
        ArmoredOutputStream outputStream = new ArmoredOutputStream(
                new BufferedOutputStream(
                        new FileOutputStream(
                                new File("/path/to/key.out"))));
        s1.encode(outputStream);
        outputStream.close();
    }

The output of pgpdump on a secret key exported from gnupg looks like this:

$ pgpdump gnupg.sec
Old: Secret Key Packet(tag 5)(920 bytes)
    Ver 4 - new
    Public key creation time - Tue May  8 15:15:42 CEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    RSA d(2046 bits) - ...
    RSA p(1024 bits) - ...
    RSA q(1024 bits) - ...
    RSA u(1024 bits) - ...
    Checksum - 3b 8c
Old: User ID Packet(tag 13)(23 bytes)
    User ID - <a class="moz-txt-link-freetext" href="xmpp:juliet@capulet.lit" moz-do-not-send="true">xmpp:juliet@...
Old: Signature Packet(tag 2)(334 bytes)
    Ver 4 - new
    Sig type - Positive certification of a User ID and Public Key packet(0x13).
    Pub alg - RSA Encrypt or Sign(pub 1)
    Hash alg - SHA256(hash 8)
    Hashed Sub: issuer fingerprint(sub 33)(21 bytes)
     v4 -    Fingerprint - 1d 01 8c 77 2d f8 c5 ef 86 a1 dc c9 b4 b5 09 cb 59 36 e0 3e
    Hashed Sub: signature creation time(sub 2)(4 bytes)
        Time - Tue May  8 15:15:42 CEST 2018
    Hashed Sub: key flags(sub 27)(1 bytes)
        Flag - This key may be used to certify other keys
        Flag - This key may be used to sign data
        Flag - This key may be used to encrypt communications
        Flag - This key may be used to encrypt storage
        Flag - This key may be used for authentication
    Hashed Sub: preferred symmetric algorithms(sub 11)(4 bytes)
        Sym alg - AES with 256-bit key(sym 9)
        Sym alg - AES with 192-bit key(sym 8)
        Sym alg - AES with 128-bit key(sym 7)
        Sym alg - Triple-DES(sym 2)
    Hashed Sub: preferred hash algorithms(sub 21)(5 bytes)
        Hash alg - SHA512(hash 10)
        Hash alg - SHA384(hash 9)
        Hash alg - SHA256(hash 8)
        Hash alg - SHA224(hash 11)
        Hash alg - SHA1(hash 2)
    Hashed Sub: preferred compression algorithms(sub 22)(3 bytes)
        Comp alg - ZLIB <RFC1950>(comp 2)
        Comp alg - BZip2(comp 3)
        Comp alg - ZIP <RFC1951>(comp 1)
    Hashed Sub: features(sub 30)(1 bytes)
        Flag - Modification detection (packets 18 and 19)
    Hashed Sub: key server preferences(sub 23)(1 bytes)
        Flag - No-modify
    Sub: issuer key ID(sub 16)(8 bytes)
        Key ID - 0xB4B509CB5936E03E
    Hash left 2 bytes - 87 ec
    RSA m^d mod n(2048 bits) - ...
        -> PKCS-1


The output of pgpdump for a key created using bc looks like this:

Old: Secret Key Packet(tag 5)(950 bytes)
    Ver 4 - new
    Public key creation time - Mon May 14 13:40:11 CEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    RSA d(2047 bits) - ...
    RSA p(1024 bits) - ...
    RSA q(1024 bits) - ...
    RSA u(1020 bits) - ...
    Checksum - ca 3d
Old: Literal Data Packet(tag 11)(16486 bytes)
    Format - unknown
    Filename - O� 3�mL�H0 0i��L!n�T��<a class="moz-txt-link-freetext" href="xmpp:alice@wonderland.lit%EF%BF%BD%014%04%13%1E%02Z%EF%BF%BDu%EF%BF%BD%02%EF%BF%BD/%EF%BF%BD" moz-do-not-send="true">xmpp:alice@...�4Z�u��/�   �
   
       ��
    �Y���t�Xo|�.&ڙ�a_3��ީ9�]S3���C`=�<.-`�s����c�%����2p�
}�;����ݠu9���
    File modified time - Wed Mar 15 21:19:39 CET 1995
pgpdump: unexpected end of file.
    Literal - ...


I noticed, that the output gets a little more "readable" (or less wrong) when I change the "isCritical" value of some of the subpackets values to "false". Is this a bug in BC?


Do you have any hint for me? Unfortunately most examples I found on the topic of key creation are more or less outdated :/

Kind regards
Paul Schaub

[1]: https://xmpp.org/extensions/xep-0373.html
[2]: https://xmpp.org/extensions/xep-0374.html
[3]: https://github.com/igniterealtime/Smack/


Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

Paul Schaub
Hi David!

Thank you very, very much for your answer :)
Unfortunately the code still doesn't work as intended :/


Am 15.05.2018 um 06:41 schrieb David Hook:
>
> With the original dump, it could not have been generated using the
> posted code. Looking at the dump it appears someone tried to role
> their own certification by following an example for adding a
> signature. Nothing else would explain the presence of the literal data
> object.

I can assure you, the key from the dump was actually created using
exactly that code. I don't think there really is a literal data object
present in the key. Instead I think the coding is somehow broken and
random bytes are interpreted as tags as other unintended packets appear
if I make slight changes to my code (like setting the "isCritical" flag
to false in some subpacket method calls).

>
> One other thing, the posted code was using SHA-256 for the checksum
> calculator - this is not allowed either, RFC 4880 specifies SHA-1
> which BC's PGP API enforces. Please make sure you are using an actual
> version of the BC APIs.

I read about that in another github issue when I researched my problem.
However, it seems like I don't get an exception using the SHA-256
calculator. Is this an indicator that I made a mistake?

Bouncycastle is included in my project using gradle like this:

dependencies {
    compile 'org.bouncycastle:bcprov-jdk15on:1.59'
    compile 'org.bouncycastle:bcpg-jdk15on:1.59'
}

> Try the following.  It works with BC 1.59.
> [...]
>

I tried that code, but unfortunately I still get keys with an error :/
Find below the actual key I generated using your code:

-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: BCPG v1.59

lQOqBFr6hWUBCACK+EjkwiEjxEfphBNNgIitKJ40MuuyU57Ss2TE9On3wF2K5gHh
TwQr8gg4wU1lSTslIvG/GA0xY57VBM9MmVfhqC6gyz+sJQCFAD2qYeK65ePwH/I2
mpcSewuRsMKZtFpqOkDsdnaoGU2L7eC1H7PvZfpVLvKm1C16dtl6Oj7pOdMgsDt9
yLv/oMiIguvPdfSsB0F29mb7JIcAjpRz1yw+mP5GoC51P9HrNY3xe58HtIHk3VYx
H7e+vfLYZKhOuKFNVlh0vDi6drkjWFBJSs61dmsVQLu/JwP1UAm7ByMjLlUHUZGg
7v71sXRRvH94URKL3nEiagsctjNfcq/rJrCLABEBAAEAB/sHM9fJmtZquMihpHmu
NSCymV3Tay9YyKTYkgvKnlBckujT8L0dNPRoCc99tGZ8SAjbo4RdSyrll56K261Q
zBTs3sPOwqDUHWxC+Gm5xiUhEai4Jze2bhCMz3B7/VelNroYDca4WrMmXew2eeMT
WT43zct3cJ2Kxvcjlzu/28SGhAOBf/BU/5J9m4ibnyiLzYV2pttSnI8znjR5b92Z
Lj3KdnQJcWd+vFGfayVIaH09qIv+kkKfxJPGdsK32cJ10a3kKDYmFbihTU2r5fGA
QYJW5Jx0Lu6/uovusgWiuQgvj/6uZI4DbcR5e+PhM1sccQ7ObaKSQbbe/RpdSDwZ
FFhJBAC2lf8ZDXnXctBR5IcI/84DNpsFtSv0Y6rweOB9w9DC5WGINiIwbeSb+YBw
2NKk9I/XO+Vp7+lhUI0273eE/6mSg7HYZU9LYrQ0A8RLgbqjN+K4Bs9Ai3pLA9f6
rLxzYI0cMuVYQYZ+Xh8OPVNC8C9+KJp2QNhqZHZXaf5kaDFweQQAwtjIhbAKG+qO
BqcaOE20by6QIH378k/VW798gRU8S/ohsMNGsU8l6K8uDeDrsZYGowy3Naq9bVV/
xAfQHR9nlXOGumfMrWrzl6H2Zfqq6yTZsjBjo9yeL4n26VIgf3KCmp+gHfYY4ftI
q/NGNEtOijFI0z10MTD4Y3xH9zyS0CMD/irjqzF0E1HEwF617Kw8zMbHaozjsYEG
xZvSJtwM3mduyxANASORLGiQcZo9t8TSZBi1YdF3CtVO2stXZY+3rD1aQWSHxbhN
hhGdr8CXZ57Wq363wi4epxVj5y+DE8+W2RTtHUWkRoMNR7MatO9Fu6fzrhJvQKr9
0miAyNtgibaGfyqQGiOR0hKyZUj2qO4egsGBXVK0GXhtcHA6YWxpY2VAd29uZGVy
bGFuZC5saXSJATQEEwEIAB4FAlr6hWUCmy8FiwkIBwIGlQoJCAsCBJYCAwECngEA
CgkQjj26DsTcw7fvVgf/Y88ABzLXTFgLo4EO1z7eBNT4Y4dJ8zyKNkcwJRVbOup1
lVNwMb65IyqnxbTXVssAHMYD9EmHzQfV1B/BlcH3DhYnVNpOoOreW93rTRcZS2Ew
UtACUBZatc+nZCyJ/2cSm862upKkYSWWBhQj7W0wj+UmnGZbbzQFZhO3zZC8OISv
X48gB9DlvRcjTypdNxfY+LQv5r4oebwABRisq9bqnfRsGt3JaoeeGS//EqppST6T
7niM96sp1dxHUKaUAqS0lxCkfUo2vv7pH8REY3t1VTvVe/d2FpMBs77wHMQSKlGZ
ovSdIgpRjJZ3TZQkMgreW31TSySr/DCZ2HwLiIp1TQ==
=8D8m
-----END PGP PRIVATE KEY BLOCK-----



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

Paul Schaub
Am 15.05.2018 um 09:18 schrieb Paul Schaub:
> Instead I think the coding is somehow broken and
> random bytes are interpreted as tags as other unintended packets appear
> if I make slight changes to my code (like setting the "isCritical" flag
> to false in some subpacket method calls).

That might also explain, why I'm occasionally encountering unknown,
invalid tags or very high version codes.

Btw: GnuPGs --list-packet command lists the generated key correctly, so
maybe this is more a pgpdump issue? Although I still find it suspect,
that pgpdump seems to be able to dump GnuPG keys but not (my)
BouncyCastle keys...


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

David Hook-3
In reply to this post by Paul Schaub

It's a strange world. I get the following dump from the sample code I sent earlier:

Old: Secret Key Packet(tag 5)(966 bytes)
    Ver 4 - new
    Public key creation time - Tue May 15 14:34:18 AEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    Sym alg - AES with 256-bit key(sym 9)
    Iterated and salted string-to-key(s2k 3):
        Hash alg - SHA1(hash 2)
        Salt - 5a 58 71 92 d5 aa 67 93
        Count - 65536(coded count 96)
    IV - c7 27 b7 17 14 93 fd 87 24 57 3c 87 5e a4 eb cc
    Encrypted RSA d
    Encrypted RSA p
    Encrypted RSA q
    Encrypted RSA u
    Encrypted SHA1 hash
Old: User ID Packet(tag 13)(23 bytes)
    User ID - [hidden email]
Old: Signature Packet(tag 2)(308 bytes)
    Ver 4 - new
    Sig type - Positive certification of a User ID and Public Key packet(0x13).
    Pub alg - RSA Encrypt or Sign(pub 1)
    Hash alg - SHA256(hash 8)
    Hashed Sub: signature creation time(sub 2)(4 bytes)
        Time - Tue May 15 14:34:18 AEST 2018
    Hashed Sub: key flags(sub 27)(critical)(1 bytes)
        Flag - This key may be used to certify other keys
        Flag - This key may be used to sign data
        Flag - This key may be used to encrypt communications
        Flag - This key may be used to encrypt storage
        Flag - This key may be used for authentication
    Hashed Sub: preferred symmetric algorithms(sub 11)(critical)(4 bytes)
        Sym alg - AES with 256-bit key(sym 9)
        Sym alg - AES with 192-bit key(sym 8)
        Sym alg - AES with 128-bit key(sym 7)
        Sym alg - Triple-DES(sym 2)
    Hashed Sub: preferred hash algorithms(sub 21)(critical)(5 bytes)
        Hash alg - SHA512(hash 10)
        Hash alg - SHA384(hash 9)
        Hash alg - SHA256(hash 8)
        Hash alg - SHA224(hash 11)
        Hash alg - SHA1(hash 2)
    Hashed Sub: preferred compression algorithms(sub 22)(critical)(3 bytes)
        Comp alg - ZLIB <RFC1950>(comp 2)
        Comp alg - BZip2(comp 3)
        Comp alg - ZIP <RFC1951>(comp 1)
    Hashed Sub: features(sub 30)(critical)(1 bytes)
        Flag - Modification detection (packets 18 and 19)
    Sub: issuer key ID(sub 16)(8 bytes)
        Key ID - 0xCD6A7A715E580998
    Hash left 2 bytes - b1 90
    RSA m^d mod n(2048 bits) - ...
        -> PKCS-1

From the Base64 below I get:

Old: Secret Key Packet(tag 5)(938 bytes)
    Ver 4 - new
    Public key creation time - Tue May 15 16:59:49 AEST 2018
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(2048 bits) - ...
    RSA e(17 bits) - ...
    RSA d(2043 bits) - ...
    RSA p(1024 bits) - ...
    RSA q(1024 bits) - ...
    RSA u(1022 bits) - ...
    Checksum - 7f 2a
Old: One-Pass Signature Packet(tag 4)(26 bytes)
    Unknown version(35)
    Sig type - unknown(91)
    Hash alg - unknown(hash 210)
    Pub alg - Reserved for Elliptic Curve(pub 18)
    Key ID - 0xB26548F6A8EE1E82
    Next packet - other than one pass signature
Old: Reserved(tag 0)(23890 bytes)
pgpdump: unexpected end of file.

gpg --list-packets for the same file, on the other hand, gives me:

:secret key packet:
    version 4, algo 1, created 1526367589, expires 0
    skey[0]: [2048 bits]
    skey[1]: [17 bits]
    skey[2]: [2043 bits]
    skey[3]: [1024 bits]
    skey[4]: [1024 bits]
    skey[5]: [1022 bits]
    checksum: 7f2a
    keyid: 8E3DBA0EC4DCC3B7
:user ID packet: [hidden email]
:signature packet: algo 1, keyid 8E3DBA0EC4DCC3B7
    version 4, created 1526367589, md5len 0, sigclass 0x13
    digest algo 8, begin of digest ef 56
    hashed subpkt 2 len 4 (sig created 2018-05-15)
    critical hashed subpkt 27 len 1 (key flags: 2F)
    critical hashed subpkt 11 len 4 (pref-sym-algos: 9 8 7 2)
    critical hashed subpkt 21 len 5 (pref-hash-algos: 10 9 8 11 2)
    critical hashed subpkt 22 len 3 (pref-zip-algos: 2 3 1)
    critical hashed subpkt 30 len 1 (features: 01)
    subpkt 16 len 8 (issuer key ID 8E3DBA0EC4DCC3B7)
    data: [2047 bits]

I'm not so sure about pgpdump as a reliable tool at this point - it appears to get into trouble parsing out the encrypted data. The key id in pgpdump is clearly wrong as well (gpg must be right, or it's so consistently wrong that it defies the laws of probability). I'm also a bit curious as to why the following lines (130 to 137 in PGPSecretKey) do not appear to be working:

if (checksumCalculator != null)
{
    if (checksumCalculator.getAlgorithm() != HashAlgorithmTags.SHA1)
    {
        throw new PGPException("only SHA1 supported for key checksum calculations.");
    }
    s2kUsage = SecretKeyPacket.USAGE_SHA1;
}
A couple of others. PGP is a packet based protocol so ArmouredOutputStream.close() does not close the underlying stream, depending on how you are using it you may find things get truncated (not happening in this case, but worth keeping in mind).   Also where does PGPSymmetricEncryptionAlgorithms, PGPHashAlgorithms, PGPCompressionAlgorithms come from?

Regards,

David

On 15/05/18 17:18, Paul Schaub wrote:
Hi David!

Thank you very, very much for your answer :)
Unfortunately the code still doesn't work as intended :/


Am 15.05.2018 um 06:41 schrieb David Hook:
With the original dump, it could not have been generated using the
posted code. Looking at the dump it appears someone tried to role
their own certification by following an example for adding a
signature. Nothing else would explain the presence of the literal data
object.
I can assure you, the key from the dump was actually created using
exactly that code. I don't think there really is a literal data object
present in the key. Instead I think the coding is somehow broken and
random bytes are interpreted as tags as other unintended packets appear
if I make slight changes to my code (like setting the "isCritical" flag
to false in some subpacket method calls).

One other thing, the posted code was using SHA-256 for the checksum
calculator - this is not allowed either, RFC 4880 specifies SHA-1
which BC's PGP API enforces. Please make sure you are using an actual
version of the BC APIs.
I read about that in another github issue when I researched my problem.
However, it seems like I don't get an exception using the SHA-256
calculator. Is this an indicator that I made a mistake?

Bouncycastle is included in my project using gradle like this:

dependencies {
    compile 'org.bouncycastle:bcprov-jdk15on:1.59'
    compile 'org.bouncycastle:bcpg-jdk15on:1.59'
}

Try the following.  It works with BC 1.59.
[...]

I tried that code, but unfortunately I still get keys with an error :/
Find below the actual key I generated using your code:

-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: BCPG v1.59

lQOqBFr6hWUBCACK+EjkwiEjxEfphBNNgIitKJ40MuuyU57Ss2TE9On3wF2K5gHh
TwQr8gg4wU1lSTslIvG/GA0xY57VBM9MmVfhqC6gyz+sJQCFAD2qYeK65ePwH/I2
mpcSewuRsMKZtFpqOkDsdnaoGU2L7eC1H7PvZfpVLvKm1C16dtl6Oj7pOdMgsDt9
yLv/oMiIguvPdfSsB0F29mb7JIcAjpRz1yw+mP5GoC51P9HrNY3xe58HtIHk3VYx
H7e+vfLYZKhOuKFNVlh0vDi6drkjWFBJSs61dmsVQLu/JwP1UAm7ByMjLlUHUZGg
7v71sXRRvH94URKL3nEiagsctjNfcq/rJrCLABEBAAEAB/sHM9fJmtZquMihpHmu
NSCymV3Tay9YyKTYkgvKnlBckujT8L0dNPRoCc99tGZ8SAjbo4RdSyrll56K261Q
zBTs3sPOwqDUHWxC+Gm5xiUhEai4Jze2bhCMz3B7/VelNroYDca4WrMmXew2eeMT
WT43zct3cJ2Kxvcjlzu/28SGhAOBf/BU/5J9m4ibnyiLzYV2pttSnI8znjR5b92Z
Lj3KdnQJcWd+vFGfayVIaH09qIv+kkKfxJPGdsK32cJ10a3kKDYmFbihTU2r5fGA
QYJW5Jx0Lu6/uovusgWiuQgvj/6uZI4DbcR5e+PhM1sccQ7ObaKSQbbe/RpdSDwZ
FFhJBAC2lf8ZDXnXctBR5IcI/84DNpsFtSv0Y6rweOB9w9DC5WGINiIwbeSb+YBw
2NKk9I/XO+Vp7+lhUI0273eE/6mSg7HYZU9LYrQ0A8RLgbqjN+K4Bs9Ai3pLA9f6
rLxzYI0cMuVYQYZ+Xh8OPVNC8C9+KJp2QNhqZHZXaf5kaDFweQQAwtjIhbAKG+qO
BqcaOE20by6QIH378k/VW798gRU8S/ohsMNGsU8l6K8uDeDrsZYGowy3Naq9bVV/
xAfQHR9nlXOGumfMrWrzl6H2Zfqq6yTZsjBjo9yeL4n26VIgf3KCmp+gHfYY4ftI
q/NGNEtOijFI0z10MTD4Y3xH9zyS0CMD/irjqzF0E1HEwF617Kw8zMbHaozjsYEG
xZvSJtwM3mduyxANASORLGiQcZo9t8TSZBi1YdF3CtVO2stXZY+3rD1aQWSHxbhN
hhGdr8CXZ57Wq363wi4epxVj5y+DE8+W2RTtHUWkRoMNR7MatO9Fu6fzrhJvQKr9
0miAyNtgibaGfyqQGiOR0hKyZUj2qO4egsGBXVK0GXhtcHA6YWxpY2VAd29uZGVy
bGFuZC5saXSJATQEEwEIAB4FAlr6hWUCmy8FiwkIBwIGlQoJCAsCBJYCAwECngEA
CgkQjj26DsTcw7fvVgf/Y88ABzLXTFgLo4EO1z7eBNT4Y4dJ8zyKNkcwJRVbOup1
lVNwMb65IyqnxbTXVssAHMYD9EmHzQfV1B/BlcH3DhYnVNpOoOreW93rTRcZS2Ew
UtACUBZatc+nZCyJ/2cSm862upKkYSWWBhQj7W0wj+UmnGZbbzQFZhO3zZC8OISv
X48gB9DlvRcjTypdNxfY+LQv5r4oebwABRisq9bqnfRsGt3JaoeeGS//EqppST6T
7niM96sp1dxHUKaUAqS0lxCkfUo2vv7pH8REY3t1VTvVe/d2FpMBs77wHMQSKlGZ
ovSdIgpRjJZ3TZQkMgreW31TSySr/DCZ2HwLiIp1TQ==
=8D8m
-----END PGP PRIVATE KEY BLOCK-----



Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

Paul Schaub
Am 15.05.2018 um 10:22 schrieb David Hook:
>
> It's a strange world.

It is :D

> I'm not so sure about pgpdump as a reliable tool at this point - it
> appears to get into trouble parsing out the encrypted data. The key id
> in pgpdump is clearly wrong as well (gpg must be right, or it's so
> consistently wrong that it defies the laws of probability).

Good to hear that I'm not crazy :D

> A couple of others. PGP is a packet based protocol so
> ArmouredOutputStream.close() does not close the underlying stream,
> depending on how you are using it you may find things get truncated
> (not happening in this case, but worth keeping in mind).

Ah, nice catch, thank you :)

> Also where does PGPSymmetricEncryptionAlgorithms, PGPHashAlgorithms,
> PGPCompressionAlgorithms come from?

Those are enums encapsulating Bouncycastles integers.

For my project I forked bouncy-gpg, so you can find the code for those
enums here:
https://github.com/vanitasvitae/pgpainless/blob/master/src/main/java/name/neuhalfen/projects/crypto/bouncycastle/openpgp/algorithms/PGPSymmetricEncryptionAlgorithms.java

>
> Regards,
>
> David

Greetings
Paul


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

Paul Schaub
I openend an issue on the pgpdump repo:
https://github.com/kazu-yamamoto/pgpdump/issues/23

I think for now I'll focus on other parts of my project, as encrypting
and decrypting works using the given keys.

Still, I'll keep you updated on any findings and keep the issue in the
back of my head :)
Thank you very much for your help!

Greetings
Paul


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

Paul Schaub
Hi!

Am 15.05.2018 um 11:03 schrieb Paul Schaub:
> I openend an issue on the pgpdump repo:
> https://github.com/kazu-yamamoto/pgpdump/issues/23

As Heiko Stamer and Kazu Yamamoto pointed out [1], BC 1.59 seems to
append the SHA1 hash (20 bytes) instead of a 2 byte checksum after the
secret key. Apparently that is against RFC-4880 and breaks pgpdump.

[1]:
https://github.com/kazu-yamamoto/pgpdump/issues/23#issuecomment-389495218


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

David Hook-3

Ask them to read the last paragraph of Section 5.5.3 RFC 4880.

" The two-octet checksum that follows the algorithm-specific portion is
   the algebraic sum, mod 65536, of the plaintext of all the algorithm-
   specific octets (including MPI prefix and data).  With V3 keys, the
   checksum is stored in the clear.  With V4 keys, the checksum is
   encrypted like the algorithm-specific data.  This value is used to
   check that the passphrase was correct.  However, this checksum is
   deprecated; an implementation SHOULD NOT use it, but should rather
   use the SHA-1 hash denoted with a usage octet of 254.  The reason for
   this is that there are some attacks that involve undetectably
   modifying the secret key."

There is further discussion on this earlier in Section 5.5.3 as well.

Regards,

David

On 16/05/18 22:15, Paul Schaub wrote:

> Hi!
>
> Am 15.05.2018 um 11:03 schrieb Paul Schaub:
>> I openend an issue on the pgpdump repo:
>> https://github.com/kazu-yamamoto/pgpdump/issues/23
> As Heiko Stamer and Kazu Yamamoto pointed out [1], BC 1.59 seems to
> append the SHA1 hash (20 bytes) instead of a 2 byte checksum after the
> secret key. Apparently that is against RFC-4880 and breaks pgpdump.
>
> [1]:
> https://github.com/kazu-yamamoto/pgpdump/issues/23#issuecomment-389495218
>


Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

David Hook-3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Actually, I think I'm confused. I was just rereading the commentary, if
the s2k usage is 0, there shouldn't be sha-1 digest.

I will have another look at this.

Regards,

David

On 17/05/18 03:27, David Hook wrote:
> > Ask them to read the last paragraph of Section 5.5.3 RFC 4880. > > "
The two-octet checksum that follows the algorithm-specific portion is >
the algebraic sum, mod 65536, of the plaintext of all the algorithm- >
specific octets (including MPI prefix and data). With V3 keys, the >
checksum is stored in the clear. With V4 keys, the checksum is >
encrypted like the algorithm-specific data. This value is used to >
check that the passphrase was correct. However, this checksum is >
deprecated; an implementation SHOULD NOT use it, but should rather > use
the SHA-1 hash denoted with a usage octet of 254. The reason for > this
is that there are some attacks that involve undetectably > modifying the
secret key." > > There is further discussion on this earlier in Section
5.5.3 as well. > > Regards, > > David > > On 16/05/18 22:15, Paul Schaub
wrote: >> Hi! >> >> Am 15.05.2018 um 11:03 schrieb Paul Schaub: >>> I
openend an issue on the pgpdump repo: >>>
https://github.com/kazu-yamamoto/pgpdump/issues/23 >> As Heiko Stamer
and Kazu Yamamoto pointed out [1], BC 1.59 seems to >> append the SHA1
hash (20 bytes) instead of a 2 byte checksum after the >> secret key.
Apparently that is against RFC-4880 and breaks pgpdump. >> >> [1]: >>
https://github.com/kazu-yamamoto/pgpdump/issues/23#issuecomment-389495218
>> > > >
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlr8cgwACgkQbqHzoYMuRw7C0gEAkpnljH/xW36kH+x5wSdZlY0d
HMRsH3dyVMD8Sb3doY4A/Rep4G7AHWwhBspdxfJC9swD7FHfPa0YOKc9iwpjEv/P
=wOwd
-----END PGP SIGNATURE-----


Reply | Threaded
Open this post in threaded view
|

Re: Invalid packet types in generated RSA PGP key

David Hook-3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Okay, so the problem was that if the checksum calculator was passed when it wasn't required the generator was it by default. The code has now been changed so that the checksum calculator has been ignored.

Thanks to https://github.com/bcgit/bc-java/pull/349

For pointing this out.

Regards,

David

On 17/05/18 04:01, David Hook wrote:
> > Actually, I think I'm confused. I was just rereading the commentary, if > the s2k usage is 0, there shouldn't be sha-1 digest. > > I will have another look at this. > > Regards, > > David > > On 17/05/18 03:27, David Hook wrote: > >> Ask them to read the last paragraph of Section 5.5.3 RFC 4880. > > " > The two-octet checksum that follows the algorithm-specific portion is > > the algebraic sum, mod 65536, of the plaintext of all the algorithm- > > specific octets (including MPI prefix and data). With V3 keys, the > > checksum is stored in the clear. With V4 keys, the checksum is > > encrypted like the algorithm-specific data. This value is used to > > check that the passphrase was correct. However, this checksum is > > deprecated; an implementation SHOULD NOT use it, but should rather > use > the SHA-1 hash denoted with a usage octet of 254. The reason for > this > is that there are some attacks that involve undetectably > modifying the > secret key." > > There is further discussion on this earlier in Section > 5.5.3 as well. > > Regards, > > David > > On 16/05/18 22:15, Paul Schaub > wrote: >> Hi! >> >> Am 15.05.2018 um 11:03 schrieb Paul Schaub: >>> I > openend an issue on the pgpdump repo: >>> > https://github.com/kazu-yamamoto/pgpdump/issues/23 >> As Heiko Stamer > and Kazu Yamamoto pointed out [1], BC 1.59 seems to >> append the SHA1 > hash (20 bytes) instead of a 2 byte checksum after the >> secret key. > Apparently that is against RFC-4880 and breaks pgpdump. >> >> [1]: >> > https://github.com/kazu-yamamoto/pgpdump/issues/23#issuecomment-389495218 > >>>>> > > >
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iF4EAREIAAYFAlr8eZ0ACgkQbqHzoYMuRw6rZAD9FIlo7X03fnOt1Oq/Hqd6G3MS
e3fnfG2rAWw2uNuadrUA/2EfbNrVAGqicazYb6NBQ849doxmJBZUJUSi8SPl8JDk
=ybiS
-----END PGP SIGNATURE-----