Cannot decode base64 with java.util.base64. Removing org.postgresql.util.Base64 has a side effect

[lchevallier69]

Please read https://stackoverflow.com/help/minimal-reproducible-example

Description
As org.postgresql.util.Base64 has been removed from the driver, it is impossible to read postgresql encoded base64 data in java

Driver Version?
42.3.2
Java Version?
Java 8
OS Version?
Windows
PostgreSQL Version?
11

the embbeded version of decoder can read this byte array without problem, but the java.utils.base64 cannot :

Logs
java.lang.IllegalArgumentException: Illegal base64 character a
at java.util.Base64$Decoder.decode0(Base64.java:714)
at java.util.Base64$Decoder.decode(Base64.java:526)

Using the following template code make sure the bug can be replicated in the driver alone.

import java.util.Base64;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;

public class Base64Test {

  public static void main(String[] args) throws SQLException {
    //		String encoded =
    // "AQIAACBqCAAAAwAAAMN20tCUhSlBarkqSp7jWEEH4Ur2BYcpQS9gelKu41hBtONRfiyJKUH2QH87xuNYQQ==";
    //		byte[] encodedBytes = encoded.getBytes();
    //		byte[] b = Base64.getDecoder().decode(encodedBytes);
    String url = "jdbc:postgresql://localhost:5432/bacasable";
    Properties props = new Properties();
    props.setProperty("user", "postgres");
    props.setProperty("password", "postgre$69");
    try (Connection conn = DriverManager.getConnection(url, props)) {
      try (Statement statement = conn.createStatement()) {
        try (ResultSet rs =
            statement.executeQuery(
                "select encode(ST_AsEWKB(geom), 'base64') as encoded from relations.reseau")) {
          if (rs.next()) {
            byte[] bytes = rs.getBytes(1);
            Base64.getDecoder().decode(bytes);
          }
        }
      }
    }
  }
}

[lchevallier69]

seems to be related to 1994

[sehrope]

What's the base64 text of the data?

[lchevallier69]

here is the String from the query
AQIAACBqCAAAAwAAAMN20tCUhSlBarkqSp7jWEEH4Ur2BYcpQS9gelKu41hBtONRfiyJKUH2QH87
xuNYQQ==

[sehrope]

Does that have a newline break in it?

[lchevallier69]

dump-bacasable-202202091710.zip

[lchevallier69]

you must execute create extension postgis before restoring the dump

[sehrope]

And previously you were using the org.postgresql.util.Base64 class to decode that data in your application code?

Specifically, I'm asking if this is happening within the driver's internal code or in your own code that is using that class.

When that class was removed it was assumed that it was only to be used internally by the driver. It wasn't considered part of the public interface of the driver.

[lchevallier69]

here the test code complete

import java.util.Base64;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;

public class Base64Test {

  public static void main(String[] args) throws SQLException {
    //		String encoded =
    // "AQIAACBqCAAAAwAAAMN20tCUhSlBarkqSp7jWEEH4Ur2BYcpQS9gelKu41hBtONRfiyJKUH2QH87xuNYQQ==";
    //		byte[] encodedBytes = encoded.getBytes();
    //		byte[] b = Base64.getDecoder().decode(encodedBytes);
    String url = "jdbc:postgresql://localhost:5432/bacasable";
    Properties props = new Properties();
    props.setProperty("user", "postgres");
    props.setProperty("password", "postgre$69");
    try (Connection conn = DriverManager.getConnection(url, props)) {
      try (Statement statement = conn.createStatement()) {
        try (ResultSet rs =
            statement.executeQuery(
                "select encode(ST_AsEWKB(geom), 'base64') as encoded from relations.reseau")) {
          if (rs.next()) {
            byte[] bytes = rs.getBytes(1);
            Base64.getDecoder().decode(bytes);
          }
        }
      }
    }
  }
}

[lchevallier69]

it seems that the string in PGAdmin is decodable, but it is not i read the result as byte array, it works properly with the org.postgresql.util.Base64 implementation.
In fact the String contains en chr(10) inside it , witch is a return char ! the byte[77] = 10

[lchevallier69]

In fact a fast way to read postgis geometries is to encode geometries encode(ST_AsEWKB(geom), 'base64') The org.postgresql.util.Base64 decode well my data but not java.util.Base64 It seems that the encode method in PG does not do a correct job, and that the org.postgresql.util.Base64 treat and corrects effectively the error I put a dump in the ticket to go further Regards De : Sehrope Sarkuni ***@***.***> Envoyé : mercredi 9 février 2022 17:23 À : pgjdbc/pgjdbc ***@***.***> Cc : Luc CHEVALLIER ***@***.***>; Author ***@***.***> Objet : [/!\ SPAM suspecté ] Re: [pgjdbc/pgjdbc] Cannot decode base64 with java.util.base64. Removing org.postgresql.util.Base64 has a side effect (Issue #2447) And previously you were using the org.postgresql.util.Base64 class to decode that data in your application code? Specifically, I'm asking if this is happening within the driver's internal code or in your own code that is using that class. When that class was removed it was assumed that it was only to be used internally by the driver. It wasn't considered part of the public interface of the driver. — Reply to this email directly, view it on GitHub<#2447 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVXU7LBZ3GB4PTS23V4C3DTU2KIG3ANCNFSM5N5Y347A>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[jorsol]

I haven't looked at this, but Base64 has 3 types of encoding schemes.

• Basic
• MIME
• URL and Filename safe

Have you tried if using getMimeDecoder() or getUrlDecoder() works?

[lchevallier69]

Well Done !
Thats works correctly, in fact we should use getMimeDecoder(), that use RFC2045 scheme
Many thanks, i'll close the issue

[vlsi]

I think we should resurrect org.postgresql.util.Base64 in @Deprecated state as the class has been removed without proper deprecation.

[sehrope]

Are we going to consider everything in org.postgresql.util.* part of the public API?

[vlsi]

Are there reasons to remove public classes/interfaces without going through the deprecation cycle?

[sehrope]

If they're not part of the public API then I see no reason to maintain them as deprecated. The class being "public" is a Java limitation to allow it to be used across package boundaries within the library. Nothing in that org.postgresql.util.* package is part of the published API of the driver so it's not unreasonable for us to remove or modify without preserving backwards compatibility.

If the class was still in the tree and we hadn't published a version without it, then I'd probably be neutral on deprecating it first if it doesn't cause any maintenance headache (e.g. in this case it's entirely self contained class). But in this specific case we already removed it so the bar is higher to add it back.