Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make buffer size configurable via properties file #2999

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ In addition to the standard connection parameters the driver supports a number o
| binaryTransfer | Boolean | true | Use binary format for sending and receiving data if possible. Setting this to false disables any binary transfer |
| binaryTransferEnable | String | "" | Comma separated list of types to enable binary transfer. Either OID numbers or names |
| binaryTransferDisable | String | "" | Comma separated list of types to disable binary transfer. Either OID numbers or names. Overrides values in the driver default set and values set with binaryTransferEnable. |
| prepareThreshold | Integer | 5 | Statement prepare threshold. A value of -1 stands for forceBinary |
| binaryTransferBufferSize | Integer | 4096 | Size of buffers for copying inputs to outputs in case of OID creation using Blobs. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it relate to all the binary transfers or for blobs only?
If the parameter is related to blobs only, I would expect the name to contain something like blob or large object or something like that.

I suggest we start tracking @since 42.7.1 so the reader can see which parameters are supported in what versions

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its only for blobs, yes. binaryTransferBlobBufferSize?

| preparedStatementCacheQueries | Integer | 256 | Specifies the maximum number of entries in per-connection cache of prepared statements. A value of 0 disables the cache. |
| preparedStatementCacheSizeMiB | Integer | 5 | Specifies the maximum size (in megabytes) of a per-connection prepared statement cache. A value of 0 disables the cache. |
| defaultRowFetchSize | Integer | 0 | Positive number of rows that should be fetched from the database when more rows are needed for ResultSet by each fetch iteration |
Expand Down
3 changes: 3 additions & 0 deletions docs/content/documentation/use.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,9 @@ A comma separated list of types to enable binary transfer. Either OID numbers or
A comma separated list of types to disable binary transfer. Either OID numbers or names.
Over-rides values in the driver default set and values set with binaryTransferEnable.

* **`binaryTransferBufferSize (`*int*`)`** *Default `4096`*\
Size of buffers for copying inputs to outputs in case of OID creation using Blobs.

* **`databaseMetadataCacheFields (`*int*`)`** *Default `65536`*\
Specifies the maximum number of fields to be cached per connection.
A value of 0 disables the cache.
Expand Down
8 changes: 8 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/PGProperty.java
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,14 @@ public enum PGProperty {
"true",
"Use binary format for sending and receiving data if possible"),

/**
* Size of buffers for copying inputs to outputs in case of OID creation using Blobs
*/
BINARY_TRANSFER_BUFFER_SIZE(
"binaryTransferBufferSize",
"4096",
"Size of buffers for copying inputs to outputs in case of OID creation using Blobs"),

/**
* Comma separated list of types to disable binary transfer. Either OID numbers or names.
* Overrides values in the driver default set and values set with binaryTransferEnable.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1731,6 +1731,14 @@ public void setXmlFactoryFactory(@Nullable String xmlFactoryFactory) {
PGProperty.XML_FACTORY_FACTORY.set(properties, xmlFactoryFactory);
}

public int getBinaryTransferBufferSize() {
return PGProperty.BINARY_TRANSFER_BUFFER_SIZE.getIntNoCheck(properties);
}

public void setBinaryTransferBufferSize(int binaryTransferBufferSize) {
PGProperty.BINARY_TRANSFER_BUFFER_SIZE.set(properties, binaryTransferBufferSize);
}

/*
* Alias methods below, these are to help with ease-of-use with other database tools / frameworks
* which expect normal java bean getters / setters to exist for the property names.
Expand Down
22 changes: 22 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/jdbc/PgConnection.java
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,11 @@ private enum ReadOnlyBehavior {
*/
private final Set<? extends Integer> binaryDisabledOids;

/**
* Buffer size for transfering binary data
*/
private final int binaryTransferBufferSize;

private int rsHoldability = ResultSet.CLOSE_CURSORS_AT_COMMIT;
private int savepointId = 0;
// Connection's autocommit state.
Expand Down Expand Up @@ -286,6 +291,19 @@ public PgConnection(HostSpec[] hostSpecs,
binaryOids.removeAll(binaryDisabledOids);
}

int userBinaryTransferBufferSize = PGProperty.BINARY_TRANSFER_BUFFER_SIZE.getInt(info);
if (userBinaryTransferBufferSize > 0) {
this.binaryTransferBufferSize = userBinaryTransferBufferSize;
} else {
String binaryTransferBufferSizeDefaultValue = castNonNull(
PGProperty.BINARY_TRANSFER_BUFFER_SIZE.getDefaultValue());
LOGGER.log(Level.WARNING, "Unsupported value for binaryTransferBufferSize: {0}, will use "
+ "default value {1}", new Object[]{userBinaryTransferBufferSize,
binaryTransferBufferSizeDefaultValue});
this.binaryTransferBufferSize =
Integer.parseInt(binaryTransferBufferSizeDefaultValue);
}

// split for receive and send for better control
Set<Integer> useBinarySendForOids = new HashSet<>(binaryOids);

Expand Down Expand Up @@ -1246,6 +1264,10 @@ public void setForceBinary(boolean newValue) {
LOGGER.log(Level.FINE, " setForceBinary = {0}", newValue);
}

public int getBinaryTransferBufferSize() {
return binaryTransferBufferSize;
}

public void setTypeMapImpl(Map<String, Class<?>> map) throws SQLException {
typemap = map;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ class PgPreparedStatement extends PgStatement implements PreparedStatement {

private @Nullable TimeZone defaultTimeZone;

private final int binaryTransferBufferSize;

PgPreparedStatement(PgConnection connection, String sql, int rsType, int rsConcurrency,
int rsHoldability) throws SQLException {
this(connection, connection.borrowQuery(sql), rsType, rsConcurrency, rsHoldability);
Expand All @@ -108,6 +110,7 @@ class PgPreparedStatement extends PgStatement implements PreparedStatement {
// TODO: this.wantsGeneratedKeysAlways = true;

setPoolable(true); // As per JDBC spec: prepared and callable statements are poolable by
this.binaryTransferBufferSize = Math.max(64 * 1024, connection.getBinaryTransferBufferSize());
}

final int maximumNumberOfParameters() {
Expand Down Expand Up @@ -1199,7 +1202,7 @@ protected long createBlob(int i, InputStream inputStream,
long oid = lom.createLO();
LargeObject lob = lom.open(oid);
OutputStream outputStream = lob.getOutputStream();
byte[] buf = new byte[4096];
byte[] buf = new byte[(int) Math.min(length, binaryTransferBufferSize)];
try {
long remaining;
if (length > 0) {
Expand Down