Add support for fingerprint generation in FileSource classes #774

major-winter · 2025-05-20T10:34:34Z

Implemented a fingerprint method in various FileSource implementations to improve upload tracking.

Introduced project-specific metadata in fingerprints and updated the upload flow for better modularity and support across different platforms, including React Native.

Implemented a `fingerprint` method in various `FileSource` implementations to improve upload tracking. Introduced project-specific metadata in fingerprints and updated the upload flow for better modularity and support across different platforms, including React Native.

major-winter · 2025-05-20T10:35:54Z

Hi @Acconut,
This is a draft PR—could you please take a look and let me know if I'm heading in the right direction? Thanks!

Acconut

Thank you very much for getting started on this! I think it's going in the right direction and I left a few comments to hopefully help you progress.

Acconut · 2025-05-20T12:15:38Z

lib/browser/fileSignature.ts

@@ -1,13 +1,9 @@
 import type { ReactNativeFile, UploadInput, UploadOptions } from '../options.js'


This entire file can probably be removed now, can't it?

This file has been removed.

Acconut · 2025-05-20T12:16:42Z

lib/node/sources/NodeStreamFileSource.ts

+  // This should likely involve reading the stream and creating a hash/checksum
+  // while preserving the stream's content for later use.
+  fingerprint(options: UploadOptions): Promise<string | null> {
+    throw new Error("fingerprint not implemented for NodeStreamFileSource")


For now, it's best to return null to keep the existing behavior and align with WebStreamFileSource.

I've modified the method to return a promise that resolves to null.

Acconut · 2025-05-20T12:23:05Z

lib/node/sources/PathFileSource.ts

+  fingerprint(options: UploadOptions): Promise<string | null> {
+    if (typeof options.fingerprint === 'function') {
+      return Promise.resolve(options.fingerprint(this._file, options))
+    }


Allowing users to modify the fingerprint is an important aspect to ensure that tus-js-client is customizable. My vision for the current fingerprint option is that its a signature like this:

fingerprint: (file: UploadInput, options: UploadOptions, sourceFingerprint: string | null) => Promise<string | null>

The callback gets the fingerprint from the file source passed and can then decide to modify it (e.g. by appending information) or to overwrite it completely.

Invoking this callback is best handled in the Upload class, so it doesn't have to be done in every implementation of FileSource. Does this make sense?

Let me summarize how I understand the direction:

We’ll extend the fingerprint option signature to include a sourceFingerprint parameter. Just to confirm: is this sourceFingerprint the value returned by the fingerprint method of the specific FileSource being used?

The fingerprint callback can then decide whether to use the sourceFingerprint as-is or modify it (e.g., by appending a project ID). Could you clarify what criteria should guide this decision?

This callback will be implemented centrally in the BaseUpload class instead of the Upload, so that both the Node and Browser environments can share the logic — and individual FileSource classes won’t need to handle user-provided fingerprint logic themselves.

From what I understand, the fingerprint option exists to let users override the default behavior and provide their own implementation. Since each FileSource can still generate a fingerprint based on its specific type, I believe the fingerprint option in the UploadOptions interface should be made optional.

We’ll extend the fingerprint option signature to include a sourceFingerprint parameter. Just to confirm: is this sourceFingerprint the value returned by the fingerprint method of the specific FileSource being used?

Yes, that's correct. sourceFingerprint can also be null if FileSource didn't supply one, e.g. for streams, allowing the user to still supply a fingerprint if they desire to do so.

The fingerprint callback can then decide whether to use the sourceFingerprint as-is or modify it (e.g., by appending a project ID). Could you clarify what criteria should guide this decision?

As you mentioned, they might want to scope uploads per project and thus embed information identifying the project in the fingerprint.

From what I understand, the fingerprint option exists to let users override the default behavior and provide their own implementation. Since each FileSource can still generate a fingerprint based on its specific type, I believe the fingerprint option in the UploadOptions interface should be made optional.

fingerprint in UploadOptions shouldn't be made optional. UploadOptions represent the type after default values are filled, where fingerprint is always set (if I remember correctly). That being said, the Upload constructor accepts Partial<UploadOptions>, so fingerprint is already optional from the user's perspective.

Thanks for the clarification! Please take a look at the updated changes.

Acconut · 2025-05-20T12:23:20Z

lib/options.ts

@@ -51,6 +51,8 @@ export type UploadInput =
 export interface UploadOptions {
  endpoint?: string

+  projectId?: string


Can you explain what this option is about?

I intend to use it to distinguish the same file across multiple projects. However, this property is better suited as part of the metadata rather than being placed here.

I don't think such a specific option is necessary, as users can just append a project ID (or any other data) using the fingerprint callback to modify the fingerprint from the file source.

Thanks for the clarification.

This field has been removed.

Acconut · 2025-05-20T12:24:35Z

lib/sources/WebStreamFileSource.ts

@@ -39,6 +39,8 @@ export class WebStreamFileSource implements FileSource {

  private _done = false

+  private _stream: ReadableStream


This property doesn't seem used. Can it be removed?

Yes, I'll remove it.

this property has been removed.

* Feat: Set Upload-Length if we know the length and it's deferred from the server. And allow resumeUpload if Upload-Defer-Length is set from server. * Feat: Rename _deferred to _uploadLengthDeferred, set value in constructor * Fix: Missed replace * Fix: Total size not calculated correctly when not last request * test: Upload-Defer-Length behaviour * Send length at end of upload, simplifying logic * Remove redundant test and improve other test --------- Co-authored-by: Marius Kleidl <marius@transloadit.com>

Bumps the npm group with 6 updates: | Package | From | To | | --- | --- | --- | | [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `22.15.3` | `22.15.29` | | [@arethetypeswrong/cli](https://github.com/arethetypeswrong/arethetypeswrong.github.io/tree/HEAD/packages/cli) | `0.17.4` | `0.18.1` | | [jasmine](https://github.com/jasmine/jasmine-npm) | `5.7.0` | `5.7.1` | | [jasmine-core](https://github.com/jasmine/jasmine) | `5.7.0` | `5.7.1` | | [puppeteer](https://github.com/puppeteer/puppeteer) | `24.7.2` | `24.9.0` | | [rollup](https://github.com/rollup/rollup) | `4.40.1` | `4.41.1` | Updates `@types/node` from 22.15.3 to 22.15.29 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Updates `@arethetypeswrong/cli` from 0.17.4 to 0.18.1 - [Release notes](https://github.com/arethetypeswrong/arethetypeswrong.github.io/releases) - [Changelog](https://github.com/arethetypeswrong/arethetypeswrong.github.io/blob/main/packages/cli/CHANGELOG.md) - [Commits](https://github.com/arethetypeswrong/arethetypeswrong.github.io/commits/@arethetypeswrong/cli@0.18.1/packages/cli) Updates `jasmine` from 5.7.0 to 5.7.1 - [Release notes](https://github.com/jasmine/jasmine-npm/releases) - [Changelog](https://github.com/jasmine/jasmine-npm/blob/main/RELEASE.md) - [Commits](jasmine/jasmine-npm@v5.7.0...v5.7.1) Updates `jasmine-core` from 5.7.0 to 5.7.1 - [Release notes](https://github.com/jasmine/jasmine/releases) - [Changelog](https://github.com/jasmine/jasmine/blob/main/RELEASE.md) - [Commits](jasmine/jasmine@v5.7.0...v5.7.1) Updates `puppeteer` from 24.7.2 to 24.9.0 - [Release notes](https://github.com/puppeteer/puppeteer/releases) - [Changelog](https://github.com/puppeteer/puppeteer/blob/main/CHANGELOG.md) - [Commits](puppeteer/puppeteer@puppeteer-v24.7.2...puppeteer-v24.9.0) Updates `rollup` from 4.40.1 to 4.41.1 - [Release notes](https://github.com/rollup/rollup/releases) - [Changelog](https://github.com/rollup/rollup/blob/master/CHANGELOG.md) - [Commits](rollup/rollup@v4.40.1...v4.41.1) --- updated-dependencies: - dependency-name: "@types/node" dependency-version: 22.15.29 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm - dependency-name: "@arethetypeswrong/cli" dependency-version: 0.18.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm - dependency-name: jasmine dependency-version: 5.7.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm - dependency-name: jasmine-core dependency-version: 5.7.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm - dependency-name: puppeteer dependency-version: 24.9.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm - dependency-name: rollup dependency-version: 4.41.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [tar-fs](https://github.com/mafintosh/tar-fs) from 3.0.8 to 3.0.9. - [Commits](mafintosh/tar-fs@v3.0.8...v3.0.9) --- updated-dependencies: - dependency-name: tar-fs dependency-version: 3.0.9 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Implemented a `fingerprint` method in various `FileSource` implementations to improve upload tracking. Introduced project-specific metadata in fingerprints and updated the upload flow for better modularity and support across different platforms, including React Native.

implement a centralized implementation for the fingerprint option removed the implementation of the fingerprint option in various sources

…classes

- added a centralized fingerprint handling method in the BaseUpload - deleted browser/fileSignature and node/fileSignature

…major-winter/tus-js-client into feat/extend_FileSource_with_fingerprint

Acconut · 2025-06-13T20:01:20Z

Thanks for the updates. I'll have a look at them in 1-2 weeks.

Acconut requested changes May 20, 2025

View reviewed changes

Vija02 and others added 10 commits May 21, 2025 14:41

resolve rebase conflict

906105c

implement ArrayBufferView fingerprint

f983c63

implement a centralized implementation for the fingerprint option removed the implementation of the fingerprint option in various sources

refactor: remove projectId from fingerprint generation in FileSource …

18fe753

…classes

feat: extend BaseUpload with default fingerprint

341742f

- added a centralized fingerprint handling method in the BaseUpload - deleted browser/fileSignature and node/fileSignature

Merge branch 'feat/extend_FileSource_with_fingerprint' of github.com:…

efda4bf

…major-winter/tus-js-client into feat/extend_FileSource_with_fingerprint

refactor: remove unused stream property from WebStreamFileSource

cfc3111

		@@ -1,13 +1,9 @@
		import type { ReactNativeFile, UploadInput, UploadOptions } from '../options.js'

		@@ -39,6 +39,8 @@ export class WebStreamFileSource implements FileSource {

		private _done = false

		private _stream: ReadableStream

Add support for fingerprint generation in FileSource classes #774

Are you sure you want to change the base?

Add support for fingerprint generation in FileSource classes #774

Uh oh!

Conversation

major-winter commented May 20, 2025

Uh oh!

major-winter commented May 20, 2025

Uh oh!

Acconut left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

major-winter May 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Acconut commented Jun 13, 2025

Uh oh!

Uh oh!

major-winter May 24, 2025 •

edited

Loading