Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NXP-30551: don't delete blob immediately after a new key has been computed asynchronously (10.10) #4935

Closed
wants to merge 1,379 commits into from

Conversation

efge
Copy link
Member

@efge efge commented Sep 9, 2021

efge and others added 30 commits November 17, 2020 00:31
Update 10.10-HF36-SNAPSHOT to 10.10-HF37-SNAPSHOT
Add unit test for snapshot dependency resolution
Using the old consumer.poll(long) ensure that the rebalancing is done,
removing the risk of commit failure during a rebalance
If the property type is different from the property value, but both
relate to Number, allow writing the property value converted to the
property type.
This cat the content of a WorkManager queue using a CSV output that
contains Work class, category and title
Update 10.10-HF37-SNAPSHOT to 10.10-HF38-SNAPSHOT
If the bulk scroller materialize more than 1m ids, records are sent
downstream to prevent a possible OOM.
troger and others added 27 commits August 5, 2021 11:30
Update 10.10-HF51-SNAPSHOT to 10.10-HF52-SNAPSHOT
Among default or elastic.
This is useful to set the right scroller on a `BulkCommand`, if its query is defined by a page provider.
If the page provider is relying on Elasticsearch, use the Elasticsearch scroller to run the bulk command query.
The bulk action is going to update content/length value when using S3BinaryManager and MongoDB
the length is taken from S3 summary and should not download the binary, no listener should be
triggered when updating the content/length of the document.
Update 10.10-HF52-SNAPSHOT to 10.10-HF53-SNAPSHOT
@efge efge closed this Sep 9, 2021
@nuxeojenkins
Copy link
Collaborator

View issues in JIRA:

  • NXP-29861: Fix random failure of testArrayConcurrentPushToEmpty on MongoDB
  • NXP-29738: Fix underlying cause of 500 during S3 upload
  • NXP-29759: Make the task endpoint use the UserManagerResolver
  • NXP-29868: Upgrade Tomcat to 9.0.40
  • NXP-29794: Resolve downstream Nuxeo Package versions from the requested resolved versions
  • NXP-29833: Check ACL name for null
  • NXP-29026: Fix Export CSV Action in Retention Search page
  • NXP-29253: Bulk actions on Retention Search menu are not correctly handled
  • NXP-29836: More robust blob upload to GCP
  • NXP-29677: Increase default number of partitions for Bulk Action and StreamWorkManager queues
  • NXP-29044: Kafka CommitFailedException while respecting poll interval generating duplicate processing
  • NXP-29840: Making writing Double property in JSON marshallers support Long values
  • NXP-29931: Fix SimpleManagedBlob.getFile
  • NXP-29924: Async automation chain call doesn't return the blob
  • NXP-29938: ZIP preview should not write to disk
  • NXP-29939: UserProfile enricher should write all schemas
  • NXP-29943: Invalid CloudFront link for versions
  • NXP-22976: Fix bookmarkable search URL on selection
  • NXP-23110: Wrong saved search when clicking Search tab
  • NXP-28903: Rich text editor inline images lost when saving
  • NXP-29946: Upgrade Tomcat to 9.0.41
  • NXP-29945: Provide a tool to analyze the content of the default WorkManager queue
  • NXP-29823: Fix update of a string in a multivalued complex with the "Single Directory Suggestion" widget
  • NXP-29928: Fix label for Event based input in Retention Rule page
  • NXP-29587: Bulk scroller should automatically turn to produceImmediate
  • NXP-29585: Have a way to limit the documents of a Bulk command
  • NXP-29959: Have a way to limit the documents of a Bulk command (10.10)
  • NXP-29991
  • NXP-30019: Align connect-client version
  • NXP-25485: Take into account the local configuration to filter the document types available for creation in WebUI
  • NXP-29810: Document.Mail should respect the setting of the Nuxeo SMTP services state
  • NXP-29865: Fix JSF Email document notification to mail all members of a group
  • NXP-30010: Improve concurrent updates removing elements from lists on MongoDB
  • NXP-29867: Fix custom HTTP code thrown by event listener being ignored when DnD'ing a file in a folder in the Web UI
  • NXP-29983: Improve constraint violation error message in REST API
  • NXP-30048: Improve concurrent delete of proxies
  • NXP-29723: Implement event information for workflowTaskReassigned and workflowTaskDelegated
  • NXP-30059: Don't use underscores in custom HTTP headers
  • NXP-30060: Backport NXP-28743 on 10.10
  • NXP-29966: Fix recomputation processes when creating a version for documents with 'Video' facet
  • NXP-30081: Add Idempotency Key support to RequestController (10.10)
  • NXP-30037: Fix CSV export for mixed content with Vocabulary metadata
  • NXP-30090: Fix algorithm to unpublish a document
  • NXP-30045: Adjust the MongoDB query maxTime with transaction timeout
  • NXP-30127: Fix plantuml rendering of Nuxeo Stream Topology in 10.10
  • NXP-28397: Fix plantuml rendering of Nuxeo Stream Topology
  • NXP-30134: Fix MongoDB query maxTime
  • NXP-28512: RenderDocument & RenderDocumentFeed not working with REST
  • NXP-30124: Upgrade Tomcat to 9.0.43
  • NXP-30120: Use document user change token for WOPI file version
  • NXP-30078: Fix compatibility of CSV export with fields including carriage return characters
  • NXP-30079
  • NXP-30111: Fix LogEntryProvider#queryLogs with QueryBuilder and repeated fields
  • NXP-29516: Allow efficient search by blob key
  • NXP-30044: Async blob digest computation
  • NXP-30043: Use new blob key for old blob used concurrently with async digest computation
  • NXP-30119: Process blobDigestUpdated event in Nuxeo Drive Server
  • NXP-30168: Execute batch upload completion outside of a transaction
  • NXP-28660: Avoid popup on console startup
  • NXP-30173: Allow logging costly S3 downloads
  • NXP-30174: MimetypeRegistryService should not download large files
  • NXP-30193: Fix cache access issue in CachingBlobStore
  • NXP-25667: Add RemoteIpValve to default Tomcat configuration
  • NXP-30100: Nuxeo health check should trace failure
  • NXP-30130: Command line executor must apply a timeout
  • NXP-30211
  • NXP-30135: Extend cluster-wide lock to Workflow model import
  • NXP-30056: Use OfficePreviewer to preview RTF documents
  • NXP-30182: Increase Bulk Status TTL on completion with error
  • NXP-28450: Allow configuration of Signature Algorithm for SAML
  • NXP-30213: Fix Redis key/value store fetch of empty key list
  • NXP-30231: Blob digest must be synthesized properly
  • NXP-30220: Add mime type for psb blobs
  • NXP-26779: Inconsistent mimetype in webui file drag and drop
  • NXP-30246: Fix bulk delete of Vocabulary metadata with spreadsheet editor
  • NXP-29032: Retention Duration fields shouldn't show individually as "required"
  • NXP-30171: Return http status code based on exception thrown
  • NXP-30253: Upgrade Tomcat to 9.0.44
  • NXP-30239: Delay the start of Quartz scheduler to avoid database constraint violations
  • NXP-25716: Simplify fulltext extraction
  • NXP-30188: Dedicate default WorkManager queue for Custom Works
  • NXP-30238: Security policy to filter annotations is not working in NEV
  • NXP-30205: Fix how enricher priority is taken into account.
  • NXP-30233: Bulk Scroller should complete command in error when query times out
  • NXP-30083: Fix CAS authentication anonymous client change of behavior introduced with NXP-23246
  • NXP-30234: Local Avro Schema Store should be shared and dynamic
  • NXP-30270: ComponentManager listeners should be called within a try/catch
  • NXP-30241: Fix upload of a Google Drive shared file
  • NXP-30274: Timeout on CommandLine Executor fails for Windows
  • NXP-30240: Prevent IllegalArgumentException during retry mechanism of AbstractWork
  • NXP-30269: Remove picture migration at startup
  • NXP-30244: Prevent Transaction Timeout during updateThumbnailListener
  • NXP-30271: Key/Value Store GC shouldn't remove cache temporary files
  • NXP-30075: Map VersionNotModifiableException to a bad request 400
  • NXP-30052: Return a HTTP 400 (bad request) when creating a document with a non-existing type
  • NXP-29626: Fix webengine multipart form fields retrieval when uploading files
  • NXP-29641: Fix component manager events on pending registrations
  • NXP-30123
  • NXP-30309: Add a parameter for enabling aliases
  • NXP-30327: Fix Connect Report unit test
  • NXP-30319: Upgrade Tomcat to 9.0.45
  • NXP-30247: When exifTool fails, file:content gets deleted
  • NXP-30128: Fix inheritance of local configuration's allowed subtypes in subfolders of configured workspace
  • NXP-30312: Prevent creation of empty thumbnail on audio file
  • NXP-30328: Fix enabling of async digest on first start
  • NXP-30314: Kafka topic not available after creation
  • NXP-30311: Component that fails to start should be unregistered
  • NXP-30333: Key/Value Store GC shouldn't remove cache intermediate files
  • NXP-29583: Allow certificates from private certificate authorities to authenticate against the elasticsearch passthrough
  • NXP-28936: Fix display of aggregates in Summary/Edit views of Saved searches
  • NXP-30304: Do not log password when failing to connect to Kafka
  • NXP-30265: Add a quiet option to nuxeoctl not to log to nuxeoctl.log
  • NXP-30250: Fix null version on doc inside folder copied from clipboard
  • NXP-30122: Fix thumbnails computation when importing a large PDF with S3 BlobProvider
  • NXP-30357: Correct mime-type in S3DirectBatchHandler
  • NXP-30395
  • NXP-30410: Upgrade Tomcat to 9.0.46
  • NXP-30407: Backport NXP-27911 to 10.10
  • NXP-30412: S3 GC must keep blobs added during GC
  • NXP-30402: Fix converter 'image2pdf'
  • NXP-30200: Manage the case where workflow definitions have changed
  • NXP-30332: Fix matching of subconverter's source MIME type
  • NXP-30260
  • NXP-30427: Retry file move with NoSuchFileException but the file exists
  • WEBUI-350: Add missing file type binding in csv document import element
  • NXP-30401: Bulk Zip Download doesn't create Audit entry for each item downloaded
  • NXP-30419: Downloadable status should rely on the restore expiration time
  • NXP-30399: Fix Transaction timeout during nuxeoDriveFileSystemDeletionListener
  • NXP-30266: Fix elasticsearch indexing of property 'ecm:proxyVersionableId' for proxies
  • NXP-30437: Fix GroupChangesTestSuite.testGroupChangesOnSyncRoot randomly failing with PostgreSQL
  • NXP-30457: Fix NPE when loading corrupted NuxeoPackage
  • NXP-30450: Support filtering when reprocessing DLQ Work
  • NXP-30441: Upgrade three.js dependency (backport to 10.10)
  • NXP-30348: Return a HTTP 404 (not found) when handling uploads with a missing associated file
  • NXP-30417: Check immediate parent when emptying trash
  • NXP-28861: Add an index on the nuxeo.aceinfo:id field for MongoDB repository
  • NXP-30219: Fix re-indexing of proxy when parent folder of target document is moved to the trash
  • NXP-30062: Handle mails with attachments without Content-Disposition
  • NXP-30434: User Favorites Not Showing in WEB UI
  • NXP-30360: Fix search results not being sorted when a quick filter is used
  • NXP-30145: Add a parameter not to save the document in the operation 'Document.SetMetadataFromBlob'
  • NXP-30444: Fix Nuxeo 10.10 build
  • NXP-30382: Tune recomputeThumbnail bulk action to avoid transaction timeout or stream failure
  • NXP-30438: Compute update thumbnail in a dedicated transaction per documents
  • NXP-30326: Fix long-running thumbnail generation for large office files
  • NXP-30477: Add mime type for mts blob
  • NXP-30475: Upgrade Tomcat to 9.0.48
  • NXP-30243: Add index on collection:documentIds to optimize RemovedCollectionWork
  • NXP-30432: Add MongoDB indexes on fetch annotation requests
  • NXP-30281: Add a MongoDB index for Drive getChangeSummary
  • NXP-30215: Add indexes for user and group directories
  • NXP-30470: Use longer Elastic timeout for write operation
  • NXP-30456: Allow User to specify SSL protocol
  • NXP-29089: Fix TestPictureResize tests with ImageMagick 6.9.10
  • NXP-30294: Possible OOM on XLS fulltext extraction with POI
  • NXP-30368: Take into account fulltext maxSize when computing text in FulltextExtractorWork
  • NXP-30465: Fix temporary blob move to transient storage when parent folders are deleted by the GC
  • NXP-30192: Handle permission restriction in DocumentModelResolver
  • NXP-30369: Process the picture views in a transaction with a dedicated timeout
  • NXP-30170: Don't recreate workflows at startup
  • NXP-30469: Add explicit timeout on Video conversion
  • NXP-30077: VideoConversionWork should handle unknown mime-type
  • NXP-30482: Register a default OAuth2 client for Nuxeo Drive
  • NXP-30500: Prevent concurrent exception during video conversion
  • NXP-30505: Upgrade Tomcat to 9.0.50
  • NXP-30463: Fix "The specified block list is invalid" error with Azure
  • NXP-29078: Stream AuditLogWriter processing cannot be parallelized
  • NXP-30479: Elasticsearch.BulkIndex should work in Multi-repository (backport)
  • NXP-28642: Fix visual bug while importing more than 1 doc file
  • NXP-30483: Fix wildcard behavior in sourceMimeType
  • NXP-29261: Allow configuration of repository/directory indexes to add, even after initial creation
  • NXP-30533: Fix text expansion of default values
  • NXP-30342: Allow for granted users to delete and trash content under retention or legal hold
  • NXP-30523: Fix WOPI thread abusive naming
  • NXP-30526: Fix picture views generation with auto versioning
  • NXP-30534: Spreadsheet editor should work with Content Security Policy
  • NXP-30484: Prevent Stream failure when invoking setProperties bulk action with invalid values
  • NXP-30515: Fix NPE when retrieving tasks for User
  • NXP-27596: Web Engine module 'EasyShare' should be overridable
  • NXP-30513: Return JSON with error message instead of HTML response for Generic HTTP 500 error
  • NXP-30499: Bulk computation should not fail when the bulk command does not exist
  • NXP-30521: Log Nuxeo version and HF on startup
  • NXP-30556: Upgrade Tomcat to 9.0.52
  • NXP-30569: StudioHotReload operation to uninstall all studio package version
  • NXP-30571: Upgrade ant-assembly-maven-plugin to 2.1.5
  • NXP-30541: Use a dedicated Work queue for ACL Propagation
  • NXP-30536: Provide options at nuxeo.conf level to tune Bulk Re-indexing
  • NXP-30366: S3 blob storage: adding depth-based prefixes to object names
  • NXP-30572: Fix permission filtering for administrators
  • NXP-30570: Bulk command scroller should be aligned on the underlying page provider
  • NXP-30573: Create a Bulk Action to populate content/length value
  • NXP-30551: Add a configurable TTL to control how long a blob remains in the storage after async key generation

@efge efge deleted the feature-NXP-30551-delay-blob-delete-10.10 branch September 9, 2021 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet