Introduce file storage for export download #130
Assignees
Labels
feature
use for describing a new feature to develop
solved
use to identify issue that has been solved (must be linked to the solving PR)
technical improvement
Technical refactor or improvement is needed
Milestone
Comments
SamuelHassine
added
feature
SamuelHassine
pushed a commit
that referenced
this issue
Oct 22, 2019
…chitecture (#130) * Starting work around minio * Introduce File manager component and use it in report * Adapt file manager UI * Prepare the export api * Refactor to move import and export files at stix entity level * Adapt view and backend to log file actions * Add file management in thread actors * Refactor functions to move everything we need at StixDomainEntity level * Remove unused info and add capacity to delete in progress export after 5 min * Adapt deletion + fix dependencies errors from yarn control * Start transformation of the worker. Provide API round queue management * Introducing Jobs and failure reporting * Re activate auto refresh * Add jobs management, massive update * Fix relayjs query typo * Introduce global import menu. Adapt the entity import and export * Fixing tests by replacing the matchAll function * Handle deletion of global imports * Remove unused import * Adapt usage of interface, update of dependencies * [frontend/api] Introduce key for internal_id and stix_id for avoid race conditions (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * Adapt work and jobs to use elastic. Worker changed to report jobs * Cleanup the grakn schema * Fix comments * Add delete button in observable enrichment * Refactoring of createdByRef, markingDef and KillChain functions * Fix duplicate and adapt readtx * Fix deepScan warning * Fix deepScan warning * Adapt schema for internal_id_key * Adapt for internal_key * [api/frontend] Enhance display and fix some bugs
SamuelHassine
pushed a commit
that referenced
this issue
Oct 22, 2019
…chitecture (#130) * Starting work around minio * Introduce File manager component and use it in report * Adapt file manager UI * Prepare the export api * Refactor to move import and export files at stix entity level * Adapt view and backend to log file actions * Add file management in thread actors * Refactor functions to move everything we need at StixDomainEntity level * Remove unused info and add capacity to delete in progress export after 5 min * Adapt deletion + fix dependencies errors from yarn control * Start transformation of the worker. Provide API round queue management * Introducing Jobs and failure reporting * Re activate auto refresh * Add jobs management, massive update * Fix relayjs query typo * Introduce global import menu. Adapt the entity import and export * Fixing tests by replacing the matchAll function * Handle deletion of global imports * Remove unused import * Adapt usage of interface, update of dependencies * [frontend/api] Introduce key for internal_id and stix_id for avoid race conditions (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * [api] Fix migations if starting fresh (#217) * Adapt work and jobs to use elastic. Worker changed to report jobs * Cleanup the grakn schema * [frontend] Working on diagram! * Fix comments * Add delete button in observable enrichment * [frontend] Working on graph * Upgrade dependencies * Refactoring of createdByRef, markingDef and KillChain functions * Fix duplicate and adapt readtx * Fix deepScan warning * Fix deepScan warning * [frontend] Handle relation to relation in the knowledge graph * Adapt schema for internal_id_key * Adapt for internal_key * [frontend] Handle relation to relation * [frontend] Graph is now working (#34) * [frontend] Graph is now working (#34) * [frontend] Add marking definitions to StixDOmainEntity edition * [frontend] Add marking definitions to StixDOmainEntity edition * [frontend] Fix relation actions icons and entities update * [api/frontend] Enhance display and fix some bugs * [frontend] Refactor the views of files management
SamuelHassine
added
the
solved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem to Solve
Currently, after asynchronous generation of an export file, the file is stored as a base64 blob in Grakn. Grakn has native limitation in the attributes length.
Current Workaround
Use current implementation.
Proposed Solution
Introduction an S3-compliant storage for files, Minio if no other backend is provided.
Additional Information
None.
The text was updated successfully, but these errors were encountered: