Skip to content

Commit

Permalink
Beta (#14)
Browse files Browse the repository at this point in the history
  • Loading branch information
georgebabarus committed Sep 7, 2020
1 parent b82d740 commit a0e7c81
Show file tree
Hide file tree
Showing 17 changed files with 411 additions and 183 deletions.
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#

# You can set these variables from the command line, and also
# from the environment for the first two. venv/bin/python -msphinx
# from the environment for the first two ./venv/bin/python -msphinx
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
Expand Down
32 changes: 23 additions & 9 deletions docs/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,31 @@
Magento filesystem component extension
======================================

Magento 2.4 version almost has already in place filesystem abstraction needed to implement and integrate new filesystem for core or custom modules.
Introduction
------------

Magento 2.4 version almost has already in place filesystem abstraction needed to implement and integrate new filesystem (eg: remote filesystems like Amazon Simple Storage Service - S3) for core or custom modules.

There are few touch points that are not abstracted enough as you could see in current documentation, and this storage extensions covers this, to allow cloud object storage service integration into Magento 2.

There are few touch points that are not abstracted enough as you could see in current documentation, and this storage extension covers this to allow Cloud object storage service integration into Magento 2.
In many cases existing Database files storage could not be useful and definitively is not a optimal implementation since it is doing a sync back to local filesystem on missing resource.

Read the `documentation <https://magento-filesystem-extension-docs.readthedocs.io>`_ to see some of the key advantages of using this Magento 2 extension to integrate with various cloud file storage services in a platform agnostic manner.
The scope of this extensions is to extract static files storage as microservice for Magento platform, decoupling files storage system from compute system.

Read the `documentation <https://docs.magento.asset42.com>`_ to see some of the key advantages of using this Magento 2 extensions to integrate with various cloud file storage services in a platform agnostic manner.

What is covered with this extensions?
---------------------------------------

* Decouple file storage system form compute systems and scale them independently.
A driver will be basically a class implementing (Magento\Framework\Filesystem\DriverInterface) basic operations: read/write files or directory.

Install all Bb_Storage, Bb_StorageOverwrites, Bb_StorageCms, Bb_StorageCatalog, Bb_StorageDownloadable modules.

Install one of the filesystem driver module eg: Bb_StorageS3.

A driver will be basically a class implementing basic operations read/write on files or directories. (see: Magento\Framework\Filesystem\DriverInterface)

Configure directory mapping to save and serve files directly from storage service.

* Mapping of any media sub-directory to various filesystem services.

Expand All @@ -31,12 +44,13 @@ Currently there are 6 Magento modules developed to achieve fully abstracted file
* Bb_Storage is the core module implementing most of business logic:

* directories mapping
* adding custom media directories for new modules
* adding custom media directories for new modules (eg: you want to store some reports in Azure Blob storage, you can configure a new directory for this report)
* use the media storage system of your choice for any given directory or subdirectory
* image resize in-place without sync back to local filesystem the file (this require having the same configuration for main directory and destination of resized files)
* this module allows only new modules to use this features, none of Magento core functionality is touched
* :term:`OOB` this module can be used in custom modules, none of Magento core features are touched
* allow new directory configuration for custom development

Bb_StorageOverwrite
Bb_StorageOverwrites

* allow Bb_Storage features on built-in media directories

Expand All @@ -57,10 +71,10 @@ Bb_StorageDownloadable

Bb_StorageS3

* implementation of Amazon S3 api as a Magento filesystem driver
* implementation of Amazon S3 like api as a Magento filesystem driver


Author
======
------

`George Babarus <https://github.com/georgebabarus>`_
26 changes: 17 additions & 9 deletions docs/source/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,27 +26,35 @@ Other Changes

.. glossary::

v1.0.0

Production ready Bb_Storage - 1 September 2020
* custom directories could be configured to use custom media storage
* Magento Core directories could be mapped to new media storage, bug multiple mapping for same directory is not production ready. Check it and report any issue.
* For now is advised to use same media storage for all subdirectories except for downloadable files for downloadable products witch could be stored in other filesystem location.


v0.2.0

Beta version - 1-July-2020
* Downloadable products
* Multiple buckets by folder mapping

v0.1.0
v0.1.0 - 15-June-2020

* This version contain only minimum code base to prove that Cloud storage service could easily and fully integrate in Magento 2 without any need for disk storage on application server.

Alpha version - 15-June-2020
* catalog product images
* WYSIWYG images managed directly in cloud storage
* Sync existing images to cloud storage

New Features
-------------
* Media storage driver for Amazon S3
* Magento core changes to improve filesystem abstraction
#New Features


* Media storage driver for Amazon S3
* Magento core changes to improve filesystem abstraction

#Other Changes

Other Changes
--------------
* Documentation created
* Documentation created

2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
'sphinx_sitemap',
'sphinxcontrib.images'
]
html_baseurl = 'https://magento-filesystem-extension-docs.readthedocs.io/'
html_baseurl = 'https://docs.magento.asset42.com/'

# html_baseurl = 'https://magento-file-storage-docs.babarus.ro/'
# sitemap_url_scheme = "{link}"
Expand Down
16 changes: 10 additions & 6 deletions docs/source/custom-development/extensibility.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ See: bb/mage-file-storage/dev/sample-files/env.php

.. code-block:: php
namespace Bb\Storage\Framework\Filesystem;
namespace Bb\Storage\Api\Framework\Filesystem;
use Bb\Storage\Model\FileInterface;
use Bb\Storage\Api\Data\FileInterface;
/**
* Class Driver
Expand All @@ -44,18 +44,21 @@ See: bb/mage-file-storage/dev/sample-files/env.php
interface DriverInterface extends \Magento\Framework\Filesystem\DriverInterface
{
/**
* Alternative for read method which is not returning the headers
* On external filesystems is easier to extract content and headers in the same call
*
* @param string $path
* @param $flag
* @param $context
* @return FileInterface
*/
public function fileGetObject(string $path, $flag = null, $context = null): FileInterface;
public function readFileObject(string $path, $flag = null, $context = null): FileInterface;
/**
* @param string $path
* @return array
*/
public function headers(string $path): array;
public function getHeaders(string $path): array;
/**
* @param $absolutePath
Expand All @@ -72,10 +75,10 @@ See: bb/mage-file-storage/dev/sample-files/env.php
public function getUrl($path): string;
/**
* @param $originalBaseUrl
* @param $baseUrl
* @return string
*/
public function rewriteBaseUrl($originalBaseUrl): string;
public function rewriteBaseUrl($baseUrl): string;
/**
* @return bool
Expand All @@ -94,4 +97,5 @@ See: bb/mage-file-storage/dev/sample-files/env.php
public function requireFallbackDirectory(): bool;
}
All configuration available and explained :ref:`here<configuration/connection>` are available for the newly implemented driver.
30 changes: 10 additions & 20 deletions docs/source/extension/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,64 +30,54 @@ Magento extension architecture
Upload images in admin area
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Uploading files form user interfaces or programmatically at product should be compatible with any customization as log as is using Magento standard interfaces.
Uploading files form user interfaces or programmatically should be compatible with any customization as log as is using Magento standard interfaces.

Nevertheless the business logic is not changed, and cloud storage services are added using regular/local filesystem interface.
Nevertheless the business logic is not changed, and cloud storage services are integrated using regular/local filesystem interface.

.. image:: _static/architecture/upload-image.png
:alt: Upload image for product or CMS blocks

.. note::
Uploading products attachments for downloadable products works just like uploading the product image showed in the above schema.

.. note::
Features: :term:`WOOB` :term:`v1.0.0`

Resized image delivery
^^^^^^^^^^^^^^^^^^^^^^

Frontend resized image delivery
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Resizing images could be performed on magento /media path similar with the case when files were on disk. The difference is that for mapped directories to a cloud storage, now, will read the files using the proper driver.

Resized images could be delivered directly from storage system after creating the resized image in the main request or return a proxy url responsible to return the image if not exist.
A better way could be to have this path behind a reverse proxy configuration. Sample configuration is provided for nginx server.

The proxy can be implemented as follow:

* nginx config to request it from storage system and create a fallback request in case of error on Magento resize script
* in case you don't have access to a web server proxy configuration there is a option to return it directly from default Magento image resize script.
* try to deliver the file from storage system by a proxy_pass call
* when missing try to process the image using Magento standard path /media/*
* save result to storage system
* return to user

.. image:: _static/architecture/frontend-image-delivery.png
:alt: Upload image for product or CMS blocks

.. note::
:term:`SWSCNO` :term:`v1.0.0`


Frontend image delivery for original images
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Original images could be delivered directly from storage system, or the CDN in front of it, by configuring the base media url in admin configuration under Store -> Configuration.

.. note::
:term:`WOOB` :term:`v1.0.0`

Infrastructure architecture
============================

Ideal infrastructure setup
--------------------------

.. warning::
For now all images within the media folder are saved in the same bucket. Having multiple buckets will allow user to use this extension for downloadable products :term:`future work`

.. image:: _static/architecture/basic-infrastructure-architecture.png
:alt: Basic infrastructure architecture


Possible optimization
---------------------

.. warning::
This is :term:`future work` and will be detailed soon.

.. image:: _static/architecture/infrastructure-architecture-improved.png
:alt: Infrastructure architecture improved

Expand Down

0 comments on commit a0e7c81

Please sign in to comment.