Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media field uses incorrect MIME type for mov/MOV video previews #38333

Closed
crystalenka opened this issue Jul 28, 2022 · 16 comments
Closed

Media field uses incorrect MIME type for mov/MOV video previews #38333

crystalenka opened this issue Jul 28, 2022 · 16 comments

Comments

@crystalenka
Copy link
Member

Steps to reproduce the issue

Requirements: com_media must be configured to allow mov or MOV video files.

  1. Find a component that supports videos for media fields. (Can't make a custom media field do this, as far as I can tell...though I'm about to see if I can fix that.)
  2. Select a video with the mov or MOV extension.
  3. Close the modal so that the video shows up the preview.

Expected result

Video displays correctly.

Actual result

Video does not display, but the video container with media buttons does.
Screen Shot of the Joomla media field, showing a blank background with a play button overlaid.

When inspecting the element, I see that it includes the source with type="video/MOV" which is an invalid mime type. The correct file type for this is video/quicktime.

If you inspect the element and change the type to the correct value, or remove the attribute altogether, you can successfully play the video:
Screen Shot of Joomla media field, this time showing a video successfully playing with the media buttons shown around it.

System information (as much as possible)

Joomla! 4.1.5 Stable [ Kuamini ] 21-June-2022 14:00 GMT
PHP 8.0.21

Additional comments

If I knew where the mime types were coming from I'd submit a PR, but I can't figure it out. It's a relatively simple fix.

@crystalenka
Copy link
Member Author

@dgrammatiko You do a lot of stuff with the media manager I think? Where do the mime types come from?

@dgrammatiko
Copy link
Contributor

It seems that it is correct in the field:

default="image/jpeg,image/gif,image/png,image/bmp,image/webp,audio/ogg,audio/mpeg,audio/mp4,video/mp4,video/mpeg,video/quicktime,video/webm,application/msword,application/excel,application/pdf,application/powerpoint,text/plain,application/x-zip"

@crystalenka
Copy link
Member Author

crystalenka commented Jul 28, 2022

Where is this coming from when it's rendered then?

Screen Shot of web inspector view showing a snipped that says type equals video/MOV

Edited to add: rendered in the back end edit view

@dgrammatiko
Copy link
Contributor

I’m on my way to the office, I’ll have a look when I get there

@crystalenka
Copy link
Member Author

Thank you!

@brianteeman
Copy link
Contributor

is it in the joomla-media-field custome element?

@crystalenka
Copy link
Member Author

is it in the joomla-media-field custome element?

Yes

@dgrammatiko
Copy link
Contributor

So it's coming from this line:

previewElementSource.type = `video/${ext}`;

which obviously will give false mime type as it tries to infer that from the extension. FWIW the media field and the related work for the additional types is still unfinished, eg #34634 (comment)
A short list of pending tasks:

  • support the adapters for video/audio/docs
  • force mime type checking for uploads (this is a huge security issue that's still somehow overlooked 🤷‍♂️)
  • convert the allowed mime types, disallowed uploads, preview, edit, etc to the a json structure as I described in the comment linked above
  • use the json structure to drive both the media manager and the media field
  • update the custom field media to support all supported types

This isn't even an exhaustive list, but I think you get the idea.

Just a note, the support for the extra types was done way too late in the J4 development and only because people kept tagging me as if I was responsible for the media manager and I got extremely annoyed. Although the foundation should be solid there are still many things missing.

@crystalenka
Copy link
Member Author

crystalenka commented Jul 28, 2022

Just a note, the support for the extra types was done way too late in the J4 development and only because people kept tagging me as if I was responsible for the media manager and I got extremely annoyed.

Apologies for adding to that! I just saw you pop up on so many media manager related items I assumed (I guess incorrectly) that you were the code owner.

This isn't even an exhaustive list, but I think you get the idea.

That's a lot. I would attempt to fix this myself but my javascript skills are not sophisticated enough yet to play in the build files. :/ Is there an easier way to address the immediate bug even though there is a lot left to do on the media manager in general?

@dgrammatiko
Copy link
Contributor

@crystalenka try this:

code
/**
 * @copyright  (C) 2018 Open Source Matters, Inc. <https://www.joomla.org>
 * @license    GNU General Public License version 2 or later; see LICENSE.txt
 */
if (!Joomla) {
  throw new Error('Joomla API is not properly initiated');
}

/**
 * Extract the extensions
 *
 * @param {*} path
 * @returns {string}
 */
const getExtension = (path) => {
  const parts = path.split(/[#]/);
  if (parts.length > 1) {
    return parts[1].split(/[?]/)[0].split('.').pop().trim();
  }
  return path.split(/[#?]/)[0].split('.').pop().trim();
};

class JoomlaFieldMedia extends HTMLElement {
  constructor() {
    super();

    this.onSelected = this.onSelected.bind(this);
    this.show = this.show.bind(this);
    this.clearValue = this.clearValue.bind(this);
    this.modalClose = this.modalClose.bind(this);
    this.setValue = this.setValue.bind(this);
    this.updatePreview = this.updatePreview.bind(this);
    this.validateValue = this.validateValue.bind(this);
    this.markValid = this.markValid.bind(this);
    this.markInvalid = this.markInvalid.bind(this);

    this.mimeType = '';
  }

  static get observedAttributes() {
    return ['type', 'base-path', 'root-folder', 'url', 'modal-container', 'modal-width', 'modal-height', 'input', 'button-select', 'button-clear', 'button-save-selected', 'preview', 'preview-width', 'preview-height'];
  }

  get type() { return this.getAttribute('type'); }

  set type(value) { this.setAttribute('type', value); }

  get basePath() { return this.getAttribute('base-path'); }

  set basePath(value) { this.setAttribute('base-path', value); }

  get rootFolder() { return this.getAttribute('root-folder'); }

  set rootFolder(value) { this.setAttribute('root-folder', value); }

  get url() { return this.getAttribute('url'); }

  set url(value) { this.setAttribute('url', value); }

  get modalContainer() { return this.getAttribute('modal-container'); }

  set modalContainer(value) { this.setAttribute('modal-container', value); }

  get input() { return this.getAttribute('input'); }

  set input(value) { this.setAttribute('input', value); }

  get buttonSelect() { return this.getAttribute('button-select'); }

  set buttonSelect(value) { this.setAttribute('button-select', value); }

  get buttonClear() { return this.getAttribute('button-clear'); }

  set buttonClear(value) { this.setAttribute('button-clear', value); }

  get buttonSaveSelected() { return this.getAttribute('button-save-selected'); }

  set buttonSaveSelected(value) { this.setAttribute('button-save-selected', value); }

  get modalWidth() { return parseInt(this.getAttribute('modal-width'), 10); }

  set modalWidth(value) { this.setAttribute('modal-width', value); }

  get modalHeight() { return parseInt(this.getAttribute('modal-height'), 10); }

  set modalHeight(value) { this.setAttribute('modal-height', value); }

  get previewWidth() { return parseInt(this.getAttribute('preview-width'), 10); }

  set previewWidth(value) { this.setAttribute('preview-width', value); }

  get previewHeight() { return parseInt(this.getAttribute('preview-height'), 10); }

  set previewHeight(value) { this.setAttribute('preview-height', value); }

  get preview() { return this.getAttribute('preview'); }

  set preview(value) { this.setAttribute('preview', value); }

  get previewContainer() { return this.getAttribute('preview-container'); }

  // attributeChangedCallback(attr, oldValue, newValue) {}

  async connectedCallback() {
    this.button = this.querySelector(this.buttonSelect);
    this.inputElement = this.querySelector(this.input);
    this.buttonClearEl = this.querySelector(this.buttonClear);
    this.modalElement = this.querySelector('.joomla-modal');
    this.buttonSaveSelectedElement = this.querySelector(this.buttonSaveSelected);
    this.previewElement = this.querySelector('.field-media-preview');

    if (!this.button || !this.inputElement || !this.buttonClearEl || !this.modalElement
      || !this.buttonSaveSelectedElement) {
      throw new Error('Misconfiguaration...');
    }

    this.button.addEventListener('click', this.show);

    // Bootstrap modal init
    if (this.modalElement
      && window.bootstrap
      && window.bootstrap.Modal
      && !window.bootstrap.Modal.getInstance(this.modalElement)) {
      Joomla.initialiseModal(this.modalElement, { isJoomla: true });
    }

    if (this.buttonClearEl) {
      this.buttonClearEl.addEventListener('click', this.clearValue);
    }

    this.supportedExtensions = Joomla.getOptions('media-picker', {});

    if (!Object.keys(this.supportedExtensions).length) {
      throw new Error('Joomla API is not properly initiated');
    }

    this.inputElement.removeAttribute('readonly');
    this.inputElement.addEventListener('change', this.validateValue);


    // Force input revalidation
    await this.validateValue({ target: this.inputElement })

    this.updatePreview();
  }

  disconnectedCallback() {
    if (this.button) {
      this.button.removeEventListener('click', this.show);
    }
    if (this.buttonClearEl) {
      this.buttonClearEl.removeEventListener('click', this.clearValue);
    }
    if (this.inputElement) {
      this.inputElement.removeEventListener('change', this.validateValue);
    }
  }

  onSelected(event) {
    event.preventDefault();
    event.stopPropagation();

    this.modalClose();
    return false;
  }

  show() {
    this.modalElement.open();

    Joomla.selectedMediaFile = {};

    this.buttonSaveSelectedElement.addEventListener('click', this.onSelected);
  }

  async modalClose() {
    try {
      await Joomla.getMedia(Joomla.selectedMediaFile, this.inputElement, this);
    } catch (err) {
      Joomla.renderMessages({
        error: [Joomla.Text._('JLIB_APPLICATION_ERROR_SERVER')],
      });
    }

    Joomla.selectedMediaFile = {};
    Joomla.Modal.getCurrent().close();
  }

  setValue(value) {
    this.inputElement.value = value;
    this.validatedUrl = value;
    this.mimeType = Joomla.selectedMediaFile.fileType;
    this.updatePreview();

    // trigger change event both on the input and on the custom element
    this.inputElement.dispatchEvent(new Event('change'));
    this.dispatchEvent(new CustomEvent('change', {
      detail: { value },
      bubbles: true,
    }));
  }

  async validateValue(event) {
    let { value } = event.target;
    if (this.validatedUrl === value || value === '') return;

    if (/^(http(s)?:\/\/).+$/.test(value)) {
      try {
        fetch(value).then((response) => {
          if (response.status === 200) {
            this.validatedUrl = value;
            this.markValid();
          } else {
            this.validatedUrl = value;
            this.markInvalid();
          }
        });
      } catch (err) {
        this.validatedUrl = value;
        this.markInvalid();
      }
    } else {
      if (/^\//.test(value)) {
        value = value.substring(1);
      }

      const hashedUrl = value.split('#');
      const urlParts = hashedUrl[0].split('/');
      const rest = urlParts.slice(1);
      fetch(`${Joomla.getOptions('system.paths').rootFull}/${value}`)
        .then((response) => response.blob())
        .then((blob) => {
          if (blob.type.includes('image')) {
            const img = new Image();
            img.src = URL.createObjectURL(blob);

            img.onload = () => {
              this.inputElement.value = `${urlParts[0]}/${rest.join('/')}#joomlaImage://local-${urlParts[0]}/${rest.join('/')}?width=${img.width}&height=${img.height}`;
              this.validatedUrl = `${urlParts[0]}/${rest.join('/')}#joomlaImage://local-${urlParts[0]}/${rest.join('/')}?width=${img.width}&height=${img.height}`;
              this.markValid();
            };
          } else if (blob.type.includes('audio')) {
            this.mimeType = blob.type;
            this.inputElement.value = value;
            this.validatedUrl = value;
            this.markValid();
          } else if (blob.type.includes('video')) {
            this.mimeType = blob.type;
            this.inputElement.value = value;
            this.validatedUrl = value;
            this.markValid();
          } else if (blob.type.includes('application/pdf')) {
            this.mimeType = blob.type;
            this.inputElement.value = value;
            this.validatedUrl = value;
            this.markValid();
          } else {
            this.validatedUrl = value;
            this.markInvalid();
          }
        })
        .catch(() => {
          this.setValue(value);
          this.validatedUrl = value;
          this.markInvalid();
        });
    }
  }

  markValid() {
    this.inputElement.removeAttribute('required');
    this.inputElement.removeAttribute('pattern');
    if (document.formvalidator) {
      document.formvalidator.validate(this.inputElement);
    }
  }

  markInvalid() {
    this.inputElement.setAttribute('required', '');
    this.inputElement.setAttribute('pattern', '/^(http://INVALID/).+$/');
    if (document.formvalidator) {
      document.formvalidator.validate(this.inputElement);
    }
  }

  clearValue() {
    this.setValue('');
    this.validatedUrl = '';
    this.inputElement.removeAttribute('required');
    this.inputElement.removeAttribute('pattern');
    if (document.formvalidator) {
      document.formvalidator.validate(this.inputElement);
    }
  }

  updatePreview() {
    if (['true', 'static'].indexOf(this.preview) === -1 || this.preview === 'false' || !this.previewElement) {
      return;
    }

    // Reset preview
    if (this.preview) {
      const { value } = this.inputElement;
      const { supportedExtensions } = this;
      if (!value) {
        this.buttonClearEl.style.display = 'none';
        this.previewElement.innerHTML = Joomla.sanitizeHtml('<span class="field-media-preview-icon"></span>');
      } else {
        let type;
        this.buttonClearEl.style.display = '';
        this.previewElement.innerHTML = '';
        const ext = getExtension(value);

        if (supportedExtensions.images.includes(ext)) type = 'images';
        if (supportedExtensions.audios.includes(ext)) type = 'audios';
        if (supportedExtensions.videos.includes(ext)) type = 'videos';
        if (supportedExtensions.documents.includes(ext)) type = 'documents';
        let previewElement;

        const mediaType = {
          images: () => {
            if (supportedExtensions.images.includes(ext)) {
              previewElement = new Image();
              previewElement.src = /http/.test(value) ? value : Joomla.getOptions('system.paths').rootFull + value;
              previewElement.setAttribute('alt', '');
            }
          },
          audios: () => {
            if (supportedExtensions.audios.includes(ext)) {
              previewElement = document.createElement('audio');
              previewElement.src = /http/.test(value) ? value : Joomla.getOptions('system.paths').rootFull + value;
              previewElement.setAttribute('controls', '');
            }
          },
          videos: () => {
            if (supportedExtensions.videos.includes(ext)) {
              previewElement = document.createElement('video');
              const previewElementSource = document.createElement('source');
              previewElementSource.src = /http/.test(value) ? value : Joomla.getOptions('system.paths').rootFull + value;
              previewElementSource.type = this.mimeType;
              previewElement.setAttribute('controls', '');
              previewElement.setAttribute('width', this.previewWidth);
              previewElement.setAttribute('height', this.previewHeight);
              previewElement.appendChild(previewElementSource);
            }
          },
          documents: () => {
            if (supportedExtensions.documents.includes(ext)) {
              previewElement = document.createElement('object');
              previewElement.data = /http/.test(value) ? value : Joomla.getOptions('system.paths').rootFull + value;
              previewElement.type = this.mimeType;
              previewElement.setAttribute('width', this.previewWidth);
              previewElement.setAttribute('height', this.previewHeight);
            }
          },
        };

        // @todo more checks
        if (this.givenType && ['images', 'audios', 'videos', 'documents'].includes(this.givenType)) {
          mediaType[this.givenType]();
        } else if (type && ['images', 'audios', 'videos', 'documents'].includes(type)) {
          mediaType[type]();
        } else {
          return;
        }

        this.previewElement.style.width = this.previewWidth;
        this.previewElement.appendChild(previewElement);
      }
    }
  }
}
customElements.define('joomla-field-media', JoomlaFieldMedia);

@crystalenka
Copy link
Member Author

@dgrammatiko I will try it when I figure out how to do the build process 😅 thank you

@dgrammatiko
Copy link
Contributor

You can create an override of the ‘media/system/js/fields/joomla-field-media.min.js’ in your active template’s js dir (ie ‘ media/templates/administrator/atum/js/system/fields/joomla-field-media.min.js’ iirc) and paste the content of the post above.

@crystalenka
Copy link
Member Author

Yup, it works! Putting it in an override did not work but I overrode the file directly and it works. I know it'll disappear in the next update unless there's a PR or something but I can confirm this did fix the issue with .mov files.

Thank you!

@dgrammatiko
Copy link
Contributor

@crystalenka couple of things here:

  • the override should work (maybe the system shouldn't be in the path?)
  • the changes are extremely minimal, like 5-6 lines
  • but the code that used to sniff the mime types didn't exist till 4.2 and this pr thus the original code was flaky
  • I'm not ok contributing to joomla anymore but at least I can fix things that obviously are my fault

Please test the PR so it might get a chance to be merged

@richard67
Copy link
Member

Closing as having a pull request. Please test #38425 . Thanks in advance.

@crystalenka
Copy link
Member Author

Thank you, @dgrammatiko. I appreciate it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants