3.3.0 release

[glensc]

• update min php requirement to 5.6.0 (@balsdorf)
• some docs how to upgrade (does one need extra disk space or not, or how to estimate needed disk space). Use league/flysystem to abstract attachments #254

[balsdorf]

Disk space question is a good one. I'll do some tests and let you know.

[glensc]

there are two kind of disk requirements:

• uprading 3.2.3 -> 3.3.0
• migrating from one storage type to another

in this issue scope, only the first one is interesting

[balsdorf]

For upgrading from 3.2.3 -> 3.3.0 there should be no additional disk space required. The upgrade process just adds the new tables and columns but leaves existing attachments in the current table. New attachments will be inserted into the new table but unless you run migrate_storage_adapter.php it won't move anything.

[glensc]

thanks for the info! i'll copy that to original PR too. should we add something to changelog.md too or as nothing special, not necessary?

[glensc]

i think i'll roll out 3.3.0 release, today-tomorrow, unless @balsdorf want to do that yourself.

[balsdorf]

Nothing is needed in changelog.md regarding disk space, but not sure if we want to add text letting people know they can switch to a different adapter? I'm undecided as I view this as an advanced feature that most people won't need so I don't know if we want to advertise it. I went ahead and added a new page in the wiki regarding it. Think this is good enough or we should advertise it in the changelog?

And please go ahead with the release when you can.

[phavel]

On the contrary, it is a important function.
I'm waiting for it and I am happy it will be released. My MySQL storage on a web host is already full because of attachments.

Thanks for the function!

[balsdorf]

@phavel I'm not worried about people like you who are reading these PRs! :) Once you try it out let me know any feedback you have, updated docs are appreciated as well.

[glensc]

@balsdorf damn, your analyze is totally wrong!

EventumAttachments does alter on issue_attachment and issue_attachment_file tables! meaning there's need for double of those tables size!

MySQL copies data to new tables using #tmp* and does rename if that succeeds. issue_attachment_file is the second biggest table in the system after support_email_body

[glensc]

@balsdorf how you propose to solve this, add the new column of issue_attachment_file table to some new table perhaps instead?

[balsdorf]

What version of MySQL are you using and what engine? For InnoDB newer than 5.6 it table adds / drops are handled INPLACE. For MyISAM, yeah it is going to be locked and copied. I read the docs and this is why I said it would not take additional disk space. I did not have a pre-release database available for testing. After your email I got a fresh snapshot of our live DB (pre-release) and applied the upgrade I saw that disk space was being used and spoke with our senior support staff. "INPLACE" for InnoDB means that the while the table is available during the alter it is still copied. Where does this leave us? If we need to avoid altering the issue_attachment_file table then yes, we will need to add a new table, say issue_attachment_path that contains the path. I can start working on this tonight. Sorry for not doing a complete test before I gave you the disk usage info, I read the docs and made an incorrect assumption.

…

On Wed, Oct 4, 2017 at 12:21 PM Elan Ruusamäe ***@***.***> wrote: @balsdorf <https://github.com/balsdorf> damn, your analyze is wrong! EventumAttachments does alter on issue_attachment and issue_attachment_file tables! meaning there's need for double of those tables size! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#300 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHzdTMR2su9IxGgCJx16VUOIOy6FZ_Wks5so76jgaJpZM4PrSiG> .

[balsdorf]

I've made the changes and submitted #302 . I'd like to do more testing tomorrow, but all my current testing looks good, including migrating data.

[glensc]

@balsdorf InnoDB is not supported in Eventum, it's default is still MyISAM. If you have it as InnoDB it's your own hack and you likely need to convert every new table manually :).

altho with introducing phinx migration tool, we could easily make engine configurable. i haven't done that myself because i still use MyISAM. i don't have viable replication/backup/restore plan using InnoDB, using mysqldump is rather lame solution. for MyISAM i use mysqlhotbackup via bacula-backup-mysql on slave instance.

would like to solve that problem first before looking into InnoDB.

[balsdorf]

Hah, sometimes I forget that people have other environments than I do. InnoDB is faster and safer (and now supports fulltext which was the reason we used to stay on MyISAM. Maybe with 4.0.0 we change the default to InnoDB. I'll ask our team what they recommend for InnoDB backups for you.

[balsdorf]

Xtrabackup is what my team recommends.
Personally we are using LVM Snapshots on one slave and mydumper on the other.

[phavel]

Once you try it out let me know any feedback you have, updated docs are appreciated as well.

Sorry, now I can't try it (development version), but I am looking forward to production version.

[glensc]

upgraded my production with snapshot build. eventum-3.2.3-189-g7b4eddae.tar.gz to be exact.

looks fine, so making release now.

[glensc]

v3.3.0 released

[glensc]

@balsdorf another issue with attachments. the upgrade was supposed not to require filesystem storage. i.e everything is still stored to database.

however it gives me error:

root@eventum ~/eventum# /tmp/eventum.phar add-att 62998 sync-20171008.log -i

                                                                              
  [Eventum_RPC_Exception]                                                     
  Impossible to create the root directory "/usr/share/eventum/var/storage/".  
                                                                              

add-attachment [-f|--filename FILENAME] [-m|--mimetype MIMETYPE] [-d|--description DESCRIPTION] [-i|--internal] [--] <issue_id> <file>

[glensc]

need to release 3.3.1 due #305

[balsdorf]

Was the issue with being unable to create the root storage directory solved by #305?

[glensc]

unable to create root directory error happened because there were no filesystem permissions. what i had in mind, i did not expect filesystem driver being used out of the box. that was my issue.

not sure if that needs to be fixed, i.e if the fix is going to make code too complex, it's not worth of it. in my system i granted the write permission. the dir is left empty with new attachment uploads, so it's not filled, but the unused driver initialization failed.

[balsdorf]

So the tarball contains the default storage directory so I'm not sure why you are seeing that error. However, it is an easy fix. I'll add it to my todo.

…

On Mon, Oct 9, 2017 at 1:32 AM Elan Ruusamäe ***@***.***> wrote: unable to create root directory error happened because there were no filesystem permissions. what i had in mind, i did not expect filesystem driver being used out of the box. that was my issue. not sure if that needs to be fixed, i.e if the fix is going to make code too complex, it's not worth of it. in my system i granted the write permission. the dir is left empty with new attachment uploads, so it's not filled, but the unused driver initialization failed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#300 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHzdcNLOAS4PiPgOrn2LCXb9mol1xhZks5sqb3jgaJpZM4PrSiG> .

[glensc]

seems you still did not get my confusion.

1. from Use league/flysystem to abstract attachments #254 i understood that the default upgrade path will be using db only storage (as pre 3.3.0)
2. but still "local" backend is configured (thus the error i got)

but as you're not going to do about it in short term, i'm going to release 3.3.1 now.