Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migration from v3 to v4 failed #908

Closed
elvisef opened this issue Feb 7, 2021 · 10 comments · Fixed by #910
Closed

migration from v3 to v4 failed #908

elvisef opened this issue Feb 7, 2021 · 10 comments · Fixed by #910
Labels
bug Something isn't working High Priority High priority issues

Comments

@elvisef
Copy link
Contributor

elvisef commented Feb 7, 2021

Detailed description of the problem [REQUIRED]

*Something is wrong with the database migration.
I tried hints from the project main github page, but something went wrong.
All permissions were good. No errors. .env file has been set successfully.

What happend after migration, is a problem with DB.
I am using Mysql, and what I found is

  1. After I created a user on the first page, (Lychee suggested), a Users table in my DB had one entry, but the ID was 0 !!! Not 1... But the primary key is there !
  2. Album tables in DB are clear, no entries there.
  3. Photos in DB are incorrect as well.*

Steps to reproduce the issue

Steps to reproduce the behavior:
Just follow steps. I did nothing more.
One thing I found, in the logs table, that the prefix was probably incorrectly written in the "env" file.
But Ive just done what was suggested during installation in the env file - to put the old prefix (but I had only the default prefix: "lychee_" in my DB), so I wrote it down there, as the file suggested. And if You look now on my screenshots, You will understand what am I talking about.

Screenshots
2021-02-07_22-08
2021-02-07_22-11
2021-02-07_22-11_1
2021-02-07_22-12
2021-02-07_22-12_1
2021-02-07_22-13
2021-02-07_22-13_1

Output of the diagnostics [REQUIRED]

*Diagnostics
-----------
Warning: Dropbox import not working. dropbox_key is empty.
Warning: You may experience problems when uploading a photo of large size. Take a look in the FAQ for details.
Warning: You may experience problems when uploading a photos of large size. Take a look in the FAQ for details.
Warning: You may experience problems when uploading a large amount of photos. Take a look in the FAQ for details.

System Information
------------------
Lychee Version (git):            master (a040cbc) - Data not in Cache
DB Version:                      4.2.1

composer install:                --no-dev
APP_ENV:                         production
APP_DEBUG:                       true

System:                          Linux
PHP Version:                     7.4
Max uploaded file size:          20971520
Max post size:                   20971520
MySQL Version:                   5.7.33

Imagick:                         1
Imagick Active:                  1
Imagick Version:                 1687
GD Version:                      2.2.5

Config Information
------------------
version:                         040201
check_for_updates:               0
sorting_Photos_col:              takestamp
sorting_Photos_order:            ASC
sorting_Albums_col:              max_takestamp
sorting_Albums_order:            ASC
imagick:                         1
skip_duplicates:                 0
small_max_width:                 0
small_max_height:                360
medium_max_width:                1920
medium_max_height:               1080
lang:                            en
layout:                          1
image_overlay:                   1
image_overlay_type:              desc
default_license:                 none
compression_quality:             90
full_photo:                      1
delete_imported:                 0
Mod_Frame:                       1
Mod_Frame_refresh:               30
thumb_2x:                        1
small_2x:                        1
medium_2x:                       1
landing_page_enable:             0
landing_owner:                   John Smith
landing_title:                   John Smith
landing_subtitle:                Cats, Dogs & Humans Photography
landing_facebook:                https://www.facebook.com/JohnSmith
landing_flickr:                  https://www.flickr.com/JohnSmith
landing_twitter:                 https://www.twitter.com/JohnSmith
landing_instagram:               https://instagram.com/JohnSmith
landing_youtube:                 https://www.youtube.com/JohnSmith
landing_background:              dist/cat.jpg
site_title:                      Lychee v4
site_copyright_enable:           1
site_copyright_begin:            2019
site_copyright_end:              2019
additional_footer_text:          
display_social_in_gallery:       0
public_search:                   0
SL_enable:                       0
SL_for_admin:                    0
public_recent:                   0
recent_age:                      1
public_starred:                  0
downloadable:                    0
photos_wraparound:               1
map_display:                     0
zip64:                           1
map_display_public:              0
map_provider:                    Wikimedia
force_32bit_ids:                 0
map_include_subalbums:           0
update_check_every_days:         3
has_exiftool:                    0
share_button_visible:            0
import_via_symlink:              0
has_ffmpeg:                      0
location_decoding:               0
location_decoding_timeout:       30
location_show:                   1
location_show_public:            0
rss_enable:                      0
rss_recent_days:                 7
rss_max_items:                   100
prefer_available_xmp_metadata:   0
editor_enabled:                  1
lossless_optimization:           0
swipe_tolerance_x:               150
swipe_tolerance_y:               250
local_takestamp_video_formats:   .avi|.mov
log_max_num_line:                1000
unlock_password_photos_with_url_param: 0
nsfw_visible:                    1
nsfw_blur:                       0
nsfw_warning:                    0
nsfw_warning_admin:              0
map_display_direction:           1
album_subtitle_type:             oldstyle

Browser and system

OS: Debian 10
Using Firefox: 78.7.0esr (64 bity)

One more thing - migration of tables seem to be OK - look:

`**************************************
*     Application In Production!     *
**************************************

 Do you really wish to run this command? (yes/no) [no]:
 > 
Migration table created successfully.
Migrating: 2014_10_12_000000_create_users_table
Migrated:  2014_10_12_000000_create_users_table (11.87ms)
Migrating: 2018_08_03_110935_create_albums_table
Migrated:  2018_08_03_110935_create_albums_table (27.78ms)
Migrating: 2018_08_03_110936_create_photos_table
Migrated:  2018_08_03_110936_create_photos_table (31.05ms)
Migrating: 2018_08_03_110942_create_configs_table
Migrated:  2018_08_03_110942_create_configs_table (10.05ms)
Migrating: 2018_08_03_111324_create_logs_table
Migrated:  2018_08_03_111324_create_logs_table (7.98ms)
Migrating: 2018_08_10_134924_move_settings
Migrated:  2018_08_10_134924_move_settings (11.47ms)
Migrating: 2018_08_15_102039_move_albums
Migrated:  2018_08_15_102039_move_albums (9.82ms)
Migrating: 2018_08_15_103716_move_photos
Migrated:  2018_08_15_103716_move_photos (3.86ms)
Migrating: 2018_10_30_135411_sharing
Migrated:  2018_10_30_135411_sharing (46.33ms)
Migrating: 2019_02_21_114356_create_pages_table
Migrated:  2019_02_21_114356_create_pages_table (7.66ms)
Migrating: 2019_02_21_114408_create_page_contents_table
Migrated:  2019_02_21_114408_create_page_contents_table (19.70ms)
Migrating: 2019_06_21_180451_create_sym_links_table
Migrated:  2019_06_21_180451_create_sym_links_table (6.60ms)
Migrating: 2019_09_28_171753_config_fix
Migrated:  2019_09_28_171753_config_fix (301.59ms)
Migrating: 2019_09_28_190822_photos_fix
Migrated:  2019_09_28_190822_photos_fix (40.05ms)
Migrating: 2019_10_01_add_livephoto_cols
Migrated:  2019_10_01_add_livephoto_cols (39.94ms)
Migrating: 2019_10_02_1400_config_map_display_public
Migrated:  2019_10_02_1400_config_map_display_public (1.01ms)
Migrating: 2019_10_03_214750_frame_refresh_in_sec
Migrated:  2019_10_03_214750_frame_refresh_in_sec (1.69ms)
Migrating: 2019_10_06_1400_config_map_providers
Migrated:  2019_10_06_1400_config_map_providers (1.39ms)
Migrating: 2019_10_06_152017_add_force_32bit_ids
Migrated:  2019_10_06_152017_add_force_32bit_ids (0.81ms)
Migrating: 2019_10_07_0900_config_map_include_sub_albums
Migrated:  2019_10_07_0900_config_map_include_sub_albums (0.69ms)
Migrating: 2019_10_09_233402_config_map_mod
Migrated:  2019_10_09_233402_config_map_mod (1.00ms)
Migrating: 2019_10_11_093442_config_check_update_every
Migrated:  2019_10_11_093442_config_check_update_every (0.80ms)
Migrating: 2019_12_02_2100_config_exiftool
Migrated:  2019_12_02_2100_config_exiftool (0.64ms)
Migrating: 2019_12_15_0700_add_share_button_visible_option
Migrated:  2019_12_15_0700_add_share_button_visible_option (19.02ms)
Migrating: 2019_12_15_1000_config_check_update_every_cat_fix
Migrated:  2019_12_15_1000_config_check_update_every_cat_fix (3.84ms)
Migrating: 2019_12_25_0600_config_exiftool_ternary
Migrated:  2019_12_25_0600_config_exiftool_ternary (7.32ms)
Migrating: 2020_01_018_2300_config_import_via_symlink
Migrated:  2020_01_018_2300_config_import_via_symlink (1.19ms)
Migrating: 2020_01_04_1200_config_has_ffmpeg
Migrated:  2020_01_04_1200_config_has_ffmpeg (6.19ms)
Migrating: 2020_01_26_1200_config_public_sorting
Migrated:  2020_01_26_1200_config_public_sorting (3.40ms)
Migrating: 2020_01_28_133201_composer_update
Migrated:  2020_01_28_133201_composer_update (1.20ms)
Migrating: 2020_02_14_0600_location_decoding
Migrated:  2020_02_14_0600_location_decoding (19.69ms)
Migrating: 2020_03_11_124417_increase_length_photo_type
Migrated:  2020_03_11_124417_increase_length_photo_type (6.35ms)
Migrating: 2020_03_17_200000_unhide_configs
Migrated:  2020_03_17_200000_unhide_configs (1.81ms)
Migrating: 2020_04_19_122905_bump_version
Migrated:  2020_04_19_122905_bump_version (1.23ms)
Migrating: 2020_04_22_155712_bump_version040002
Migrated:  2020_04_22_155712_bump_version040002 (1.14ms)
Migrating: 2020_04_29_000250_bump_version040003
Migrated:  2020_04_29_000250_bump_version040003 (1.46ms)
Migrating: 2020_05_12_114228_rss
Migrated:  2020_05_12_114228_rss (1.01ms)
Migrating: 2020_05_12_161427_bump_version040005
Migrated:  2020_05_12_161427_bump_version040005 (1.19ms)
Migrating: 2020_05_19_174233_config_prefer_available_xmp_metadata
Migrated:  2020_05_19_174233_config_prefer_available_xmp_metadata (0.86ms)
Migrating: 2020_05_26_135052_bump_version040006
Migrated:  2020_05_26_135052_bump_version040006 (1.58ms)
Migrating: 2020_06_04_104605_config_editor_enabled
Migrated:  2020_06_04_104605_config_editor_enabled (0.74ms)
Migrating: 2020_07_11_104605_config_lossless_optimization
Migrated:  2020_07_11_104605_config_lossless_optimization (1.24ms)
Migrating: 2020_07_11_184605_update_licences
Migrated:  2020_07_11_184605_update_licences (11.82ms)
Migrating: 2020_07_26_085322_config_swipe_tolerance
Migrated:  2020_07_26_085322_config_swipe_tolerance (1.39ms)
Migrating: 2020_07_29_132731_config_local_takestamp
Migrated:  2020_07_29_132731_config_local_takestamp (1.32ms)
Migrating: 2020_08_21_123622_add_smart_tag_album_cols
Migrated:  2020_08_21_123622_add_smart_tag_album_cols (25.71ms)
Migrating: 2020_10_09_130043_bump_version040007
Migrated:  2020_10_09_130043_bump_version040007 (1.62ms)
Migrating: 2020_10_15_104504_add_log_max_num_line
Migrated:  2020_10_15_104504_add_log_max_num_line (0.76ms)
Migrating: 2020_10_15_161346_sort_image_per_album
Migrated:  2020_10_15_161346_sort_image_per_album (25.15ms)
Migrating: 2020_11_12_183345_config_password_url_param_for_smart_album
Migrated:  2020_11_12_183345_config_password_url_param_for_smart_album (1.42ms)
Migrating: 2020_11_19_231553_bump_version040008
Migrated:  2020_11_19_231553_bump_version040008 (1.70ms)
Migrating: 2020_12_12_203153_migrate_admin_user
Migrated:  2020_12_12_203153_migrate_admin_user (6.28ms)
Migrating: 2020_12_12_203831_create_web_authn_tables
Migrated:  2020_12_12_203831_create_web_authn_tables (21.90ms)
Migrating: 2020_12_18_162100_bump_version040009
Migrated:  2020_12_18_162100_bump_version040009 (0.97ms)
Migrating: 2020_12_18_162155_add_nsfw_album
Migrated:  2020_12_18_162155_add_nsfw_album (21.27ms)
Migrating: 2020_12_18_163800_bump_version040010
Migrated:  2020_12_18_163800_bump_version040010 (0.88ms)
Migrating: 2020_12_24_022307_bump_version040100
Migrated:  2020_12_24_022307_bump_version040100 (0.87ms)
Migrating: 2020_12_26_153220_nested_set_for_albums
Migrated:  2020_12_26_153220_nested_set_for_albums (35.60ms)
Migrating: 2021_01_09_163715_remove_max_min_takestamps
Migrated:  2021_01_09_163715_remove_max_min_takestamps (26.06ms)
Migrating: 2021_01_12_122546_bump_version040200
Migrated:  2021_01_12_122546_bump_version040200 (1.04ms)
Migrating: 2021_01_18_103729_add_album_cover
Migrated:  2021_01_18_103729_add_album_cover (13.39ms)
Migrating: 2021_01_20_113912_bump_version040201
Migrated:  2021_01_20_113912_bump_version040201 (0.98ms)
Migrating: 2021_01_24_231904_fix-rotation
Migrated:  2021_01_24_231904_fix-rotation (2.32ms)
Migrating: 2021_01_27_085903_config_map_display_direction
Migrated:  2021_01_27_085903_config_map_display_direction (1.33ms)
Migrating: 2021_01_30_111736_display_takedate
Migrated:  2021_01_30_111736_display_takedate (0.89ms)
Migrating: 2064_12_25_0000_generate_installed_log
Migrated:  2064_12_25_0000_generate_installed_log (1.96ms)

I think the problem lies in album tables.
Not everything is being moved...

@ildyria
Copy link
Member

ildyria commented Feb 7, 2021

After I created a user on the first page, (Lychee suggested), a Users table in my DB had one entry, but the ID was 0 !!! Not 1... But the primary key is there !

0 is a good thing as this is the ID for the admin user.

@ildyria
Copy link
Member

ildyria commented Feb 7, 2021

change the DB_LYCHEE_OLD_PREFIX to nothing and do php artisan migration:refresh :)

@elvisef
Copy link
Contributor Author

elvisef commented Feb 8, 2021

After the command You mentioned, it crashes:
2021-02-08_12-13_1

It says, as You see, that this command has no "migration" namespace.

I tried to do it once more. I cleaned a database, just old tables were left.
Then I prepared these points:

4 Generate your secret key with php artisan key:generate
5 Migrate your database with php artisan migrate to create a new database or migrate an existing Lychee installation to the latest framework.

I've regenerated the key, just in case.
Then while trying to migrate, using the command in point 5, It crashes (!). It didn't crash the first time yesterday I did this.
So as I see it searches for a column name that does not exist in the old table "albums". Why ?

More errors:
2021-02-08_12-12
2021-02-08_12-13

While trying to reinstall, I see this:
2021-02-08_12-16

@ildyria
Copy link
Member

ildyria commented Feb 8, 2021 via email

@ildyria ildyria added bug Something isn't working High Priority High priority issues labels Feb 8, 2021
ildyria added a commit that referenced this issue Feb 8, 2021
* fixes #908
* add regression test for mysql (on distribution only, good enough)
@ildyria
Copy link
Member

ildyria commented Feb 8, 2021

@elvisef if you do a git pull to update your files and redo the migration (like you did "I tried to do it once more. I cleaned a database, just old tables were left.") it should be working now.

So as I see it searches for a column name that does not exist in the old table "albums"

No it is searching for a column _rgt in the new albums table. This was a changed made with version 4.2 and was not in version 4.0. However because we added some code, album addition via "models" requires those column to exist... This is now fixed in the migration, you do not need to worry about it. 👍

@ildyria
Copy link
Member

ildyria commented Feb 8, 2021

That being done, I want to say that I really appreciate the extensive information your provided us with to solve this issue. Big thanks again.

@elvisef
Copy link
Contributor Author

elvisef commented Feb 9, 2021

It works ! Thank You : )
I am really happy to have this working again and to help You solve the issue.

I always used v3 version, because there was no need to change it.
But some day... I wanted to test the new one.

So what I did now:

  • [1] removed new tables in Lychee database.
  • [2] git pull in Lychee's directory. (it managed to do migration automatically ? I think it just did it..)
  • [3] doing php artisan migrate:refresh made things clear

..and the main login screen is back now : )

By the way, I have a question, how to force changing login data ?
I don't know why, but I can't login, and the login is encrypted as well so...

Just to force changing or re-creating a username and pass ? Should I remove it from the users table ?

@d7415
Copy link
Contributor

d7415 commented Feb 9, 2021

By the way, I have a question, how to force changing login data ?

https://lycheeorg.github.io/docs/faq.html#how-to-reset-username-and-password

@elvisef
Copy link
Contributor Author

elvisef commented Feb 9, 2021

By the way, I have a question, how to force changing login data ?

https://lycheeorg.github.io/docs/faq.html#how-to-reset-username-and-password

Thanks, works like a charm.

@ildyria
Copy link
Member

ildyria commented Feb 9, 2021

  • [2] git pull in Lychee's directory. (it managed to do migration automatically ? I think it just did it..)

Yes. composer install is adding a git hook on pull/merge, as a result, if you do git pull it will follow with composer install --no-dev (if you have installed without dev originally) and php artisan migrate to update the database.

You can disable this behavior by adding a file .NO_AUTO_COMPOSER_MIGRATE at the root of your Lychee installation...

We added that because we had a lot of complaints such as "I am up-to-date" but I get XXX error... and of course the database was not updated... This is also why we now have a page that forces database update if the files version is newer than the database version. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working High Priority High priority issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants