Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration from SBL and SSM #3

Closed
nilshoerrmann opened this issue Jul 2, 2014 · 88 comments
Closed

Migration from SBL and SSM #3

nilshoerrmann opened this issue Jul 2, 2014 · 88 comments

Comments

@nilshoerrmann
Copy link

See mazedigital/association_ui_selector#5.

@jdsimcoe
Copy link

@nilshoerrmann I agree that having a Wiki with an ability to post scripts would be helpful in accomplishing this task. I don't need a dedicated extension per-se, but I think at least having a path to successfully migrate would make life a bunch easier! Thanks for all the immense work on this Association stuff. It really is wonderful to see it come together.

@nilshoerrmann
Copy link
Author

There is a little Symphony code sprint this Wednesday and this may be a good time to take a look at extension migrations as @brendo will be around.

If you'd like to help with this (ideas, implementation, testing), please leave a short note. Thanks!

/cc @animaux

@animaux
Copy link

animaux commented Aug 11, 2014

@nilshoerrmann I’ll be around for testing!

@jdsimcoe
Copy link

I should be around as well. I've had very good progress on getting SSM moved over to Associations and almost have it completed for my large database.

@nilshoerrmann
Copy link
Author

Nice!

It would be helpful, if you could post issues you encountered so far.
I was thinking of some kind of FAQ list we could add to the repository's wiki to guide people. And I think it should be possible to bundle an upgrade script – I've thought about it an it should be less complicated than I thought before.

@nilshoerrmann
Copy link
Author

Let's start collecting migration steps:

From Select Box Link to Association Field

  1. Install Association Field
  2. Rename sym_fields_selectbox_link to sym_fields_association.
  3. In sym_fields rename all selectbox_link types to association.

@nilshoerrmann
Copy link
Author

From Subsection Manager to Association Field

This needs to be tested, but I think these are the steps for Subsection Manager:

  1. Install Association Field.
  2. If you're using a custom caption, install Reflection Field and recreate your caption.
  3. Resave your entries to populate the Reflection fields.
  4. For each entry in the sym_fields_subsectionmanager table, create an entry in sym_fields_association with the same field_id. Remove sym_fields_subsectionmanager when you're done.
  5. In sym_fields rename all subsectionmanager types to association.
  6. Go to your section settings, add the missing settings (related fields, allow multiple etc.) and save your section.

@animaux
Copy link

animaux commented Aug 13, 2014

Sounds good at first sight. Should I give it a try then?

@nilshoerrmann
Copy link
Author

Yes, but make a database copy first, will you? :)

The biggest question is: will Symphony correctly display the associations in the interface after you saved your sections (table links, association drawer).

@animaux
Copy link

animaux commented Aug 13, 2014

I have several copies by now :) OK!

putting on white gloves

@animaux
Copy link

animaux commented Aug 13, 2014

@nilshoerrmann is the current reflection field now supporting root and workspace params? Or the integration branch?

@nilshoerrmann
Copy link
Author

Not sure, if Andrew has been working on it. We haven't.
Maybe @brendo has some time left to look at it.

@animaux
Copy link

animaux commented Aug 13, 2014

Hmm. This looks like it, doesntit? symphonists/reflectionfield@f74287e

@nilshoerrmann
Copy link
Author

Try it out :)

Am 13.08.2014 um 10:32 schrieb Alexander Rutz notifications@github.com:

Hmm. This looks like it, doesntit? symphonists/reflectionfield@f74287e


Reply to this email directly or view it on GitHub.

@animaux
Copy link

animaux commented Aug 13, 2014

For each entry in the sym_fields_subsectionmanager table, create an entry in sym_fields_association with the same field_id. Remove sym_fields_subsectionmanager when you're done.

Shouldn’t the subsection_id of the ssm field become into the related_field_id of the association field also?

@nilshoerrmann
Copy link
Author

No, because the first is a section id and the second is a field id.

@animaux
Copy link

animaux commented Aug 13, 2014

Ah ok.

@animaux
Copy link

animaux commented Aug 13, 2014

And the id?

@nilshoerrmann
Copy link
Author

Will be set automatically.

@animaux
Copy link

animaux commented Aug 13, 2014

Ok. I get an id=1 whcih makes me a bit suspicious :)

@nilshoerrmann
Copy link
Author

You're getting what?

@animaux
Copy link

animaux commented Aug 13, 2014

The newly created entry in sym_fields_association automatically got the id of 1 (via phpMyAdmin)

@nilshoerrmann
Copy link
Author

Yes, that's correct because it's the first entry in that table.

@animaux
Copy link

animaux commented Aug 13, 2014

Hmn, ok, I thought it was a system-wide unique id.

@animaux
Copy link

animaux commented Aug 13, 2014

Remove sym_fields_subsectionmanager when you're done.

Maybe it’s better to only delete it’s entries in order not to confuse the SSM-uninstaller?

@nilshoerrmann
Copy link
Author

No, it shouldn't matter because SSM checks the existence first:
https://github.com/hananils/subsectionmanager/blob/master/extension.driver.php#L589

@animaux
Copy link

animaux commented Aug 13, 2014

Goody!

@animaux
Copy link

animaux commented Aug 13, 2014

Success! Will have to check for further issues, but basically it seems to work!

Hooray!

@nilshoerrmann
Copy link
Author

Great!

@jdsimcoe
Copy link

I cannot imagine a solution that will neither create very complex code (that would be bad) nor very slow XML output (that would not be better).

I suspected this might be the case but I feel like using output parameters works efficiently and limits complicated datasource calls and XML. This was always the standard way of linking datasources outside of SSM, SBL, and now Association.

@munki-boy
Copy link

The reflection field way I tried turned out to be a good quick fix for a site that's already nested, I mean from a migration point-of-view. I have images that have an "owner" in a separate section with name and URL. I was actually able to write

<a href="{entry/url}">{entry/name}</a>

directly into the reflection field then only had to add disable-output-escaping to my existing value-of in my template.

@jdsimcoe
Copy link

That’s cool. So you just supplied that output to your front-end pages?

On Friday, August 15, 2014 at 7:34 AM, munki-boy wrote:

The reflection field way I tried turned out to be a good quick fix for a site that's already nested, I mean from a migration point-of-view. I have images that have an "owner" in a separate section with name and URL. I was actually able to write
{entry/name}
directly into the reflection field then only had to add disable-output-escaping to my existing value-of in my template.


Reply to this email directly or view it on GitHub (#3 (comment)).

@munki-boy
Copy link

Yes, I got confused because the Reflection field part seems to work, although its preview code in the section can look messed up depending what you type in - must close some tag in the actual backend page prematurely.

The output is I think a string of HTML rather than actual HTML, and when reading old posts on getsymphony people seemed to have been messing with CDATA tags, I couldn't figure that out but just tried exactly this

<xsl:value-of select="image-owner" disable-output-escaping="yes"/>

Where image-owner is the name of the reflection field in the image-owner section and in this case I'm fetching it through the association field that's in a parent section called photos. In the datasource association output I selected "image-owner" rather than "name" or "url" being the three fields that are in my image owners section.

TBH I did type a lot of other stuff first that didn't work well, especially the CDATA idea that I was sure would be the correct thing to do. I assume I did that wrong and it should work.

@munki-boy
Copy link

Now I migrated an actual site I find that the selector will only show items that are already chosen.
For example if I had previously in SSM selected say Red, Green, Blue they will be shown already associated. If I then go and click into the search/select box nothing will be available, even if there's other colours like Purple in the colour section.

If I then remove say Red and re-save then in the selector only Green, Blue will be available to choose. If I remove all and re-save then the thing just says none found.

I'm pretty sure in earlier tests without migration it used to go and get the range of all possible things you would want to choose.

Doesn't seem to matter if I choose None, Search or Sortable except the UI design changes.

I'm feeling production ready apart from this hurdle.

@nilshoerrmann
Copy link
Author

I'm pretty sure in earlier tests without migration it used to go and get the range of all possible things you would want to choose.

Yes, that's how it's supposed to work. Looks like the interface is not able the fetch the related options.

  • Have a look at your browser console to see if there are any errors.
  • Backup your database, then delete all entries from sym_sections_association and resave all your sections. See if that helps.

@munki-boy
Copy link

Ok, I tried the console first but didn't see any errors there, I thought that was the case because I often cause errors myself :)

I deleted all the rows from sym_sections_association then re-saved all my sections. This re-populated the table but both columns for Interface and Editor became all "NULL", previously some had been chosen to be the selector or editor.

The problem with not being able to find or choose from the association box remained the same. With either "none found" for previously non-associated sections and just the already chosen items in other sections.

After re-saving my sections where I briefly returned to

...symphony/blueprints/sections/

I saw a console error but I'd already left the page and only had time to see it before the page refreshed, when I went back it wasn't there.

The error said something similar to "a.document default owner is null" and was part of symphony.min.js

Sorry about this vagueness I only saw it for a second and not sure if it's relevant, probably not I think.

Perhaps I'll try a quick empty site and check that's working ok, with my current versions of everything.

@munki-boy
Copy link

Is it possible that this

For each entry in the sym_fields_subsectionmanager table, create an entry in sym_fields_association with the same field_id. Remove sym_fields_subsectionmanager when you're done.

Went wrong? My sym_sections_association doesn't seemt to have a "field_id" at the moment, it has

id
parent_section_id
parent_section_field_id
child_section_id
child_section_field_id
hide_association
interface
editor

It seems to be only the selector box that isn't populating. The site is working and the data is there in the frontend.

@nilshoerrmann
Copy link
Author

(I reformatted your last comment a bit to make it more readable.)

Is it possible that this … went wrong?

sym_fields_association and sym_sections_association are two different things. The first is the Association field, the other is Symphony's global association storage.

Sorry about this vaguenes …

From your reports, I can only say that the interface wasn't able to fetch the associated entries. But I have no clue why – especially because there are no errors logged in the console.

@munki-boy
Copy link

Tried a fresh install with no data and that worked as expected when I added some data. So I've just ported everything into that install and it's continued to work.

Must be something in the migrated site that breaks things but I can't see what's different there. I did have a bit of a rough time upgrading SSM before migrating though, it was an beta version initially.

The next migration is much more data but the site has been kept up-to-date with SSM, so I'll see if that goes more smoothly.

I intend to go live with these migrated sites, is there anything you think might be a major change that might make it difficult to install the final versions of the association extensions later?

@munki-boy
Copy link

I started on my larger site which had a section with one SBL and several SSM, I did the migration as Nils said, and in the order listed here. That is SBL first then SSM.

One thing I noticed that might matter to some but not to me in this case. Where the SBL had no item selected. In my case the SBL was to choose a "parent section" so top-level sections did not have a parent.

Anyway, where that was just empty or blank, it's now become a '0' zero character in the Association Selector. I'm not targeting that anywhere so it doesn't matter for me but maybe for others?

Also, if you don't go and re-connect the associations in your section editor for some sections, when you visit the pages with list of entries in the section you can get a SQL error just like Pat has shown here in an earlier version of Symphony:

http://www.getsymphony.com/discuss/thread/102012/

Or rather if that part of the section doesn't have any association selected and re-saved in the section editor. I'm not sure if it does this for every section. It seems a bit harsh to break this way but I guess it only happens when migrating as there's something already in the tables.

@nilshoerrmann
Copy link
Author

One thing I noticed that might matter to some but not to me in this case. Where the SBL had no item selected. In my case the SBL was to choose a "parent section" so top-level sections did not have a parent.

Anyway, where that was just empty or blank, it's now become a '0' zero character in the Association Selector. I'm not targeting that anywhere so it doesn't matter for me but maybe for others?

There was a change in the way Select Box Link handles empty values a while back. My guess is that what you see is more related to this change than to your migration from one field to the other.

Or rather if that part of the section doesn't have any association selected and re-saved in the section editor. I'm not sure if it does this for every section. It seems a bit harsh to break this way but I guess it only happens when migrating as there's something already in the tables.

I'm not sure, if I understand this problem correctly. Can you please elaborate?

@munki-boy
Copy link

After the migration process, if a person doesn't click the required "Values" in the association fields in a section and re-save (in blueprints/sections). If you then visit a the page that lists the entries for the section (publish/whatever), that page cannot be viewed and a mySQL error is displayed about sort order. The error is just like the one shown above in the link and the situation sound similar too.

I imagine the process that prepares the page view is unable to sort an empty list into 'Asc' order as it tried to do.

Going back into the section editor and highlighting the "Values" in the box fixes the error. Assuming again that this provided a some items that could be sorted into 'Asc' order. So I suppose clicking any of the Values, even the wrong ones would populate this list with things that could be sorted.

I'd already done some when I noticed so I'm not sure if it happens in all cases or if other fields in the section cause problems. I posted the link above as I fixed the problem before having copied the error to paste in here.

@munki-boy
Copy link

This is the error I get in my site:

Symphony Fatal Database Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') ORDER BY t1.`sortorder` ASC' at line 4

SELECT SQL_CACHE t1.* FROM ar6_fields AS `t1` WHERE 1 AND id IN () ORDER BY t1.`sortorder` ASC

It's only the first time after migration, if I add the values to fix the problem then go and try to take them away again then I'm not allowed to save the section as "This is a required field." is displayed.

So the section is "expected" to have values selected but straight after migration it might not have.

@munki-boy
Copy link

I'm trying to get output params to filter further DS by as Jonathan suggested above for sections that are nested more than one level.

If I select in Associated Content then I get the data but no output param. So if I go and choose $ds-section.entry as a parameter then it does appear as a param but it is duplicated as in:

    <ds-section.entry>
        <item handle="6">6</item>
        <item handle="6">6</item>
    </ds-section.entry>

If I remove from Associated Content then the duplicate dissapears, but then so does the content I want to access. Is there a way to get the Associated Content item as a param to filter another DS by?

@munki-boy
Copy link

I've taken the associated content away and kept the normal DS output then filtered the normal way. This works fine as you'd expect.

@munki-boy
Copy link

Hmm this means that sort order of items in the association select doesn't have any effect as my DS filter isn't coming from the association. :sadface

@nilshoerrmann
Copy link
Author

The only solution I see is to rethink the backend structure: do all these things really have to be handled inline? While I cannot answer that question, it may be an idea to take a look at Order Entries to sort entries in the section itself. But that really depends on your project.

One thing I don't know: do output parameters retain the sort order?

@munki-boy
Copy link

I started thinking that too, order entries would work but is dirty in my clean new world of association.

No DS parameter output from association output at all? This seems fundamental.

I'll go and look at sort order, gimme a sec...

@nilshoerrmann
Copy link
Author

No DS parameter output from association output at all? This seems fundamental.

That's up to the Data Source you are using Association Output with.

@munki-boy
Copy link

Yes in my case sort order is represented in the association output.

That's up to the Data Source you are using Association Output with.

Well it's the standard DS I can't get any params to filter another DS by from the Association Output. If I could then I'd filter by them and be happy :)

@nilshoerrmann
Copy link
Author

I see, misunderstanding.

@munki-boy
Copy link

The output seems to be through $ds-section.entry which seems ok except the standard ds output is already coming through there so I get two of the same results in the output as shown above.

There's no way to have just the Association Output to filter by, you can have the standard output or both but not just association output on it's own as a filterable param. I'm wanting something like $ds-association.section.entry to be in the list of parameter to output for filtering.

As it stands sort order in association output works fine but I can only use it in XSLT and not as a DS filter.

@munki-boy
Copy link

I've finished my tricky migration and everything that worked before, works now. With the addition of a nice association editor interface throughout.

It seems the association output sometimes can't output the data of nested sections if that data is coming from a further association field in that nested section - that is to say in a direct grandparent/parent/child relationship. When things get weird switch to a default separate datasource filter. But - a child section of that is not a direct descendant of the top section so can resume use of association output. Maybe useful on certain clumps of data.

Of course the field/selector/editor parts can be used throughout regardless of association output.

@bzerangue
Copy link
Member

Is it possible to add a migration script from subsectionmanager field to association field like @nilshoerrmann did when he deprecated the mediathek field where you could migrate from mediathek field to the subsectionmanager field?

This would be great feature to make the migration process easier. Especially when so many of us used and relied on the subsectionmanager field.

@nilshoerrmann
Copy link
Author

It would be possible to do this to migrate from SBL to AF, but a direct upgrade from SSM to AF is not possible due to the different feature set (this was different when Mediathek became Subsection Manager). For the backend, you'll have to replace custom captions manually and you'll have to work around the lack of content filtering (see #12). On the frontend, your XML will change because AF doesn't support nested output out of the box – so you'll have to migrate to Association Output on that side.

Formalising the guidelines from this thread and adding them to the readme would be the best we can do.

@bzerangue
Copy link
Member

Ok. Thank you

On Mon, Jan 5, 2015 at 8:42 AM, Nils Hörrmann notifications@github.com
wrote:

It would be possible to do this to migrate from SBL to AF, but a direct upgrade from SSM to AF is not possible due to the different feature set (this was different when Mediathek became Subsection Manager). For the backend, you'll have to replace custom captions manually and you'll have to work around the lack of content filtering (see #12). On the frontend, your XML will change because AF doesn't support nested output out of the box – so you'll have to migrate to Association Output on that side.

Formalising the guidelines from this thread and adding them to the readme would be the best we can do.

Reply to this email directly or view it on GitHub:
#3 (comment)

@tonyarnold
Copy link

Sorry to resurrect an old issue, but I'm making this move now. Can I select more than one "Value" for an association? Say I have a description and a file in my associated section — can I select both of them? I'm guessing there are some limitations here I'm unaware of.

Right, this is a question for the selection/editing UI extensions — sorry guys!

@animaux
Copy link

animaux commented Feb 13, 2015

@tonyarnold I am guessing you mean if it possible to show the description and linked file of an associated section in the selector interface? If that’s what you mean the answer is: yes, by using a reflection field to compile the info you need, and select this field in the association field that links to the section.

Have a look here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants