Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLIP: Using Dexterity for Plone Site #2454

Open
jaroel opened this issue Jun 24, 2018 · 26 comments
Open

PLIP: Using Dexterity for Plone Site #2454

jaroel opened this issue Jun 24, 2018 · 26 comments

Comments

@jaroel
Copy link
Member

@jaroel jaroel commented Jun 24, 2018

PLIP (Plone Improvement Proposal)

Responsible Persons

Proposer: Roel Bruggink

Seconder:

Abstract

Replace the CMFPlone Portal class with a Dexterity implementation.

Motivation

Moving from a CMF item to a Dexterity container will allow us to edit the site root directly using the DX framework including support for behaviors.

This makes it possible to add the rich text behavior to the site root. This way we don’t need to use extra documents to select as default views. This is the last step we need to be able to go with a full folderish objects site and get rid of the “select item as default view” which is a huge usability improvement

Assumptions

Proposal & Implementation

Replace the base class for Products.CMFPlone.Portal.PloneSite with a DX container.
Development is initially done on the latest stable (5.1) branch of Products.CMFPlone. We'll move to unstable (5.2) when this is done or if 5.2 is released, whichever is earlier.

Deliverables

PLIP buildout config can be found here: https://github.com/plone/buildout.coredev/blob/6.0/plips/plip-2454-dx-siteroot.cfg

Jenkins job: https://jenkins.plone.org/view/PLIPs/job/plip-dx-siteroot-3.8/

Change Products.CMFPlone.Portal.PloneSite from CMF to DX. #2612
Various supportive changes in -at least- plone/plone.app.contenttypes#475 and plone/plone.dexterity#85.

Risks

Currently unknown.

Participants

Roel Bruggink
Alessandro Pisa

@jensens
Copy link
Member

@jensens jensens commented Jun 25, 2018

How do you plan to migrate the Site Root?

@jaroel
Copy link
Member Author

@jaroel jaroel commented Jun 25, 2018

I haven't given it too much thought atm.
I copied over an existing Plone 5.1 Data.fs into my development stuff and at least it loaded the state correctly from the ZODB, had the correct mro for the Plone Site root and rendered the site just fine.

Something like this should do using a view/script on the zope root for each Plone site:

class MigrateRootToDx(BrowserView):

    def __call__(self):
        from Acquisition import aq_base
        from plone.protect.auto import safeWrite
        request = self.request

        print("Start migration Plone Sites root to DX")
        for site in self.context.objectValues('Plone Site'):
            safeWrite(site, request)
            print("Running for {}".format(site))

            # Initialise the btree.
            if '_tree' not in site.__dict__:
                print("Initialising btrees")
                site._initBTrees()

            # Migrate existing portal content.
            print("Migrate content")
            for obj_meta in site._objects:
                obj_id = obj_meta['id']
                print("Migrating object {}".format(obj_id))
                # Load the content item, tool, or whatever
                obj = getattr(site, obj_id)
                # insert it into the btree.
                # Use _setOb so we don't reindex stuff. the paths stay the same.
                site._setOb(obj_id, aq_base(obj))
                # Drop the attribute
                del site.__dict__[obj_id]

            # Drop no-longer needed bookkeeping
            print("Clean up bookkeeping data")
            del site.__dict__['_objects']
            site._p_changed = True

I do need to check what bookkeeping can be dropped.

edit: I've just tested the above and that seems to Just Works (tm).

>>> sorted(app.Plone.keys())
['_Access_arbitrary_user_session_data_Permission', '_Access_contents_information_Permission', '_Access_inactive_portal_content_Permission', '_Access_session_data_Permission', '_Add_portal_content_Permission', '_Add_portal_folders_Permission', '_Add_portal_member_Permission', '_Allow_sendto_Permission', '_CMFEditions__Access_previous_versions_Permission', '_CMFEditions__Apply_version_control_Permission', '_CMFEditions__Checkout_to_location_Permission', '_CMFEditions__Revert_to_previous_versions_Permission', '_CMFEditions__Save_new_version_Permission', '_Change_local_roles_Permission', '_Content_rules__Manage_rules_Permission', '_Copy_or_Move_Permission', '_Delete_comments_Permission', '_Delete_objects_Permission', '_Delete_own_comments_Permission', '_Edit_comments_Permission', '_FTP_access_Permission', '_List_folder_contents_Permission', '_List_portal_members_Permission', '_List_undoable_changes_Permission', '_Mail_forgotten_password_Permission', '_Manage_properties_Permission', '_Modify_portal_content_Permission', '_Modify_view_template_Permission', '_Portlets__Manage_own_portlets_Permission', '_Portlets__Manage_portlets_Permission', '_Portlets__View_dashboard_Permission', '_Reply_to_item_Permission', '_Request_review_Permission', '_Review_comments_Permission', '_Review_portal_content_Permission', '_Search_ZCatalog_Permission', '_Set_own_password_Permission', '_Set_own_properties_Permission', '_Show_Toolbar_Permission', '_Undo_changes_Permission', '_Use_Database_Methods_Permission', '_Use_external_editor_Permission', '_Use_mailhost_services_Permission', '_Use_version_control_Permission', '_View_Groups_Permission', '_View_History_Permission', '_View_Permission', '_View_management_screens_Permission', '_WebDAV_Lock_items_Permission', '_WebDAV_Unlock_items_Permission', '_WebDAV_access_Permission', '__ZCacheManager_ids__', '__ac_local_roles__', '__ac_roles__', '__allow_groups__', '__annotations__', '__before_publishing_traverse__', '__before_traverse__', '__error_log__', '_components', '_count', '_mt_index', '_owner', '_plone_app_contenttypes__Add_Collection_Permission', '_plone_app_contenttypes__Add_Document_Permission', '_plone_app_contenttypes__Add_Event_Permission', '_plone_app_contenttypes__Add_File_Permission', '_plone_app_contenttypes__Add_Folder_Permission', '_plone_app_contenttypes__Add_Image_Permission', '_plone_app_contenttypes__Add_Link_Permission', '_plone_app_contenttypes__Add_News_Item_Permission', '_plone_app_event__Import_Ical_Permission', '_plone_app_multilingual__Manage_Translations_Permission', '_plone_resource__Export_ZIP_file_Permission', '_plone_resourceeditor__Manage_Sources_Permission', '_properties', '_tree', '_v__providedBy__', 'cmf_uid', 'contributors', 'creation_date', 'creators', 'default_page', 'description', 'effective_date', 'expiration_date', 'format', 'id', 'language', 'modification_date', 'rights', 'subject', 'title']

>>> app.Plone.portal_catalog
<CatalogTool at /Plone/portal_catalog>

>>> app.Plone.portal_catalog()
[<Products.ZCatalog.Catalog.mybrains object at 0x10b27e598>, <Products.ZCatalog.Catalog.mybrains object at 0x10b27e0b8>, <Products.ZCatalog.Catalog.mybrains object at 0x10b258668>, <Products.ZCatalog.Catalog.mybrains object at 0x10b258940>, <Products.ZCatalog.Catalog.mybrains object at 0x10b258bb0>]

>>> app.Plone.portal_catalog()[0]
<Products.ZCatalog.Catalog.mybrains object at 0x10b258328>

>>> app.Plone.portal_catalog()[0].getObject()
<Document at /Plone/front-page>

>>> 'portal_catalog' in app.Plone.__dict__
False
@jaroel
Copy link
Member Author

@jaroel jaroel commented Jun 25, 2018

We'll need to add the Plone Site FTI and fix the edit method alias when upgrading.
Need to check plone.app.multilingual as we won't have a front-page object anymore.

@jensens jensens added this to New (drafts) in PLIPs Jun 26, 2018
@ebrehault
Copy link
Member

@ebrehault ebrehault commented Jun 26, 2018

@jaroel The FWT is very positive about this PLIP.

It would be useful to get more information about the motivations (what could be the benefits? What kind of behaviors could be usefully used on the Plone Site object) as it would help to communicate about this important improvement.

Note: we will vote on the PLIP during our next meeting.

@sunew
Copy link
Contributor

@sunew sunew commented Jun 26, 2018

for one, selectablecontrainstypes behavior would be super useful:)

@robgietema
Copy link
Member

@robgietema robgietema commented Jul 2, 2018

This makes it possible to add the rich text behavior to the site root. This way we don’t need to use extra documents to select as default views. This is the last step we need to be able to go with a full folderish objects site and get rid of the “select item as default view” which is a huge usability improvement. I can second this proposal.

@ebrehault
Copy link
Member

@ebrehault ebrehault commented Jul 10, 2018

@jaroel, the @plone/framework-team approved this PLIP

@gforcada gforcada moved this from New (drafts) to In Process (approved) in PLIPs Jul 10, 2018
@jaroel
Copy link
Member Author

@jaroel jaroel commented Jul 10, 2018

@ebrehault that's great news, thanks!

@tkimnguyen
Copy link
Member

@tkimnguyen tkimnguyen commented Jul 11, 2018

How does this work? "This is the last step we need to be able to go with a full folderish objects site and get rid of the 'select item as default view'" In CastleCMS we got rid of default items by using Mosaic and including a rich text tile on the Folder content type default layout

@tisto
Copy link
Member

@tisto tisto commented Aug 7, 2018

@jaroel would you mind giving us an update on the current status? What is missing before we can review your code? Do you plan to integrate this in Plone 5.2 or later?

@tisto
Copy link
Member

@tisto tisto commented Aug 7, 2018

@tkimnguyen in Plone-React we got rid of default items as well. The new tiles-based "Page" type is folderish by default as well.

@jaroel
Copy link
Member Author

@jaroel jaroel commented Aug 8, 2018

@tisto In practice it seems to work just fine. I'm resolving the test failures when I have time.
See https://jenkins.plone.org/view/PLIPs/job/plip-dx-siteroot/ for the current state.

You can check the changes done so far using the PLIP buildout config https://github.com/plone/buildout.coredev/blob/5.1/plips/plip-2454-dx-siteroot.cfg .

I wouldn't see this as a blocker for 5.2. I'd not even consider this for 5.2, unless people have used this in production somewhere.

@jaroel
Copy link
Member Author

@jaroel jaroel commented Aug 19, 2018

@gforcada or @tisto I'm getting timeout accessing the ftp server on jenkins (https://jenkins.plone.org/view/PLIPs/job/plip-dx-siteroot/12/testReport/plone.app/testing/layers_rst/)
Any idea what could be going on there?

@jaroel
Copy link
Member Author

@jaroel jaroel commented Feb 11, 2019

FYI I'm back on this. Primary goal is to make this work for 5.2+, as that seems stable enough now.

@tisto tisto changed the title Using Dexterity for Plone Site PLIP: Using Dexterity for Plone Site Feb 12, 2019
@tisto tisto added this to the Plone 6.0 milestone Feb 12, 2019
@tisto
Copy link
Member

@tisto tisto commented Feb 27, 2019

@jaroel we do not plan to do a 5.3 release so this would be Plone 6 then. I added the label...

@jaroel
Copy link
Member Author

@jaroel jaroel commented May 27, 2020

@jensens FYI:

  1. So far bin/instance starts and I have a functional Plone site root.
  2. The tests failures from plone/plone.supermodel@c1822bb are back
  3. https://jenkins.plone.org/view/PLIPs/job/plip-dx-siteroot-3.8/ has some failures
  4. The ZMI (/manage) doesn't reflect the normal interface for the site root
    5. The Products.CMFEditions failures make me sad. They have disappeared somehow, which makes me sad
@esteele
Copy link
Member

@esteele esteele commented Jan 26, 2021

What's the current status of this? It looks like tests are currently green.

@jaroel
Copy link
Member Author

@jaroel jaroel commented Jan 26, 2021

I've just updated the branches in the repos I needed.
The "upgrade script" #2454 (comment) should be added somewhere. This could be in Products.CMFPlone.Portal.__getstate__ or as a normal upgrade step.
The FWT should review what has been done, and people must test it with actual production data.

To be honest I don't really have time nor incentive currently to work on this except for once in a blue moon. I'm fairly certain it would be more effective if someone like @ale-rt would champion this PLIP.

@ale-rt
Copy link
Member

@ale-rt ale-rt commented Feb 5, 2021

I will work on this at the Not-an-Alpine-City-Sprint

@jaroel
Copy link
Member Author

@jaroel jaroel commented Feb 6, 2021

@ale-rt I'm sure this is in very capable hands, thank you very much Alessandro!
I'm more than happy to answer questions.

@esteele
Copy link
Member

@esteele esteele commented Apr 20, 2021

@ale-rt I see you've taken care of the upgrade step. Is this ready for FWT review? If not, are there additional tasks I (or others) can help out with?

@ale-rt
Copy link
Member

@ale-rt ale-rt commented Apr 20, 2021

Hi @esteele! I am planning to recheck this on the 23rd of April (there is a Zope sprint scheduled).
I am not using this on production yet but so far I did not spot any issue.
Hi really would like somebody from the @plone/framework-team to pick this up and review it.

@ale-rt
Copy link
Member

@ale-rt ale-rt commented Apr 20, 2021

And of course testing by anybody: if anyone can check that their DB run fine after this switch it would be super cool.

@pbauer
Copy link
Member

@pbauer pbauer commented Apr 20, 2021

I tested it against a Plone 6 (master) database with p.a.multilingual and that failed initially with expected tracebacks and I could not open the portal in the ZMI.

After accessing http://localhost:8080/Plone/portal_setup/manage_upgrades I managed to run the two upgrade-steps "Be sure that the Plone Site FTI is a dexterity one" and "Make the Plone Site a dexterity container". Then the site seemed to work fine only the FTI of Plone Site has lost the default_view property language-switcher and was reset to listing_view.

What freaks me out is that some test failed in unexpected ways: The content of [i for i in portal._tree] was not the same as portal.keys() and portal.contentValues() failed with:

>>> m.contentValues()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/pbauer/workspace/dynajet_relaunch/src-mrd/Products.CMFCore/src/Products/CMFCore/PortalFolder.py", line 202, in contentValues
    return [item[1] for item in self.contentItems(filter)]
  File "/Users/pbauer/workspace/dynajet_relaunch/src-mrd/Products.CMFCore/src/Products/CMFCore/PortalFolder.py", line 188, in contentItems
    return self._filteredItems(ids, filter)
  File "/Users/pbauer/workspace/dynajet_relaunch/src-mrd/Products.CMFCore/src/Products/CMFCore/PortalFolder.py", line 174, in _filteredItems
    obj = get(id)
  File "/Users/pbauer/.cache/buildout/eggs/plone.folder-3.0.3-py3.8.egg/plone/folder/ordered.py", line 73, in _getOb
    raise AttributeError(e)
AttributeError: 'de'
@ale-rt
Copy link
Member

@ale-rt ale-rt commented Apr 21, 2021

Thanks @pbauer for reporting this back.
About the upgrade, were you able to run the http://localhost:8120/Plone/@@plone-upgrade or that was not helpful because you were already using Plone 6?
I will take care of copying the default_view and maybe some other properties during the upgrade.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
PLIPs
In Process (approved)
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants