New Attribute List Approach

[paultvis]

Hi,

I just wanted to give my thoughts on the new approach to attributes and their creation. I understand from a data perspective that it is much, much cleaner having the list values stored in another table, however it has really impacted usability and workflow.

We are just about to create a new attribute which has about 50-60 unique values. Previously we would have had the list created elsewhere and copied values into the values section of the attribute form. That would have taken a minute or two to create (copy, paste, copy, paste, etc). Now however we need to go through multiple clicks and form opens and submits to create each list entry. It also appears that duplicate values are allowed in the list meaning that there is a risk of attributes not being set up correctly. Previously if you tried to create a duplicate an error would be raised. Navigation is not helped by the fact that the add item button is at the top, so to add a new item you need to scroll to the bottom to see the last entry, then scroll back to the top to add the next (this is opposite to Create list, which is at the bottom, meaning you have to scroll through all the existing lists to be able to add a new one - you might consider reversing those)

By my estimate looking at the guy creating the list, a 5 min job is now in the 20-30min timeframe due to the extra navigation and checking he is having to do to make sure there are no duplicates.

As an indication of the increase in time here, we have a site with technical products. As we sort and organise each category we will probably need to create 3-5 new attributes with numbers of values ranging from 3 or 4 up to 50-60. So far we have sorted about 20 out of 250 categories so the additional amount of time needed here is huge.

I just wanted to give you my thoughts and some feedback on the changes.

Paul

[marcus-at-localhost]

I have to agree with @paultvis for a different reason:
While I understand the decision for the data schema, it a) breaks the API again for 3rd party clients (without a warning), it adds a load of new queries to the API (query attributes, parse them, select the list fields, query those for the MultienumOptions etc.) It's one thing to do that on the client side (eg the PIM itself), it's another doing that on the server, which adds to the build time of the catalog. Even with caching, fetching the whole list at once (if it's not too many items) the initial hit is massive.

I really appreciate all the work and how the software evolves, but the breaking API changes are killing me.
I'm in the process to build a Postman Test Suite, but with the massive number of endpoints/objects, I'm behind and it's a bit overwhelming :D.

Thanks though. m.

Back to fixing our app.

[paultvis]

Agreed, I too appreciate the work that goes into this and would like to add that since starting to use Atro we have really been able to progress our catalogue cleansing. We had pretty much given up in Akeneo and the move over has been a huge success. We work in the data layer a lot which has its challenges but it sounds like working through the APIs would not always solve that.

I have just seen your reply @atrolex and I did say to the guy setting up the attribute (between the swearing) that this is probably the first part of an agile journey and that things sometimes get worse before they get better. Completely agreed with @marcus-at-localhost that the no warning to changes is a pain. It is a lottery whether the automated updates are going to bring joy or pain! Not sure how feasible it would be, but some kind of high level view of the roadmap and summary or sprint planning would be really useful - just to allow us to prepare.

Just to reiterate (after what probably seems negative, but was meant constructively), the overall experience with Atro has been great!

Paul

[marcus-at-localhost]

@paultvis turn off automatic updates. That helps to take a deep breath, after reading the changelogs and then roll the dice :)
Also I was wrong, @rratsun laid out the changes: https://github.com/atrocore/atropim/releases/tag/1.8.3 - it's just not always as clear what it means (in terms of how it affects API/UI/UX since it is an complex app). And of course we all have a staging system to test everything before, not? :)
Else, @atrolex and @rratsun are very responsive and helpful.

[atrolex]

Thank you both for a valuable feedback. We will discuss everything internally and will decide how to proceed. What we have done was just i first increment, on which we can build additional functionality, also in terms of usability.

[atrolex]

We have discussed this topic internally and see no need for urgent changes now. Reason we have implemented this new data type for attributes was, that we wanted options to be managed as data and not as meta data. Data Types enum and multi enum store the options in metadata, so by the nature they should be used to manage states, statuses etc, very simple value lists.
Attributes have nothing to do with "states" and can have additional properties like color, image, description etc (you can configure additional fields for it).

Additionally:

1. Uniqueness of options is realized in the paid module "Advanced Data Types"
2. You can create lists from the attribute, from the list interface https://demo.atropim.com/#ExtensibleEnum
3. Now you can use import feeds to import your options, what was previously impossible
4. Also, now you can refer to the same lists both in multiple attributes and fields. You can easily change names through mass updates.
5. To quickly create options, you can duplicate existing values.
6. For attributes with multilingual options on the product detail page you would see the value only in one language, what is enough (via API you can get translations for any language).

For the future we plan to implement the functionality to use additional key (eg in the header), which would lead to extension of the response, so you get not only OptionID, but also the values, if needed in wished languages.

[atrolex]

Next big change will be refactoring of data type Unit, so be prepared :) In the DB for an attribute value we will store the UnitID, and not "cm2" anymore. Data Type Unit will be removed, it will be possible to Add Measure to a Float, Integer or Range field, what will make them to what is Unit now.
Range will be added as a new Data Type.
We are thinking about data type Currency, currencies could be managed as Units for Measures and Multipliers (=Exchange Rates) are already available there.

[marcus-at-localhost]

Would it be possible to deprecate old data types over a longer time and leave the old implementation as is until Atrocore 2.0 comes out (see semantic versioning)? New data types can still be introduced, but it would ensure that 3rd party clients don't break right away.
That would help to migrate those clients that use the API, gradually. Else we are faced with holding back updates and fall behind, or breaking the api clients and go into emergency fixing mode.

And the change you describe sounds like there is another endpoint to call. It would be great if there is an option (via ?select=field... ?) to include the units into one call and not call 10 different endpoints to compose them myself, for one product :-/ (this is a bit out of scope of this discussion, I know ;-))

Thanks!

[rratsun]

I think in this case we can keep backward compatibility. Field type Currency we will keep without changes, for now. Field type Unit we will change in a background, but for API, I suppose, everything will be almost the same. Maybe we just add one more additional column and that's all.

[marcus-at-localhost]

@rratsun just updated to 1.6.6, and I had no issues with the Unit to Int/Float (+Unit) field transformation. Thanks for the good work.
Before the update, I finally set up a dev instance with the same data as the live instance and wrote some tests in Postman to check changes in the API. It helped, but the amount of fields is a bit overwhelming.
Anyway, just wanted to give some feedback.

PS: Just for the record, if someone stumbles upon it: if you clone the whole application within the same server instance, don't forget to change appId in data/config.php else the cronjob won't work. Took me a little bit to figure that out :).

[rratsun]

@marcus-at-localhost please, update to 1.6.8. We have fixed a bug in migration.