add-on store: add ability to evaluate add-ons from NVDA #15576
Comments
|
For now, I don't find a way to programatically convert an issue to a discussion. But imo the issue number, or the corresponding PR, maybe stored in the json metadata file of each add-on, so that the URL can be opened from the actions menu. Also, reactions maybe used to make easy to speakers of different languages to understand if reviewers think that the add-on is right, and this maybe documented. |
This comment was marked as resolved.
This comment was marked as resolved.
|
cc @jcsteh due to the following quotation, in case you want to provide feedback about the store.
I clarify that I'm not very convinced about promoting popular add-ons since other ones maybe better, so I'd prefer to talk about quality and not popularity, but my proposal is to provide a way to see reactions and who reviewed add-ons. |
|
@nvdaes I support your efforts here, I think users will both benefit from this,
and have desired it.
That said, does github have to be the platform used for these kind of reviews?
There may be better alternatives.
Users are still "afraid" of GitHub, and most do anything they can to avoid it.
So you are automatically selecting for only the more technically inclined and
dedicated users.
That might be desirable from some prospectives, but not from all prospectives.
It may be worth looking into other platforms, or mechanisms.
|
|
Thanks Luke: In the alternatives section, I mentioned that people may poooost reviews to websites or the addon mailing list, and this doesn"t require an extra efforz since this option is possible at this moment. But comments that I have read point out that the store may contain harmful code, summarizing what I have understand, and so I think that github is the best option now, since technical users may provide more reliable reviews at this point.Enviado desde mi iPhoneEl 5 oct 2023, a las 3:25, Luke Davis ***@***.***> escribió:
@nvdaes I support your efforts here, I think users will both benefit from this,
and have desired it.
That said, does github have to be the platform used for these kind of reviews?
There may be better alternatives.
Users are still "afraid" of GitHub, and most do anything they can to avoid it.
So you are automatically selecting for only the more technically inclined and
dedicated users.
That might be desirable from some prospectives, but not from all prospectives.
It may be worth looking into other platforms, or mechanisms.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
This has been a requested a couple times, and we have an internal issue for this. If we are to use GitHub, I think the simplest way would be to use GitHub discussions in nvaccess/addon-datastore. Though, the best way to do this would be if NV Access creates our own server backend and reviews exist within the store. |
|
I think that, at least for now, we can generate a disckussion for each submission and make stuff in addondatastorevalidation, an create an action to consult the discussion. I can work on this if this is triaged.Enviado desde mi iPhoneEl 6 oct 2023, a las 2:13, Sean Budd ***@***.***> escribió:
This has been a requested a couple times.
If we are to use GitHub, I think the simplest way would be to use GitHub discussions in nvaccess/addon-datastore.
We could generate a discussion on submission for each add-on ID and a new comment for each version.
Though, the best way to do this would be if NV Access creates our own server backend and reviews exist within the store.
However, this would be a major piece of work and is unlikely to be completed in the near future.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
seanbudd
added
p5
|
To simplify things, why not allow submitters, or only trusted submitters, to put a "Thumb up" or "Thumb down"? |
|
I would avoid generating a discussion for each submission and instead use one per add-on ID and a comment per version. We also need to add this to the metadata and implement an add-on action in NVDA to open the reviews page. |
|
I would also suggest waiting until this discussion pans out before starting implementation |
|
ok, i agree and will test on my fork, unless you indicate me that you don"t want help for this. i"ll comment my tests here with links.Enviado desde mi iPhoneEl 6 oct 2023, a las 3:13, Sean Budd ***@***.***> escribió:
@nvdaes
I would avoid generating a discussion for each submission and instead use one per add-on ID and a comment per version.
One per submission could get very noisy, and most reviews would be better centralised to an add-on itself not a particular version. People will miss reviews if it's a different discussion per version.
We also need to add this to the metadata and implement an add-on action in NVDA to open the reviews page.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
It also might be worth testing the different discussion types. "Q&A" (reviews are oriented around voting more) or "Polls" (to rate the add-on) might be worth trying out |
|
One question: |
|
i think that we shouldn"t restrict creativity of reviewers. imo the fact that an addon is not reviewed due to complexity is also meaningful for people. Other add-ons may not be reviewed due to dirty code etc.. anyway I would encourage to provide reactions and very short comments in github discussions, and give complete feedback to authors, taking care about reporting security issues privately.Enviado desde mi iPhoneEl 6 oct 2023, a las 3:27, ruifontes ***@***.***> escribió:
One question:
The review will be about security, code quality, functionality or what?
I still remember the discussions about NVDAExtensionGlobalPlugin add-on...
It is a very complex add-on? Sure it is...
Patch too much NVDA functions? Yes, it does...
It is usefull? Yes, it is...
How it should be classified?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
The only parts that require a server, are the user management (to keep just
anyone from leaving a review), and the review form submission (that allows an
authenticated user's form submission to update the database).
That could be done with a cloud applet.
I am imagining the use of GitHub pages to display the output as a static site,
with updates being managed through something like a Netlify service applet
connected to the repo hosting the site.
There are people doing review type things with single JS functions on Jekyll
backed sites, although I don't know if they incorporate the user authentication
part.
Something like a vague combination of this:
https://github.com/robinmetral/jekyll-book-review
with the concept behind this:
https://stackoverflow.com/questions/49900416/implementing-a-evaluation-system-using-jekyll
|
|
I think that General category is easier to use. Also, for reference, here's a |
|
I've created a discussion also using the linked GitHub Action, not just GitHub Script, and it's great since it can return the discussion url aid. But we should use the announcement category, not the General one, so just maintainers can create discussions under this category. Then we may use the add-on id as the title,to check if a discussion for this particular add-on exists or not. |
|
Noelia, you wrote:
we should use the announcement category, not the General one, so
just maintainers can create discussions under this category.
If I understand well, maintainers of a specific add-on can create
a discussion to that add-on...
But, lets suppose I create a discussion for one of our add-ons...
next day some reviewer, with or without reason, submit a very
negative review...
I, as owner of the discussion, can delete this comment, right?
If so, it is not very good...
Best regards,
Rui Fontes
NVDA portuguese team
Message
ID: ***@***.***>
[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "#15576 (comment)",
"url": "#15576 (comment)",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]
|
|
Hi Rui. You wrote:
No, I meant maintainers of addon-datastore repo, not add-on maintainers. The discussion would be created automatically after an issue to osubmit an add-on is opened. The announcement category would ensure that we can list discussion in that category with the title set to the add-on id. If we don't use the announcements category, everyone may create discussion with a title set to add-on ids, so it would be difficult to search for the discussion related to an add-on via GitHub Actions. |
|
Ah! Ok! That way is good!
Best regards,
Rui Fontes
NVDA portuguese team
Às 14:29 de 08/10/2023, Noelia Ruiz
Martínez escreveu:
Hi Rui. You wrote:
If I understand well, maintainers of a specific
add-on can create
a discussion to that add-on...
But, lets suppose I create a discussion for one of our
add-ons...
next day some reviewer, with or without reason, submit a very
negative review...
I, as owner of the discussion, can delete this comment, right?
No, I meant maintainers of addon-datastore repo, not
add-on maintainers. The discussion would be created
automatically after an issue to osubmit an add-on is opened. The
announcement category would ensure that we can list discussion
in that category with the title set to the add-on id. If we
don't use the announcements category, everyone may create
discussion with a title set to add-on ids, so it would be
difficult to search for the discussion related to an add-on via
GitHub Actions.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message
ID: ***@***.***>
[
{
***@***.***": "http://schema.org",
***@***.***": "EmailMessage",
"potentialAction": {
***@***.***": "ViewAction",
"target": "#15576 (comment)",
"url": "#15576 (comment)",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
***@***.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]
|
|
I think that, instead of trying to list and filter all discussions under the announcement category, maybe better to find the add-on id of the submission and maintain a discussions.json file in the root of the repo, similar to submitters.json, to find out if the corresponding discussion exists. So, the first PR for this maybe done in the datastore repo. I'm discussing this here in case feedback is provided before proceeding with a PR. |
|
That system makes sense @nvdaes |
|
I'm stucked trying to add a comment to an existing discussion. Creating the discussion if it doesn't exists and save the discussion ID and discussion URL associated with each addon id seems easier. GitHub workflow to add discussion and comment Here's the workflow run Here's the created discussion |
|
@nvdaes - does the action we use to add comments to issues and pull requests not work for discussions? My assumption is that it should be the same to add a comment to a discussion as adding to an issue. This should be much easier https://github.com/marketplace/actions/create-or-update-comment |
|
This action to make comments on issues or pull request doesn"t work for discussions: we need to ouse graphql and issues and prs support rest too.Enviado desde mi iPhoneEl 11 oct 2023, a las 1:52, Sean Budd ***@***.***> escribió:
@nvdaes - does the action we use to add comments to issues and pull requests not work for discussions? My assumption is that it should be the same to add a comment to a discussion as adding to an issue. This should be much easier
https://github.com/marketplace/actions/create-or-update-comment
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Shouldn't the comment be created after the discussion is created? |
|
@nvdaes I think the parameters fed in https://github.com/nvdaes/addon-datastore/blob/master/.github/workflows/checkAndSubmitAddonMetadata.yml#L253C13-L253C33 should be encapsulated in quotes, e.g. |
|
yes, my approach is to check if the discussion exists, and in this case we try to creating the comment providing the body and discussion id, found in discussions.json file, and if the discussion doesn"t exist it"s created and the id and url are saved in the json file, associated with the addon id like "reportSymbols": discussionid: foo, urr: https://... of course with quotations and braces.Enviado desde mi iPhoneEl 11 oct 2023, a las 3:19, Sean Budd ***@***.***> escribió:
Shouldn't the comment be created after the discussion is created?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
This should fix the issue you raise here
|
|
i"ll try it in a few hours from PC. thanksEnviado desde mi iPhoneEl 11 oct 2023, a las 3:29, Sean Budd ***@***.***> escribió:
@nvdaes I think the parameters fed in https://github.com/nvdaes/addon-datastore/blob/master/.github/workflows/checkAndSubmitAddonMetadata.yml#L253C13-L253C33 should be encapsulated in quotes, e.g.
addDiscussionComment(input: {body: "17.0.0", discussionId: "D_kwDODeDxhs4AV09t", clientMutationId: "GitHub Script action"}) {\n' +
This should fix the issue you raise here
I'm stucked trying to add a comment to an existing discussion. Creating the discussion if it doesn't exists and save the discussion ID and discussion URL associated with each addon id seems easier.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Finally the comment has been created. I think that one problem was that in the payload we need to specify comment { body } or other fields, but not just the word comment, in contrast with clientMutationId, which is not an object. Please let me know how to specify the repository id and category id for nvaccess/addon-datastore. For example, you may want to store this data as secrets. |
|
The repository ID should be fetchable depending on the context that the action is run in. I can create a category soon, but I'd like to consider using the polls feature. Being able to vote on ratings out of 5 for an add-on would be useful in my opinion, and I don't understand the negatives in using this discussion type. |
|
I think we should add it to the metadata so that a user can open the reviews from the add-on store. |
|
we need the category id which is a long value, and I have a workflow to get it, and this worked for my repo. i will try to find out the pools category id for nvaccess/addondatastore, and will send a pr with the final workflow.Enviado desde mi iPhoneEl 11 oct 2023, a las 5:56, Sean Budd ***@***.***> escribió:
The repository ID should be fetchable depending on the context that the action is run in.
I can create a category soon, but I'd like to consider using the polls feature. Being able to vote on ratings out of 5 for an add-on would be useful in my opinion, and I don't understand the negatives in using this discussion type.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
I've created a category here: https://github.com/nvaccess/addon-datastore/discussions/categories/add-on-reviews |
|
ok then my next step will be to find out the category id, and make a pr on addon-datastore repo sending the workflow and and empty discussions.json file to store addon ids with discussion ids and url, then address review, and proceed with validation repo and then maybe with NVDA creating an action"Enviado desde mi iPhoneEl 11 oct 2023, a las 6:25, Sean Budd ***@***.***> escribió:
@nvdaes
I've created a category here: https://github.com/nvaccess/addon-datastore/discussions/categories/add-on-reviews
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
@seanbudd , I've created pr nvaccess/addon-datastore#1757 |
Related to nvaccess/nvda#15576 Summary of the issue Add-ons submitted to the store don't require a previous human review, though some checks are performed automatically to ensure conformance with a predefined json schema, and to avoid the acceptance of add-ons which declare to be tested with non existing API versions. Though security cannot be warranted, a known place to make and consult human reviews can be provided. This maybe useful to search for feedback of users and reviewers before installing an add-on. Development process GitHub Actions have been used to create a discussion for each addonId, and a comment for each submission (add-on version),after add-on metadata is merged to the main branch of the addon-datastore repo. Some info about created discussions, associated with add-on ids, will be stored in a discussions.json file. Also, the URL corresponding with the comment for each submission will be stored in the add-on metadata.
|
@seanbudd , thanks for merging pr 1757 in addon-datastore repo. I've submitted readFeeds 26.0.0, and the corresponding discussion has been created at I'll try to submit other related pull requests. |
Closes #15576 Summary of the issue: The add-on store doesn't have a mechanism to see and provide feedback for add-ons. Description of user facing changes Now, it's possible to open a webpage to provide or see feedback for add-ons from an action added to the add-on store. The review URL is shown in the details panel for add-ons including a reviewUrl in metadata. Description of development approach An action has been added to provide feedback for add-ons, in a similar way to the approach followed for the license URL. Also, this has been added to the details panel.
Part of issue #15576. Summary of the issue: Now, add-ons can be reviewed from GitHub discussions, linked in the Available add-ons tab of the store. Description of user facing changes This improves documentation for this feature: Explained that this action can be found in the Available add-ons tab. Remove reference to comment on addon versions, since comments aren't longer added to add-ons metadata.
Is your feature request related to a problem? Please describe.
Add-ons don't require subjective reviews to be included in the store, and feedback can be sent directly to authors. This makes difficult to find feedback about add-ons from other users/code reviewers, specially if an add-on seems to be OK.
Describe the solution you'd like
In the actions menu for each add-on, it should be possible to open the issue corresponding to the add-on, or a GitHub discussion, which seems to make easier to be translated into other languages thanks to a GitHub feature, so that people can informa that an add-on is right, and users can consult this knowing the reviewer who provided the feedback.
Describe alternatives you've considered
The add-ons mailing list can be used, and also reviewers may post comments on different websites, encouraging people to use the store and specific add-ons.
Additional context
I've read negative comments by people complaining about the fact that the store can not be secure since add-ons are approved automatically, thoug seems that people previously complained since when add-ons were reviewed, some of them weren't accepted or considered.
The text was updated successfully, but these errors were encountered: