Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Word requests for contextual sentences #1148

Open
kamillel opened this issue May 14, 2016 · 22 comments

Comments

@kamillel
Copy link

commented May 14, 2016

Feature: a user submits a word in a foreign language to get back sentences created by native speakers to show how this word can be used.

This feature has been requested many times, I compiled information from the GSoC 2015 page, user scenarios and Trang's use cases and I tried to re-use already existing similar behaviour patterns whenever possible to integrate it as seamlessly as possible with the current structure.

This feature has been called 'wishlist', 'word request'... I chose 'request' because it's clearer than 'wish' and I removed 'word' because it could be misleading, as users could ask for phrases or even sentences. Because Tatoeba is primarily about text, in my opinion it's obvious that the 'default' request is request about text, not about features or questions (if we had a page for feature requests, it should state 'feature requests').

Use case 1: I want to make a request
Access by clicking on my profile > My Requests
1_add_request

It works in a similar way as adding sentences.

Form:

  • Request: word or phrase (or even sentence) I want to submit
  • Language: word language
  • Notes: optional - can be used to ask for a specific context (e.g 'orange as a colour', or 'orange as a fruit')
  • Status: public (default) or unlisted. Unlisted means the requests won't appear on the general browsing list, so users can have more control on who can see their requests.

When I add a new request, it appears in the 'Requests added' list below (disappears if I leave the page).

Sidebar:
Requests links:

  • Add a request (current page)
  • All requests (open & closed & favourite)
  • Open requests
  • Closed requests
  • Favourite requests (optional)

Tips:
To explain how the feature works

Notes

  • This form could also be accessible from the search page. For example from the sidebar: 'Didn't find what you were looking for? > Add a request', and the request form would be pre-populated with the search inputs (word and language).
  • When a user submit a request, there could be a confirmation pop-up saying 'A similar request was already submitted for [word] with this context [notes]. Do you will want to submit your request? [Yes] [Cancel]'

Use case 2: I want to check the sentences added to my requests
Access by clicking on 'All requests'
2_all_requests

I get a list of all my open, closed and favourite requests.

I can filter them by language.

For each request, I see the language, the notes, the sentences that have been added already and the user.
If I click on the hashtag I'm going to the request page (in case I want to share the link).

For the requests I raised, I can edit, delete, close or re-open them.
For the requests I added to my favourites, I can 'un-favourite' them or add an example sentence.

If there is more than X requests, a pagination will appear. (I only put 4 on the wireframes to make it lighter) Maybe we could start with 10?

Use case 3: I want to add sentences for existing requests
Sentence list
Accessed by clicking on menu: Contribute > Add sentences
3_add_sentences-button

I see the open (public) requests.
I can filter them by language or by users.
If there are more than X (10?) a pagination will appear.
For each request, I can add them to my favourites or add a sentence by clicking on an the + icon.

Note:

  • In this wireframe, I grouped 'Add new sentences' and 'Add new sentences to answer a request' under the same page, so there would be only one entry in the menu: 'Add sentence' (because in both cases that what users are going to end up doing it).
    We could also split it in two: keep the 'Add new sentences' as it is, and add a separate 'Add new sentences for requests' (shorter: 'Answer requests?). Let me know what you prefer :)
  • I used an icon to be closer to the 'Translate' sentence look and feel as these mechanisms are quite close (icon could be different).

Sentence form
Accessed by clicking on the + for the request I want to answer
4_answer_request

I can see the request text, the language, user, notes and already added sentences.
I can add my sentence on the text field and click OK to add it.
It will appear at the top of the sentences list.

@LokSyut

This comment has been minimized.

Copy link

commented May 14, 2016

Perhaps, when you submit a request, it might be good to have an option to choose a specific user you want to add a sentence, if that’s not too difficult to organize.
And thanks again!

@alanfgh

This comment has been minimized.

Copy link
Contributor

commented May 15, 2016

This looks good. Nice work!

My initial comments:
(1) I understand why you wanted to avoid wordiness, but I guarantee that if you only call these "requests", people will submit feature requests here, too. The option I favor is to call them "word requests" in menu items and page titles, and refer to either "Word", "Word/phrase", or "Word or phrase" in the appropriate field. I think this is consistent with how we use the term "sentence" to refer to an entity that is usually a sentence but can contain multiple sentences; nevertheless, we are consistent in calling it a "sentence". Note that even if we call it a "word request", some people will misuse the feature, but I think it will be far fewer than if we simply call it a "request".

(2) I think some people will want to add a word request for multiple languages at a time. Will they be able to choose multiple languages from the drop-down list?

(3) Note that the rest of the website uses the English spelling for "favorite". I'll save my other comments about wording for later, when we're closer to consensus on how the UI will look.

I'll let you know if I have other comments on the basic UI after looking at your proposal some more.

@kamillel

This comment has been minimized.

Copy link
Author

commented May 15, 2016

Thank you for your feedbacks!

@LokSyut : it sounds like a good idea, users who want to keep their requests unlisted are likely to have a specific user in mind to add sentences
I'm not sure yet how the 'notifications' system could work (to notify the native speaker he/she received a word request), maybe as part of the messages if it makes sense, but that's also a question for the developers I'm not sure what is possible or not in this area

@alanfgh:
(1) Thinking back on it I think I agree with you, I was over-simplified it. Using 'word request' as a menu item / link sounds better as long as we can use more explicit/lengthy titles

(2) For simplicity sake for the first implementation, I would say no because I'm not sure how widely this would be used as this means having the same word with the same spelling in multiple languages, but maybe it's just my feeling and if many users request this we could add it

(3) It could be great if a native English speaker could review all of the wording :)

@trang

This comment has been minimized.

Copy link
Member

commented May 15, 2016

Thank you again for your work @kamillel :) Having pictures to think upon really helps. Here's my feedback.

Another approach for requests?

It occurred to me that maybe we should take a different approach for collecting requests. It seems more useful if we put an emphasis on the fact that users create requests around vocabulary that they are learning, and therefore the "requests" are more of a "vocabulary tracking list".

Among other things:

  • My requests could become My vocabulary.
  • Add a request could become Add a vocabulary item.

The problem if we take the "requests" approach, is that I feel we're sort of giving users the expectation that we can provide example sentences for any request, in a relatively short period of time. This is achievable for languages where we have active contributors, but it won't be the case for a lot of languages. If their requests remain unanswered for days, then weeks, users may feel discouraged from creating requests at all.

If however we take the "vocabulary list" approach, then users will always feel a use for the feature: even if their requests is never fulfilled, they can still use Tatoeba to simply keep track of which vocabulary they've been learning so far.

Status: public/unlisted

It is unclear to me how users would share unlisted requests so this is an option we may want to drop for the first iteration.

I personally think to implement this, we would need to provide a way for users to create lists of requests, and the public/unlisted status would be set on the list level rather than on the request level. It would be too cumbersome if users have to specify the status of each request individually, and share each unlisted request individually.

Implementing lists of requests will require a quite significant additional effort, which is why I'm suggesting to leave this on the side for now.

Request ownership

I don't think we need to associate a request to one specific user. A request can belong to several users, and instead of using the "favorites" concept, I think we should use the "+1" concept.
If I "+1" your request, I should be able to add my own notes, and you should no longer be able to edit the request content.

Roughly something like this:

(+1)                  requested by 1 person
____________________________________________
[eng] up the wall

* kamillel's notes: ...........

After I've +1'ed your request:

(x)                   requested by 2 people
____________________________________________
[eng] up the wall

* kamillel's notes: ...........
* trang's notes: ...........

I can click (x) if I change my mind and don't want to request it anymore. Then it goes back to the previous state.

@kamillel

This comment has been minimized.

Copy link
Author

commented May 16, 2016

@trang: thank you for your feebacks!

Vocabulary list
This approach makes a lot of sense, it's more "self-explanatory" and we avoid the confusion we were already feeling around the term 'requests' and even 'words'.

Status
I wasn't entirely sure what was possible in this area, it makes sense to leave it for a second iteration, there is quite a lot to think of on this to come with a neat solution

Request ownership
I was thinking that one request was owned by one user, in case two users ask for the same word but with a totally different context (e.g with homonyms). But maybe it's a not a real issue and users may prefer having many examples showing different uses (and in any case it could be easier to see if this is relevant once we see how the feature is really used). I will need to work on this flow to see how it fits with the open/closed lists, I may bug on Gitter :)

@trang trang added the enhancement label May 18, 2016

@kamillel

This comment has been minimized.

Copy link
Author

commented May 21, 2016

I changed 'request' references to 'vocabulary item', added the multiple ownership approach and used more the 'translating' mechanism.

Use case 1: I want to make a request
1_add_request

  • I removed the public/unlisted status for the first implementation as it requires a lot more thought before being implemented

Use case 2: I want to check the sentences added to my requests
2_all_requests

As suggested by Trang, I removed the single ownership concept:

  • as a user, there is no more difference between items that I personally created or that I am 'following'
  • the number next to the star is the number of users following the item, it's on the right hand side where users usually are
  • I can follow/unfollow an item by clicking on the star (like with favourites, the star is full if in my list, empty if not)

That means that items could en up being 'orphans'. I'm not sure if it's an issue or not but they could be automatically deleted if it is.

About the edit function, I don't know if it's possible to have two scenarios:

  • if a user is the single owner and no sentences have been added: the user can edit any part of the vocabulary item
  • once there are multiple 'owners' or sentences have been added: users can only edit their own notes
    But maybe for the first implementation, we could leave it open (so any owner could edit any part of it) and if it's start getting messy, add this restriction

In the same way it would only be possible to delete a request if there is only 1 user and no sentences.

Use case 3: I want to add sentences for existing requests

After a lot mulling over, I think the user goal is closer to translating (helping others) than adding sentences (asking for help), so I rearrange the wireframes in this way:

3_add_sentences
4_answer_request

The first interface would be a search and the second would be the list.
Adding a new contextual sentence would work 'inline' (within the search list), as adding a translation works.

Vocabulary items are ordered by the number of people requesting them (high to low).
If there are notes from different users, they all appearing below the vocabulary item, and if any sentences have been added, they appear below the notes.

Note;
As an enhancement, it could be nice to have an entry in 'Browse', so users could browse vocabulary items and add them to their lists.

@trang

This comment has been minimized.

Copy link
Member

commented May 24, 2016

There's one use case missing in the wireframes: the possibility for users to link existing sentences to their vocabulary. But we can leave this for later I think.

We can implement this feature with the following steps:

  • Step 1: Users can create a list of vocabulary that they are learning.
  • Step 2: Users can link sentences to their vocabulary.
  • Step 3: Users can request sentences for their vocabulary.

Here are the scenarios I have in mind.

Starting point

  • I'm reading an article in a foreign language.
  • I collect new vocabulary from the article by creating vocabulary items in Tatoeba.

Scenario 1

This scenario is more or less the ideal case, where I have control over the sentences that are connected to my vocabulary and I can explicitly request sentences.

  • For each item I create, I search in Tatoeba if there are existing sentences I can link to the vocabulary.
  • If I find good sentences, I link them to the item.
  • If I don't find good sentences, or if I don't find any sentence at all, I open a request for native speakers to create sentences for this item.
  • I can check somewhere if there are new sentences created for my requests.
  • From there I can decide whether or not I actually want to link these new sentences to my vocabulary items.

Scenario 2

This is a possible scenario for the first iteration. It focuses on implementing the "Step 1".

  • For each vocabulary item I create, Tatoeba will perform a search of sentences containing this vocabulary item.
  • The top results will be displayed to me (optional), and Tatoeba will store the number of results for this search.
  • Contributors can access a page with all the vocabulary items that were created (with possibly a filter to show only the items of a certain user). In this list, the items with the lowest number of search results will be displayed first, giving more visibility to items that have no results.
  • As they browse the list, contributors can add new sentences for vocabulary that have no sentences, or a low number of sentences.
  • Whenever I go to my vocabulary list, I can see the number of search results for a certain item, and I can click on the item to trigger the search, to see more precisely which sentences are returned for the search.

In this scenario don't control which items are linked to my vocabulary, instead we use the search engine as an intermediate: each of my vocabulary items are linked to all the search results for that item. This has some obvious limitations but it can be enough for a start.
I also cannot request explicitly for sentences. All the vocabulary items implicitly carry a request, and the fact that some items have less search results than other will serve as prioritization criteria.


There are more things we could drop for the first iteration.

  • The edit function. If a user makes a typo, instead of editing, they can remove the item and create a new one.
    When we do implement the edit function, we should actually allow all users to edit their items (even if other users are following the item). From the user's perspective, it will look like they are editing, but in fact what will happen is that the item will be removed from the user's vocabulary list, and a new item will be created and added to the vocabulary list.
  • The number of followers. At the beginning it will not matter a lot how many people are following a certain vocabulary item. What is more important is whether or not Tatoeba can provide example sentences for a certain item.
    We could also skip the "follow" function at first as well. If a user wants an existing item to be part of their vocabulary list, then would simply copy it manually and add it as a new item.
  • The notes. We'll need more time to figure out use cases for the notes. Maybe users would rather have private notes than public notes. I don't think for instance that I would use notes to specify the meaning in which I want a word to be used. I will usually want the word to be illustrated in as many different contexts as possible.
  • The open/closed status. We'll need more time to define how this status will be used. What happens when a user closes a request? What happens if a user never closes a request even though there are plenty of sentences for it?

If all of this makes sense to you, then I'd suggest we continue discussing the details in a separate issue that would cover the things to implement in the first iteration. I'll let you create it, and add your updated wireframes in it.

We can keep the current issue open to discuss about the next iterations.

@kamillel

This comment has been minimized.

Copy link
Author

commented May 29, 2016

Many thanks for your feedbacks, this approach makes a lot of sense, it was bugging me that previously the vocabulary items were a bit disconnected from the rest of the site, now it feels more integrated.

I don't think the concept of 'followers/requesters' will be needed any more, it was just a way of ordering items (to show the most 'needed' first) but using the amount of sentences sounds a more relevant way of ordering items for now.

I agree for the notes, I struggled to find good examples while making the wireframes, I think that something we need users' inputs on. Once people start using the requests we will see if there is a case for it.

I created a new issue for the first implementation: #1166

@ckjpn

This comment has been minimized.

Copy link

commented Jul 3, 2016

The Suggested Improvements.

I'd suggest changing the ALL CAPS to standard sentence capitalization, and also adding a period at the end of the sentence. It would also be a good idea to change this to the standard link color used throughout the website.

Why (The Problems)

  1. ALL CAPS looks like SHOUTING and is ALSO SLOWER TO READ.
  2. We ask members to contribute sentences with proper punctuation, so the website itself should use proper punctuation.
  3. It's always a good idea to have a standard user interface across a website.

URL and Screenshot

https://tatoeba.org/eng/sentences/add

screen shot 2016-07-03 at 10 06 00

(Note that the English wording for this could perhaps be better, but this is understandable.)

@ckjpn

This comment has been minimized.

Copy link

commented Jul 3, 2016

I'd also suggest using the standard link colors used throughout the website for these links, too, for the same reason as mentioned above. (#1148 (comment))

https://tatoeba.org/eng/vocabulary/of/TRANG
screen shot 2016-07-03 at 10 16 40

While, I find the current link color, light green, difficult to read, if you make sure that all the link colors on the website are standardized, it will be easier to change all the links in the future.

@ckjpn

This comment has been minimized.

Copy link

commented Jul 3, 2016

Problem

We likely need a way to prevent spam-like word requests.

Someone has already submitted multiple requests across languages mentioning a product name, WhatsApp.

This could potentially become a major problem and irritation to members.

2 Possible Approaches to Preventing This

  1. Make it possible to easily see who has requested items.
  2. Make it possible for Admins and possibly Corpus Maintainers to remove such items.

Perhaps other solutions could be considered, too.

@RyckRichards

This comment has been minimized.

Copy link
Member

commented Jul 3, 2016

@ckjpn

This comment has been minimized.

Copy link

commented Jul 3, 2016

Problem

Multiple near duplicate submissions are possible, depending on capitalization.

URL and Screenshot

https://dev.tatoeba.org/eng/vocabulary/of/CK

screen shot 2016-07-03 at 10 49 14

Note

This is not an immediate problem, perhaps, but it is something that likely needs to be dealt with. It could be ignored, of course, and just assume members will use the most logical capitalization of words.

It might be possible just to convert everything to UPPERCASE or lowercase. All lowercase is how most people likely search, but ALL UPPERCASE avoids the problem of seeing words like "boston" which should be capitalized.

@ckjpn

This comment has been minimized.

Copy link

commented Jul 3, 2016

Problem

An exact match for sentence counts implies that a word isn't covered, though it might be in another form of the word.

For example, on the dev site someone asked for "armpit" which shows 0 sentences, but the more normal way we use this word in the plural "armpits" has a number of sentence examples.

https://dev.tatoeba.org/eng/vocabulary/add_sentences/eng
screen shot 2016-07-03 at 10 50 54

Notes

It's probably still a good idea to maintain the exact search.
Maybe this is a problem that could just be ignored, but it is something that will likely cause some confusion.

Possible Solution.

Add a column to the right of the "0 SENTENCES", perhaps calling the column "fuzzy search" or something like that. In this column, have a link the the search for "armpit" without the equal sign (="armpit").

@sabretou

This comment has been minimized.

Copy link

commented Jul 3, 2016

Regarding exact search, it occurred to me that we should probably mention in the Tips section on the right side of the page that this system is based on exact search. I stress this because the Vocabulary system, by its nature, favours languages that do not feature heavy inflection.

For languages like Marathi or Sanskrit, which strongly inflect nouns, users would probably require a wildcard-based search at the very least.

@alexanderk409

This comment has been minimized.

Copy link

commented Jul 3, 2016

Vocabulary items that already have examples.

I think we should hide the words from the list of wanted items if they already have examples. As a native speaker I would prefer to contribute something that isn't in the database yet. If someone just wants more examples with a certain word I think he should be able to indicate how many sentence with that word he wants. It's something annoying when you accomplish someone's request but the word still remains in the list.

@ckjpn

This comment has been minimized.

Copy link

commented Jul 4, 2016

Perhaps the text on these 2 pages that link to the same page should be the same.

"Sentences Wanted"

  1. top screenshot URL
    https://tatoeba.org/eng/sentences/add
  2. bottom screenshot URL
    https://tatoeba.org/eng/vocabulary/of/CK
  3. URL of the link
    https://tatoeba.org/eng/vocabulary/add_sentences

screen shot 2016-07-04 at 16 05 44

@ckjpn

This comment has been minimized.

Copy link

commented Jul 5, 2016

Problem

When filtering by language, often the page is blank with no idication why it is blank.

https://tatoeba.org/eng/vocabulary/add_sentences/kor

screen shot 2016-07-06 at 08 21 39

Possible Solution

Put a message saying something like the following.

There are no requests for Korean.
There are no requests for this language.

There are no requests for Korean that do not already have at least 10 sentences.
There are no requests for this language that do not already have at least 10 sentences.

@zachleigh

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2016

Problem

In the current dev branch, adding a duplicate vocabulary item causes a sql error and screen freeze, but no message is given to user. In model vocabulary.php

    public function addItem($lang, $text) {
        $text = trim($text);
        if (empty($text) || empty($lang)) {
            return null;
        }
        $numSentences = $this->_getNumberOfSentences($lang, $text);
        $hash = murmurhash3($lang.$text);
        $data = array(
            'id' => $hash,
            'lang' => $lang,
            'text' => $text,
            'numSentences' => $numSentences
        );

        $this->save($data);
        $this->UsersVocabulary->add($hash, CurrentUser::get('id'));

        return $data;
    }

id in data needs to be unique.

Possible solutions:

  • Wrap $this->save($data); in a try/catch block and $return an error if sql throws error
  • Check for unique hash value before saving and return error

In the controller, check for 'error' index on $result:

        $result = $this->Vocabulary->addItem($lang, $text);
        if (!isset($result['error'])) {
                $hexValue = unpack('H*', $result['id']);
                $result['id'] = str_pad($hexValue[1], 32, '0');
                $numSentences = $result['numSentences'];
                $result['numSentencesLabel'] = format(
                    __n('{number} sentence', '{number} sentences', $numSentences, true),
                    array('number' => $numSentences)
                );
        }
        $this->set('result', $result);

        $this->layout = 'json';

And then pick up the error in add.ctrl.js.

@jiru

This comment has been minimized.

Copy link
Member

commented Aug 23, 2016

@zachleigh You can also use CakePHP’s data validation system to make CakePHP to check for uniqueness before saving. This way, the error is not reported by SQL but CakePHP, and thus easier to catch and handle.

@zachleigh

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2016

Thanks @jiru Ill look into that. Looks like a better option than my two suggestions.

@trang If you want, Ill handle this issue after completing #1278. Do you want me to make a separate issue for it or just keep it in here?

@trang

This comment has been minimized.

Copy link
Member

commented Aug 23, 2016

@trang If you want, Ill handle this issue after completing #1278.

That'd be great :)

Do you want me to make a separate issue for it or just keep it in here?

Separate issue would be better, to avoid overwhelming this thread, but in the end it doesn't matter too much. What's important for me is that the problems and solutions are documented and can be traced. So you could just reference to your comment above in your pull request, and that'd be fine too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.