Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. #930

wappowers · 2017-09-28T00:19:27Z

This is to address #891. I also added a "get cookie expiry" keyword, which I needed for the acceptance testing, and seems useful in any case. However, if that is an issue let me know.

wappowers · 2017-09-28T00:32:03Z

Firefox specific issue is causing test failure. I will look into this.

wappowers · 2017-09-28T04:12:50Z

It looks like this is failing because of a known issue with how Firefox gets the expiry from cookies. It is slated to be fixed in Firefox 56.
mozilla/geckodriver#463
https://bugzilla.mozilla.org/show_bug.cgi?id=1372595

aaltat · 2017-09-28T06:19:54Z

src/SeleniumLibrary/keywords/cookie.py

@@ -53,11 +53,22 @@ def get_cookie_value(self, name):
        raise ValueError("Cookie with name %s not found." % name)

    @keyword
-    def add_cookie(self, name, value, path=None, domain=None, secure=None):
+    def get_cookie_expiry(self, name):


I don't exactly right now remember what the Selenium get_cookie returns, but if it's a dictionary (like it looks like) it might be feasible to write a keyword which return the whole cookie object. Or write a keyword which will return specific key from the cookie. Example like:

| Get Cookie | name | Get Cookie Value | name | key |

The later example is enhancing existing keyword and therefore it would be good keep the backwards compatible as much as possible. There by default it should return the cookie value key.

aaltat · 2017-09-28T06:32:17Z

test/acceptance/keywords/cookies.robot

@@ -48,9 +48,14 @@ Get Cookies When There Are None
    ${cookies}=    Get Cookies
    Should Be Equal    ${cookies}    ${EMPTY}

+Get Cookie Expiry Set By Selenium


FF indeed seems to have a problem with cookies, please add a tag to the test Known_Issue_Firefox and CI will automatically mark this test as noncritical for Firefox.

Did you happen to try with other browsers, like IE, Edge or Safari, that do they suffer from the same problem?

aaltat · 2017-09-28T06:35:00Z

Adding the expiry argument looks good, but for the new keyword I think we should create more generic solution. See my comments for more detailed response

pekkaklarck · 2017-09-28T06:44:51Z

On a quick look the PR look very good. Few comments and questions related to it below:

Adding a new keyword to get the expiry value sounds like a good idea. Instead of adding a keyword for exactly that purpose, I would consider adding a keyword like Get Cookie or Get Cookie Information that returns an object with all information the cookie has as attributes. It could then be used in test data like this:
```
${cookie} =    Get Cookie    args    here
Log    ${cookie.name}
Should Be Equal    ${cookie.expiry}    ${expected}
```
Am I right that the expiry value means certain date? In that case it would be nice to be able to give it also in format like 2017-09-28 09:35:12. There's robot.utils.parse_time that we could use here similarly as SL already uses timestr_to_secs. That parse_time also handles nice formats like NOW and NOW + 1 day out-of-the-box.
New keywords and new/changed arguments need a note like This keyword is new in SeleniumLibrary 3.0. or Prior to SeleniumLibrary 3.0 the 'expiry' argument had no effect..
Did I get it right that the Firefox bug is only related to reading the value or does it affect also setting it? Either way, adding a note that the functionality doesn't work with Firefox < 56 ought to be enough. On CI the test can be disabled with Known Issue Firefox tag.

It seems @aaltat already commented about these topics. Next time we need to synchronize. =)

pekkaklarck · 2017-09-28T07:49:44Z

I discussed this with @aaltat and we agreed that support for setting expiry would be great in SL 3.0. As a result I assigned issue #891 into SL 3.0 rc 1 scope. We plan to get the rc 1 out next week. Do you @wappowers have time to work with this issue before that?

I also submitted separate issue #932 about getting all cookie information. I think that's better than adding separate Get Cookie Expiry keyword. If adding a keyword that returns all information as an object turns out to be too much work for SL 3.0, we can also change the current Get Cookie Value to allow returning other information than the value. That would be enough to test setting the expiry value, and would also be trivial to implement:

    def get_cookie_value(self, name, info='value'):
        cookie = self.browser.get_cookie(name)
        if cookie is not None:
            return cookie[info]
        raise ValueError("Cookie with name %s not found." % name)

wappowers · 2017-09-28T13:09:10Z

@aaltat @pekkaklarck Thanks for the feedback! I will definitely have time to work on this in the next few days, so getting this in shape by the end of the weekend is no problem.

The Firefox bug is only affecting retrieving the cookie expiry, it looks like it is setting it correctly.

I have tested this on Safari as well, and the time is being set correctly, but Safari stores the time in a milliseconds format, so the acceptance test is failing. I will see if there is something that can be done with this, I imagine there is. I have not checked this on the Microsoft browsers yet.

Since I need some way to get the cookie expiry value, if I try to address #932 as well, would it be ok to keep that in this PR? I will also look at allowing the expiry settings to be more flexible using robot.utils.parse_time.

pekkaklarck · 2017-09-28T13:20:48Z

Sounds great! I'm fine either keeping implementation of #932 in this PR or having it in another one. In general it's cleaner to have one feature per PR, but this time it's pretty hard to test one without the other.

Did I get it right that Safari stores expiry as milliseconds but others use seconds? That's pretty strange. I think we should always return it in the same format, but then there's a question which format to use. I'll add a separate comment about that into #932.

pekkaklarck · 2017-09-28T13:34:32Z

So, I proposed using Python's standard datetime object to represent expiry in #932. If we do that, then the same format should obviously be supported also when setting expiry. Unfortunately robot.utils.parse_time doesn't support datetime (it really should!). Either we can check the expiry type first, or we can use Robot's DateTime library that supports also that format. That library is included since RF 2.8.5, and we have RF 2.8.7 as the minimum requirement, so it ought to be safe to use it.

…ded.

wappowers · 2017-09-28T21:26:04Z

I made a few changes to try and get this ready. The new 'Get Cookie' keyword returns a dictionary of the named cookie, and this could address #932. However, the acceptance test is failing when using Robot 2.8.7 because the cookie is a dictionary, and it look like they dictionary support was added to Builtin with 2.9.0. I can update the acceptance test to use the Collections library, or another way if there is a better approach.

The other change was to have the expiry set using a datetime format. However, the Selenium 'get cookie' command is still returning the cookie in epoch time. I opted not to reformat that before returning it to the user, since altering the data returned felt off to me. However, I can do that if it is the preferred choice. I don't love the way I handled this, but it was the best I could come up with.

The other failure in the acceptance tests was a random one with javascript in Firefox. I cannot imagine my changes affected that, but I will look into it to be sure.

pekkaklarck · 2017-09-28T22:19:54Z

I don't have time to really look at this now but one comment: Instead of returning the cookie informaion as a dict, I think you should return a separate Cookie or CookieInformation object which has all the cookie information as its attributes. That would be much easier to handle in test data using the extended variable syntax and implementation wouldn't be too hard:

class CookieInformation(object):

    def __init__(self, name, value, expiry, other, stuff, ...):
        self.name = name
        self.value = value
        ...

    def __str__(self):
        return '{}={}'.format(self.name, self.value)

…brary

… a dictionary.

wappowers · 2017-09-29T05:14:51Z

Thanks @pekkaklarck, I made the change you suggested, and it solved some of my other problems, like needing to use the collections library for dictionary compatibility. I could not find another example of a keyword file with more than one 'class' in it. So if implemented this incorrectly I can fix it.

pekkaklarck · 2017-09-29T08:04:12Z

Apparently no other keyword has yet needed such a model object, and I think they are used a lot less than they should in general. One good example is Robot's Process library that returns a handy result object. And yeah, having the class in the module that needs it is just fine.

wappowers · 2017-09-29T16:00:50Z

If you have time @aaltat, this is passing the acceptance tests, and I think it is ready to be looked at again. Thanks.

aaltat · 2017-10-01T19:24:10Z

src/SeleniumLibrary/keywords/cookie.py

+        If no cookie is found with `name`, this keyword fails.
+        """
+        cookie = self.browser.get_cookie(name)
+        if cookie is not None:


Write: if cookie:

aaltat · 2017-10-01T19:25:56Z

src/SeleniumLibrary/keywords/cookie.py

+        """
+        cookie = self.browser.get_cookie(name)
+        if cookie is not None:
+            cookie_information = CookieInformation(cookie['name'], cookie['value'], cookie['domain'], cookie['path'],


Please follow pep8 in line length recommendation in new code. Only the keyword examples can normally exceed the line length.

aaltat · 2017-10-01T19:47:35Z

src/SeleniumLibrary/keywords/cookie.py

        self.browser.add_cookie(new_cookie)
+
+
+class CookieInformation(object):


It is better ro write CookieInformation class like this:

class CookieInformation(object): def __init__(self, **cookie_dict): for key in cookie_dict: setattr(self, key, cookie_dict[key])

And write line 64 and 65 like this:
return CookieInformation(**cookie)

Because selenium cookie is a dictionary and at least in Windows get_cookie does not return expiry if it not defined.

First of all, such a generic __init__ would be even better implemented like this:

def __init__(self, data): self.__dict__.update(data)

I'm don't think that is a good idea in this context, though:

Explicit is better than implicit. We are saving few lines of code but hiding information. Being able to use this object you needed to know what info the Selenium method returned.

Making it sure we always return same info regardless what browsers return is important.

Then something like this could work better:

class CookieInformation(object): def __init__(self, cookie): self.domain = cookie['domain'] if 'domain' in cookie else None self.expiry = cookie['expiry'] if 'expiry' in cookie else None self.httpOnly = cookie['httpOnly'] if 'httpOnly' in cookie else None self.name = cookie['name'] if 'name' in cookie else None self.path = cookie['path'] if 'path' in cookie else None self.secure = cookie['secure'] if 'secure' in cookie else None self.value = cookie['value'] if 'value' in cookie else None @property def full_info(self): return self.__dict__

aaltat · 2017-10-01T19:48:06Z

src/SeleniumLibrary/keywords/cookie.py

+        self.http = http
+        self.secure = secure
+
+    def __str__(self):


The __str__ method is not needed.

If actually is. Robot automatically logs return values from kws and without this method the message is useless. Should be discussed what info to show, though. Could only show name and value and keep the message short or return all the info. I'd probably keep __str__ simple but could add a separate property showing everything that could be used like

Log ${cookie.full_info}

if needed.

I did not know that it is needed. Most of the time, I am interested about the name and value, because for me they are the most important information which I need. I think we can leave it as is and Pekka idea about full_info could be useful.

aaltat · 2017-10-01T20:07:11Z

test/acceptance/keywords/cookies.robot

+    [Documentation]    Get Cookie Value Set By Selenium
+    [Tags]  Known Issue Firefox
+    ${cookie_dict}=    Get Cookie    another
+    ${expiry} =  Convert To Integer  1822148495


One can not trust that expiry value is constant. Please set the cookie expiry first and then read it.

aaltat · 2017-10-01T20:22:33Z

src/SeleniumLibrary/keywords/cookie.py

@@ -67,4 +81,22 @@ def add_cookie(self, name, value, path=None, domain=None, secure=None):
        # Secure should be True or False
        if is_truthy(secure):
            new_cookie['secure'] = secure
+        if is_truthy(expiry):
+            expiry_datetime = datetime.strptime(expiry, '%Y-%m-%d %H:%M:%S')


It would be better to use Robot Framework DataTime library for better support different time formats.

Example something like this could work:

from robot.libraries.DateTime import convert_date ... expiry_datetime = int(convert_date('20140528 12:05:03.111', result_format='epoch'))

Then the keyword documentation could say that expiry supports same formats as DateTime library,

aaltat · 2017-10-01T20:27:25Z

The code looks generally OK, but there are some changes needed to cover some cases.

wappowers · 2017-10-02T04:38:13Z

@aaltat Thank you for the feedback, it was very helpful. I have made the changes you requested, except for those that you are discussing with @pekkaklarck. If those conversations result in additional changes being required I can definitely make them.

I have my Firefox tests failing off and on because of the Confirm Action test. It may be related to #934. But since this is a javascript test, I do not see how my changes could be related.

aaltat · 2017-10-02T18:43:03Z

src/SeleniumLibrary/keywords/cookie.py

+        self.http = http
+        self.secure = secure
+
+    def __str__(self):


I did not know that it is needed. Most of the time, I am interested about the name and value, because for me they are the most important information which I need. I think we can leave it as is and Pekka idea about full_info could be useful.

aaltat · 2017-10-02T19:11:16Z

src/SeleniumLibrary/keywords/cookie.py

        self.browser.add_cookie(new_cookie)
+
+
+class CookieInformation(object):


Then something like this could work better:

class CookieInformation(object): def __init__(self, cookie): self.domain = cookie['domain'] if 'domain' in cookie else None self.expiry = cookie['expiry'] if 'expiry' in cookie else None self.httpOnly = cookie['httpOnly'] if 'httpOnly' in cookie else None self.name = cookie['name'] if 'name' in cookie else None self.path = cookie['path'] if 'path' in cookie else None self.secure = cookie['secure'] if 'secure' in cookie else None self.value = cookie['value'] if 'value' in cookie else None @property def full_info(self): return self.__dict__

aaltat · 2017-10-02T19:15:25Z

Github seems to hide the outdated diff discussion by default, which is not nice. To be sure, this is thing what I wanted to say for the CookieInformation class.

Something like this could work then:


class CookieInformation(object):

    def __init__(self, cookie):
        self.domain = cookie['domain'] if 'domain' in cookie else None
        self.expiry = cookie['expiry'] if 'expiry' in cookie else None
        self.httpOnly = cookie['httpOnly'] if 'httpOnly' in cookie else None
        self.name = cookie['name'] if 'name' in cookie else None
        self.path = cookie['path'] if 'path' in cookie else None
        self.secure = cookie['secure'] if 'secure' in cookie else None
        self.value = cookie['value'] if 'value' in cookie else None

    @property
    def full_info(self):
        return self.__dict__

Other changes looks good to go.

If you want to be mentioned when we publish next release, please add your name to the the https://github.com/robotframework/SeleniumLibrary/blob/master/CHANGES.rst file.

pekkaklarck · 2017-10-02T19:41:46Z

This isn't a big deal, but in my opinion model objects like CookieInformation should explicitly list arguments they expect rather than getting in some other data that they need to handle somehow. Model objects being independent on certain external data presentation makes them easier to use in different context and to unit test without mocking/stubbing. Arguments accepted by model objects should, when feasible, also have default values to make using them even more flexible. This I'd prefer code like this:

class CookieInformation(object):

    def __init__(self, name, value, expiry=None, ...):
        self.name = name
        self.value = value
        self.expiry = # code to handle expiry somehow
        # ...

This could then be used, for example, like:

cookie = CookieInformation(data['name'], data['value'], data.get('expiry'), ...)

Having __str__ is obviously not mandatory, but it's really useful. Also, if we add separate full_info property for logging purposes, it should return a string.

…ew changes.

wappowers · 2017-10-03T04:23:52Z

I updated this with the changes @aaltat suggested and added a full_info property that returns a string as @pekkaklarck requested.

Add 'expiry' to 'add cookie' keyword and add get cookie expiry keyword.

a242eae

aaltat requested changes Sep 28, 2017

View reviewed changes

pekkaklarck mentioned this pull request Sep 28, 2017

Fix setting cookie expiry date #891

Closed

Datetime format is now used for expiry setting. Get cookie keyword ad…

3938b59

…ded.

Using Collections library for dictionary reliant acceptance test.

1e9b12a

wappowers added 2 commits September 28, 2017 22:42

Merge branch 'master' of https://github.com/robotframework/SeleniumLi…

7f556b6

…brary

Refactor to have 'get cookie' return an extended variable rather than…

52f4db4

… a dictionary.

wappowers changed the title ~~Add 'expiry' to 'add cookie' keyword and add 'get cookie expiry' keyword.~~ Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. Sep 29, 2017

aaltat requested changes Oct 1, 2017

View reviewed changes

wappowers added 2 commits October 1, 2017 23:06

Updated to use Robot Framework DateTime library, and PEP8 changes.

56730c3

Acceptance tests pass regardless of timezone.

63f0209

aaltat requested changes Oct 2, 2017

View reviewed changes

Added name to CHANGES.rst file.

6b57846

Added full_info option to CookieInformation class, and some code revi…

dfa3fab

…ew changes.

aaltat merged commit d506f19 into robotframework:master Oct 5, 2017

aaltat mentioned this pull request Oct 5, 2017

Add keyword to get all cookie information #932

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. #930

Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. #930

wappowers commented Sep 28, 2017

wappowers commented Sep 28, 2017

wappowers commented Sep 28, 2017 •

edited

Loading

aaltat Sep 28, 2017

aaltat Sep 28, 2017

aaltat commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017 •

edited

Loading

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 29, 2017

pekkaklarck commented Sep 29, 2017

wappowers commented Sep 29, 2017

aaltat Oct 1, 2017

aaltat Oct 1, 2017

aaltat Oct 1, 2017

pekkaklarck Oct 1, 2017

aaltat Oct 2, 2017

aaltat Oct 1, 2017

pekkaklarck Oct 1, 2017

aaltat Oct 2, 2017

aaltat Oct 1, 2017

aaltat Oct 1, 2017

aaltat commented Oct 1, 2017

wappowers commented Oct 2, 2017

aaltat Oct 2, 2017

aaltat Oct 2, 2017

aaltat commented Oct 2, 2017

pekkaklarck commented Oct 2, 2017 •

edited

Loading

wappowers commented Oct 3, 2017

		self.browser.add_cookie(new_cookie)


		class CookieInformation(object):

Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. #930

Add 'expiry' to 'add cookie' keyword and add 'get cookie' keyword. #930

Conversation

wappowers commented Sep 28, 2017

wappowers commented Sep 28, 2017

wappowers commented Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaltat commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017 • edited Loading

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 28, 2017

pekkaklarck commented Sep 28, 2017

wappowers commented Sep 29, 2017

pekkaklarck commented Sep 29, 2017

wappowers commented Sep 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaltat commented Oct 1, 2017

wappowers commented Oct 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaltat commented Oct 2, 2017

pekkaklarck commented Oct 2, 2017 • edited Loading

wappowers commented Oct 3, 2017

wappowers commented Sep 28, 2017 •

edited

Loading

pekkaklarck commented Sep 28, 2017 •

edited

Loading

pekkaklarck commented Oct 2, 2017 •

edited

Loading