Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Working with cookies #175

Closed
lazywithclass opened this Issue · 19 comments

8 participants

@lazywithclass

It would be nice to have helper functions to help work with cookies.

The basics being:

  1. saving cookies from within CasperJS not just from the command line
  2. retrieving cookies to use stored session
  3. deleting cookies

For example: I am testing the app we're developing, it's login protected, it would be nice to have only one test do the login and let the others use the generated cookie.

Thanks for the great work! :P

P.S.: I think I might help with this, I would need a few hints about where you would add this, so that I don't do it my way and find the result inappropriate with the existing architecture.

@n1k0
Owner

Nice idea, but there's no really convenient API in PhantomJS for doing so… Yeah, we could use document.cookie but it's very very limited…

As a side note, this guy has suggested a nice API in his own PhantomJS fork https://github.com/neraliu/phantomjs/tree/improvedcookiesapi — but it's unfortunately unmerged yet :/

I wonder if we shouldn't wait for some official support of cookies in PhantomJS… What do you think?

@lazywithclass

I think that the best way would be to ask Ariya Hidayat about his plans.
If he's going to merge soon then I think you should wait, otherwise if he has different plans you should evaluate options:

  • giving your users the chance to use a phantomjs fork of their choice (I don't know if currently it's easy or painful)
  • depend on some external lib that eases the process of working with cookies
  • writing an API from scratch (brrr :D)

At the end it just depends on what features you think users would find useful to have, I can say what I would find useful:

  • find myself logged in when I instruct CasperJS from the CLI
  • find myself logged in when I instruct CasperJS from its API
  • read the information on the cookie
  • delete the cookie for a fresh start

In this order of priority.
These are my opinions on the subject, but I'm not that into the project so I might be saying nonsense here :P
I will look into this tornight, to have a better understanding of the subject.

@n1k0
Owner

giving your users the chance to use a phantomjs fork of their choice

You can do that already, just set the PHANTOMJS_EXECUTABLE environment variable to the path of the phantomjs executable you want to use.

depend on some external lib that eases the process of working with cookies

The problem is they will use the document.cookie string, very limited (lack of many useful informations, probably for security purpose)

I think that the best way would be to ask Ariya Hidayat about his plans.

He looks to be very busy atm, and he won't be releasing any major API change until next planned version, in two months :/

find myself logged in when I instruct CasperJS from the CLI

Can't using the --cookies-file option helps? From what I've understood from it, it allows to set the path for storing the cookie file; so you may be able to reuse these? (untested)

Anyway, I'll be thinking of a way to ease reading cookies, but right now I don't feel it well :x

@sqren

An API for handling cookies has been implemented in Phantomjs. This makes it possible to work with httponly cookies in Phantomjs, and it could be really useful to add this feature to Casperjs as well.

I've made a simple change to createPage() in the casper.js module. I've added the following lines:

casper.setCookies = page.setCookies;
casper.cookies = page.cookies;

This way casper gets access to phantoms cookie methods. There is probably a better wat to do it, but as for now, it works. Maybe this should be implemented properly into Casperjs?

@n1k0
Owner

Why not let users accessing these methods through the page property? eg:

casper.then(function() {
    var cookies = this.page.cookies;
    this.page.setCookies(...);
});
@sqren

That's probably a better idea, to avoid polluting the casper namespace.

@n1k0
Owner

So I'm gonna close this one.

@n1k0 n1k0 closed this
@abbood

interesting exchange you guys.. I noticed that If I login using my browser, then typing document.cookie from the chrome dev view gives me something like this:

"key1=value1"; "key2=value2"

but if I output the contents of the same cookie using the syntax above ie

casper.then(function() {
    var cookies = this.page.cookies;
});

then I get every key/value pair above wrapped like this:

domain: .. expires: .. expiry:.. httponly:.. name:.. path:.. secure:.. value..

where name is key and value is value.. I wonder why that's happening?

@n1k0
Owner

@abbood that's the way phantomjs works with cookies

@abbood

yeah no biggie.. easily solved by this script:

var cookie = [];
$(wrappedCookies).each(function() {
    cookie.push(this.name + '=' + this.value);
});
@abbood

@n1k0 so this is a question about how to persist cookies from one casperjs page to another..

so basically i got a nodejs file that spawns casperjs as a worker to do certain tasks.. one is to login, once logged in I store the cookie in a file.

when i spawn the next casper worker.. i want it to to use the cookie rather having to login again.. both these methods failed:

  1. when i spawn the worker capserjs I add the --cookies-file=./cookiefilename ie
var child = spawn('casperjs',['monsterScrape2.js','--cookies-file=./'+cookieFileName]);```

2. within the casperjs worker file.. I make it read and set the cookie from a file ie

```javascript
var casper = require('casper').create();
var cookieFileName = 'monsterCookie.txt';

// grab cookies from file 
var fs = require('fs');
var utils = require('utils');
var cookies = fs.read(cookieFileName);

casper.page.setCookies(cookies); 

casper.start('https://domain/page.html', function() {
    //FAIL! cookies aren't used here
    this.debugHTML();
});

casper.run();

notes:
1. it was mentioned earlier in #39 that start removes cookies from the page? if so how do I prevent that?
2. I know that sessions persist within the same phantomjs page object (see here https://gist.github.com/abbood/5347252) and same happens within the same casperjs page object (see here https://gist.github.com/abbood/5347287)
3. keep in mind that I store cookies as is in the file (ie without any json/cookie parsing at all).. so my cookie file looks exactly like this

[General]
cookies="@Variant(\0\0\0\x7f\0\0\0\x16QList<QNetworkCookie>\0\0\0\0\x1\0\0\0\n\0\0\0YCNTR=LB; expires=Tue, 09-Apr-2013 17:12:05 GMT; domain=.recruiter.domain.com; path=/\0\0\0qUID=13eb22f-2.21.171.120-1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0]UIDR=1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0[R_LANG=en; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.domain.com; path=/\0\0\0\x94\x43=4gpUmUGr2jgDrs4xOJVrGaNbD8DtYSd1E6quyLhe3E4F3EAGhbRJucnDgRVDeHh0; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.monstergulf.com; path=/\0\0\0\x94WT_FPC=id=20cf093f17f2c6f3d041365495136954:lv=1365495369854:ss=1365495136954; expires=Fri, 07-Apr-2023 08:16:09 GMT; domain=.domain.com; path=/\0\0\0\xc4\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTI4NzQ5NzQ0LjMwMjkxMjYxAAAAAAAAAAABAAAAmyoBAMo+ZFHhPWRRAQAAAAJWAADKPmRR4T1kUQAAAAA-; expires=Thu, 09-Apr-2015 16:16:10 GMT; domain=statse.domain.com; path=/\0\0\0Yv1st=CE061E87215F2D73; expires=Wed, 19-Feb-2020 14:28:00 GMT; domain=.domain.com; path=/\0\0\0\x84\x43OOKIE_ID=178.135.147.39-2368749744.30291261; expires=Fri, 07-Apr-2023 16:16:11 GMT; domain=cookie.domain.com; path=/DCS000065_7K5I\0\0\0\xbe\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTIzNjg3NDk3NDQuMzAyOTEyNjEAAAAAAAABAAAAQQAAAM0+ZFHNPmRRAQAAAAEAAADNPmRRzT5kUQAAAAA-; expires=Fri, 07-Apr-2023 16:16:13 GMT; domain=cookie.domain.com; path=/)"
@n1k0
Owner

@abbood there's no WebPage#setCookies() method. Can you please try to run your casper test file from the command line (no node spawn) using the --cookies-file option and see if it works?

If it's not, feel free to open an issue.

@abbood

there isn't a webpage#setCookies()? but in this thread you said this:

Why not let users accessing these methods through the page property? eg:

casper.then(function() {
    var cookies = this.page.cookies;
    this.page.setCookies(...);
});

and I did try this without the spawn method (ie on it's own)..

anyways I opened a new issue here #445

@n1k0
Owner

but in this thread you said this

I was mistaked, PhantomJS doesn't provide this method. It was just an example which occurs being invalid. The main idea was to let the developer deals with cookies using what the PhantomJS API actually offers.

@abbood

ok just to confirm: this.page.setCookie(..) doesn't work.. but this.page.cookies does work (I know b/c I already use it in my casperjs code)..

@myimedia

Writing and restoring the cookies:

login.js

//your usual casper stuff...
var fs = require('fs');
//do your login stuff...
casper.run(function() { 
  var cookies = JSON.stringify((this.page.cookies)); 
  fs.write('cookies.txt', cookies, 'w'); 
  this.exit();                                                                      
});

next-test.js

var casper = require('casper').create();
var endpoint = "http://yourdomain/";

var fs = require('fs');
var cookiefile = 'cookies.txt';
var cookies = JSON.parse(fs.read(cookiefile));

casper.on('remote.message', function(message) {
  console.log(message);
}); //this is how I determined my session wasn't carrying 

casper.start(endpoint, function() {

  this.page.cookies = cookies;

  //more stuff...

});

casper.run();
@clochix

FYI, here are the functions I use to load cookies and save them https://gist.github.com/clochix/5967978
The file format seems compatible with the one used by browsers.

@jonathancummins

Do any of these work arounds fix the problem? Im having no success with cookies.

@davidlinse
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.