Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Working with cookies #175

Closed
lazywithclass opened this Issue Jul 9, 2012 · 22 comments

Comments

Projects
None yet

It would be nice to have helper functions to help work with cookies.

The basics being:

  1. saving cookies from within CasperJS not just from the command line
  2. retrieving cookies to use stored session
  3. deleting cookies

For example: I am testing the app we're developing, it's login protected, it would be nice to have only one test do the login and let the others use the generated cookie.

Thanks for the great work! :P

P.S.: I think I might help with this, I would need a few hints about where you would add this, so that I don't do it my way and find the result inappropriate with the existing architecture.

Owner

n1k0 commented Jul 9, 2012

Nice idea, but there's no really convenient API in PhantomJS for doing so… Yeah, we could use document.cookie but it's very very limited…

As a side note, this guy has suggested a nice API in his own PhantomJS fork https://github.com/neraliu/phantomjs/tree/improvedcookiesapi — but it's unfortunately unmerged yet :/

I wonder if we shouldn't wait for some official support of cookies in PhantomJS… What do you think?

I think that the best way would be to ask Ariya Hidayat about his plans.
If he's going to merge soon then I think you should wait, otherwise if he has different plans you should evaluate options:

  • giving your users the chance to use a phantomjs fork of their choice (I don't know if currently it's easy or painful)
  • depend on some external lib that eases the process of working with cookies
  • writing an API from scratch (brrr :D)

At the end it just depends on what features you think users would find useful to have, I can say what I would find useful:

  • find myself logged in when I instruct CasperJS from the CLI
  • find myself logged in when I instruct CasperJS from its API
  • read the information on the cookie
  • delete the cookie for a fresh start

In this order of priority.
These are my opinions on the subject, but I'm not that into the project so I might be saying nonsense here :P
I will look into this tornight, to have a better understanding of the subject.

Owner

n1k0 commented Jul 9, 2012

giving your users the chance to use a phantomjs fork of their choice

You can do that already, just set the PHANTOMJS_EXECUTABLE environment variable to the path of the phantomjs executable you want to use.

depend on some external lib that eases the process of working with cookies

The problem is they will use the document.cookie string, very limited (lack of many useful informations, probably for security purpose)

I think that the best way would be to ask Ariya Hidayat about his plans.

He looks to be very busy atm, and he won't be releasing any major API change until next planned version, in two months :/

find myself logged in when I instruct CasperJS from the CLI

Can't using the --cookies-file option helps? From what I've understood from it, it allows to set the path for storing the cookie file; so you may be able to reuse these? (untested)

Anyway, I'll be thinking of a way to ease reading cookies, but right now I don't feel it well :x

Contributor

sqren commented Oct 6, 2012

An API for handling cookies has been implemented in Phantomjs. This makes it possible to work with httponly cookies in Phantomjs, and it could be really useful to add this feature to Casperjs as well.

I've made a simple change to createPage() in the casper.js module. I've added the following lines:

casper.setCookies = page.setCookies;
casper.cookies = page.cookies;

This way casper gets access to phantoms cookie methods. There is probably a better wat to do it, but as for now, it works. Maybe this should be implemented properly into Casperjs?

Owner

n1k0 commented Oct 14, 2012

Why not let users accessing these methods through the page property? eg:

casper.then(function() {
    var cookies = this.page.cookies;
    this.page.setCookies(...);
});
Contributor

sqren commented Oct 15, 2012

That's probably a better idea, to avoid polluting the casper namespace.

Owner

n1k0 commented Oct 19, 2012

So I'm gonna close this one.

@n1k0 n1k0 closed this Oct 19, 2012

abbood commented Apr 7, 2013

interesting exchange you guys.. I noticed that If I login using my browser, then typing document.cookie from the chrome dev view gives me something like this:

"key1=value1"; "key2=value2"

but if I output the contents of the same cookie using the syntax above ie

casper.then(function() {
    var cookies = this.page.cookies;
});

then I get every key/value pair above wrapped like this:

domain: .. expires: .. expiry:.. httponly:.. name:.. path:.. secure:.. value..

where name is key and value is value.. I wonder why that's happening?

Owner

n1k0 commented Apr 7, 2013

@abbood that's the way phantomjs works with cookies

abbood commented Apr 7, 2013

yeah no biggie.. easily solved by this script:

var cookie = [];
$(wrappedCookies).each(function() {
    cookie.push(this.name + '=' + this.value);
});

abbood commented Apr 9, 2013

@n1k0 so this is a question about how to persist cookies from one casperjs page to another..

so basically i got a nodejs file that spawns casperjs as a worker to do certain tasks.. one is to login, once logged in I store the cookie in a file.

when i spawn the next casper worker.. i want it to to use the cookie rather having to login again.. both these methods failed:

  1. when i spawn the worker capserjs I add the --cookies-file=./cookiefilename ie
var child = spawn('casperjs',['monsterScrape2.js','--cookies-file=./'+cookieFileName]);```

2. within the casperjs worker file.. I make it read and set the cookie from a file ie

```javascript
var casper = require('casper').create();
var cookieFileName = 'monsterCookie.txt';

// grab cookies from file 
var fs = require('fs');
var utils = require('utils');
var cookies = fs.read(cookieFileName);

casper.page.setCookies(cookies); 

casper.start('https://domain/page.html', function() {
    //FAIL! cookies aren't used here
    this.debugHTML();
});

casper.run();

notes:

  1. it was mentioned earlier in n1k0#39 that start removes cookies from the page? if so how do I prevent that?
  2. I know that sessions persist within the same phantomjs page object (see here https://gist.github.com/abbood/5347252) and same happens within the same casperjs page object (see here https://gist.github.com/abbood/5347287)
  3. keep in mind that I store cookies as is in the file (ie without any json/cookie parsing at all).. so my cookie file looks exactly like this
[General]
cookies="@Variant(\0\0\0\x7f\0\0\0\x16QList<QNetworkCookie>\0\0\0\0\x1\0\0\0\n\0\0\0YCNTR=LB; expires=Tue, 09-Apr-2013 17:12:05 GMT; domain=.recruiter.domain.com; path=/\0\0\0qUID=13eb22f-2.21.171.120-1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0]UIDR=1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0[R_LANG=en; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.domain.com; path=/\0\0\0\x94\x43=4gpUmUGr2jgDrs4xOJVrGaNbD8DtYSd1E6quyLhe3E4F3EAGhbRJucnDgRVDeHh0; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.monstergulf.com; path=/\0\0\0\x94WT_FPC=id=20cf093f17f2c6f3d041365495136954:lv=1365495369854:ss=1365495136954; expires=Fri, 07-Apr-2023 08:16:09 GMT; domain=.domain.com; path=/\0\0\0\xc4\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTI4NzQ5NzQ0LjMwMjkxMjYxAAAAAAAAAAABAAAAmyoBAMo+ZFHhPWRRAQAAAAJWAADKPmRR4T1kUQAAAAA-; expires=Thu, 09-Apr-2015 16:16:10 GMT; domain=statse.domain.com; path=/\0\0\0Yv1st=CE061E87215F2D73; expires=Wed, 19-Feb-2020 14:28:00 GMT; domain=.domain.com; path=/\0\0\0\x84\x43OOKIE_ID=178.135.147.39-2368749744.30291261; expires=Fri, 07-Apr-2023 16:16:11 GMT; domain=cookie.domain.com; path=/DCS000065_7K5I\0\0\0\xbe\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTIzNjg3NDk3NDQuMzAyOTEyNjEAAAAAAAABAAAAQQAAAM0+ZFHNPmRRAQAAAAEAAADNPmRRzT5kUQAAAAA-; expires=Fri, 07-Apr-2023 16:16:13 GMT; domain=cookie.domain.com; path=/)"
Owner

n1k0 commented Apr 10, 2013

@abbood there's no WebPage#setCookies() method. Can you please try to run your casper test file from the command line (no node spawn) using the --cookies-file option and see if it works?

If it's not, feel free to open an issue.

abbood commented Apr 10, 2013

there isn't a webpage#setCookies()? but in this thread you said this:

Why not let users accessing these methods through the page property? eg:

casper.then(function() {
    var cookies = this.page.cookies;
    this.page.setCookies(...);
});

and I did try this without the spawn method (ie on it's own)..

anyways I opened a new issue here n1k0#445

Owner

n1k0 commented Apr 10, 2013

but in this thread you said this

I was mistaked, PhantomJS doesn't provide this method. It was just an example which occurs being invalid. The main idea was to let the developer deals with cookies using what the PhantomJS API actually offers.

abbood commented Apr 10, 2013

ok just to confirm: this.page.setCookie(..) doesn't work.. but this.page.cookies does work (I know b/c I already use it in my casperjs code)..

Writing and restoring the cookies:

login.js

//your usual casper stuff...
var fs = require('fs');
//do your login stuff...
casper.run(function() { 
  var cookies = JSON.stringify((this.page.cookies)); 
  fs.write('cookies.txt', cookies, 'w'); 
  this.exit();                                                                      
});

next-test.js

var casper = require('casper').create();
var endpoint = "http://yourdomain/";

var fs = require('fs');
var cookiefile = 'cookies.txt';
var cookies = JSON.parse(fs.read(cookiefile));

casper.on('remote.message', function(message) {
  console.log(message);
}); //this is how I determined my session wasn't carrying 

casper.start(endpoint, function() {

  this.page.cookies = cookies;

  //more stuff...

});

casper.run();
Contributor

clochix commented Jul 10, 2013

FYI, here are the functions I use to load cookies and save them https://gist.github.com/clochix/5967978
The file format seems compatible with the one used by browsers.

Do any of these work arounds fix the problem? Im having no success with cookies.

Contributor

davidlinse commented Oct 22, 2013

same here.. works as expected for me..

On Tue, Oct 22, 2013 at 10:55 PM, Jonathan Cummins <notifications@github.com

wrote:

Do any of these work arounds fix the problem? Im having no success with
cookies.


Reply to this email directly or view it on GitHubhttps://github.com/n1k0/casperjs/issues/175#issuecomment-26847115
.

hubpan pushed a commit to hubpan/casperjs that referenced this issue Feb 7, 2014

@andrewfaria i use your script to get the cookies.but restoring the cookies didn't success.

leahy16 commented May 31, 2016 edited

@andrewfaria likewise. Your restore cookies code doesn't work for me either (I'm using Casper 1.1.1)
@qileilove the following works for me with Casper 1.1.1

// GET COOKIES
var cookies = phantom.cookies; 
// returns an array of cookies, where each cookie is a JavaScript object

// SET COOKIES
for (var i = 0; i < cookies.length; i++) {
   phantom.addCookie(cookies[i]); 
}

Docs:
http://phantomjs.org/api/phantom/property/cookies.html
http://phantomjs.org/api/phantom/method/add-cookie.html

noinlijin commented Nov 15, 2016 edited

// save Cookie

var cookies = JSON.stringify(phantom.cookies);
var fs = require('fs');
fs.write("cookie.txt", cookies, "w");

// use Cookie

var cookieFileName = 'cookie.txt';
var fs = require('fs');
var cookies = fs.read(cookieFileName);
phantom.cookies = JSON.parse(cookies);
casper.start("https://twitter.com/",function () {
    casper.capture("1.png")
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment