Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add mongodb backend #5

Merged
merged 36 commits into from

2 participants

@ithinkihaveacat

No description provided.

added some commits
@ithinkihaveacat Ignore node_modules dcc67fb
@ithinkihaveacat Make mongodb available 6871eb8
@ithinkihaveacat Add mongodb dependency 32853db
@ithinkihaveacat Add infrastructure to support multiple backends 6a8f2ee
@ithinkihaveacat CS fixes 60d780e
@ithinkihaveacat Rename knock() -> group(), add new knock() 3852b64
@ithinkihaveacat Restructure to support multiple backend caches 005108d
@ithinkihaveacat Merge remote-tracking branch 'origin/master' into add-mongodb-backend 344610f
@ithinkihaveacat Remove support for 0.6
Think this doesn't work on 0.6 because in 0.6, server.close() does not
take a callback. (Have to attach the close event separately?)
f7ff0c2
@ithinkihaveacat CS fixes f7c4c45
@ithinkihaveacat CS fixes 89ec6b7
@ithinkihaveacat Add afirst() e5b3465
@ithinkihaveacat Fix utils.js, simple.js 4d3c2e0
@ithinkihaveacat Disable cache, filter tests 0056b96
@ithinkihaveacat Simplify
Remove function that's a bit too clever for its own good.
49e78fd
@ithinkihaveacat Fix test/cache.js 912daf6
@ithinkihaveacat Fix test/filter.js a590000
@ithinkihaveacat CS fixes 8422539
@ithinkihaveacat Support only-if-cached 4ed3ae2
@ithinkihaveacat Improve robustness of header handling 126e03c
@ithinkihaveacat Restructure files
Break fishback.js into separate files.
49ad5c6
@ithinkihaveacat Refactor handling of the 'request' event 991309e
@ithinkihaveacat Make req/res filtering happen on events
The way it should've been from the beginning (!).
185b617
@ithinkihaveacat Remove all network activity from tests
Will add some end-to-end tests back in later.
5669374
@ithinkihaveacat CS fixes, and remove dead functions 1db0c71
@ithinkihaveacat Update package.json, remove fishback-* scripts ca18d97
@ithinkihaveacat Move Client() into separate file 2e41b9d
@vid

Hi development team,

I'd really like to use a nodej caching proxy with hooks (for bayeux messages in my case), is this ready for use now or do you have a timeline?

thanks!

— users

@ithinkihaveacat Simply main loop
Make "backends" (cache, HTTP client) request() signature match that of
fishback.request.  This makes a bunch of stuff simpler.
a583aec
@ithinkihaveacat

@vid What sort of hooks do you want? I've added a lot of hooks to this branch, because they're used internally. The request listener itself will fire newRequest and newResponse events, and the cache and HTTP client backend will also fire similar events.

Note that you can't alter request or response bodies via the event listeners, only status codes and headers. (The system is designed to transform headers, basically.)

@ithinkihaveacat Add listener during Proxy creation
It's incorrect to add while processing each request!
ad68599
@vid

I'd have to install and work with it more to better understand. Does it work now? I notice you removed the startup examples so wasn't sure how to best get it going.

thanks!

@vid
vid commented

still hoping you might provide a simple "start it up" example, if it's currently functional (that's not clear).

in case you have a moment to help!, I've completely fallen for nodejs and am taking a deep dive (sometimes feeling like I'm drowning, hah hah). the project I'm working on is a research project where we identify entities from websites. I want to use a caching proxy to avoid abusing the websites, and to inject some js instrumentation into the pages.

@ithinkihaveacat ithinkihaveacat merged commit b56e131 into master

1 check passed

Details default The Travis build passed
@ithinkihaveacat

@vid I've added an example back in now, but please note that this code is all pretty rough! What were you wanting to do? If you can let me know what you're wanting to do in some detail I can give you some idea how to achieve it, or indeed let you know if fishback is suitable!

@vid
vid commented

thanks for asking! what we're doing is working with content specific web sites, such as health and science. we're using a backend text mining tool, and would like to find a way to interactively annotate pages, and update what's to be annotated. We messed around with userscripts for a while, but they have a number of drawbacks. I think a proxy approach will be better. I specifically want a caching server, in order to avoid abusing web sites, but also so updated content can be stored and I can index the content. But I need to inject some javascript instrumentation to served pages. I also want to be able to intercept calls, and add client-server interaction. I would also like to support https.

I've managed to get everything but caching and https working based on https://github.com/ayurmedia/proxy. thanks @ayurmedia! Working around issues related to gzipped pages was interesting, and it still has some glitches with some sites. It uses bayeux pubsub. I can upload that if interested. But I'd rather not reinvent the caching wheel. However I'm not sure if you're interested in rewriting content, and how https fits in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 21, 2013
  1. Ignore node_modules

    authored
  2. Make mongodb available

    authored
  3. Add mongodb dependency

    authored
Commits on Jan 24, 2013
  1. CS fixes

    authored
Commits on Jan 25, 2013
  1. Remove support for 0.6

    authored
    Think this doesn't work on 0.6 because in 0.6, server.close() does not
    take a callback. (Have to attach the close event separately?)
  2. CS fixes

    authored
Commits on Feb 15, 2013
  1. CS fixes

    authored
Commits on Feb 16, 2013
  1. Add afirst()

    authored
Commits on Feb 17, 2013
Commits on Feb 18, 2013
  1. Simplify

    authored
    Remove function that's a bit too clever for its own good.
  2. Fix test/cache.js

    authored
  3. Fix test/filter.js

    authored
  4. CS fixes

    authored
Commits on Feb 21, 2013
  1. Restructure files

    authored
    Break fishback.js into separate files.
Commits on Feb 22, 2013
  1. Make req/res filtering happen on events

    authored
    The way it should've been from the beginning (!).
Commits on Feb 23, 2013
  1. Remove all network activity from tests

    authored
    Will add some end-to-end tests back in later.
Commits on Feb 24, 2013
Commits on Feb 28, 2013
  1. Simply main loop

    authored
    Make "backends" (cache, HTTP client) request() signature match that of
    fishback.request.  This makes a bunch of stuff simpler.
  2. Add listener during Proxy creation

    authored
    It's incorrect to add while processing each request!
Commits on Mar 3, 2013
  1. Emit 'endHead' before 'newResponse'

    authored
    To give those listening for endHead events a change to modify the head
    earlier.
Commits on Mar 4, 2013
  1. Fix MongoDb client

    authored
    Now works with cache.js.
Commits on Mar 5, 2013
  1. Update README

    authored
Something went wrong with that request. Please try again.