Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WordPress proxy #2632

Closed
jimaek opened this issue Dec 6, 2014 · 59 comments
Closed

WordPress proxy #2632

jimaek opened this issue Dec 6, 2014 · 59 comments

Comments

@jimaek
Copy link
Member

jimaek commented Dec 6, 2014

A new feature that I always wanted to have is finally here.
We can now automatically proxy any file from the WordPress repository. All you have to do is specify the name and version.

For example:
http://cdn.jsdelivr.net/wp/1000eb/tags/3.3.1/javascripts/admin.js
http://cdn.jsdelivr.net/wp/wp-slimstat/tags/3.8.5/wp-slimstat.js

This feature can simplify the integration of jsDelivr for all developers. Now its ridiculously easy to add a CDN option to any plugin.

It works by proxying http://plugins.svn.wordpress.org/ and adding the correct content-type headers. So WordPress takes care of the security audit and adding new versions.

The proxy for now allows only css,js and font files to be proxied.
Please test it out and let me know what you think. Are there any bugs? Any features missing?

@getusedtoit As a WP dev what do you think?

@pnommensen
Copy link
Contributor

Timing-Allow-Origin: * header is missing. Also, add public to cache-control header for consistency.

@jimaek
Copy link
Member Author

jimaek commented Dec 6, 2014

Added Timing-Allow-Origin and CORS

@ajkj
Copy link
Contributor

ajkj commented Dec 8, 2014

there might be broken img/font/etc issue. which use relative img/font path in css file.
i experienced same issue in xe-core(CMS) and fixed it by adding image/swf/font files.

@jimaek
Copy link
Member Author

jimaek commented Dec 8, 2014

It depends but technically relative paths should work just fine.

@ajkj
Copy link
Contributor

ajkj commented Dec 8, 2014

ex) http://cdn.jsdelivr.net/wp/arabic-font/tags/1.0/css/arabic-fonts.css

http://jsfiddle.net/qtrjks50/

@pnommensen
Copy link
Contributor

It's because right now only css|js is permitted.

@ajkj
Copy link
Contributor

ajkj commented Dec 8, 2014

sorry, i forgot beta status.

@pnommensen
Copy link
Contributor

I don't see why to not permit commonly used files :) images, fonts, css, js. When we made the nginx config we wanted to try with these two file types first.

@jimaek
Copy link
Member Author

jimaek commented Dec 8, 2014

All font files are now allowed too. Keep up the testing :)

@as-com
Copy link
Contributor

as-com commented Dec 8, 2014

How about themes?

@as-com
Copy link
Contributor

as-com commented Dec 8, 2014

Also, are you going to update the jsDelivr WordPress plugin to use this feature?

@tomByrer
Copy link
Contributor

tomByrer commented Dec 9, 2014

I don't see why to not permit commonly used files :) images, fonts, css, js.
How about themes?

From a user-performance point of view, comes to a certain point where it is better not to host everything outsite the originating URL.

@pnommensen
Copy link
Contributor

@tomByrer Can you give an example? Basically this also means using a CDN for WordPress (like maxcdn, cloudfront) with common setup like wp-super-cache and w3tc hurts performance.

@jimaek
Copy link
Member Author

jimaek commented Dec 9, 2014

How about themes?

I dont plan hosting them. Most people buy their themes from third-party markets, so there is no way to proxy everything.

Also, are you going to update the jsDelivr WordPress plugin to use this feature?

If I find an affordable freelancer to do that then yes

@tomByrer
Copy link
Contributor

tomByrer commented Dec 9, 2014

@pnommensen Each browser only has a few 'spare' HTTP connections for serving outside Originating Hostname. 4-11 last I looked. Browsers have 6-8 max connections for the originating, so if the only file served from there is the index.htm & all the other files are CDN hosted, then the other 5-7 HTTP connections sit there unused.

Really important for WP sites, since most have tracking spam'social sharing buttons' & other service plugins, which usually eat up several outbound HTTP connections already.

YMMV, best to test.

@jimaek
Copy link
Member Author

jimaek commented Dec 9, 2014

Even if jsDelivr serves 100% of the plugin files it will never be the only hostname for the website. Dont forget the theme files, images, other crap they add. So the problem you are describing will never happen. I can only see benefits from the speed of our CDN and parallelization across multiple hostnames (jsDelivr+website domain)

@tomByrer
Copy link
Contributor

tomByrer commented Dec 9, 2014

Dont forget the theme files, images, other crap they add

Yes, thus my "don't host the entire theme on a CDN" caution. You're right, when it comes to WP, usually not a worry; Orig Hostname HTTPs are usually saturated.

our CDN and parallelization across multiple hostnames

That was the point of my lightning talk at Pivotal Labs. :)

Though hosting an essential JS or CSS might be faster on Orig Host vs CDN; really it is a last-minute test & tweak.

@ghost
Copy link

ghost commented Dec 13, 2014

This is such a great feature, I'm going to add it to 3.9 :) how long does it take for your proxy to detect a new version on the repository? You guys rock. I will recommend that we use this at my day job at the City University of New York, for our plugins ;)

@jimaek
Copy link
Member Author

jimaek commented Dec 13, 2014

Please dont push it to production yet. I want to make 100% sure that nothing will break if used live :)

There is nothing to detect, once you tag a version and it appears on WordPress its automatically available on jsDelivr. So its instant, no waiting at all.

Test it out if you can and let me know if you find any issues.
We are also going to contact WordPress.org in case they want to make this feature semi-official.

@ghost
Copy link

ghost commented Dec 14, 2014

Okay cool, I'll just test it locally for now then. Getting recognition from WP would be huge :)

@ghost
Copy link

ghost commented Dec 15, 2014

It looks stable enough to me :) I've been testing it under various scenarios, and so far it's working as expected. Let me know when you think it's ready for prime time! I can't wait to implement it in Slimstat.

@ajkj
Copy link
Contributor

ajkj commented Dec 15, 2014

what do you think of changing dir name to wordpress-plugins or wordpress-plugins-mirror?
nobody will guess it's wordpress-plugin-cdn-url with dir name 'wp'. and i think it's too general naming.

@jimaek
Copy link
Member Author

jimaek commented Dec 15, 2014

Why do you say that? wp is a popular acronym of WordPress. Since jsDelivr is a CDN it makes it clear that wp contains WordPess related files on our CDN.
Plus wordpress-plugins is too big.

And why anyone needs to guess anything, The devs will read what it is on our docs and the users that will see it on their source code are not our problem (although most will understand what it is)

Sorry I just dont follow your logic :)

@ghost
Copy link

ghost commented Dec 15, 2014

I vote for leaving it as 'wp' ;) Shorter and more intuitive.

@ajkj
Copy link
Contributor

ajkj commented Dec 15, 2014

there is already wordpress dir which contain whole wordpress.
i thought naming wp-plugins or wordpress-plugins would fit in naming convention.
but as you said, it also doesn't matter because it's just CDN url.

@ghost
Copy link

ghost commented Dec 17, 2014

Please ping me when you think this is ready for prime time!

@jimaek
Copy link
Member Author

jimaek commented Dec 18, 2014

Honestly I dont see any issues, I was just waiting for people to report their problems.

I guess you can push this live and let me know how it goes :)

@jimaek
Copy link
Member Author

jimaek commented Dec 30, 2014

Awesome! Now the hard part is to create a page on our new website explaining this new feature.

@as-com
Copy link
Contributor

as-com commented Dec 30, 2014

Are we going to use the github wiki?

@jimaek
Copy link
Member Author

jimaek commented Dec 30, 2014

I was talking about this jsdelivr/www.jsdelivr.com#28

@ghost
Copy link

ghost commented Jan 7, 2015

I just released version 3.9.2 to fix a bug discovered by one of our users, and everything worked out perfectly right away. This is really a killer app!

@jimaek
Copy link
Member Author

jimaek commented Jan 7, 2015

@getusedtoit Awesome! Unfortunately very few devs will know about this because we suck at promoting new features :p

@as-com
Copy link
Contributor

as-com commented Jan 7, 2015

I know! Let's just put a huge pop up on jsdelivr.com! There's a 100% chance if people noticing! There's also a 100% chance of annoying users! :-P

On Wed, Jan 7, 2015 at 4:19 AM, Dmitriy Akulov notifications@github.com
wrote:

@getusedtoit Awesome! Unfortunately very few devs will know about this because we suck at promoting new features :p

Reply to this email directly or view it on GitHub:
#2632 (comment)

@guix77
Copy link

guix77 commented Jan 17, 2015

wow I just had that idea looking at my sites PageSpeed audits. Glad you did it ! Keep it up :)

@ghost
Copy link

ghost commented Feb 1, 2015

Hi guys, I have a quick question about this functionality. I see that the following URL is working as expected:

http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.js

Is it safe to assume that it will be available in the foreseeable future just like the "tags" version? I usually push the latest stable both as a tag and in trunk (since I use github for development), and by using this trick, I wouldn't have to worry about including the version number in the URL anymore.

Thank you in advance!

@pnommensen
Copy link
Contributor

Yeah, should work great :)

Patrick Nommensen
http://nginx.com

On Feb 1, 2015, at 8:39 AM, Get Used to IT notifications@github.com wrote:

Hi guys, I have a quick question about this functionality. I see that the following URL is working as expected:

http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.js http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.js
Is it safe to assume that it will be available in the foreseeable future just like the "tags" version? I usually push the latest stable both as a tag and in trunk (since I use github for development), and by using this trick, I wouldn't have to worry about including the version number in the URL anymore.


Reply to this email directly or view it on GitHub #2632 (comment).

@ghost
Copy link

ghost commented Feb 1, 2015

👍

@as-com
Copy link
Contributor

as-com commented Feb 1, 2015

There is an issue with caching, though. I believe the CDNs, origin, and browsers cache a copy of it for an indefinite amount of time. I think it is necessary to include the version number in the tag version for cache-busting.

On Sun, Feb 1, 2015 at 11:41 AM, Patrick Nommensen
notifications@github.com wrote:

Yeah, should work great :)

Patrick Nommensen
http://nginx.com

On Feb 1, 2015, at 8:39 AM, Get Used to IT notifications@github.com wrote:

Hi guys, I have a quick question about this functionality. I see that the following URL is working as expected:

http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.js http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.js
Is it safe to assume that it will be available in the foreseeable future just like the "tags" version? I usually push the latest stable both as a tag and in trunk (since I use github for development), and by using this trick, I wouldn't have to worry about including the version number in the URL anymore.


Reply to this email directly or view it on GitHub #2632 (comment).


Reply to this email directly or view it on GitHub:
#2632 (comment)

@ghost
Copy link

ghost commented Feb 2, 2015

Good point!

@jimaek
Copy link
Member Author

jimaek commented Feb 3, 2015

Yes, dont use trunk because it will always show older versions. You can use some wordpress variable with the plugin's version to automate the process.

@hubdotcom
Copy link
Contributor

Amazing feature, well done!

@ghost
Copy link

ghost commented May 14, 2015

Hi guys,

we pushed an update to our plugin, however the CDN is serving the old file for some reason:

https://plugins.trac.wordpress.org/browser/wp-slimstat/trunk/wp-slimstat.min.js

versus

http://cdn.jsdelivr.net/wp/wp-slimstat/trunk/wp-slimstat.min.js

Could you please look into this?

@jimaek
Copy link
Member Author

jimaek commented May 14, 2015

Dont use the trunk, use the tags. Trunk gets cached and needs purging and its not recommended to use it. This is much better:
http://cdn.jsdelivr.net/wp/wp-slimstat/tags/4.0.2/wp-slimstat.min.js

You can use a wp function to automatically get the plugin's version and insert it into URLs, fully automated.

P.S. I purged the trunk as well.

@ghost
Copy link

ghost commented May 14, 2015

Hi jimaek,

thank you for your prompt response. The reason why I'm using trunk is that many of our users use caching plugins for WordPress. Which means that the source code of the page is "frozen" for days. When we release a new update, the URL in the cached version of the page will still point to the previous one, thus breaking the functionality for all those users. We've tried to warn our users by displaying messages that invite them to clear their WP cache, but many times people end up submitting a support request anyway, lamenting that the tracker is not working. Our goal was to minimize those requests to our support team, by finding a way to keep the URL always the same.

What solution could be implemented to address this issue? (other than telling our users to clear their cache every time they upgrade).

Thank you again!

@jimaek
Copy link
Member Author

jimaek commented May 14, 2015

Have you tried something like this http://wordpress.stackexchange.com/questions/7112/w3-total-cache-cache-refresh-programmatically ?

When you update plugins W3 Total Cache will notify the user as well that he needs to clear his cache.

Anyway if none of this solutions work then shoot me an email and I will give you access to our purging API. You will be able to use trunk and purge when you release a new version

@ghost
Copy link

ghost commented May 14, 2015

Interesting link, thank you for sharing. Unfortunately not everybody uses W3. Some of our users use SuperCache, others use HyperCache, etc. I wish it was that easy :( How much work would it be to have trunk purged let's say once every 2 hours. If that doesn't work, I'm okay with purging it through the API. Let me know what works best for you :)

@jimaek
Copy link
Member Author

jimaek commented May 14, 2015

You can set a cronjob on your server and ping our API to purge the cache whenever you want. But if you purge every 2 hours then the CDN will get much slower because every 2 hours all 70 nodes will need to re-cache the file.

@ghost
Copy link

ghost commented May 14, 2015

You're right, and I don't want that :) I'll just do it manually whenever I release an update. That works for me. I'll send you a private message. Again, thank you so much for this service and for being on top of things!

@jimaek
Copy link
Member Author

jimaek commented May 14, 2015

np :)

@as-com
Copy link
Contributor

as-com commented May 14, 2015

I think most caching plugins automatically clear the page cache when a
plugin is installed or updated. It's possible that people disabled that
functionality or they are simply using stupid caching plugins.

Also, there is the possibility that some lazy/ignorant people don't update
your plugin and there are bugs caused by incompatible JS.
On Thu, May 14, 2015 at 11:31 AM Dmitriy Akulov notifications@github.com
wrote:

np :)


Reply to this email directly or view it on GitHub
#2632 (comment).

@hubdotcom
Copy link
Contributor

Setting as trunk would cause some issues with non updated installs. Let's say you introduce new code in wp-slimstat.min.js that is not backwards compatible - it will break any installs running on previous versions of the plugin (since the plugin always requests the latest js file).

@ghost
Copy link

ghost commented Jun 1, 2015

@hubdotcom Good point. Major changes to the tracker are quite rare, I must say. Most of the time, we tweak the code to fix bugs in the JS code, which means that even people who don't update Slimstat, will get the new tracking script automatically. And of course, those who have their own rationale for not wanting to upgrade the plugin, can always turn the CDN setting off, and serve the script from their website ;)

@hubdotcom
Copy link
Contributor

Fair enough - just using this for the tracking script makes perfect sense!

@jimaek jimaek changed the title WordPress proxy beta WordPress proxy Jan 18, 2016
@jimaek
Copy link
Member Author

jimaek commented May 7, 2017

@jimaek jimaek closed this as completed May 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants