Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ember-*** packages include the tmp dir. #4199

Closed
ChALkeR opened this issue Jun 1, 2015 · 18 comments
Closed

ember-*** packages include the tmp dir. #4199

ChALkeR opened this issue Jun 1, 2015 · 18 comments

Comments

@ChALkeR
Copy link

ChALkeR commented Jun 1, 2015

Disclaimer: I do not use the package, this was found during automated npm packages downloading.

Migrated from emberjs/ember.js#11316 after IRC disussion.

ember-*** npm packages commonly include tmp (and .idea) directories.
tmp directories have sizes from 0 MiB to 621 MiB per package. They include duplicate copies of some scripts, and I suspect that are not actually needed in the distribution. I suspect that these directories grow over time (from release to release).

This seems to be a common problem for all ember-*** packages, hence reporting here.

List of packages (may be incomplete) with the tmp directory size in bytes:

651622411       ember-shepherd-0.1.1.tgz/tmp
547791776       ember-accessible-icon-0.0.3.tgz/tmp
238722262       ember-cli-flot-0.0.3.tgz/tmp
184765255       emberx-select-1.1.2.tgz/tmp
180687876       ember-cli-trackjs-0.1.3.tgz/tmp
122985898       ember-cli-sync-for-each-1.0.8.tgz/tmp
84743005        ember-i18next-0.2.0.tgz/tmp
84043588        ember-cli-skycon-0.0.3.tgz/tmp
84022828        ember-cli-jquery-classy-loader-0.0.2.tgz/tmp
78177959        ember-cli-auto-register-0.2.1.tgz/tmp
57715286        emberx-slider-1.0.3.tgz/tmp
55423165        ember-cli-filter-by-query-1.0.2.tgz/tmp
47370328        ember-cli-copyable-0.9.3.tgz/tmp
38571430        ember-credit-cards-0.0.2.tgz/tmp
36264369        ember-foundation-apps-0.0.3.tgz/tmp
35792350        ember-cli-panels-0.0.6.tgz/tmp
31687314        ember-cli-uncharted-describe-models-0.0.6.tgz/tmp
26192768        ember-cli-rx-0.2.5.tgz/tmp
23656533        ember-cli-poly-component-0.0.5.tgz/tmp
23162805        ember-semantic-analysis-0.0.2.tgz/tmp
19926529        ember-spin-button-0.1.4.tgz/tmp
19826585        ember-cli-static-pages-0.0.5.tgz/tmp
19789187        ember-state-services-0.0.4.tgz/tmp
18049399        ember-cli-array-sequence-1.1.1.tgz/tmp
17465260        ember-poetic-crud-mixins-1.0.1.tgz/tmp
17106891        ember-easy-form-extensions-0.2.10.tgz/tmp
16998849        ember-cli-injection-0.2.1.tgz/tmp
15758956        ember-cli-mapkit-0.1.0.tgz/tmp
15514798        ember-component-css-0.1.2.tgz/tmp
15214810        ember-cli-showdown-2.1.4.tgz/tmp
15105683        ember-cli-siren-adapter-1.0.0.tgz/tmp
13754954        ember-cli-accounting-0.1.1.tgz/tmp
12865229        ember-modals-0.4.0.tgz/tmp
12512120        ember-authr-0.0.1.tgz/tmp
12307605        ember-cli-uncharted-ajax-0.0.6.tgz/tmp
12150522        ember-cli-conditional-compile-0.0.4.tgz/tmp
11882944        ember-pikaday-0.6.0.tgz/tmp
11334130        ember-redirect-1.1.0.tgz/tmp
10156373        ember-breadcrumbs-0.1.6.tgz/tmp
9557614         ember-cli-scrolling-content-1.1.1.tgz/tmp
9393431         ember-cli-firebase-hosting-0.1.2.tgz/tmp
9280354         ember-cli-chosen-0.0.3.tgz/tmp
9125369         ember-timetree-2.0.1.tgz/tmp
8523072         ember-cli-stained-by-children-0.4.0.tgz/tmp
8011447         ember-cli-file-creator-0.1.0.tgz/tmp
7598409         ember-data-fixture-adapter-1.0.0.tgz/tmp
7245142         ember-cli-filepicker-0.1.3.tgz/tmp
7052179         ember-cli-bugsnag-0.0.9.tgz/tmp
6405953         ember-cli-chartist-0.2.7.tgz/tmp
5765901         ember-aviary-0.0.7.tgz/tmp
5521195         ember-magic-man-0.0.9.tgz/tmp
5448914         ember-cli-table-0.0.2.tgz/tmp
4956285         ember-slider-0.0.3.tgz/tmp
4208830         ember-css-routes-0.0.2.tgz/tmp
2172055         ember-fsm-0.4.2.tgz/tmp
451008          emberui-0.6.0.tgz/tmp
379234          ember-cli-ember-table-0.1.1.tgz/tmp
200533          ember-cpm-1.3.2.tgz/tmp
137843          ember-cli-ghost-app-0.1.0.tgz/tmp
8689            ember-loader-0.0.20.tgz/tmp
8201            ember-cli-gravatar-3.0.1.tgz/tmp
4105            ember-try-0.0.5.tgz/tmp
4105            ember-percentages-1.0.2.tgz/tmp
4105            ember-list-view-0.0.6.tgz/tmp
4105            ember-linkify-1.2.1.tgz/tmp
4105            ember-keen-tracking-0.0.7.tgz/tmp
4105            ember-keen-querying-0.0.7.tgz/tmp
4105            ember-instance-scope-0.0.2.tgz/tmp
4105            ember-get-helper-0.0.5.tgz/tmp
4105            ember-gauge-0.0.6.tgz/tmp
4105            ember-feature-flags-1.0.0.tgz/tmp
4105            ember-drop-0.0.1.tgz/tmp
4105            ember-deploy-couchbase-0.1.1.tgz/tmp
4105            ember-cli-summernote-0.1.15.tgz/tmp
4105            ember-cli-styleless-tokenfield-0.0.4.tgz/tmp
4105            ember-cli-pure-1.0.0.tgz/tmp
4105            ember-cli-placeholdit-0.0.3.tgz/tmp
4105            ember-cli-make-concoction-0.0.5.tgz/tmp
4105            ember-cli-google-street-view-0.1.2.tgz/tmp
4105            ember-cli-deploy-rsync-0.0.1.tgz/tmp
4105            ember-cli-dates-1.1.1.tgz/tmp
4105            ember-cli-crudtable-0.4.2.tgz/tmp
4105            ember-cli-cloudinary-images-0.1.0.tgz/tmp
4105            ember-cli-bootstrap-switch-0.1.7.tgz/tmp
4105            ember-cli-autoprefixer-0.4.0.tgz/tmp
4105            ember-cli-alcatraz-client-0.1.1.tgz/tmp
4105            ember-cli-active-link-wrapper-0.0.2.tgz/tmp

I found tmp dir in 87 packages and .idea dir in 64 packages. There could be more.

@rwjblue
Copy link
Member

rwjblue commented Jun 1, 2015

This was fixed in #3539, and has been included in versions since ember-cli@0.2.1. Once these packages update to > 0.2.1 (unless they opt out of the upgrade) they will properly ignore tmp and `dist.

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

I reported that to three packages already. If you want to fix that per-package, you could file appropriate issues yourself.

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

That's about 3 GiB in total, btw =).

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

Ok. I made a list, informed you. If anyone is willing to file appropriate issues to all 87 packages that I mentioned (minus the 3 that I already informed) they are free to do so.

That should be probably done by people who use/care about the listed packages.

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

6ac1822 — fix commit.

@rwjblue What about .idea, btw? That one is very minor, but still.

@rwjblue
Copy link
Member

rwjblue commented Jun 1, 2015

@ChALkeR - Sure, seems somewhat editor specific, but it doesn't bother anyone having extra stuff in the ignore file. Mind making a PR?

@stefanpenner
Copy link
Contributor

issues opened for repos I could find.

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

@stefanpenner Great, thanks! =)

@stefanpenner
Copy link
Contributor

@ChALkeR cool bot. It would be useful for some of our own checks and feedback tools. Is it OSS?

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

@stefanpenner That's not a bot. I just downloaded about 1/3 of the packages from npm with wget (latest versions of relatively recent packages, in fact everything published with npm2, that's pretty much everything from nodejs 0.10.x times and later), then extracted them.

I needed that for other reasons (code usage checks).

That some packages are very large for some reason caught my eye then, and I checked them with filelight (but you could usa du | sort for the same thing). The list above was produced by du ember*/tmp.

@stefanpenner
Copy link
Contributor

@ChALkeR makes sense. I have been meaning to pull down all ember-* packages and do some analysis on them. I was hoping I could piggy back off your work ;)

Anyways this was one of the data points I wanted to analysis and provide feedback for, so you beat me to it thanks :)

@ChALkeR
Copy link
Author

ChALkeR commented Jun 1, 2015

@stefanpenner I started with downloading the JSON from https://skimdb.npmjs.com/registry/_design/scratch/_view/byField and parsing it (warning: ~230 MiB).

Then I filtered the packages list (by npm version), formed a plain list of the download links, joined them with "\n" and fed that list to wget.

@odoe
Copy link

odoe commented Jun 1, 2015

That was a pretty good exercise in getting familiar with some addons. If I typo'd a couple of issue numbers, my apologies. but I submitted PRs in this list where I saw them missing.

@stefanpenner
Copy link
Contributor

this inspired me to hack together: https://github.com/stefanpenner/all-addons

@ChALkeR
Copy link
Author

ChALkeR commented Aug 24, 2015

An update:

547791776       npm.v2    ember-accessible-icon-0.0.3.tgz/tmp
318107235       npm.v1    ember-pretenderify-0.0.21.tgz/tmp
238722262       npm.v2    ember-cli-flot-0.0.3.tgz/tmp
122985898       npm.v2    ember-cli-sync-for-each-1.0.8.tgz/tmp
122783201       npm.v1    ember-introjs-0.2.3.tgz/tmp
109219176       npm.v1    ember-cli-web-workers-0.0.0-alpha.1.tgz/tmp
92869061        npm.v2    ember-i18next-0.3.1.tgz/tmp
84043588        npm.v2    ember-cli-skycon-0.0.3.tgz/tmp
84022828        npm.v2    ember-cli-jquery-classy-loader-0.0.2.tgz/tmp
66429015        npm.v1    ember-group-by-0.0.2.tgz/tmp
55292143        npm.v2    ember-cli-filepicker-0.1.95.tgz/tmp
50702191        npm.v2    ember-hopscotch-0.1.0.tgz/tmp
36264369        npm.v2    ember-foundation-apps-0.0.3.tgz/tmp
35792350        npm.v2    ember-cli-panels-0.0.6.tgz/tmp
31687314        npm.v2    ember-cli-uncharted-describe-models-0.0.6.tgz/tmp
24324110        npm.v1    ember-computed-indirect-0.2.0.tgz/tmp
23656533        npm.v2    ember-cli-poly-component-0.0.5.tgz/tmp
19826585        npm.v2    ember-cli-static-pages-0.0.5.tgz/tmp
18049399        npm.v2    ember-cli-array-sequence-1.1.1.tgz/tmp
15758956        npm.v2    ember-cli-mapkit-0.1.0.tgz/tmp
15493388        npm.v2    ember-key-responder-0.4.0.tgz/tmp
13581314        npm.v1    ember-pardon-0.1.1.tgz/tmp
12512120        npm.v2    ember-authr-0.0.1.tgz/tmp
12307605        npm.v2    ember-cli-uncharted-ajax-0.0.6.tgz/tmp
9393431         npm.v2    ember-cli-firebase-hosting-0.1.2.tgz/tmp
9389970         npm.v2    ember-fryctoria-1.0.0-alpha.2.tgz/tmp
9125369         npm.v2    ember-timetree-2.0.1.tgz/tmp
8523072         npm.v2    ember-cli-stained-by-children-0.4.0.tgz/tmp
8253058         npm.v1    ember-slick-0.0.1.tgz/tmp
8011447         npm.v2    ember-cli-file-creator-0.1.0.tgz/tmp
7747067         npm.v1    ember-htmlbars-tags-1.0.0.tgz/tmp
7461034         npm.v1    ember-flash-message-2000-0.0.8.tgz/tmp
7184488         npm.v2    ember-cli-tooltipster-0.0.9.tgz/tmp
6625952         npm.v1    ember-cli-json-schema-0.0.1.tgz/tmp
6089704         npm.v1    ember-cli-build-notifications-0.0.3.tgz/tmp
5765901         npm.v2    ember-aviary-0.0.7.tgz/tmp
5521195         npm.v2    ember-magic-man-0.0.9.tgz/tmp
5448914         npm.v2    ember-cli-table-0.0.2.tgz/tmp
4956285         npm.v2    ember-slider-0.0.3.tgz/tmp
4208830         npm.v2    ember-css-routes-0.0.2.tgz/tmp
4074202         npm.v1    ember-cli-fastly-0.0.4.tgz/tmp
137843          npm.v2    ember-cli-ghost-app-0.1.0.tgz/tmp
8689            npm.v2    ember-loader-0.0.20.tgz/tmp
4105            npm.v2    ember-pagination-ui-0.0.3.tgz/tmp
4105            npm.v2    ember-list-view-0.0.6.tgz/tmp
4105            npm.v2    ember-keen-tracking-0.0.7.tgz/tmp
4105            npm.v2    ember-keen-querying-0.0.7.tgz/tmp
4105            npm.v2    ember-instance-scope-0.0.2.tgz/tmp
4105            npm.v2    ember-drop-0.0.1.tgz/tmp
4105            npm.v2    ember-deploy-couchbase-0.1.1.tgz/tmp
4105            npm.v2    ember-cli-summernote-0.1.15.tgz/tmp
4105            npm.v2    ember-cli-styleless-tokenfield-0.0.4.tgz/tmp
4105            npm.v2    ember-cli-placeholdit-0.0.3.tgz/tmp
4105            npm.v2    ember-cli-paypal-0.0.3.tgz/tmp
4105            npm.v2    ember-cli-make-concoction-0.0.5.tgz/tmp
4105            npm.v2    ember-cli-fxos-0.1.4.tgz/tmp
4105            npm.v2    ember-cli-deploy-rsync-0.0.1.tgz/tmp
4105            npm.v2    ember-cli-bootstrap-switch-0.1.7.tgz/tmp
4105            npm.v2    ember-cli-alcatraz-client-0.1.1.tgz/tmp
4105            npm.v2    ember-cli-active-link-wrapper-0.0.2.tgz/tmp
4105            npm.v1    ember-cli-fill-murray-t1mmen-0.1.2.tgz/tmp
4105            npm.v1    ember-cli-async-button-0.5.0.tgz/tmp

This list is based on one week old data and includes the packages that were missing in the initial list: this should be complete or almost complete, while the first list was partial.

Previous list excluded all the npm.v1 entries and maybe some more.

Still more than 2 GiB of garbage.

@stefanpenner
Copy link
Contributor

@ChALkeR thanks for the update, the default addon blueprint contains the correct ignore info. Addons that are offending, should have issues opened themselves. Not sure we can do anything else from this side.

@ChALkeR
Copy link
Author

ChALkeR commented Sep 15, 2015

@stefanpenner I'm not asking you to change anything. That was a status update (featuring a full list), and you reopened the issue =).

I reported issues to some of those packages, hopefully someone else will do the same for the rest of them.

@stefanpenner
Copy link
Contributor

@ChALkeR yup, i appreciate the status report, just justifying my closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants