Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upDebian package is necessary. #3576
Comments
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
aral
Jun 11, 2017
I was just about to open an issue for this also. Having a Debian package would make it exponentially easier to set up your own instance and would result in a considerable rise in the instance count.
Imagine:
How to set up Mastodon on Ubuntu (CloudScale.ch, Digital Ocean, etc.)
sudo apt-get updatesudo apt-get install mastodon
aral
commented
Jun 11, 2017
|
I was just about to open an issue for this also. Having a Debian package would make it exponentially easier to set up your own instance and would result in a considerable rise in the instance count. Imagine: How to set up Mastodon on Ubuntu (CloudScale.ch, Digital Ocean, etc.)
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jun 11, 2017
I am expert (but just a few ) of debian package, and maybe it is easy to create for mastodon once, but a package you must maintain takes much time.Can be changed as the way to build be changing.
matyapiro31
commented
Jun 11, 2017
|
I am expert (but just a few ) of debian package, and maybe it is easy to create for mastodon once, but a package you must maintain takes much time.Can be changed as the way to build be changing. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ProgVal
Jun 11, 2017
Contributor
maybe it is easy to create for mastodon once
I am not sure about this. Mastodon has 927 nodejs dependencies and 201 ruby dependencies.
I am not sure about this. Mastodon has 927 nodejs dependencies and 201 ruby dependencies. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Gargron
Jun 11, 2017
Member
Mastodon has 927 nodejs dependencies and 168 ruby dependencies
This is not unusual for a project of this scale. This doesn't stop Gitlab from being a debian/ubuntu package.
Generally for this thread: https://github.com/chef/omnibus
This is not unusual for a project of this scale. This doesn't stop Gitlab from being a debian/ubuntu package. Generally for this thread: https://github.com/chef/omnibus |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jun 11, 2017
I think it may not contain so many packages in node.js and ruby to assign on the package setting directly.
Those package manager also need editing to specify dependencies.
matyapiro31
commented
Jun 11, 2017
|
I think it may not contain so many packages in node.js and ruby to assign on the package setting directly. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
guillaume-uH57J9
Jun 11, 2017
I am not sure about this. Mastodon has 927 nodejs dependencies and 201 ruby dependencies.
That's good point. I would think twice about installing a package if apt-get asks me "Installing 1128 packages for dependencies. Do you want to continue ? "
Rationalizing nodjs/ruby dependencies in Mastodon (ie reducing their #), or making some of them optional, would go a long way toward packaging.
guillaume-uH57J9
commented
Jun 11, 2017
That's good point. I would think twice about installing a package if apt-get asks me "Installing 1128 packages for dependencies. Do you want to continue ? " Rationalizing nodjs/ruby dependencies in Mastodon (ie reducing their #), or making some of them optional, would go a long way toward packaging. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
aral
Jun 11, 2017
@guillaume-uH57J9 Use case: I want to install Mastodon on my server.
For this use case, I don’t really care what the details of the installation are: Could be 1 dependency, could be 1 million dependencies – the beauty of having a package installer is that I don’t have to care; it’s a black box. My concern is: can I write a single command, hit OK, and boom – a little while later – I have Mastodon installed. Yay! (Much rejoicing, etc.) :)
That said, of course, by all means, please reduce dependencies, etc., if it streamlines the overall process but it’s not a prerequisite for this or the bottleneck that this issue aims to solve. (The bottleneck to new instance adoption is installation complexity and time.) The number of dependencies isn’t really a factor given the tremendous value a package will provide in terms of reducing installation time from hours to minutes – orders of magnitude – and upgrades to a single command.
aral
commented
Jun 11, 2017
•
|
@guillaume-uH57J9 Use case: I want to install Mastodon on my server. For this use case, I don’t really care what the details of the installation are: Could be 1 dependency, could be 1 million dependencies – the beauty of having a package installer is that I don’t have to care; it’s a black box. My concern is: can I write a single command, hit OK, and boom – a little while later – I have Mastodon installed. Yay! (Much rejoicing, etc.) :) That said, of course, by all means, please reduce dependencies, etc., if it streamlines the overall process but it’s not a prerequisite for this or the bottleneck that this issue aims to solve. (The bottleneck to new instance adoption is installation complexity and time.) The number of dependencies isn’t really a factor given the tremendous value a package will provide in terms of reducing installation time from hours to minutes – orders of magnitude – and upgrades to a single command. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jun 11, 2017
Most important role of packaging is to separate the environment for running from the env for building.
Can reduce npm package when running isnt what i know; maybe ruby package is unnecessary then.
matyapiro31
commented
Jun 11, 2017
|
Most important role of packaging is to separate the environment for running from the env for building. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
aral
Jun 11, 2017
@matyapiro31 Again, as far as my use case for this goes (and I’m happy to open a separate issue if you feel I’m hijacking your use case) :) the most important reason for this is to reduce installation time on the most popular hosting platforms (Ubuntu-based) from hours to minutes (an order of magnitude change that will impact instance adoption rates substantially) and maintenance to a single command. If there are any advantages for developers, that’s an additional bonus but my concern here is in encouraging instance adoption.
aral
commented
Jun 11, 2017
|
@matyapiro31 Again, as far as my use case for this goes (and I’m happy to open a separate issue if you feel I’m hijacking your use case) :) the most important reason for this is to reduce installation time on the most popular hosting platforms (Ubuntu-based) from hours to minutes (an order of magnitude change that will impact instance adoption rates substantially) and maintenance to a single command. If there are any advantages for developers, that’s an additional bonus but my concern here is in encouraging instance adoption. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jun 11, 2017
For example, http-server is convenient package for testing, but not necessary when run:
$ npm install -g http-server
Mastodon doesn't use http-server , but there are package installed for similar reason.
matyapiro31
commented
Jun 11, 2017
|
For example, http-server is convenient package for testing, but not necessary when run:
Mastodon doesn't use http-server , but there are package installed for similar reason. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
commented
Jun 11, 2017
|
@Gargron could you list up what packages are used now? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Gargron
Jun 11, 2017
Member
I do not want to list or justify Mastodon's runtime dependencies. We do not try to make the project bigger on purpose, and everyone who is contributing to the codebase sees that we need the dependencies that we are using. What you are describing ("convenient package for testing, but not necessary when run") is already covered by environments: development, test, production. Dependencies are already sorted accordingly, and production dependencies do not include anything which is not used in production.
You also shouldn't worry about them, I think. What all of these "packaging" tools do, they manage system packages (think postgres, redis, node, protobuf). Ruby and Node dependencies still get installed the usual way, with no difference to manual installation. For a list of system dependencies, look at the production guide: https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Production-guide.md#general-dependencies
|
I do not want to list or justify Mastodon's runtime dependencies. We do not try to make the project bigger on purpose, and everyone who is contributing to the codebase sees that we need the dependencies that we are using. What you are describing ("convenient package for testing, but not necessary when run") is already covered by environments: development, test, production. Dependencies are already sorted accordingly, and production dependencies do not include anything which is not used in production. You also shouldn't worry about them, I think. What all of these "packaging" tools do, they manage system packages (think postgres, redis, node, protobuf). Ruby and Node dependencies still get installed the usual way, with no difference to manual installation. For a list of system dependencies, look at the production guide: https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Production-guide.md#general-dependencies |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ProgVal
Jun 11, 2017
Contributor
everyone who is contributing to the codebase sees that we need the dependencies that we are using
I think some of them could be avoided.
I am not very good at Javascript, but I think the dependency on classnames could be removed, for example.
It was added in 860ffc0, apparently just to replace button ${this.props.secondary ? 'button-secondary' : ''} ${this.props.block ? 'button--block' : ''} by:
classNames('button', this.props.className, {
'button-secondary': this.props.secondary,
'button--block': this.props.block,
})
I know it is a small dependency, but it should be possible to remove it at very little cost.
I think some of them could be avoided. It was added in 860ffc0, apparently just to replace
I know it is a small dependency, but it should be possible to remove it at very little cost. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jun 11, 2017
maybe this helps us to create a package.
plus:
/Aptfile and /Vagrantfile can be reduced by writing dependencies.
matyapiro31
commented
Jun 11, 2017
|
maybe this helps us to create a package. plus: |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
commented
Jun 15, 2017
|
@Gargron Finally, you think package is necessary? |
nolanlawson
added
deployment
enhancement
labels
Jun 16, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
akuckartz
commented
Jun 20, 2017
|
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
guillaume-uH57J9
Jun 25, 2017
I don’t really care what the details of the installation are: Could be 1 dependency, could be 1 million dependencies
Some users might not care, and it won't change much at installation time.
But for package maintainers this is day and night. Packaging an application + 1 dependency, and maintaining them, is a relatively simple/quick task.
Packaging 1 application + 1 billion dependencies (and maintaining them) cannot be accomplished by one person. It'd require a whole team.
guillaume-uH57J9
commented
Jun 25, 2017
Some users might not care, and it won't change much at installation time. But for package maintainers this is day and night. Packaging an application + 1 dependency, and maintaining them, is a relatively simple/quick task. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Jehops
Jul 29, 2017
Contributor
For the FreeBSD package we punted on the node dependencies. All the OS and ruby/rails dependencies are covered with pkg install mastodon, but then the user still has to do, e.g., sudo su - mastodon; yarn install; RAILS_ENV=production rails assets:precompile. Doing the job of npm/yarn is too ambitious (for me) at this point; we would have to bake a similar infrastructure we have for dealing with ruby packages into the FreeBSD ports/package system.
While it's not a perfect package, most of the feedback I have received from Mastodon administrators running FreeBSD has been positive.
|
For the FreeBSD package we punted on the node dependencies. All the OS and ruby/rails dependencies are covered with While it's not a perfect package, most of the feedback I have received from Mastodon administrators running FreeBSD has been positive. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Gargron
Jul 29, 2017
Member
@Jehops I actually don't really understand why bundler and yarn dependencies cannot be bundled in the package. There is literally no chance that running bundle install or yarn install will yield a different result unless the Mastodon lockfiles change (e.g. in a new release). That's the exact purpose of lockfiles - reproducible, deterministic bundling.
|
@Jehops I actually don't really understand why bundler and yarn dependencies cannot be bundled in the package. There is literally no chance that running |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Jehops
Jul 29, 2017
Contributor
@Gargron, for each rails dependency, there is a one-to-one correspondence with a FreeBSD port/package and each of those FreeBSD ports/packages handles dependencies. Basically, the FreeBSD port/package system is used in place of gem and bundler. That that sort of solution for the yarn dependencies would be nice, but a lot of work. It may be feasible in the short term to make one omnibus mastodon-yarn-dependencies package.
|
@Gargron, for each rails dependency, there is a one-to-one correspondence with a FreeBSD port/package and each of those FreeBSD ports/packages handles dependencies. Basically, the FreeBSD port/package system is used in place of gem and bundler. That that sort of solution for the yarn dependencies would be nice, but a lot of work. It may be feasible in the short term to make one omnibus mastodon-yarn-dependencies package. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Jul 30, 2017
@Gargron mastodon's package usage design is not better to manage package.
We are forced to use newer version of node, ruby, yarn etc than which in distribution packages.
It makes us very complicated to control mastodon package.
There are at least 5 package varieties:
native packages for mastodon,npm packages,
gem packages, native package for gem,
native packages in sub repositories,
and we must check if those packages has bad affect on other packages, this is the reason why very difficult to maintain package.
matyapiro31
commented
Jul 30, 2017
|
@Gargron mastodon's package usage design is not better to manage package. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
matyapiro31
Oct 11, 2017
I now somehow understand how mastodon works, however it's still very difficult to create debian package.
Mastodon depends on many software that is not packaged correctly for debian. Supplying these packages with appropriate update is very difficult.
matyapiro31
commented
Oct 11, 2017
|
I now somehow understand how mastodon works, however it's still very difficult to create debian package. Mastodon depends on many software that is not packaged correctly for debian. Supplying these packages with appropriate update is very difficult. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ghost
Nov 20, 2017
-
FYI there is a RFP for this on debian's side https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859741
-
"Mastodon depends on many software that is not packaged correctly for debian" If anyone has identified any of these packages, either this thread or the thread I linked to above would be appropriate places to mention them so we can start filing RFPs and getting those packages in debian, too.
-
"these prerequisites change all the time" That's the whole point of debian - to have a stable basis of development, something that we can actually audit and a known 'good state' that we can build from in case someone puts showstopper bugs in.
ghost
commented
Nov 20, 2017
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ProgVal
Nov 20, 2017
Contributor
If anyone has identified any of these packages, either this thread or the thread I linked to above would be appropriate places to mention them so we can start filing RFPs and getting those packages in debian, too.
Ruby:
- source -> ruby-method-source, ruby-source-map ??
- ruby -> ruby
- pkg-config -> ruby-pkg-config
- puma -> puma
- rails -> ruby-rails
- uglifier -> ruby-uglifier
- hamlit-rails
- pg -> ruby-pg
- pghero
- dotenv-rails
- fog-aws -> ruby-fog-aws
- fog-core -> ruby-fog-core
- fog-local -> ruby-fog-local
- fog-openstack -> ruby-fog-openstack
- paperclip
- paperclip-av-transcoder
- active_model_serializers
- addressable -> ruby-addressable
- bootsnap
- browser -> ruby-browser
- charlock_holmes
- iso-639
- cld3
- devise -> ruby-devise
- devise-two-factor -> ruby-devise-two-factor
- doorkeeper -> ruby-doorkeeper
- fast_blank
- goldfinger
- hiredis -> ruby-hiredis
- redis-namespace -> ruby-redis-namespace
- htmlentities -> ruby-htmlentities
- http -> ruby-http
- http_accept_language
- httplog
- idn-ruby
- kaminari -> ruby-kaminari
- link_header
- mime-types -> ruby-mime-types, ruby-mime-types-data
- nokogiri -> ruby-nokogiri
- nsa
- oj -> ruby-oj
- ostatus2
- ox -> ruby-ox,
- pundit -> ruby-pundit
- rabl -> ruby-rabl, ruby-rabl-rails
- rack-attack -> ruby-rack-attack
- rack-cors -> ruby-rack-cors
- rack-timeout
- rails-i18n -> ruby-rails-i18n
- rails-settings-cached
- redis -> ruby-redis
- mario-redis-lock
- rqrcode -> ruby-rqrcode
- ruby-oembed -> ruby-oembed
- sanitize -> ruby-sanitize
- sidekiq -> ruby-sidekiq
- sidekiq-scheduler
- sidekiq-unique-jobs
- sidekiq-bulk
- simple-navigation -> ruby-simple-navigation
- simple_form
- sprockets-rails -> ruby-sprockets-rails
- strong_migrations
- twitter-text -> ruby-twitter-text
- tzinfo-data
- webpacker
- webpush
- json-ld-preloaded
- rdf-normalize
nodejs:
- babel-plugin-preval
- postcss-smart-import
- webpack-merge
- axios
- autoprefixer
- babel-plugin-transform-react-remove-prop-types
- substring-trie
- babel-plugin-syntax-dynamic-import
- dotenv
- react-textarea-autosize
- cross-env
- rimraf -> node-rimraf
- emoji-mart
- array-includes
- react-redux-loading-bar
- mkdirp -> node-mkdirp
- react-router-scroll-4
- postcss-loader
- font-awesome
- react-immutable-proptypes
- react-overlays
- react-redux
- babel-plugin-transform-runtime
- prop-types
- react-motion
- http-link-header
- node-sass
- intl
- webpack-bundle-analyzer
- object-fit-images
- sass-loader
- offline-plugin
- babel-plugin-transform-decorators-legacy
- intl-messageformat
- postcss-object-fit-images
- react-hotkeys
- escape-html -> node-escape-html
- resolve-url-loader
- babel-preset-env
- npmlog -> node-npmlog
- webpack
- babel-plugin-transform-react-inline-elements
- babel-plugin-lodash
- babel-plugin-transform-react-jsx-self
- immutable
- tiny-queue
- redux-immutable
- classnames
- babel-loader
- is-nan -> node-number-is-nan
- uuid -> libjs-node-uuid, node-node-uuid
- express -> node-express
- react-dom
- webpack-manifest-plugin
- path-complete-extname
- lodash -> node-lodash
- babel-plugin-react-intl
- style-loader
- react-intl
- react-toggle
- pg
- detect-passive-events
- requestidlecallback
- redis -> node-node-redis
- babel-plugin-transform-class-properties
- stringz
- reselect
- extract-text-webpack-plugin
- compression-webpack-plugin
- redux
- babel-plugin-transform-object-rest-spread
- react-notification
- js-yaml -> node-js-yaml
- css-loader
- glob -> node-glob
- mark-loader
- websocket.js
- punycode -> node-punycode
- redux-thunk
- intersection-observer
- react-swipeable-views
- babel-plugin-transform-es2015-modules-commonjs
- throng
- react
- es6-symbol
- babel-plugin-transform-react-jsx-source
- rails-ujs
- uws
- react-router-dom
- marky
- object-assign -> node-object-assign
- intl-relativeformat
- react-immutable-pure-component
- babel-core -> node-babel-core (buster)
- babel-preset-react
- precss
- file-loader
Ruby:
nodejs:
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ghost
Nov 20, 2017
Things you can check off
-
(at least for (contrib:buster), sid) babel-core: https://packages.debian.org/hu/sid/node-babel-core
-
ruby-dotenv seems to be what could otherwise be called ruby-rails-dotenv ... https://gemnasium.com/gems/dotenv-rails seems to suggest that they are the same.
-
hamlit-rails - status: RFP
-
pghero : status : RFP
-
paperclip : status : RFP
-
ruby-cucumber-expressions (required by paperclip) status : RFP
-
ruby-av (required by paperclip-av-transcoder) status : RFP
-
ruby-cocaine(required by ruby-av) status : in debian! only in buster/sid though. https://packages.debian.org/source/buster/ruby-cocaine .
-
ruby-av-transformer status : RFP
Things you can probably add to that list:
- ruby-html2haml (required to build? hamlit-rails) status: in debian.
- ruby-appraisal (required to build? hamlit-rails) status: in debian, but woefully behind. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=882278
- ruby-rspec (required for ruby-appraisal) status in debian: in stable
- ruby-shoulda (required by paperclip) status in debian: in stable
- ruby-mocha (required by paperclip) status in debian: in stable
- ruby-aws-sdk (required by paperclip) status in debian: in stable
- ruby-bourne (required by paperclip) status in debian: in stable
- ruby-cucumber-rails (required by paperclip) status in debian: in stable
- ruby-coveralls (required by ruby-cucumber-expressions) status in debian: in stable
- pry ruby-coderay ruby-method-source (required by ruby-cucumber-expressions) status in debian: in stable
- launchy (required by paperclip) status in debian: in stable
- ruby-fakeweb (required by paperclip) status in debian: in stable
- ruby-generator-spec (required by paperclip) status in debian: in stable
- ruby-simplecov (required by ruby-climate-control ) status in debian: in stable
- ruby-climate-control (required by ruby-cocaine) status : in debian stable
ghost
commented
Nov 20, 2017
•
|
Things you can check off
Things you can probably add to that list:
|
matyapiro31 commentedJun 5, 2017
•
edited
Edited 1 time
-
matyapiro31
edited Jun 5, 2017 (most recent)
mastodon has no Debian package.
when debian package for mastodon is created, we're able to shrink Dockerfile simply and absorb the difference among version and distribution.