Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

split up libextra (move some out to rustpkg packages, some to libstd) #8784

Closed
thestinger opened this issue Aug 27, 2013 · 14 comments · Fixed by #12896

Comments

@thestinger
Copy link
Contributor

commented Aug 27, 2013

The plan seems to be that we'll move some of this out to rustpkg, and move a conservative set of modules to libstd that we are willing to stabilize/standardize.

As far as I can tell there's still no issue open for this, so here we are!

@thestinger

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2013

Tagging as an RFC, because we need to go through the modules and decide whether each one is a reasonable module to clean up and freeze with a stable (mostly) interface.

I think Go is a good case study here because they have lots of the cryptography and Unicode support outside of the main repository.

@brson

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2013

Nominating.

@brson

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2013

cc #9343

I think there are three destinations for extra modules:

  • Move to std. Probably very few should go in std.
  • Move to it's own package on GitHub, integrate with Rust CI.
  • Move to it's own crate in the main repo - probably just for things that rustc depends on.

Some notes:

  • I've long resisted moving test into std because it pulls in many other modules.
  • ebml is required by rustc but shouldn't really be used by anything else. Does it get it's own crate in the main repo?
@catamorphism

This comment has been minimized.

Copy link
Contributor

commented Oct 17, 2013

1.0, backwards-compat

@Seldaek

This comment has been minimized.

Copy link
Contributor

commented Oct 31, 2013

Just two cents from a user here. From what I have seen in other language the "batteries not included" approach can be quite frustrating. In node for example many very simple things require a package, and since there isn't really a list of packages blessed by the core language team or even by the community, newcomers can end up having to try many packages of dubious quality before they find one that fits. I can fully understand the fact that you need to reduce the scope to stabilize the core ASAP, but I hope you keep this in mind and think of a strategy for the future.

Either create a rust organization on github where blessed packages can be moved to, or maybe even rebuild a libextra later where popular and proven packages can be pulled in to form a stronger base language distribution (kind of like in PHP where anything you can possibly want is included, though I don't think we'd need to go as far).

@brson

This comment has been minimized.

Copy link
Contributor

commented Jan 26, 2014

@Seldaek The intent is to have an offical rust incubator (http://github.com/rust-lang) where the official packages reside, along with a process for promoting community packages to official packages. Whether crates reside in the installed distribution or are built from remote repos, I hope we have a situation where it's clear what crates you should use for many common tasks and adding them to your project is easy. So I fully agree with you.

bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
@liigo

This comment has been minimized.

Copy link
Contributor

commented Jan 26, 2014

Does libextra (or the split crates) belongs to Rust's "standard library"?

bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
bors added a commit that referenced this issue Jan 26, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
bors added a commit that referenced this issue Jan 27, 2014
It was decided a long, long time ago that libextra should not exist, but rather its modules should be split out into smaller independent libraries maintained outside of the compiler itself. The theory was to use `rustpkg` to manage dependencies in order to move everything out of the compiler, but maintain an ease of usability.

Sadly, the work on `rustpkg` isn't making progress as quickly as expected, but the need for dissolving libextra is becoming more and more pressing. Because of this, we've thought that a good interim solution would be to simply package more libraries with the rust distribution itself. Instead of dissolving libextra into libraries outside of the mozilla/rust repo, we can dissolve libraries into the mozilla/rust repo for now.

Work on this has been excruciatingly painful in the past because the makefiles are completely opaque to all but a few. Adding a new library involved adding about 100 lines spread out across 8 files (incredibly error prone). The first commit of this pull request targets this pain point. It does not rewrite the build system, but rather refactors large portions of it. Afterwards, adding a new library is as simple as modifying 2 lines (easy, right?). The build system automatically keeps track of dependencies between crates (rust *and* native), promotes binaries between stages, tracks dependencies of installed tools, etc, etc.

With this newfound buildsystem power, I chose the `extra::flate` module as the first candidate for removal from libextra. While a small module, this module is relative complex in that is has a C dependency and the compiler requires it (messing with the dependency graph a bit). Albeit I modified more than 2 lines of makefiles to accomodate libflate (the native dependency required 2 extra lines of modifications), but the removal process was easy to do and straightforward.

---

Testing-wise, I've cross-compiled, run tests, built some docs, installed, uninstalled, etc. I'm still working out a few kinks, and I'm sure that there's gonna be built system issues after this, but it should be working well for basic use!

cc #8784
@olsonjeffery

This comment has been minimized.

Copy link
Contributor

commented Jan 27, 2014

I think serialize and json are good candidates to move out into a serialize or serialization crate. Are there any order Encoder/Decoder impls in extra ?

@huonw

This comment has been minimized.

Copy link
Member

commented Jan 27, 2014

ebml.

@dmanescu

This comment has been minimized.

Copy link
Contributor

commented Jan 28, 2014

I'm working on arena and glob.

dmanescu added a commit to dmanescu/rust that referenced this issue Jan 28, 2014
dmanescu added a commit to dmanescu/rust that referenced this issue Jan 29, 2014
In line with the dissolution of libextra - rust-lang#8784 - moves arena to its own library libarena.
Changes based on PR rust-lang#11787. Updates .gitignore to ignore doc/arena.
dmanescu added a commit to dmanescu/rust that referenced this issue Jan 29, 2014
In line with the dissolution of libextra - rust-lang#8784 - moves glob to its own library libglob.
Changes based on PR rust-lang#11787. Updates .gitignore to ignore doc/glob.
bors added a commit that referenced this issue Jan 29, 2014
In line with the dissolution of libextra - #8784 - this moves arena and glob into
their own respective modules. Updates .gitignore with the entries
doc/{arena,glob} in accordance.
@seanmonstar

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2014

Could this mean libcrypto could come back?

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Jan 29, 2014

That's an interesting question! I think we'd have to talk about how "official" the libcrypto library is and whether we'd want to consider its interface a candidate for stabilization. You may want to post to the mailing list soliciting opinions about introducing libcrypto into the standard distribution.

Our stance has been to not include crypto in the standard library, but I don't think we made a decision about the standard distribution in general.

@brson

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2014

I think we mostly don't want to write crypto in rust. Bindings to good crypto may be welcome.

@flaper87

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2014

I agree with @brson here. Crypto sounds like something that should be implemented / maintained outside Rust.

bors added a commit that referenced this issue Feb 21, 2014
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 21, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 21, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 21, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
bors added a commit that referenced this issue Feb 22, 2014
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 22, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 24, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 24, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
bors added a commit that referenced this issue Feb 24, 2014
This also inverts the dependency between libserialize and libcollections.

cc #8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Feb 24, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
bors added a commit that referenced this issue Feb 24, 2014
This also inverts the dependency between libserialize and libcollections.

cc #8784
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 14, 2014
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.

Closes rust-lang#8784
Closes rust-lang#12413
Closes rust-lang#12576
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 14, 2014
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.

Closes rust-lang#8784
Closes rust-lang#12413
Closes rust-lang#12576
bors added a commit that referenced this issue Mar 15, 2014
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.

Closes #8784
Closes #12413
Closes #12576
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
Done as a part of rust-lang#8784.

Signed-off-by: OGINO Masanori <masanori.ogino@gmail.com>
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
This also inverts the dependency between libserialize and libcollections.

cc rust-lang#8784
jxs added a commit to jxs/rust that referenced this issue Apr 2, 2014
This commit shreds all remnants of libextra from the compiler and standard
distribution. Two modules, c_vec/tempfile, were moved into libstd after some
cleanup, and the other modules were moved to separate crates as seen fit.

Closes rust-lang#8784
Closes rust-lang#12413
Closes rust-lang#12576
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.