New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow library.json to specify sources other than PlatformIO's Repository #461

Closed
dave-newson opened this Issue Jan 16, 2016 · 6 comments

Comments

Projects
None yet
6 participants
@dave-newson

dave-newson commented Jan 16, 2016

The Problem

If I want to use a library.json to manage my projects 3rd party dependencies, then these 3rd party libraries;

  • must already have a library.json file
  • must be registered on PlatformIOs repository.

Desired Functionality

Let's assume that, in a perfect world:

  • my library.json manages all the third party dependencies in my project.
  • after I run platformio update all dependencies are fulfilled.

The desire is to eliminate the need to manual checkouts, clones, code downloads, etc. In 90% of common use cases I want to just run platformio update and have all the magic taken care of.

Use Case 1 - I want to use a library which isn't on the PlatformIO repository

There are always going to be cases where I want to include a library in my code but the libraries author has not included a library.json file in their source.

This may be because:

  • they don't known about PlatformIO
  • they haven't gotten around to putting it in the registry
  • they have some objection to using PlatformIO at all

Frankly, and this is the important bit; Who cares? I should not require their consent to perform what is basically git clone. The library.json file is a helper for PlatformIOs repository, and not necessary for code to function.

The official solution to this problem is "submit a library.json PR to the 3rd party", but come on. That could take weeks to get actioned, and in the mean time I have to manually include the dep. That's hardly a solution.

Use Case 2 - I want to use a specific fork

Say I've followed the standard GitHub approach of forking, branching, creating a change and then PRing it back to the main third party project. That's nice, and I'm a good open-source citizen.

Problem is, this change won't be available in the library via PlatformIO until it gets merged back in, if it gets merged back in.

I could submit my fork up to PlatformIO's repository, so now we have "arduino-stepper-dave-newson", but frankly that's just silly. Do we want people to do that for every hotfix and branch? No, I don't believe we do.

Use Case 3: Source code isn't on VCS at all.

There's also cases where libraries have been put online as .zip files by authors born in the 50s and who still write their documentation in .txt files and put things on .edu domains. These libraries simply may never end up on a VCS like GitHub (licensing, anyone?).

The only way around this right now is a manual install.

Desired Change

This approach is stolen directly from the PHP package manager Composer.

I can specify required dependencies in PlatformIO's library.json as follows:

"dependencies":
[
    {
        "name": "derp",
    },

Please also allow me to override the libraries source with something akin to:

"repositories": [
    {
        "name": "derp",
        "url": "https://github.com/dave-newson/derp.git",
        "type": "git"
    }
],

The name property indicates to the PlatformIO executable that I want the dependency derp to come from the specified GitHub repository. This way I can use forks and branches of my own.

In additional, PlatformIO shouldn't require this repository to contain a library.json file. It should just checkout the entire repo wholesale and dump it into my project.

Again, stolen from the likes of Composer, a similar approach can be used to drag in zip archives:

"repositories": [
    {
        "type": "package",
        "package": {
            "name": "derp",
            "version": "master",
            "dist": {
                "type": "zip",
                "url": "http://example.com/src/derp.zip",
                "reference": "master"
            },
        }
    }
]

Obviously PlatformIO requires its own approach and json structure, but this would enable people to include from any git-based repository or zip archive, without the need for registration with PlatformIO.
This would greatly expand its capabilities and enable faster adoption.

This system could also allow expansion for compatibility with any other VCS like Mercurial or SVN.

@ivankravets ivankravets added this to the 3.0.0 milestone Jan 16, 2016

@ivankravets ivankravets self-assigned this Jan 16, 2016

@oskargargas

This comment has been minimized.

Show comment
Hide comment
@oskargargas

oskargargas Jan 19, 2016

On top of that please allow checkout of specific version based on git (or other vcs) tag or commit hash.
Ruby's Bundler and iOS Cocoapods (which uses the same library file format) are perfect example whats needed in terms of dependency version management and custom sources checkout.

oskargargas commented Jan 19, 2016

On top of that please allow checkout of specific version based on git (or other vcs) tag or commit hash.
Ruby's Bundler and iOS Cocoapods (which uses the same library file format) are perfect example whats needed in terms of dependency version management and custom sources checkout.

@ivankravets

This comment has been minimized.

Show comment
Hide comment
@ivankravets

ivankravets Jan 19, 2016

Member

@oskargargas Of course, I'll implement it.

Member

ivankravets commented Jan 19, 2016

@oskargargas Of course, I'll implement it.

@furyfire

This comment has been minimized.

Show comment
Hide comment
@furyfire

furyfire Jan 26, 2016

The design specs for PHP's composer should apply directly to platformio.
https://getcomposer.org/doc/
Maybe it is time for the library's you require to be placed in the project folder and not platformio's user folder?

What about resolving dependecies? Can package A require package B which will also be automatically installed and updated?

furyfire commented Jan 26, 2016

The design specs for PHP's composer should apply directly to platformio.
https://getcomposer.org/doc/
Maybe it is time for the library's you require to be placed in the project folder and not platformio's user folder?

What about resolving dependecies? Can package A require package B which will also be automatically installed and updated?

@ivankravets

This comment has been minimized.

Show comment
Hide comment
@ivankravets

ivankravets Jan 26, 2016

Member

@furyfire

Maybe it is time for the library's you require to be placed in the project folder and not platformio's user folder?

Thanks! See #475

What about resolving dependecies? Can package A require package B which will also be automatically installed and updated?

These features are partially implemented in the PlatformIO 2.0. However, this functionality will be rewritten from the scratch in PlatformIO 3.0. See https://github.com/platformio/platformio/issues?utf8=✓&q=is%3Aopen+milestone%3A3.0.0+label%3Alib

Member

ivankravets commented Jan 26, 2016

@furyfire

Maybe it is time for the library's you require to be placed in the project folder and not platformio's user folder?

Thanks! See #475

What about resolving dependecies? Can package A require package B which will also be automatically installed and updated?

These features are partially implemented in the PlatformIO 2.0. However, this functionality will be rewritten from the scratch in PlatformIO 3.0. See https://github.com/platformio/platformio/issues?utf8=✓&q=is%3Aopen+milestone%3A3.0.0+label%3Alib

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd commented Feb 8, 2016

👍

@cslamar

This comment has been minimized.

Show comment
Hide comment
@cslamar

cslamar commented Apr 23, 2016

👍

ivankravets added a commit that referenced this issue Sep 9, 2016

Merge branch 'develop' into feature/unicode-issue-771
* develop:
  Fix incorrect line order when converting from INO to CPP and pointer is used
  Fix unit test
  Notify about `version` field when creating library
  Add support for SparkFun Blynk Board
  Return valid exit code from ``plaformio test`` command
  Disable SSL Server-Name-Indication for Python < 2.7.9
  Version bump to 3.0.1 (issue #772)
  Disable temporary SSL for PlatformIO services // Resolve #772
  Version bump to 3.0.0 (issues #770, #766, #747, #730, #765, #640, #659, #742, #459, #542, #763, #759, #753, #757, #749, #748, #745, #519, #709, #743, #413, #498, #410, #740, #361, #414, #554, #732, #588, #475, #461, #101, #719, #721, #537, #415, #522, #289, #556, #570, #456, #617, #432, #408, #479, #667, #510)
  Fix menu height for  docs
  Fix issue with multiple archives when linking firmware
  Add migration guide for PIO2 to PIO3
  Search libraries by headers/includes with ``platformio lib search --header`` option
  Update pio run command examples
  Add Unit Testing Demo
  Update PIO Plus badge title and link
  Add PlatformIO Plus badge
  Add links to PlatformIO Plus
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment