Improved readme readability. Fixed some problems with packaging and p…

…ublishing

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## v0.1.2
 - Added support for installing Python on most Linux distros
 - Fixed a bug related to creating `pyflow` directory
+- Fixed a bug in specifying package url with the `publish` command.
 
 ## v0.1.1
 - Fixed a bug, where spaces could prevent console scripts from being installed

CONTRIBUTING.md

@@ -0,0 +1,18 @@
+Thanks for your interest in contributing to Pyflow. Bug reports, API improvements, 
+performance improvements and new features are all welcome - as issues, or PRs.
+
+Required tools to build and test:
+- Rust
+- Clippy
+- Rustfmt
+
+Before submitting a PR, please run `cargo fmt`, `cargo clippy`, and `cargo test`.
+
+Recommended starting points:
+- Open issues
+- Dependency graphs that don't resolve correctly
+- Adding and improving tests
+- Bugs or nvonvenient behavior you've encountered
+- Friction points in your workflow not already addressed
+- `//todo` comments in code
+

README.md

@@ -6,23 +6,20 @@
 
 #### *Simple is better than complex* - The Zen of Python
 
-This tool manages Python installations and dependencies. It implements
-[PEP 582 -- Python local packages directory](https://www.python.org/dev/peps/pep-0582/)
-and [Pep 518 (pyproject.toml)](https://www.python.org/dev/peps/pep-0518/). 
-Dependencies are stored in the project directory → `__pypackages__` → `3.7`(etc) → `lib`.
+This tool manages Python installations and dependencies.
 
-You don't need Python installed to use it; it will
-install the specified version of Python if not already installed. 
-
-It includes convenience features, like running standalone files in their
-own environment with no config, and running project functions directly from the CLI.
-
-**Goal**: Make using and publishing Python projects as simple as possible. Understanding
+**Goals**: Make using and publishing Python projects as simple as possible. Understanding
 Python environments shouldn't be required to use dependencies safely. We're attempting
 to fix each stumbling block in the Python workflow, so that it's as elegant
 as the language itself.
 
-Only works with Python ≥ 3.4. 
+You don't need Python or any other tools installed to use Pyflow.
+
+It can run standalone scripts in their
+own environments with no config, and project functions directly from the CLI.
+
+It implements [PEP 582 -- Python local packages directory](https://www.python.org/dev/peps/pep-0582/)
+and [Pep 518 (pyproject.toml)](https://www.python.org/dev/peps/pep-0518/), and supports Python ≥ 3.4.  
 
 
 ## Installation
@@ -31,15 +28,14 @@ Only works with Python ≥ 3.4.
 or
 [this deb](https://github.com/David-OConnor/pyflow/releases/download/0.1.1/pyflow_0.1.1_amd64.deb) .
 
-
 - **A different Linux distro:** Download this [standalone binary](https://github.com/David-OConnor/pyflow/releases/download/0.1.1/pyflow-linux)
  and place it somewhere
 accessible by the system PATH. For example, `/usr/bin`.
 
 - **If you have [Rust](https://www.rust-lang.org) installed**: Run `cargo install pyflow`.
 
 - **Mac:**  Build from source using the instructions near the bottom of this page,
- or install via `cargo`. If able, please build and PR a Mac binary.
+ or install via `cargo`. If able, please PR the binary.
 
 ## Quickstart
 - *(Optional)* Run `pyflow init` in an existing project folder, or `pyflow new projname` 
@@ -50,29 +46,31 @@ or add dependencies to it.
 this file will be created if it doesn't exist.
 - Run `pyflow` or `pyflow myfile.py` to run Python.
 
+
 ## Quick-and-dirty start for quick-and-dirty scripts
-- Add the line `__requires__ = [numpy, requests]` somewhere in the script, where `numpy` and `requsts` are dependencies.
+- Add the line `__requires__ = [numpy, requests]` somewhere in your script, where `numpy` and 
+`requsts` are dependencies.
 Run `pyflow script myscript.py`, where `myscript.py` is the name of your script.
 This will set up an isolated environment for this script, and install
-dependencies as required, without altering any other environment. This is a safe way
+dependencies as required. This is a safe way
 to run one-off Python files that aren't attached to a project, but have dependencies.
 
 
 ## Why add another Python manager?
-`Pipenv` and `Poetry` both address part of this problem.
- Some reasons why this tool is different:
+`Pipenv` and `Poetry` both address part of Pyflow's *raison d'être*.
+ Some reasons why this is different:
 
  - It automatically manages Python installations and environments. You specify a Python version
- in `pyproject.toml` (if ommitted, the pyflow asks), and pyflow ensures that version is used. 
- If it's not installed, pyflow downloads a binary, and uses that.
+ in `pyproject.toml` (if ommitted, itasks), and ensures that version is used. 
+ If the version's not installed, Pyflow downloads a binary, and uses that.
  If multiple installations are found for that version, it asks which to use.
 
-- By not using Python to install or run, it remains intallation-agnostic and 
-environment-agnostic. This is important for making setup and use as simple and decison-free as
+- By not using Python to install or run, it remains environment-agnostic. 
+This is important for making setup and use as simple and decison-free as
  possible. It's especially important on Linux, where there may be several versions
 of Python installed, with different versions and access levels. This avoids
 complications, especially for new users. It's common for Python-based CLI tools
-to not run properly when installed from `pip` due to the `PATH` 
+to not run properly when installed from `pip` due to the `PATH` or user directories
 not being configured in the expected way.
 
 - Its dependency resolution and locking is faster due to using a cached
@@ -81,17 +79,16 @@ on the incomplete data available on the [pypi warehouse](https://github.com/pypa
 
 - It keeps dependencies in the project directory, in `__pypackages__`. This is subtle, 
 but reinforces the idea that there's
-no hidden state to be concerned with.
+no hidden state.
 
-- It will always use the specified version of Python. This is a notable problem, for example,
- with `Poetry`; it
+- It will always use the specified version of Python. This is a notable limitation in `Poetry`; it
 may pick the wrong installation (eg Python2 vice Python3), with no obvious way to change it.
 
 - Multiple versions of a dependency can be installed, allowing resolution
 of conflicting sub-dependencies. (ie: Your package requires `Dep A>=1.0` and `Dep B`.
 `Dep B` requires Dep `A==0.9`) There are many cases where `Poetry` and `Pipenv` will fail
-to resolve dependencies, but we're able to by doing this. Try it for yourself with a few
- random dependencies from [pypi](https://pypi.org/); there's a good change you'll
+to resolve dependencies. Try it for yourself with a few
+ random dependencies from [pypi](https://pypi.org/); there's a good chance you'll
  hit this problem using `Poetry` or `Pipenv`. Limitations: This will not work for
 some compiled dependencies, and attempting to package something using this will
 trigger an error.
@@ -107,24 +104,22 @@ The commands may be long depending on the path of virtual envs and projects,
 and it requires modifying the state of the terminal for each project, each time
 you use it, which you may find inconvenient or inelegant.
 
-If you're satisified with an existing flow, there may be no reason to change, but
 I think we can do better. This is especially relevant for new Python users
-who haven't groked venvs, or are unaware of the hazards of working with a system Python.
+who don't understand venvs, or are unaware of the hazards of working with a system Python.
 
 `Pipenv` improves the workflow by automating environment use, and 
-allowing reproducable dependency resolution. `Poetry` improves upon `Pipenv's` API,
+allowing reproducable dependency graphs. `Poetry` improves upon `Pipenv's` API,
 speed, and dependency resolution, as well as improving
 the packaging and distributing process by using a consolidating project config. Both
- are sensitive to the Python environment used to run them. This tool
-attempts to improve upon both in the areas listed in the section above. Its goal is to be
-as intuitive as possible.
+ are sensitive to the Python environment used to run them, and won't work
+ correctly if it's not as expected. 
 
 `Conda` addresses these problems elegantly, but maintains a separate repository
 of binaries from `PyPi`. If all packages you need are available on `Conda`, it may
 be the best solution. If not, it requires falling back to `Pip`, which means 
 using two separate package managers.
 
-When building and deploying packages, a set of degenerate files are 
+When building and deploying packages, a set of overlapping files are 
 traditionally used: `setup.py`, `setup.cfg`, `requirements.txt` and `MANIFEST.in`. We use
 `pyproject.toml` as the single-source of project info required to build
 and publish.
@@ -170,7 +165,7 @@ diffeqpy = "1.1.0"
 ```
 The `[tool.pyflow]` section is used for metadata. The only required item in it is
  `py_version`, unless
-building and distributing a package. The `[tool.pypyackage.dependencies]` section
+building and distributing a package. The `[tool.pyflow.dependencies]` section
 contains all dependencies, and is an analog to `requirements.txt`. 
 
 You can specify `extra` dependencies, which will only be installed when passing
@@ -306,15 +301,14 @@ check for resolutions, then vary children as-required down the hierarchy. We don
 
 
 ## Not-yet-implemented
-- Installing from sources other than `pypi` (eg repos)
-- Installing multiple versions of a dependency may not work if it uses compiles code
+- Installing from sources other than `pypi` (eg repos, paths)
 - The lock file is missing some info like hashes
 - Adding a dependency via the CLI with a specific version constraint, or extras.
-- Developer requirements
+- Developer dependencies
 - Packaging and publishing projects that use compiled extensions
 - Dealing with multiple-installed-versions of a dependency that uses importlib
 or dynamic imports
-
+- Install Python on Mac
 
 ## Building and uploading your project to PyPi
 In order to build and publish your project, additional info is needed in
@@ -330,11 +324,13 @@ description = "Small, but packs a punch!"
 homepage = "https://everything.math"
 repository = "https://github.com/raz/everythingkiller"
 license = "MIT"
+keywords = ["nanotech", "weapons"]
 classifiers = [
     "Topic :: System :: Hardware",
     "Topic :: Scientific/Engineering :: Human Machine Interfaces",
 ]
 scripts = { activate = "jeejah:activate" }
+python_requires=">=3.6"
 
 
 [tool.pyflow.dependencies]
@@ -370,7 +366,7 @@ not install correctly.
 - Most of the features here are already provided by a range of existing packages,
 like the ones in the table above.
 - The field of contributers is expected to be small, since it's written in a different language.
-- Dependency managers like Pipenv and Poetry work well enough for many cases, and 
+- Dependency managers like Pipenv and Poetry work well enough for many cases,
 have dedicated dev teams, and large userbases.
 - Conda in particular handles many things this does quite well.
 

src/build.rs

@@ -48,13 +48,18 @@ fn _serialize_py_dict(hm: &HashMap<String, Vec<String>>) -> String {
 
 /// Creates a temporary file which imitates setup.py
 fn create_dummy_setup(cfg: &crate::Config, filename: &str) {
+    let cfg = cfg.clone();
+
     let version = match cfg.version {
         Some(v) => v.to_string(),
         None => "".into(),
     };
-    let cfg = cfg.clone();
 
-    println!("CFG: {:#?}", &cfg);
+    let mut keywords = String::new();
+    for kw in &cfg.keywords {
+        keywords.push_str(" ");
+        keywords.push_str(kw);
+    }
 
     let data = format!(
         r#"import setuptools
@@ -73,14 +78,16 @@ setuptools.setup(
     long_description_content_type="text/markdown",
     url="{}",
     packages=setuptools.find_packages(),
-
+    keywords="{}",
     classifiers={},
-
-    entry_points={{
-        "console_scripts": ,
-    }},
+    python_requires="{}",
 )
 "#,
+
+//            entry_points={{
+//        "console_scripts": ,
+//    }},
+
         cfg.readme_filename.unwrap_or_else(|| "README.md".into()),
         cfg.name.unwrap_or_else(|| "".into()),
         version,
@@ -89,8 +96,10 @@ setuptools.setup(
         cfg.license.unwrap_or_else(|| "".into()),
         cfg.description.unwrap_or_else(|| "".into()),
         cfg.homepage.unwrap_or_else(|| "".into()),
+        keywords,
         serialize_py_list(&cfg.classifiers),
         //        serialize_py_list(&cfg.console_scripts),
+        cfg.python_requires.unwrap_or_else(|| "".into()),
 
         // todo:
         //            extras_require="{}",
@@ -160,14 +169,14 @@ pub(crate) fn publish(bin_path: &PathBuf, cfg: &crate::Config) {
     let repo_url = cfg
         .package_url
         .clone()
-        .unwrap_or_else(|| "https://test.pypi.org/legacy".to_string());
+        .unwrap_or_else(|| "https://test.pypi.org/legacy/".to_string());
 
     println!("Uploading to {}", repo_url);
     Command::new(bin_path.join("twine"))
         .args(&[
             "upload",
-            // todo - test repo / setting repos not working.
-            //            &format!("--repository-url {}/", repo_url),
+            "--repository-url",
+            &repo_url,
             "dist/*",
         ])
         .status()

src/dep_resolution.rs

@@ -433,7 +433,7 @@ fn find_constraints(
 
 /// We've determined we need to add all the included packages, and renamed all but one.
 fn make_renamed_packs(
-    vers_cache: &HashMap<String, (String, Version, Vec<Version>)>,
+    _vers_cache: &HashMap<String, (String, Version, Vec<Version>)>,
     deps: &[Dependency],
     //    all_deps: &[Dependency],
     name: &str,
@@ -577,7 +577,7 @@ pub fn resolve(
             // Find what constraints are driving each dep that shares a name.
             let constraints = find_constraints(reqs, &result, &deps);
 
-            let names: Vec<String> = deps.iter().map(|d| d.version.to_string()).collect();
+            let _names: Vec<String> = deps.iter().map(|d| d.version.to_string()).collect();
             //            println!("(dbg): Multiple versions found for {}: {:#?}", &name, names);
             let inter = dep_types::intersection_many(&constraints);
 

src/main.rs

@@ -180,6 +180,7 @@ pub struct Config {
     //    entry_points: HashMap<String, Vec<String>>, // todo option?
     scripts: HashMap<String, String>, //todo: put under [tool.pyflow.scripts] ?
                                       //    console_scripts: Vec<String>, // We don't parse these; pass them to `setup.py` as-entered.
+    python_requires: Option<String>,
 }
 
 impl Config {
@@ -485,8 +486,8 @@ version = "0.1.0"
 description = ""
 author = ""
 
-pyackage_url = "https://test.pypi.org"
-# pyackage_url = "https://pypi.org"
+pyackage_url = "https://test.pypi.org/legacy/"
+# pyackage_url = "https://pypi.org/legacy/"
 
 
 [tool.pyflow.dependencies]

src/util.rs

@@ -221,7 +221,8 @@ pub fn find_console_scripts(bin_path: &Path) -> Vec<String> {
 pub fn merge_reqs(added: &[String], cfg: &crate::Config, cfg_filename: &str) -> Vec<Req> {
     let mut added_reqs = vec![];
     for p in added.iter() {
-        match Req::from_str(&p, false) {
+        let trimmed = p.replace(',', "");
+        match Req::from_str(&trimmed, false) {
             Ok(r) => added_reqs.push(r),
             Err(_) => abort(&format!("Unable to parse this package: {}. \
                     Note that installing a specific version via the CLI is currently unsupported. If you need to specify a version,\
@@ -476,7 +477,7 @@ pub fn prompt_list<T: Clone + ToString>(
 
     let input = match input {
         Ok(ip) => ip,
-        Err(e) => {
+        Err(_) => {
             abort("Please try again; enter a number like 1 or 2 .");
             unreachable!()
         }