Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ChannelSpec #2870

Merged
merged 53 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
e585ce1
Make find_slash_and_platform private-lib
AntoinePrv Sep 25, 2023
e189e93
Add ChannelSpec
AntoinePrv Sep 25, 2023
02beb7a
Add ChannelSpec::Type
AntoinePrv Sep 25, 2023
2b27731
Add abs_path_or_url_to_url
AntoinePrv Sep 26, 2023
7116740
Add ChannelSpec path normalization and type detection
AntoinePrv Sep 26, 2023
87a9520
Add ChannelSpec::location
AntoinePrv Sep 26, 2023
0babd93
Allow any platform string in ChannelSpec
AntoinePrv Sep 27, 2023
66872c5
Channel::platforms is a flat_set
AntoinePrv Sep 27, 2023
f7064fb
Move flat_set_caster to own file
AntoinePrv Sep 28, 2023
602e690
More flat_set in Channel
AntoinePrv Sep 28, 2023
f578e56
Regenerate Python stubs
AntoinePrv Sep 28, 2023
506d3c6
Plug ChannelSpec into Channel
AntoinePrv Sep 28, 2023
29b3bf4
Resolve path in ChannelSpec
AntoinePrv Sep 28, 2023
fac00f7
Fix env export channels
AntoinePrv Sep 29, 2023
875569d
No resolve symlink in ChannelSpec
AntoinePrv Sep 29, 2023
d075383
Fix tests on Windows
AntoinePrv Oct 3, 2023
9c69a44
Add ChannelSpec attribute extractors
AntoinePrv Oct 3, 2023
1723c5e
Add raw domain ChannelSpec test
AntoinePrv Oct 4, 2023
634ca7d
Use URL in read_channel_configuration
AntoinePrv Oct 4, 2023
7e982fd
Fix local channel name
AntoinePrv Oct 4, 2023
e2ee9d9
Improve path_to_url
AntoinePrv Oct 4, 2023
ab1dadd
Construct from Channel from URL
AntoinePrv Oct 4, 2023
886cd03
Properly parse channel alias
AntoinePrv Oct 4, 2023
06efd80
Fix wrong move
AntoinePrv Oct 5, 2023
a747686
Channel alias is a CondaURL
AntoinePrv Oct 4, 2023
d6ec077
Add defaulted scheme to URL
AntoinePrv Oct 5, 2023
a833a3a
Add defaulted host to URL
AntoinePrv Oct 5, 2023
cc276aa
CondaURL tokens can only be at the begining
AntoinePrv Oct 6, 2023
50ad356
Add CondaURL::path_without_token
AntoinePrv Oct 6, 2023
ad79d6b
Add option to remove credentials in URLs
AntoinePrv Oct 6, 2023
cba884e
Add empty cretendial URL tests
AntoinePrv Oct 9, 2023
f195c70
Add str Credential options
AntoinePrv Oct 9, 2023
87fc107
Add credential option to CondaURL::str
AntoinePrv Oct 9, 2023
45dd4dd
Add split_prefix/suffix
AntoinePrv Oct 9, 2023
b42a059
Add ChannelContext::from_package_path
AntoinePrv Oct 9, 2023
853adbc
URL::pretty_path only on Windows
AntoinePrv Oct 10, 2023
b6fadec
Add ChannelContext::from_path
AntoinePrv Oct 10, 2023
27b6055
Do not resolve paths in ChannelSpec
AntoinePrv Oct 10, 2023
9bc90d2
Change mapping order in path resolving
AntoinePrv Oct 10, 2023
15285c4
Compute URL canonical_name in read_channel_configurations
AntoinePrv Oct 10, 2023
dc08c30
Add URL::authority credentials
AntoinePrv Oct 10, 2023
1b282e1
Rename private URL::authentification > URL::authentifaction_elems
AntoinePrv Oct 10, 2023
5c4e458
Refactor URL::authority
AntoinePrv Oct 10, 2023
26ed420
Refactor some read_channel_configurations
AntoinePrv Oct 10, 2023
9903c62
Remove empty path URL case
AntoinePrv Oct 11, 2023
5b4c32c
Add path_is_prefix
AntoinePrv Oct 11, 2023
684402c
Fix url_match
AntoinePrv Oct 11, 2023
830527c
Add flat_set doctest printer
AntoinePrv Oct 11, 2023
120adf4
Add CondaURL doctest printer
AntoinePrv Oct 11, 2023
243dd1e
Alias dynamic_platform_set in ChannelSpec
AntoinePrv Oct 12, 2023
6c334e6
Use util:concat
AntoinePrv Oct 12, 2023
2a75f1c
Restore subdir in matchspecs
AntoinePrv Oct 12, 2023
d8c6391
Make find_slash_and_platform private
AntoinePrv Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions libmamba/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ set(LIBMAMBA_SOURCES
${LIBMAMBA_SOURCE_DIR}/specs/conda_url.cpp
${LIBMAMBA_SOURCE_DIR}/specs/version.cpp
${LIBMAMBA_SOURCE_DIR}/specs/version_spec.cpp
${LIBMAMBA_SOURCE_DIR}/specs/channel_spec.cpp
${LIBMAMBA_SOURCE_DIR}/specs/repo_data.cpp
# Core API (low-level)
${LIBMAMBA_SOURCE_DIR}/core/singletons.cpp
Expand Down Expand Up @@ -236,6 +237,7 @@ set(LIBMAMBA_PUBLIC_HEADERS
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/conda_url.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/version.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/version_spec.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/channel_spec.hpp
${LIBMAMBA_INCLUDE_DIR}/mamba/specs/repo_data.hpp
# Core API (low-level)
${LIBMAMBA_INCLUDE_DIR}/mamba/core/activation.hpp
Expand Down
41 changes: 29 additions & 12 deletions libmamba/include/mamba/core/channel.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
#include <string>
#include <string_view>
#include <utility>
#include <vector>

#include "mamba/specs/conda_url.hpp"
#include "mamba/util/flat_set.hpp"

namespace mamba
{
Expand All @@ -25,6 +25,10 @@ namespace mamba
{
class RepoChecker;
}
namespace specs
{
class ChannelSpec;
}

std::vector<std::string> get_known_platforms();

Expand All @@ -40,11 +44,11 @@ namespace mamba

~Channel();

const std::string& scheme() const;
std::string_view scheme() const;
const std::string& location() const;
const std::string& name() const;
const std::string& canonical_name() const;
const std::vector<std::string>& platforms() const;
const util::flat_set<std::string>& platforms() const;
std::optional<std::string> auth() const;
std::optional<std::string> user() const;
std::optional<std::string> password() const;
Expand All @@ -56,9 +60,9 @@ namespace mamba
std::string base_url() const;
std::string platform_url(std::string platform, bool with_credential = true) const;
// The pairs consist of (platform,url)
std::vector<std::pair<std::string, std::string>>
util::flat_set<std::pair<std::string, std::string>>
platform_urls(bool with_credential = true) const;
std::vector<std::string> urls(bool with_credential = true) const;
util::flat_set<std::string> urls(bool with_credential = true) const;

private:

Expand All @@ -70,14 +74,25 @@ namespace mamba
std::string_view user = {},
std::string_view password = {},
std::string_view token = {},
std::string_view package_filename = {}
std::string_view package_filename = {},
util::flat_set<std::string> platforms = {}
);

Channel(
specs::CondaURL url,
std::string location,
std::string name,
std::string canonical_name,
util::flat_set<std::string> platforms = {}
);

const specs::CondaURL& url() const;

specs::CondaURL m_url;
std::string m_location;
std::string m_name;
std::string m_canonical_name;
std::vector<std::string> m_platforms;
util::flat_set<std::string> m_platforms;

// This is used to make sure that there is a unique repo for every channel
mutable std::unique_ptr<validation::RepoChecker> p_repo_checker;
Expand Down Expand Up @@ -116,7 +131,7 @@ namespace mamba
const Channel& make_channel(const std::string& value);
std::vector<const Channel*> get_channels(const std::vector<std::string>& channel_names);

const Channel& get_channel_alias() const;
const specs::CondaURL& get_channel_alias() const;
const channel_map& get_custom_channels() const;

Context& context() const
Expand All @@ -128,7 +143,7 @@ namespace mamba

Context& m_context;
ChannelCache m_channel_cache;
Channel m_channel_alias;
specs::CondaURL m_channel_alias;
channel_map m_custom_channels;
multichannel_map m_custom_multichannels;

Expand All @@ -137,16 +152,18 @@ namespace mamba
const multichannel_map& get_custom_multichannels() const;

Channel make_simple_channel(
const Channel& channel_alias,
const specs::CondaURL& channel_alias,
const std::string& channel_url,
const std::string& channel_name,
const std::string& channel_canonical_name
);

Channel from_url(std::string_view url);
Channel from_any_path(specs::ChannelSpec&& spec);
Channel from_package_path(specs::ChannelSpec&& spec);
Channel from_path(specs::ChannelSpec&& spec);
Channel from_url(specs::ChannelSpec&& spec);
Channel from_name(const std::string& name);
Channel from_value(const std::string& value);
Channel from_alias(std::string_view alias);
};

} // namespace mamba
Expand Down
10 changes: 10 additions & 0 deletions libmamba/include/mamba/fs/filesystem.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -377,11 +377,21 @@ namespace mamba::fs
return m_path.extension();
}

u8path lexically_normal() const
{
return m_path.lexically_normal();
}

u8path lexically_relative(const u8path& base) const
{
return m_path.lexically_relative(base);
}

u8path lexically_proximate(const u8path& base) const
{
return m_path.lexically_proximate(base);
}

//---- Modifiers ----

void clear() noexcept
Expand Down
95 changes: 95 additions & 0 deletions libmamba/include/mamba/specs/channel_spec.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
// Copyright (c) 2023, QuantStack and Mamba Contributors
//
// Distributed under the terms of the BSD 3-Clause License.
//
// The full license is in the file LICENSE, distributed with this software.

#ifndef MAMBA_SPECS_CHANNEL_SPEC_HPP
#define MAMBA_SPECS_CHANNEL_SPEC_HPP

#include <string>
#include <string_view>

#include "mamba/util/flat_set.hpp"

namespace mamba::specs
{
/**
* Channel specification.
*
* This represent the string that is passed by the user to select a channel.
* It needs to be resolved in order to get a final URL/path.
* This is even true when a full URL or path is given, as some authentification information
* may come from login database.
*
* Note that for a string to be considered a URL, it must have an explicit scheme.
* So "repo.anaconda.com" is considered a name, similarily to "conda-forge" and not a URL.
* This is because otherwise it is not possible to tell names and URL appart.
*/
class ChannelSpec
{
public:

enum class Type
{
/**
* A URL to a full repo strucuture.
*
* Example "https://repo.anaconda.com/conda-forge".
*/
URL,
/**
* A URL to a single package.
*
* Example "https://repo.anaconda.com/conda-forge/linux-64/pkg-0.0-bld.conda".
*/
PackageURL,
/**
* An (possibly implicit) path to a full repo strucuture.
*
* Example "/Users/name/conda-bld", "./conda-bld", "~/.conda-bld".
*/
Path,
/**
* An (possibly implicit) path to a single-package.
*
* Example "/tmp/pkg-0.0-bld.conda", "./pkg-0.0-bld.conda", "~/pkg-0.0-bld.tar.bz2".
*/
PackagePath,
/**
* A relative name.
*
* It needs to be resolved using a channel alias, or a custom channel.
* Example "conda-forge", "locals", "my-channel/my-label".
*/
Name,
};

static constexpr std::string_view default_name = "defaults";
static constexpr std::string_view platform_separators = "|,;";

using dynamic_platform_set = util::flat_set<std::string>;

[[nodiscard]] static auto parse(std::string_view str) -> ChannelSpec;

ChannelSpec() = default;
ChannelSpec(std::string location, dynamic_platform_set filters, Type type);

[[nodiscard]] auto type() const -> Type;

[[nodiscard]] auto location() const& -> const std::string&;
[[nodiscard]] auto location() && -> std::string;
[[nodiscard]] auto clear_location() -> std::string;

[[nodiscard]] auto platform_filters() const& -> const dynamic_platform_set&;
[[nodiscard]] auto platform_filters() && -> dynamic_platform_set;
[[nodiscard]] auto clear_platform_filters() -> dynamic_platform_set;

private:

std::string m_location = std::string(default_name);
dynamic_platform_set m_platform_filters = {};
Type m_type = {};
};
}
#endif
75 changes: 69 additions & 6 deletions libmamba/include/mamba/specs/conda_url.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ namespace mamba::specs
public:

using StripScheme = util::detail::StripScheme;
using HideConfidential = util::detail::HideConfidential;
using Credentials = util::detail::Credentials;
using Encode = util::detail::Encode;
using Decode = util::detail::Decode;

Expand All @@ -34,15 +34,18 @@ namespace mamba::specs
explicit CondaURL(util::URL&& url);
explicit CondaURL(const util::URL& url);

using Base::scheme_is_defaulted;
using Base::scheme;
using Base::set_scheme;
using Base::clear_scheme;
using Base::user;
using Base::set_user;
using Base::clear_user;
using Base::password;
using Base::set_password;
using Base::clear_password;
using Base::authentication;
using Base::host_is_defaulted;
using Base::host;
using Base::set_host;
using Base::clear_host;
Expand All @@ -52,16 +55,50 @@ namespace mamba::specs
using Base::authority;
using Base::path;
using Base::pretty_path;
using Base::set_path;
using Base::clear_path;
using Base::append_path;
using Base::query;
using Base::set_query;
using Base::clear_query;
using Base::fragment;
using Base::set_fragment;
using Base::clear_fragment;

/**
* Set the path from a not encoded value.
*
* All '/' are not encoded but interpreted as separators.
* On windows with a file scheme, the colon after the drive letter is not encoded.
* A leading '/' is added if abscent.
* If the path contains only a token, a trailing '/' is added afterwards.
*/
void set_path(std::string_view path, Encode::yes_type = Encode::yes);

/** Set the path from an already encoded value.
*
* A leading '/' is added if abscent.
* If the path contains only a token, a trailing '/' is added afterwards.
*/
void set_path(std::string path, Encode::no_type);

/**
* Append a not encoded sub path to the current path.
*
* Contrary to `std::filesystem::path::append`, this always append and never replace
* the current path, even if @p subpath starts with a '/'.
* All '/' are not encoded but interpreted as separators.
* If the final path contains only a token, a trailing '/' is added afterwards.
*/
void append_path(std::string_view path, Encode::yes_type = Encode::yes);

/**
* Append a already encoded sub path to the current path.
*
* Contrary to `std::filesystem::path::append`, this always append and never replace
* the current path, even if @p subpath starts with a '/'.
* If the final path contains only a token, a trailing '/' is added afterwards.
*/
void append_path(std::string_view path, Encode::no_type);

/** Return the Conda token, as delimited with "/t/", or empty if there isn't any. */
[[nodiscard]] auto token() const -> std::string_view;

Expand All @@ -76,6 +113,29 @@ namespace mamba::specs
/** Clear the token and return ``true`` if it exists, otherwise return ``false``. */
auto clear_token() -> bool;

/** Return the encoded part of the path without any Conda token, always start with '/'. */
[[nodiscard]] auto path_without_token(Decode::no_type) const -> std::string_view;

/** Return the decoded part of the path without any Conda token, always start with '/'. */
[[nodiscard]] auto path_without_token(Decode::yes_type = Decode::yes) const -> std::string;

/**
* Set the path from an already encoded value, without changing the Conda token.
*
* A leading '/' is added if abscent.
*/
void set_path_without_token(std::string_view path, Encode::no_type);

/**
* Set the path from an not yet encoded value, without changing the Conda token.
*
* A leading '/' is added if abscent.
*/
void set_path_without_token(std::string_view path, Encode::yes_type = Encode::yes);

/** Clear the path without changing the Conda token and return ``true`` if it exists. */
auto clear_path_without_token() -> bool;

/** Return the platform if part of the URL path. */
[[nodiscard]] auto platform() const -> std::optional<Platform>;

Expand Down Expand Up @@ -138,7 +198,8 @@ namespace mamba::specs
/** Clear the package and return true if it exists, otherwise return ``false``. */
auto clear_package() -> bool;

using Base::str;
/** Return the full, exact, encoded URL. */
[[nodiscard]] auto str(Credentials credentials = Credentials::Show) const -> std::string;

/**
* Return the full decoded url.
Expand All @@ -147,18 +208,20 @@ namespace mamba::specs
* asset.
* @param strip_scheme If true, remove the scheme and "localhost" on file URI.
* @param rstrip_path If non-null, remove the given charaters at the end of the path.
* @param hide_confidential If true, hide password and tokens in the decoded string.
* @param credentials If true, hide password and tokens in the decoded string.
* @param credentials Decide to keep, remove, or hide passwrd, users, and token.
*/
[[nodiscard]] auto pretty_str(
StripScheme strip_scheme = StripScheme::no,
char rstrip_path = 0,
HideConfidential hide_confidential = HideConfidential::no
Credentials credentials = Credentials::Show
) const -> std::string;


private:

void set_platform_no_check_input(std::string_view platform);
void ensure_path_without_token_leading_slash();

friend auto operator==(const CondaURL&, const CondaURL&) -> bool;
};
Expand Down
Loading