Skip to content
Permalink
Browse files

Add possibility for user to fetch specific packages from other sources

- config right now for parsing packages, which should be installed from sources where they normally would not be installed from

see: #23
Add possibility for user to fetch specific packages from other sources

see: #23
Append aurman config to README

Explain the possibility of how to choose an explicit source for a package in case of multiple available sources

see: #23
Add newlines to README


Fix typo in README


Do not forget already ignored things


Add --do_everything

Flag so that aurman handles also -u for repo packages
Merge branch 'other_repos' into develop
  • Loading branch information...
polygamma committed Mar 7, 2018
1 parent f6908f1 commit 6c02ba3db9f803f08fddf54cba390a910f7534d3
Showing with 276 additions and 34 deletions.
  1. +33 −0 README.md
  2. +66 −4 src/aurman/classes.py
  3. +3 −0 src/aurman/help_printing.py
  4. +64 −20 src/aurman/main.py
  5. +29 −9 src/aurman/main_solver.py
  6. +2 −1 src/aurman/parse_args.py
  7. +79 −0 src/aurman/parsing_config.py
@@ -73,6 +73,38 @@ you may specify more than one package, space separated

- `--holdpkg_conf`: append packages from the pacman.conf to `--holdpkg`

- `--do_everything`: `-u` for repo packages will also be handled by `aurman`, not by `pacman`.
may be useful if you are using the `aurman` config to fetch repo packages from other repos than they would normally be installed from.
this is in general **not** recommended!

## Config
You may use the file `aurman_config` under `$XDG_CONFIG_HOME/aurman` (fallback to `~/.config/aurman` in case of no `$XDG_CONFIG_HOME`) as config for aurman.

### config options
#### choose between multiple package sources
By default `aurman` assumes the following priorities in case of multiple available packages with the same names (high to low):
- repository package as listed in the pacman.conf - see https://www.archlinux.org/pacman/pacman.conf.5.html#_repository_sections
> The order of repositories in the configuration files matters; repositories listed first will take precedence over those listed later in the file when packages in two repositories have identical names, regardless of version number.
- aur packages
If one wants to override this priority, it has to be done via the aurman config.
For aur packages create a section `[aur_packages]` and list the names of the aur packages.
For repo packages create a section `[repo_packages]` and list the names of the repo packages followed by `=` and the name of the repo.
Example:
```ini
[aur_packages]
aur_package_name
other_aur_package_name
[repo_packages]
repo_package_name=repo_one
other_repo_package_name=repo_two
```
## Features
- threaded sudo loop in the background so you only have to enter your password once
@@ -84,6 +116,7 @@ you may specify more than one package, space separated
- let the user see and edit all needed PKGBUILDs before any of the building of AUR packages starts
- fetching of needed pgp keys for package building
- pacman --search for repo and aur packages (aur results sorted by popularity)
- differentiate between the possible sources to install packages from in case of same names in different known repos and/or the aur
## Dependency solving description including benchmarks
https://github.com/polygamma/aurman/wiki/Description-of-the-aurman-dependency-solving
@@ -9,6 +9,7 @@
from aurman.aur_utilities import is_devel, get_aur_info
from aurman.coloring import aurman_status, aurman_note, aurman_error, aurman_question, Colors
from aurman.own_exceptions import InvalidInput, ConnectionProblem
from aurman.parsing_config import packages_from_other_sources
from aurman.utilities import strip_versioning_from_name, split_name_with_versioning, version_comparison, ask_user
from aurman.wrappers import expac, makepkg, pacman

@@ -248,10 +249,13 @@ def get_packages_from_expac(expac_operation: str, packages_names: Sequence[str],
if "Q" in expac_operation:
formatting = list("nvDHoPReGw")
repos = []
repo_dict = {}
else:
assert "S" in expac_operation
formatting = list("nvDHoPReGr")
repos = Package.get_known_repos()
# packages the user wants to install from another repo
repo_dict = packages_from_other_sources()[1]

expac_return = expac(expac_operation, formatting, packages_names)
return_dict = {}
@@ -290,15 +294,42 @@ def get_packages_from_expac(expac_operation: str, packages_names: Sequence[str],
assert "S" in expac_operation
to_expand['repo'] = splitted_line[9]

if to_expand['name'] in return_dict and repos.index(return_dict[to_expand['name']].repo) < repos.index(
to_expand['repo']):
continue
# continue if we explicitly want a package from a specific repo
# and the package is not from that repo
# or if the order of the repos in pacman.conf tells us to
if to_expand['name'] in return_dict:
if to_expand['name'] in repo_dict:
if to_expand['repo'] == repo_dict[to_expand['name']]:
pass
elif return_dict[to_expand['name']].repo != repo_dict[to_expand['name']]:
if repos.index(return_dict[to_expand['name']].repo) < repos.index(to_expand['repo']):
continue

elif repos.index(return_dict[to_expand['name']].repo) < repos.index(to_expand['repo']):
continue

if to_expand['name'] in to_expand['conflicts']:
to_expand['conflicts'].remove(to_expand['name'])

return_dict[to_expand['name']] = Package(**to_expand)

# check if all repos the user gave us are actually known
for repo_package_name in repo_dict:
if repo_package_name not in return_dict:
aurman_error("Package {} "
"not known in any repo".format(Colors.BOLD(Colors.LIGHT_MAGENTA(repo_package_name))))
raise InvalidInput("Package {} "
"not known in any repo".format(Colors.BOLD(Colors.LIGHT_MAGENTA(repo_package_name))))

package_repo = return_dict[repo_package_name].repo
if package_repo != repo_dict[repo_package_name]:
aurman_error("Package {} not found in repo {}"
"".format(Colors.BOLD(Colors.LIGHT_MAGENTA(repo_package_name)),
Colors.BOLD(Colors.LIGHT_MAGENTA(repo_dict[repo_package_name]))))
raise InvalidInput("Package {} not found in repo {}"
"".format(Colors.BOLD(Colors.LIGHT_MAGENTA(repo_package_name)),
Colors.BOLD(Colors.LIGHT_MAGENTA(repo_dict[repo_package_name]))))

return list(return_dict.values())

def __init__(self, name: str, version: str, depends: Sequence[str] = None, conflicts: Sequence[str] = None,
@@ -1102,6 +1133,11 @@ def get_installed_packages() -> List['Package']:
:return: A list containing the installed packages
"""
repo_packages_names = set(expac("-S", ('n',), ()))

# packages the user wants to install from aur
aur_names = packages_from_other_sources()[0]
repo_packages_names -= aur_names

installed_packages_names = set(expac("-Q", ('n',), ()))
installed_repo_packages_names = installed_packages_names & repo_packages_names
unclassified_installed_names = installed_packages_names - installed_repo_packages_names
@@ -1116,6 +1152,10 @@ def get_installed_packages() -> List['Package']:
# installed aur packages
installed_aur_packages_names = set(
[package.name for package in Package.get_packages_from_aur(list(unclassified_installed_names))])
for name in aur_names:
if name not in installed_aur_packages_names:
aurman_error("Package {} not found in AUR!".format(Colors.BOLD(Colors.LIGHT_MAGENTA(name))))
raise InvalidInput("Package {} not found in AUR!".format(Colors.BOLD(Colors.LIGHT_MAGENTA(name))))

if installed_aur_packages_names:
return_list.extend(
@@ -1153,6 +1193,9 @@ def __init__(self, packages: Sequence['Package']):

self.append_packages(packages)

def recreate_dicts(self):
self.__init__(list(self.all_packages_dict.values()))

def append_packages(self, packages: Sequence['Package']):
"""
Appends packages to this system.
@@ -1293,19 +1336,38 @@ def append_packages_by_name(self, packages_names: Sequence[str]):

packages_names = set([strip_versioning_from_name(name) for name in packages_names])
packages_names_to_fetch = [name for name in packages_names if name not in self.all_packages_dict]
aur_names = packages_from_other_sources()[0]
for name in packages_names:
if name in packages_names_to_fetch:
continue

if name not in aur_names:
continue

package = self.all_packages_dict[name]
if package.type_of is not PossibleTypes.AUR_PACKAGE and package.type_of is not PossibleTypes.DEVEL_PACKAGE:
packages_names_to_fetch.append(name)

deleted_while_appending = False
while packages_names_to_fetch:
fetched_packages = Package.get_packages_from_aur(packages_names_to_fetch)
self.append_packages(fetched_packages)

deps_of_the_fetched_packages = []
for package in fetched_packages:
deps_of_the_fetched_packages.extend(package.relevant_deps())
if package.name in self.all_packages_dict:
del self.all_packages_dict[package.name]
deleted_while_appending = True

self.append_packages(fetched_packages)

relevant_deps = list(set([strip_versioning_from_name(dep) for dep in deps_of_the_fetched_packages]))

packages_names_to_fetch = [dep for dep in relevant_deps if dep not in self.all_packages_dict]

if deleted_while_appending:
self.recreate_dicts()

def are_all_deps_fulfilled(self, package: 'Package', only_make_check: bool = False,
only_depends: bool = False, print_reason: bool = False) -> bool:
"""
@@ -154,3 +154,6 @@ def __repr__(self):
only_aurman_points.append(HelpOption(["--holdpkg_conf"],
"append packages from the pacman.conf to"
" {}".format(Colors.LIGHT_GREEN("--holdpkg"))))
only_aurman_points.append(HelpOption(["--do_everything"],
"{} will be handled by aurman for repo packages, too"
"".format(Colors.LIGHT_GREEN("-u"))))
@@ -10,6 +10,7 @@
from aurman.help_printing import aurman_help
from aurman.own_exceptions import InvalidInput
from aurman.parse_args import PacmanOperations, parse_pacman_args
from aurman.parsing_config import read_config, packages_from_other_sources
from aurman.utilities import acquire_sudo, version_comparison, search_and_print
from aurman.wrappers import pacman, expac

@@ -69,6 +70,12 @@ def process(args):
noconfirm = pacman_args.noconfirm # if --noconfirm
search = pacman_args.search # list containing the specified strings for -s and --search
solution_way = pacman_args.solution_way # if --solution_way
do_everything = pacman_args.do_everything # if --do_everything

try:
read_config() # read config - available via AurmanConfig.aurman_config
except InvalidInput:
return

not_remove = pacman_args.holdpkg # list containing the specified packages for --holdpkg
# if --holdpkg_conf append holdpkg from pacman.conf
@@ -105,27 +112,44 @@ def process(args):
# if user just wants to search
if search:
if not repo:
installed_system = System(System.get_installed_packages())
try:
installed_system = System(System.get_installed_packages())
except InvalidInput:
return
else:
installed_system = None
search_and_print(search, installed_system, str(pacman_args), repo, aur)
return

# groups are for pacman
groups = pacman("-Sg", True, sudo=False)
groups_chosen = []
for name in packages_of_user_names[:]:
if name in groups:
groups_chosen.append(name)
packages_of_user_names.remove(name)

# in case of sysupgrade or groups to install and not --aur, call pacman
if (sysupgrade or groups_chosen) and not aur:
if not aur:
groups = pacman("-Sg", True, sudo=False)
for name in packages_of_user_names[:]:
if name in groups:
groups_chosen.append(name)
packages_of_user_names.remove(name)

# pacman call in the beginning of the routine
if not aur \
and (sysupgrade and (not do_everything or pacman_args.refresh) or groups_chosen):
if not sudo_acquired:
acquire_sudo()
sudo_acquired = True
pacman_args_copy = deepcopy(pacman_args)
pacman_args_copy.targets = groups_chosen
# aurman handles the update
if do_everything:
pacman_args_copy.sysupgrade = False
# ignore packages from other sources for sysupgrade
packages_from_other_sources_ret = packages_from_other_sources()
names_to_ignore = packages_from_other_sources_ret[0]
for name_to_ignore in packages_from_other_sources_ret[1]:
names_to_ignore.add(name_to_ignore)
for already_ignored in pacman_args_copy.ignore:
names_to_ignore |= set(already_ignored.split(","))
if names_to_ignore:
pacman_args_copy.ignore = [",".join(names_to_ignore)]
try:
pacman(str(pacman_args_copy), False)
except InvalidInput:
@@ -144,18 +168,21 @@ def process(args):
aurman_status("initializing {}...".format(Colors.BOLD("aurman")), True)

# analyzing installed packages
installed_system = System(System.get_installed_packages())
try:
installed_system = System(System.get_installed_packages())
except InvalidInput:
return

if installed_system.not_repo_not_aur_packages_list:
aurman_status("the following packages are neither in known repos nor in the aur")
for package in installed_system.not_repo_not_aur_packages_list:
aurman_note("{}".format(Colors.BOLD(Colors.LIGHT_MAGENTA(package))))

# fetching upstream repo packages...
if not aur:
try:
upstream_system = System(System.get_repo_packages())
else:
upstream_system = System(())
except InvalidInput:
return

# fetching needed aur packages
if not repo:
@@ -165,6 +192,12 @@ def process(args):
names_of_installed_aur_packages.extend([package.name for package in installed_system.devel_packages_list])
upstream_system.append_packages_by_name(names_of_installed_aur_packages)

# remove known repo packages in case of --aur
if aur:
for package in upstream_system.repo_packages_list:
del upstream_system.all_packages_dict[package.name]
upstream_system = System(list(upstream_system.all_packages_dict.values()))

# sanitize user input
try:
sanitized_names = upstream_system.sanitize_user_input(packages_of_user_names)
@@ -236,11 +269,15 @@ def process(args):
else:
concrete_packages_to_install.append(package)

# in case of sysupgrade and not --repo fetch all installed aur packages, of which newer versions are available
if sysupgrade and not repo:
installed_aur_packages = [package for package in installed_system.aur_packages_list]
installed_aur_packages.extend([package for package in installed_system.devel_packages_list])
for package in installed_aur_packages:
# in case of sysupgrade fetch all installed packages, of which newer versions are available
if sysupgrade:
installed_packages = []
if not repo:
installed_packages.extend([package for package in installed_system.aur_packages_list])
installed_packages.extend([package for package in installed_system.devel_packages_list])
if not aur:
installed_packages.extend([package for package in installed_system.repo_packages_list])
for package in installed_packages:
# must not be that we have not received the upstream information
assert package.name in upstream_system.all_packages_dict
upstream_package = upstream_system.all_packages_dict[package.name]
@@ -298,6 +335,9 @@ def process(args):
if not sudo_acquired:
acquire_sudo()

# repo packages to install from other sources
repo_packages_dict = packages_from_other_sources()[1]

# generate pacman args for the aur packages
pacman_args_copy = deepcopy(pacman_args)
pacman_args_copy.operation = PacmanOperations.UPGRADE
@@ -324,7 +364,11 @@ def process(args):
as_explicit_container.add(package.name)

pacman_args_copy = deepcopy(pacman_args)
pacman_args_copy.targets = [package.name for package in package_chunk]
pacman_args_copy.targets = [package.name for package in package_chunk if
package.name not in repo_packages_dict]

pacman_args_copy.targets.extend(["{}/".format(repo_packages_dict[package.name]) + package.name
for package in package_chunk if package.name in repo_packages_dict])
pacman_args_copy.asdeps = True
pacman_args_copy.asexplicit = False
try:
@@ -352,7 +396,7 @@ def process(args):
def main():
try:
# auto completion
if argv[1] == "--auto_complete":
if len(argv) >= 2 and argv[1] == "--auto_complete":
possible_completions()
return

Oops, something went wrong.

0 comments on commit 6c02ba3

Please sign in to comment.
You can’t perform that action at this time.