Skip to content
Browse files

Stream pkg INSERTs rather than caching everything.

Use libarchive (which we were linking against but not using) to handle
streaming the remote pkg_summary.  The copy of decompress.c is no longer

We now stream one pkg_info record at a time, rather than loading the
entire contents in and then processing them all in one go.  They are
still processed as part of the same commit, so there is no performance
penalty (if anything this version is slightly faster).

Using the SmartOS 2015Q1 x86_64 repository as a test corpus, heap usage
for a clean "pkgin update" with this change goes down from 75MB to 29MB.
  • Loading branch information
jperkin committed Jul 16, 2015
1 parent 8530c8a commit bb51acbab56929b7c40bb3a450e276de1a268a3d
Showing with 266 additions and 447 deletions.
  1. +1 −2
  2. +0 −196 external/decompress.c
  3. +0 −3 external/lib.h
  4. +265 −246 summary.c
@@ -8,8 +8,7 @@ SRCS= main.c summary.c tools.c pkgindb.c depends.c actions.c \
pkgindb_queries.c pkg_str.c sqlite_callbacks.c selection.c \
pkg_check.c pkg_infos.c preferred.c
# included from libinstall
SRCS+= automatic.c decompress.c dewey.c fexec.c global.c \
opattern.c pkgdb.c var.c
SRCS+= automatic.c dewey.c fexec.c global.c opattern.c pkgdb.c var.c
# included from openssh
SRCS+= progressmeter.c

This file was deleted.

@@ -441,9 +441,6 @@ lpkg_t *alloc_lpkg(const char *);
lpkg_t *find_on_queue(lpkg_head_t *, const char *);
void free_lpkg(lpkg_t *);

/* Extract input if compressed to NUL terminated buffer (not counted) */
int decompress_buffer(const char *, size_t, char **, size_t *);

/* Parse NUL terminated inputed, argument is strlen of the input */
struct pkg_vulnerabilities *parse_pkg_vulnerabilities(const char *, size_t, int);
/* Read pkg_vulnerabilities from file */

0 comments on commit bb51acb

Please sign in to comment.
You can’t perform that action at this time.