Skip to content
Commits on Apr 11, 2016
  1. @anarazel

    Allow Pin/UnpinBuffer to operate in a lockfree manner.

    anarazel committed Apr 10, 2016
    Pinning/Unpinning a buffer is a very frequent operation; especially in
    read-mostly cache resident workloads. Benchmarking shows that in various
    scenarios the spinlock protecting a buffer header's state becomes a
    significant bottleneck. The problem can be reproduced with pgbench -S on
    larger machines, but can be considerably worse for queries which touch
    the same buffers over and over at a high frequency (e.g. nested loops
    over a small inner table).
    
    To allow atomic operations to be used, cram BufferDesc's flags,
    usage_count, buf_hdr_lock, refcount into a single 32bit atomic variable;
    that allows to manipulate them together using 32bit compare-and-swap
    operations. This requires reducing MAX_BACKENDS to 2^18-1 (which could
    be lifted by using a 64bit field, but it's not a realistic configuration
    atm).
    
    As not all operations can easily implemented in a lockfree manner,
    implement the previous buf_hdr_lock via a flag bit in the atomic
    variable. That way we can continue to lock the header in places where
    it's needed, but can get away without acquiring it in the more frequent
    hot-paths.  There's some additional operations which can be done without
    the lock, but aren't in this patch; but the most important places are
    covered.
    
    As bufmgr.c now essentially re-implements spinlocks, abstract the delay
    logic from s_lock.c into something more generic. It now has already two
    users, and more are coming up; there's a follupw patch for lwlock.c at
    least.
    
    This patch is based on a proof-of-concept written by me, which Alexander
    Korotkov made into a fully working patch; the committed version is again
    revised by me.  Benchmarking and testing has, amongst others, been
    provided by Dilip Kumar, Alexander Korotkov, Robert Haas.
    
    On a large x86 system improvements for readonly pgbench, with a high
    client count, of a factor of 8 have been observed.
    
    Author: Alexander Korotkov and Andres Freund
    Discussion: 2400449.GjM57CE0Yg@dinodell
Commits on May 24, 2015
  1. @bmomjian

    pgindent run for 9.5

    bmomjian committed May 23, 2015
Commits on May 20, 2015
  1. @hlinnaka

    Collection of typo fixes.

    hlinnaka committed May 20, 2015
    Use "a" and "an" correctly, mostly in comments. Two error messages were
    also fixed (they were just elogs, so no translation work required). Two
    function comments in pg_proc.h were also fixed. Etsuro Fujita reported one
    of these, but I found a lot more with grep.
    
    Also fix a few other typos spotted while grepping for the a/an typos.
    For example, "consists out of ..." -> "consists of ...". Plus a "though"/
    "through" mixup reported by Euler Taveira.
    
    Many of these typos were in old code, which would be nice to backpatch to
    make future backpatching easier. But much of the code was new, and I didn't
    feel like crafting separate patches for each branch. So no backpatching.
Commits on Jan 29, 2015
  1. @anarazel

    Align buffer descriptors to cache line boundaries.

    anarazel committed Jan 29, 2015
    Benchmarks has shown that aligning the buffer descriptor array to
    cache lines is important for scalability; especially on bigger,
    multi-socket, machines.
    
    Currently the array sometimes already happens to be aligned by
    happenstance, depending how large previous shared memory allocations
    were. That can lead to wildly varying performance results after minor
    configuration changes.
    
    In addition to aligning the start of descriptor array, also force the
    size of individual descriptors to be of a common cache line size (64
    bytes). That happens to already be the case on 64bit platforms, but
    this way we can change the struct BufferDesc more easily.
    
    As the alignment primarily matters in highly concurrent workloads
    which probably all are 64bit these days, and the space wastage of
    element alignment would be a bit more noticeable on 32bit systems, we
    don't force the stride to be cacheline sized on 32bit platforms for
    now. If somebody does actual performance testing, we can reevaluate
    that decision by changing the definition of BUFFERDESC_PADDED_SIZE.
    
    Discussion: 20140202151319.GD32123@awork2.anarazel.de
    
    Per discussion with Bruce Momjan, Tom Lane, Robert Haas, and Peter
    Geoghegan.
Commits on Aug 30, 2014
  1. @anarazel

    Make backend local tracking of buffer pins memory efficient.

    anarazel committed Aug 30, 2014
    Since the dawn of time (aka Postgres95) multiple pins of the same
    buffer by one backend have been optimized not to modify the shared
    refcount more than once. This optimization has always used a NBuffer
    sized array in each backend keeping track of a backend's pins.
    
    That array (PrivateRefCount) was one of the biggest per-backend memory
    allocations, depending on the shared_buffers setting. Besides the
    waste of memory it also has proven to be a performance bottleneck when
    assertions are enabled as we make sure that there's no remaining pins
    left at the end of transactions. Also, on servers with lots of memory
    and a correspondingly high shared_buffers setting the amount of random
    memory accesses can also lead to poor cpu cache efficiency.
    
    Because of these reasons a backend's buffers pins are now kept track
    of in a small statically sized array that overflows into a hash table
    when necessary. Benchmarks have shown neutral to positive performance
    results with considerably lower memory usage.
    
    Patch by me, review by Robert Haas.
    
    Discussion: 20140321182231.GA17111@alap3.anarazel.de
Commits on Aug 25, 2014
  1. @anarazel

    Fix typos in some error messages thrown by extension scripts when fed…

    anarazel committed Aug 25, 2014
    … to psql.
    
    Some of the many error messages introduced in 458857c missed 'FROM
    unpackaged'. Also e016b72 and 45ffeb7 forgot to quote extension
    version numbers.
    
    Backpatch to 9.1, just like 458857c which introduced the messages. Do
    so because the error messages thrown when the wrong command is copy &
    pasted aren't easy to understand.
Commits on Aug 22, 2014
  1. @anarazel
Commits on Aug 21, 2014
  1. @anarazel

    Add pinning_backends column to the pg_buffercache extension.

    anarazel committed Aug 22, 2014
    The new column shows how many backends have a buffer pinned. That can
    be useful during development or to diagnose production issues
    e.g. caused by vacuum waiting for cleanup locks.
    
    To handle upgrades transparently - the extension might be used in
    views - deal with callers expecting the old number of columns.
    
    Reviewed by Fujii Masao and Rajeev rastogi.
Commits on Jul 14, 2014
  1. @nmisch

    Add file version information to most installed Windows binaries.

    nmisch committed Jul 14, 2014
    Prominent binaries already had this metadata.  A handful of minor
    binaries, such as pg_regress.exe, still lack it; efforts to eliminate
    such exceptions are welcome.
    
    Michael Paquier, reviewed by MauMau.
Commits on Jul 10, 2014
  1. @bmomjian

    Adjust blank lines around PG_MODULE_MAGIC defines, for consistency

    bmomjian committed Jul 10, 2014
    Report by Robert Haas
Commits on Apr 18, 2014
  1. @petere

    Create function prototype as part of PG_FUNCTION_INFO_V1 macro

    petere committed Apr 18, 2014
    Because of gcc -Wmissing-prototypes, all functions in dynamically
    loadable modules must have a separate prototype declaration.  This is
    meant to detect global functions that are not declared in header files,
    but in cases where the function is called via dfmgr, this is redundant.
    Besides filling up space with boilerplate, this is a frequent source of
    compiler warnings in extension modules.
    
    We can fix that by creating the function prototype as part of the
    PG_FUNCTION_INFO_V1 macro, which such modules have to use anyway.  That
    makes the code of modules cleaner, because there is one less place where
    the entry points have to be listed, and creates an additional check that
    functions have the right prototype.
    
    Remove now redundant prototypes from contrib and other modules.
Commits on Jan 27, 2014
  1. Relax the requirement that all lwlocks be stored in a single array.

    Robert Haas committed Jan 27, 2014
    This makes it possible to store lwlocks as part of some other data
    structure in the main shared memory segment, or in a dynamic shared
    memory segment.  There is still a main LWLock array and this patch does
    not move anything out of it, but it provides necessary infrastructure
    for doing that in the future.
    
    This change is likely to increase the size of LWLockPadded on some
    platforms, especially 32-bit platforms where it was previously only
    16 bytes.
    
    Patch by me.  Review by Andres Freund and KaiGai Kohei.
Commits on Aug 30, 2012
  1. @alvherre

    Split tuple struct defs from htup.h to htup_details.h

    alvherre committed Aug 30, 2012
    This reduces unnecessary exposure of other headers through htup.h, which
    is very widely included by many files.
    
    I have chosen to move the function prototypes to the new file as well,
    because that means htup.h no longer needs to include tupdesc.h.  In
    itself this doesn't have much effect in indirect inclusion of tupdesc.h
    throughout the tree, because it's also required by execnodes.h; but it's
    something to explore in the future, and it seemed best to do the htup.h
    change now while I'm busy with it.
Commits on Oct 12, 2011
  1. @tglsfdc

    Throw a useful error message if an extension script file is fed to psql.

    tglsfdc committed Oct 12, 2011
    We have seen one too many reports of people trying to use 9.1 extension
    files in the old-fashioned way of sourcing them in psql.  Not only does
    that usually not work (due to failure to substitute for MODULE_PATHNAME
    and/or @extschema@), but if it did work they'd get a collection of loose
    objects not an extension.  To prevent this, insert an \echo ... \quit
    line that prints a suitable error message into each extension script file,
    and teach commands/extension.c to ignore lines starting with \echo.
    That should not only prevent any adverse consequences of loading a script
    file the wrong way, but make it crystal clear to users that they need to
    do it differently now.
    
    Tom Lane, following an idea of Andrew Dunstan's.  Back-patch into 9.1
    ... there is not going to be much value in this if we wait till 9.2.
Commits on Sep 1, 2011
  1. @bmomjian
Commits on Feb 14, 2011
  1. @tglsfdc

    Avoid use of CREATE OR REPLACE FUNCTION in extension installation files.

    tglsfdc committed Feb 13, 2011
    It was never terribly consistent to use OR REPLACE (because of the lack of
    comparable functionality for data types, operators, etc), and
    experimentation shows that it's now positively pernicious in the extension
    world.  We really want a failure to occur if there are any conflicts, else
    it's unclear what the extension-ownership state of the conflicted object
    ought to be.  Most of the time, CREATE EXTENSION will fail anyway because
    of conflicts on other object types, but an extension defining only
    functions can succeed, with bad results.
  2. @tglsfdc

    Convert contrib modules to use the extension facility.

    tglsfdc committed Feb 13, 2011
    This isn't fully tested as yet, in particular I'm not sure that the
    "foo--unpackaged--1.0.sql" scripts are OK.  But it's time to get some
    buildfarm cycles on it.
    
    sepgsql is not converted to an extension, mainly because it seems to
    require a very nonstandard installation process.
    
    Dimitri Fontaine and Tom Lane
Commits on Nov 23, 2010
  1. @petere
Commits on Sep 22, 2010
  1. @mhagander
Commits on Sep 20, 2010
  1. @mhagander
Commits on Jun 11, 2009
  1. @bmomjian
Commits on Aug 14, 2008
  1. @hlinnaka
Commits on Nov 15, 2007
  1. @bmomjian

    pgindent run for 8.3.

    bmomjian committed Nov 15, 2007
Commits on Nov 13, 2007
  1. @bmomjian
Commits on Nov 11, 2007
  1. @bmomjian

    Make /contrib install/uninstall script consistent:

    bmomjian committed Nov 11, 2007
    	remove transactions
    	use create or replace function
    	make formatting consistent
    	set search patch on first line
    
    Add documentation on modifying *.sql to set the search patch, and
    mention that major upgrades should still run the installation scripts.
    
    Some of these issues were spotted by Tom today.
Commits on Nov 10, 2007
  1. @bmomjian
  2. @bmomjian

    Move most /contrib README files into SGML. Some still need conversion

    bmomjian committed Nov 10, 2007
    or will never be converted.
Commits on Jul 16, 2007
  1. @tglsfdc

    Fix pg_buffercache to release buffer partition locks in reverse order,

    tglsfdc committed Jul 16, 2007
    and add a note about why.  This is not tremendously important right now,
    probably, but it will get more urgent if NUM_BUFFER_PARTITIONS is increased
    as much as proposed.
Commits on Jun 26, 2007
  1. @tglsfdc

    Fix PGXS conventions so that extensions can be built against Postgres

    tglsfdc committed Jun 26, 2007
    installations whose pg_config program does not appear first in the PATH.
    Per gripe from Eddie Stanley and subsequent discussions with Fabien Coelho
    and others.
Commits on Apr 7, 2007
  1. @bmomjian

    Add the usage count statistics to the information available in

    bmomjian committed Apr 7, 2007
    contrib/pgbuffercache.
    
    Greg Smith
Commits on Oct 22, 2006
  1. @tglsfdc
Commits on Oct 19, 2006
  1. @tglsfdc

    Clean up local redeclarations of variables with DLLIMPORT, per report

    tglsfdc committed Oct 19, 2006
    from Magnus that MSVC complains about this.
Commits on Oct 4, 2006
  1. @bmomjian

    pgindent run for 8.2.

    bmomjian committed Oct 4, 2006
Commits on Jul 23, 2006
  1. @tglsfdc

    Split the buffer mapping table into multiple separately lockable

    tglsfdc committed Jul 23, 2006
    partitions, as per discussion.  Passes functionality checks, but
    I don't have any performance data yet.
Commits on May 30, 2006
  1. @tglsfdc

    Magic blocks don't do us any good unless we use 'em ... so install one

    tglsfdc committed May 30, 2006
    in every shared library.
Something went wrong with that request. Please try again.