Permalink
Commits on Sep 22, 2016
  1. Preparing for 2.0.4

    Klacke Wikstrom committed Sep 22, 2016
Commits on Sep 20, 2016
  1. Add support for salted hashes in users credentials

    Now, it is possible to use a random salt to hash passwords. The salt must be
    provided, encoded in base64.
    In yaws configuration is must follow the format:
    
      user = "User:{Algo}$Salt$Hash"
    
    And in .yaws_file:
    
      {"User", "Algo", "Salt", "Hash"}.
    
    Using clear passwords or hashes without salt is still possible. Yaws startup
    script always uses salted hashes
    capflam committed Sep 13, 2016
  2. Remove support for Erlang/OTP R14 and R15 releases

    Now, R16B01 or higher is required.
    capflam committed Sep 13, 2016
  3. Add support of hashed passwords in yaws configuration and in .yaws_au…

    …th files
    
    Now, it is possible to used hashed password to define user's credential. In the
    yaws configuration, you can add following lines is <auth> sections:
    
      user = "User:{Algo}Base64Hash"
    
    and in .yaws_auth files:
    
      {"User", "Algo", "Base64Hash"}.
    
    Algo is one of: md5 | ripemd160 | sha | sha224 | sha256 | sha384 | sha512
    
    and Base64Hash is the result of the hash functions, encoded in base64. For
    example:
    
      "/N4rLtula/QIYB+3If6bXDONEO5CnqBPrlURto+/j7k=" for "bar" password.
    
    yaws startup script has been updated to add a way to generate user's credential:
    
      $> yaws --auth --algo sha256 USER
      baz
    
      User's credential successfully generated:
          Put this line in your Yaws config (in <auth> section): user = "USER:{sha256}uqWglk0zIPvAxqkiFARTyFE+okq4/QV3A0gEqWckgJY="
    
          Or in a .yaws_auth file: {"USER", "sha256", "uqWglk0zIPvAxqkiFARTyFE+okq4/QV3A0gEqWckgJY="}.
    capflam committed Sep 13, 2016
  4. Remove sensitive information from crash messages in yaws_outmod

    This is probably a bad idea to keep yaws_outmd to print crash messages in
    production. This module was developped to ease debugging. But, to avoid
    problems, as far as possible, authentication information are removed, if any.
    capflam committed Sep 12, 2016
  5. Store the hash of passwords in #auth{} records instead of the clear v…

    …ersion
    
    So, if passwords leak (in logs, in crash messages or whatever), it will be
    significantly harder for hacker to use it. Passwords are hashed using SHA256.
    capflam committed Sep 12, 2016
  6. Revert "Move username/password tuples from server state to an ETS table"

    This reverts commit 37f44d0.
    
    There are many flaws with this patch. First, the ETS table used to store
    user/password tuples is always created during the configuration parsing (or
    during the loading in embedded mode). So during a configuration reload, the old
    server configurations and the new one never match because the table id in
    used, when Yaws configuration is reloaded, the servers are restarted, even when
    there is no changes.
    
    Then, the ETS table is owned by the process that parses the configuration (or
    that loads it in embedded mode). So the table is deleted when this process
    dies. This happens when the configuration is reloaded using the startup
    script. This is a huge bug, because after a reload, all requests requiring a
    user/password authentication will crash because the table is gone.
    
    And finally, ETS table owned by old server configurations is never garbaged.
    capflam committed Sep 12, 2016
  7. Fix a leak when a server conf is updated because of a forgotten ets t…

    …able ref
    
    Each server owns an ets table to store some information about files and scripts
    and to cache small static files. When the server configuration is updated, the
    old #sconf record is replaced by a new one and a new ets table is created. But
    the old one was never deleted. So we could have a proliferation of forgotten ets
    tables leading to a memoring leak after successive updates.
    
    So now, instead of creating a new ets table when a server configuration is
    updated, we keep the old one. This avoids useless creation of ets tables and
    fixes the memory leak.
    capflam committed Sep 12, 2016
  8. Support strong/basic_validation options in yaws_compile

    Because these options don't generate binary code, the corresponding '<erl>'
    block is ignored. So, in the result of yaws_compile:compile/2, following spec
    
        {mod, Line, Script, NumChars,  Mod, Fun}
    
    will be replaced by
    
        {skip, NumChars}
    capflam committed Sep 19, 2016
  9. Remove temporary files generated during scripts parsing when job is done

    When .yaws scripts are parsed, each "<erl>" block produces a temporary erlang
    module (i.e a .erl file). These modules are compiled and used during page
    rendering.
    
    Instead of keeping these files til the next parsing, we remove them just after
    the compilation.
    capflam committed Sep 16, 2016
  10. Support user-defined options to compile .yaws scripts

    The function yaws_compile:compile_file/2 has been exported. It can be used to
    set some additional compiler options. By default, following options are used:
    
      * [binary,return_errors] during normal scripts compilation, when yaws is
        running
    
      * [binary,return_errors,return_warnings,debug_info] during a check
        (yaws --check)
    
    So the additional options must be compatible with these ones. For example, 'P',
    'S' or 'E' options must not be used. There is no check to validate provided
    options.
    capflam committed Sep 16, 2016
  11. Fix small typo in testsuites

    capflam committed Sep 15, 2016
  12. Rewrite yaws_compile module to improve .yaws script parsing

    yaws_compile modules has been rewritten from scratch to be more flexible. Now,
    it can handle following syntaxes:
    
        <erl>out(_A) -> {html, "test1"}. </erl>
    
        <b><erl>out(_A) -> {html, "test2"}. </erl></b>
    
        <b><erl>
          out(_A) -> {html, "test3"}.
        </erl></b>
    
    In addition, it support strange tag syntaxes with spaces and newlines, like:
    
        <erl
           module =
             "my_module"
         >
           out(_A) -> {html, "test"}.
        </erl
              >
    
    The sames syntaxes are supported for verbatim blocks of course.
    
    Unknown attributes in erl tags (all except "module") are ignored instead of
    crashing the Yaws process. When a module name is provided for an erl block, we
    check it to be sure that it will not override an existing module (owned by
    another yaws script or a "real" module, like yaws !).
    
    About verbatim blocks, it is now possible to add HTML attributes that will be
    keep after substitution. For example:
    
         <verbatim class="code" id="listing-1">
            ...
         </verbatim>
    
    will be replaced by:
    
         <pre class="code" id="listing-1">
            ...
         </pre>
    
    Finally, about performance, the new parser tends to be faster in many cases when
    one file is compiled. But it is really faster when many scripts are compiled in
    parallel. The old parser used a global lock on the node. So when a script was
    parsed, all others was blocked. Now, a lock is set per script. So two different
    script can be parsed in same time.
    capflam committed Sep 14, 2016
Commits on Sep 14, 2016
  1. Better support for erts version >= 10

    Nico committed with capflam Sep 13, 2016
Commits on Sep 13, 2016
  1. Rewrite opening of report.log so that previous file is not overwritten

    when Yaws starts.
    This is particularly a pain when Yaws permanently restarts and that
    the report.log file contains the reason of the crash.
    
    The use of error_logger_file_h:init(File) was opening the file as [write]
    This patch reverts it to [append], while trying to handle both types of
    results (pre R18 and post R18)
    Nico committed Sep 13, 2016
Commits on Sep 12, 2016
  1. Do not build yaws anymore for R14 releases on travis

    There are some incompatibilities with common_test and inets. So, for now builds
    on R14B03 and R14B04 are disabled.
    capflam committed with capflam Sep 12, 2016
Commits on Sep 8, 2016
  1. Fix the configuration reload when SNI is enabled

    This fixes #286.
    capflam committed with capflam Sep 8, 2016
Commits on Sep 2, 2016
  1. Upload build artifacts producted by Travis on external repo

    Because Travis logs are not suffisant to find bugs when the testsuites execution
    fail, we upload build artifcats on an extern repository.
    
    For now, this is done on github, in the repository
    capflam/yaws-travis-build-artifacts. There is no versionning, we just create new
    release for each travis build and add assets in its release.
    capflam committed Sep 1, 2016
  2. Refactor the testsuite to use common_test instead of a in-house frame…

    …work
    
    The testsuite has been rewritten to use Erlang common_test framework.
    Internally, eunit is used to check results. The TS can be launched with the
    'make check' command. It is only available with the autotools.
    
    testsuite/README.md describes how the testsuite works, how to launch specific
    suites or testcases and how to write new tests.
    
    ibrowse dependency has been removed. So it is possible to run the TS without
    connectivity. We use a custom HTTP client to do checks. So it is easy to adapt
    it when needed. The API to write new tests is in the module
    testsuite/testsuite.erl.
    capflam committed Aug 5, 2016
  3. Rework how random ports are handled

    It is possible to let Yaws choose the listening port of a server by leaving it
    to 0. Now, this port is chosen during the configuration parsing. There are 2
    reasons to do it this way:
    
      * First, the #sconf record has the real listening port instead of 0. This
        avoids the checks to get it. This is also convenient to have it when the
        configuration is retrieved or dumped.
    
      * Then, if there are 2 or more servers configured to use a random port, each
        one will have its own port. Each server will be isolated in its own group.
        So it is possible to have in same time a HTTP and a HTTPS vhost using a
        random port.
    
    There is a drawback when the configuration is reloaded. All servers will be
    restarted and will use a new listening port. But random ports are used for test
    purposes so this is not really an issue.
    
    We use IANA range for dynamic or private ports (49152 -> 65535) to choose a
    port. We try to find the first free port in this range. An environment variable
    is used to know the last assigned random port (attached to the kernel
    application, so it is possible to reserve ports manually before Yaws is
    started).
    
    This new feature is mainly used by the new testsuite.
    capflam committed with capflam Aug 12, 2016
  4. Check "Transfer-Encoding" header first, before "Content-Length" header

    Usually, when "Transfer-Encoding" header is set to "chunk", "Content-Length"
    header is not set. So it is safe to check these headers in any order. In Yaws,
    "Content-Length" header was checked first.
    But, some clients (e.g. inets) set the "Content-Length" header to 0 when the
    body is chunked. So, in this situation, the body was skipped by Yaws. And with
    persistant connections, subsequent requests was rejected.
    
    So, to be compliant with these buggy clients, we check "Transfer-Encoding"
    header first to know if the body is chunked. Then, if not, we check
    "Content-Length" header to know the body size.
    capflam committed with capflam Aug 12, 2016
  5. Use MOD:module_info/1 instead of erlang:function_exported/3 in yaws_d…

    …ynopts
    
    erlang:function_exported/3 doesn't load the requested module. So, it always
    returns false if the module was not previously loaded. So, in certain
    circumstances, yaws_dynopts returns the wrong value.
    Instead, we use MOD:module_info/1 to retrieve the same information. So, we are
    sure to load the module.
    capflam committed with capflam Aug 12, 2016
Commits on Aug 22, 2016
  1. Fix #283: FastCGI protocol error for empty HTTP body requests

    In my setup I receive the following 500 error when POSTing an empty
    request to any PHP script.
    
      CGI failure: {"recv from application server failed",closed}
    
    My investigations showed that the FCGI_STDIN <<>> frame was sent twice
    and the FastCGI server closes the connection right away, (most of the
    time) before having time to anwser.
    
    The FastCGI protocol describes the end of a stream as an empty record
    for that stream. What Yaws does is it basically closes the stream
    twice. Moreover, a stream is described as zero or more non-empty
    records, followed by an empty record.
    
    http://www.mit.edu/~yandros/doc/specs/fcgi-spec.html#S3.3
    
      "A stream record is part of a stream, i.e. a series of zero or more
      non-empty records (length != 0) of the stream type, followed by an
      empty record (length == 0) of the stream type."
    
    http://www.nongnu.org/fastcgi/#id2854157
    
      "Furthermore, it will forward any POST data it receives from the
      client to the application via the STDIN stream. Once all data have
      been forwarded, an empty STDIN packet is sent to close the stream."
    
    Though the specification is not as precise as an RFC, sending two
    empty records is not in the spirit of the spec.
    
    Add special handling for empty body requests to
    yaws_cgi:fcgi_pass_through_client_data/1.
    ofavre committed with vinoski Aug 22, 2016
Commits on Jul 25, 2016
  1. Security flaw http://httpoxy.org/ fixed

    A security flaw with HTTP_PROXY fixed. When we now construct the
    cgi env variables, we just skip the Proxy header.
    Reported by dominic@varspool.com
    Klacke Wikstrom committed Jul 25, 2016
Commits on Jul 10, 2016
  1. Fix typos in shopcart.erl

    vinoski committed Jul 10, 2016
Commits on Jul 8, 2016
  1. randome/random.

    cmeiklejohn committed with capflam Jul 8, 2016
Commits on Jul 2, 2016
  1. Fix #274: remove doc ref to is_default

    The is_default feature was removed a long long time ago but was still
    mentioned in yaws.tex, so remove it. Replace it with a reference to
    pick_first_virthost_on_nomatch.
    vinoski committed Jul 2, 2016
  2. Merge pull request #272 from klacke/etnt/sconf_gconf_set_functions

    Adding set functions for the sconf and gconf records
    committed on GitHub Jul 2, 2016
Commits on Jul 1, 2016
  1. Do not decode request path in logs

    Log files are openned in raw mode, so UTF-8 characters (>255) cannot be
    written. Their bytewise representation should be used instead. But, there is no
    good reason to decode paths. Some analyzing tools could be ascii oriented and
    this requires an extra encoding to get the original request.
    
    So now, no decoding is done on the request path in access and auth logs.
    capflam committed with capflam Jun 1, 2016
  2. Fix the UTF-8 handling for directories and files in yaws_ls

    This patch fixes the commit 8c0773f. See
    8c0773f for details.
    capflam committed with capflam Jun 1, 2016
Commits on Jun 30, 2016
  1. Preparing for rel 2.0.3

    Klacke committed Jun 30, 2016
  2. Manage modules/functions deprecated in Erlang/OTP 19.0

    'crypto:rand_bytes/1' and the 'random' module have been deprecated in Erlang/OTP
    19.0. Instead, we must use, respectively, 'crypto:strong_rand_bytes' and the
    'rand' module.
    
    Fix #277
    capflam committed with capflam Jun 30, 2016