Permalink
Commits on Nov 18, 2016
  1. Fix yaws_dynopts:compare_version/2 function

    capflam committed Nov 17, 2016
    It failed to compare versions of different length.
    
    Ref. #292
  2. Use a dynamic wrapper around ssl:connection_information/2

    capflam committed Nov 18, 2016
    This function has been introduced int Erlang/OTP 18. So for previous releases we
    need to have a wrapper function.
    
    Ref. #292
  3. Remove dynamic wrapper on rand_bytes/1 function

    capflam committed Nov 17, 2016
    Because crypto:rand_bytes/1 is deprecated since releases 19, we had added a
    wrapper function in yaws_dynopts module to use crypto:strong_rand_bytes/1 when
    possible. This was mandatory for R14 and R15 releases. But, since we have
    removed the support for these releases, the wrapper function is useless.
    
    Now, use use crypto:strong_rand_bytes/1 directly.
    
    Ref #292
  4. Fix a typo in yaws_ls

    capflam committed Nov 18, 2016
  5. Fix a typo in yaws_config

    capflam committed Nov 18, 2016
  6. Specify the port in proc_cleanup_SUITE testsuite

    capflam committed Nov 18, 2016
    Instead of using the default port (8000) in proc_cleanup_SUITE testsuite, we use
    a reserved port.
Commits on Oct 30, 2016
  1. Remove forgotten debug messages

    capflam committed Oct 30, 2016
    [ci skip]
  2. Add support of unicode characters in the configuration file

    capflam committed Oct 30, 2016
    By default, Erlang VM opens files using latin1 encoding. So it was impossible to
    use Docroot path with unicode characters. Though, there is an option to open
    file using unicode encoding.
    
    So now, it is possible to choose. With the yaws script, the option '--encoding'
    can be used to choose between latin1 and unicode. It is also possible to set the
    yaws application environment variable '{encoding, latin1|unicode}'.
    
    If you load the configuration file by hand, using yaws_config:load/1, you must
    set #env.encoding field. By default, latin1 is used.
    
    IMPORTANT: if you use Docroot path with unicode characters, you MUST use
    quotes. For example:
    
       docroot = "/tmp/ẅẅẅ"
    
    Ref: issue #290
Commits on Sep 22, 2016
  1. Preparing for 2.0.4

    Klacke Wikstrom
    Klacke Wikstrom committed Sep 22, 2016
Commits on Sep 20, 2016
  1. Add support for salted hashes in users credentials

    capflam committed Sep 13, 2016
    Now, it is possible to use a random salt to hash passwords. The salt must be
    provided, encoded in base64.
    In yaws configuration is must follow the format:
    
      user = "User:{Algo}$Salt$Hash"
    
    And in .yaws_file:
    
      {"User", "Algo", "Salt", "Hash"}.
    
    Using clear passwords or hashes without salt is still possible. Yaws startup
    script always uses salted hashes
  2. Remove support for Erlang/OTP R14 and R15 releases

    capflam committed Sep 13, 2016
    Now, R16B01 or higher is required.
  3. Add support of hashed passwords in yaws configuration and in .yaws_au…

    capflam committed Sep 13, 2016
    …th files
    
    Now, it is possible to used hashed password to define user's credential. In the
    yaws configuration, you can add following lines is <auth> sections:
    
      user = "User:{Algo}Base64Hash"
    
    and in .yaws_auth files:
    
      {"User", "Algo", "Base64Hash"}.
    
    Algo is one of: md5 | ripemd160 | sha | sha224 | sha256 | sha384 | sha512
    
    and Base64Hash is the result of the hash functions, encoded in base64. For
    example:
    
      "/N4rLtula/QIYB+3If6bXDONEO5CnqBPrlURto+/j7k=" for "bar" password.
    
    yaws startup script has been updated to add a way to generate user's credential:
    
      $> yaws --auth --algo sha256 USER
      baz
    
      User's credential successfully generated:
          Put this line in your Yaws config (in <auth> section): user = "USER:{sha256}uqWglk0zIPvAxqkiFARTyFE+okq4/QV3A0gEqWckgJY="
    
          Or in a .yaws_auth file: {"USER", "sha256", "uqWglk0zIPvAxqkiFARTyFE+okq4/QV3A0gEqWckgJY="}.
  4. Remove sensitive information from crash messages in yaws_outmod

    capflam committed Sep 12, 2016
    This is probably a bad idea to keep yaws_outmd to print crash messages in
    production. This module was developped to ease debugging. But, to avoid
    problems, as far as possible, authentication information are removed, if any.
  5. Store the hash of passwords in #auth{} records instead of the clear v…

    capflam committed Sep 12, 2016
    …ersion
    
    So, if passwords leak (in logs, in crash messages or whatever), it will be
    significantly harder for hacker to use it. Passwords are hashed using SHA256.
  6. Revert "Move username/password tuples from server state to an ETS table"

    capflam committed Sep 12, 2016
    This reverts commit 37f44d0.
    
    There are many flaws with this patch. First, the ETS table used to store
    user/password tuples is always created during the configuration parsing (or
    during the loading in embedded mode). So during a configuration reload, the old
    server configurations and the new one never match because the table id in
    used, when Yaws configuration is reloaded, the servers are restarted, even when
    there is no changes.
    
    Then, the ETS table is owned by the process that parses the configuration (or
    that loads it in embedded mode). So the table is deleted when this process
    dies. This happens when the configuration is reloaded using the startup
    script. This is a huge bug, because after a reload, all requests requiring a
    user/password authentication will crash because the table is gone.
    
    And finally, ETS table owned by old server configurations is never garbaged.
  7. Fix a leak when a server conf is updated because of a forgotten ets t…

    capflam committed Sep 12, 2016
    …able ref
    
    Each server owns an ets table to store some information about files and scripts
    and to cache small static files. When the server configuration is updated, the
    old #sconf record is replaced by a new one and a new ets table is created. But
    the old one was never deleted. So we could have a proliferation of forgotten ets
    tables leading to a memoring leak after successive updates.
    
    So now, instead of creating a new ets table when a server configuration is
    updated, we keep the old one. This avoids useless creation of ets tables and
    fixes the memory leak.
  8. Support strong/basic_validation options in yaws_compile

    capflam committed Sep 19, 2016
    Because these options don't generate binary code, the corresponding '<erl>'
    block is ignored. So, in the result of yaws_compile:compile/2, following spec
    
        {mod, Line, Script, NumChars,  Mod, Fun}
    
    will be replaced by
    
        {skip, NumChars}
  9. Remove temporary files generated during scripts parsing when job is done

    capflam committed Sep 16, 2016
    When .yaws scripts are parsed, each "<erl>" block produces a temporary erlang
    module (i.e a .erl file). These modules are compiled and used during page
    rendering.
    
    Instead of keeping these files til the next parsing, we remove them just after
    the compilation.
  10. Support user-defined options to compile .yaws scripts

    capflam committed Sep 16, 2016
    The function yaws_compile:compile_file/2 has been exported. It can be used to
    set some additional compiler options. By default, following options are used:
    
      * [binary,return_errors] during normal scripts compilation, when yaws is
        running
    
      * [binary,return_errors,return_warnings,debug_info] during a check
        (yaws --check)
    
    So the additional options must be compatible with these ones. For example, 'P',
    'S' or 'E' options must not be used. There is no check to validate provided
    options.
  11. Fix small typo in testsuites

    capflam committed Sep 15, 2016
  12. Rewrite yaws_compile module to improve .yaws script parsing

    capflam committed Sep 14, 2016
    yaws_compile modules has been rewritten from scratch to be more flexible. Now,
    it can handle following syntaxes:
    
        <erl>out(_A) -> {html, "test1"}. </erl>
    
        <b><erl>out(_A) -> {html, "test2"}. </erl></b>
    
        <b><erl>
          out(_A) -> {html, "test3"}.
        </erl></b>
    
    In addition, it support strange tag syntaxes with spaces and newlines, like:
    
        <erl
           module =
             "my_module"
         >
           out(_A) -> {html, "test"}.
        </erl
              >
    
    The sames syntaxes are supported for verbatim blocks of course.
    
    Unknown attributes in erl tags (all except "module") are ignored instead of
    crashing the Yaws process. When a module name is provided for an erl block, we
    check it to be sure that it will not override an existing module (owned by
    another yaws script or a "real" module, like yaws !).
    
    About verbatim blocks, it is now possible to add HTML attributes that will be
    keep after substitution. For example:
    
         <verbatim class="code" id="listing-1">
            ...
         </verbatim>
    
    will be replaced by:
    
         <pre class="code" id="listing-1">
            ...
         </pre>
    
    Finally, about performance, the new parser tends to be faster in many cases when
    one file is compiled. But it is really faster when many scripts are compiled in
    parallel. The old parser used a global lock on the node. So when a script was
    parsed, all others was blocked. Now, a lock is set per script. So two different
    script can be parsed in same time.
Commits on Sep 14, 2016
  1. Better support for erts version >= 10

    Nico authored and capflam committed Sep 13, 2016
Commits on Sep 13, 2016
  1. Rewrite opening of report.log so that previous file is not overwritten

    Nico
    Nico committed Sep 13, 2016
    when Yaws starts.
    This is particularly a pain when Yaws permanently restarts and that
    the report.log file contains the reason of the crash.
    
    The use of error_logger_file_h:init(File) was opening the file as [write]
    This patch reverts it to [append], while trying to handle both types of
    results (pre R18 and post R18)
Commits on Sep 12, 2016
  1. Do not build yaws anymore for R14 releases on travis

    capflam committed Sep 12, 2016
    There are some incompatibilities with common_test and inets. So, for now builds
    on R14B03 and R14B04 are disabled.
Commits on Sep 8, 2016
  1. Fix the configuration reload when SNI is enabled

    capflam committed Sep 8, 2016
    This fixes #286.
Commits on Sep 2, 2016
  1. Upload build artifacts producted by Travis on external repo

    capflam committed Sep 1, 2016
    Because Travis logs are not suffisant to find bugs when the testsuites execution
    fail, we upload build artifcats on an extern repository.
    
    For now, this is done on github, in the repository
    capflam/yaws-travis-build-artifacts. There is no versionning, we just create new
    release for each travis build and add assets in its release.
  2. Refactor the testsuite to use common_test instead of a in-house frame…

    capflam committed Aug 5, 2016
    …work
    
    The testsuite has been rewritten to use Erlang common_test framework.
    Internally, eunit is used to check results. The TS can be launched with the
    'make check' command. It is only available with the autotools.
    
    testsuite/README.md describes how the testsuite works, how to launch specific
    suites or testcases and how to write new tests.
    
    ibrowse dependency has been removed. So it is possible to run the TS without
    connectivity. We use a custom HTTP client to do checks. So it is easy to adapt
    it when needed. The API to write new tests is in the module
    testsuite/testsuite.erl.
  3. Rework how random ports are handled

    capflam committed Aug 12, 2016
    It is possible to let Yaws choose the listening port of a server by leaving it
    to 0. Now, this port is chosen during the configuration parsing. There are 2
    reasons to do it this way:
    
      * First, the #sconf record has the real listening port instead of 0. This
        avoids the checks to get it. This is also convenient to have it when the
        configuration is retrieved or dumped.
    
      * Then, if there are 2 or more servers configured to use a random port, each
        one will have its own port. Each server will be isolated in its own group.
        So it is possible to have in same time a HTTP and a HTTPS vhost using a
        random port.
    
    There is a drawback when the configuration is reloaded. All servers will be
    restarted and will use a new listening port. But random ports are used for test
    purposes so this is not really an issue.
    
    We use IANA range for dynamic or private ports (49152 -> 65535) to choose a
    port. We try to find the first free port in this range. An environment variable
    is used to know the last assigned random port (attached to the kernel
    application, so it is possible to reserve ports manually before Yaws is
    started).
    
    This new feature is mainly used by the new testsuite.
  4. Check "Transfer-Encoding" header first, before "Content-Length" header

    capflam committed Aug 12, 2016
    Usually, when "Transfer-Encoding" header is set to "chunk", "Content-Length"
    header is not set. So it is safe to check these headers in any order. In Yaws,
    "Content-Length" header was checked first.
    But, some clients (e.g. inets) set the "Content-Length" header to 0 when the
    body is chunked. So, in this situation, the body was skipped by Yaws. And with
    persistant connections, subsequent requests was rejected.
    
    So, to be compliant with these buggy clients, we check "Transfer-Encoding"
    header first to know if the body is chunked. Then, if not, we check
    "Content-Length" header to know the body size.
  5. Use MOD:module_info/1 instead of erlang:function_exported/3 in yaws_d…

    capflam committed Aug 12, 2016
    …ynopts
    
    erlang:function_exported/3 doesn't load the requested module. So, it always
    returns false if the module was not previously loaded. So, in certain
    circumstances, yaws_dynopts returns the wrong value.
    Instead, we use MOD:module_info/1 to retrieve the same information. So, we are
    sure to load the module.
Commits on Aug 22, 2016
  1. Fix #283: FastCGI protocol error for empty HTTP body requests

    ofavre authored and vinoski committed Aug 22, 2016
    In my setup I receive the following 500 error when POSTing an empty
    request to any PHP script.
    
      CGI failure: {"recv from application server failed",closed}
    
    My investigations showed that the FCGI_STDIN <<>> frame was sent twice
    and the FastCGI server closes the connection right away, (most of the
    time) before having time to anwser.
    
    The FastCGI protocol describes the end of a stream as an empty record
    for that stream. What Yaws does is it basically closes the stream
    twice. Moreover, a stream is described as zero or more non-empty
    records, followed by an empty record.
    
    http://www.mit.edu/~yandros/doc/specs/fcgi-spec.html#S3.3
    
      "A stream record is part of a stream, i.e. a series of zero or more
      non-empty records (length != 0) of the stream type, followed by an
      empty record (length == 0) of the stream type."
    
    http://www.nongnu.org/fastcgi/#id2854157
    
      "Furthermore, it will forward any POST data it receives from the
      client to the application via the STDIN stream. Once all data have
      been forwarded, an empty STDIN packet is sent to close the stream."
    
    Though the specification is not as precise as an RFC, sending two
    empty records is not in the spirit of the spec.
    
    Add special handling for empty body requests to
    yaws_cgi:fcgi_pass_through_client_data/1.