tool.html

<!DOCTYPE html>
<html>
    <head>
        <title>
            Tools manifesto
        </title>
        <link rel="stylesheet"
              href="Style.css" />
        <link rel="shortcut icon"
              href="favicon.png"
              type="image/png" />
    </head>
    <body>
        <h1 id="top">Tools manifesto</h1>
        <p>
            version 2019-08-19 by <a href="/">Artem Pronichkin</a><br/>
            submit a
            <a href="https://github.com/pronichkin/pronichkin.github.io/issues">
            comment</a> (&ldquo;issue&rdquo;) or
            <a href="https://github.com/pronichkin/pronichkin.github.io/blob/master/tool.html">
            edit</a>
        </p>

        <h2>tl;dr</h2>
        <p>
            This is about creating great tools (including scripts.) If you 
            develop, maintain, own a tool or utility, or repeatedly ask others
            to use some tool, or build requirements for tools used in your processes,
            please consider the below criteria.
        </p>
        <p>
            These requirements are based on years of experience working with most
            security-cautions customers, such as banks or government agencies across
            the world.
        </p>
        <p>
            In a very short way, all of the below can be reduced to: <i>do not
            require or assume more than needed, and document everything which is
            truly required.</i>
        </p>

        <center>
            <img src="imadethese.jpg" alt="I made these (meme)" /><br />
            <small>
                <a href="http://theawkwardyeti.com/comic/gall-bladders-day/">credit: the Awkward Yeti</a>
            </small>
        </center>

        <h2 id="Contents">Contents</h2>
        <p>
            <ul>
                <li>
                    <a href="#problem">The problem with today's tools</a>
                </li>
                <li>
                    <a href="#scope">Scope of the problem</a>
                </li>
                <li>
                    <a href="#1. Whitelisting">1.&nbsp;Compliance with Application Whitelisting solutions</a>
                </li>
                <li>
                    <a href="#2. Core">2.&nbsp;Server Core support</a>
                </li>
                <li>
                    <a href="#3. Credentials">3.&nbsp;Current user credentials</a>
                </li>
                <li>
                    <a href="#4. Internet">4.&nbsp;Internet connectivity</a>
                </li>
                <li>
                    <a href="#5. Dependencies external">5.&nbsp;External dependencies</a>
                </li>
                <li>
                    <a href="#6. Dependencies inbox">6.&nbsp;Inbox dependencies</a>
                </li>
                <li>
                    <a href="#7. Remote">7.&nbsp;Remote operations</a>
                </li>
                <li>
                    <a href="#8. Permissions">8.&nbsp;Permissions and privileges (local Administrator)</a>
                </li>
                <li>
                    <a href="#9. Distribution">9.&nbsp;Distribution mechanisms</a>
                </li>
            </ul>
        </p>

        <h2 id="problem">The problem with today's tools</h2>
        <p>
            When supporting customers or working with commercial support organizations
            on behalf of customers, it's quite common to be asked for running a
            certain tool to troubleshoot or gather logs. It sounds very easy for
            support people, and hence they tend to
            &ldquo;overrequest&rdquo;&mdash;that is, ask for using more tools than
            needed at certain point.
        </p>
        <p>
            Unfortunately, not all tools are created equal, and some of them might be
            difficult to run in certain environment&mdash;especially when it's fairly
            locked down. While certainly possible, request
            to run such tools might delay troubleshooting significantly, because it
            would require to change production system configuration (e.g. include
            specific binary into whitelisting policy.) Hence, we should avoid such
            tools (that is, the tools which fail to comply with the requirements
            provided below)&mdash;unless absolutely necessary and reasonable to proceed.
        </p>
        <p>
            I'm not saying that each and every customer would impose the constraints
            listed below. In fact, most of the customers would not, or maybe they
            will have only one or two of these requirements. However, the list below
            is meant to be comprehensive and failsafe&mdash;that is, if you succeed
            to comply with all of these requirements, you can be confident your tool
            won't face any obstacles and can be recommended for broad use.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="scope">Scope of the problem</h2>
        <p>
            These requirements are not specific to troubleshooting
            or diagnostic tools. I mention those types of tools first because I have
            personally been bitten by this before many times. When you have a
            time-sensitive problem and call for support, the last thing you want is
            to be asked for running a tool which fails to execute. So that instead
            of troubleshooting the actual problem, you start troubleshooting the
            tool. And at the end that might require changes to production environment
            which are often not easy (either technically or operationally), or might
            impact the repro.
        </p>
        <p>
            However, the below requirements are
            in fact universal and can (or even should) be applied to any tools or
            utilities that
            we, as an industry, expect customers to run&mdash;such as monitoring,
            hardware configuration,
            audit assessment, system deployment and configuration, etc.
        </p>
        <p>
            Let me put this straight. If your tool fails to comply with the
            requirements listed below, running it might both be <b>painful</b> and
            take long for certain customers. They might spend hours or even days
            figuring out how to make the tool work. Even if it&apos;s not
            precisely your fault (the tool works as designed and as needed), you
            probably <i>could</i> make life easier for all parties if you were at
            least <i>aware</i> of constrains of some customers&apos; environments.
        </p>
        <p>
            That&apos;s the whole point of this writeup. It&apos;s really not to
            put a blame on anyone. (Certainly not on <i>you.)</i> It&apos;s to rise
            awareness.
        </p>
        <p>
            Last but not least, I am a Windows guy and I work for Microsoft. Hence
            the below explanations are sometimes Windows-centric and mention several
            Windows-specific technologies as examples. However, the nature of various
            constraints
            imposed by the customers are really cross-platform, regardless of the
            operating system or tools being used. Please consider the below
            requirements even if your tools are intended to run on different operating
            system&mdash;though you might need to substitute some technology names
            with their equivalents or counterparts.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>
        <hr />

        <h2 id="1. Whitelisting">1.&nbsp;Compliance with Application Whitelisting solutions</h2>
        <p>
            Ideally, the tool should come inbox with the operating system (or other
            product that it's intended to work with&mdash;e.g. database software.)
            Even if
            you plan to update it frequently, having the initial version inbox at
            first place (merely as a placeholder) would help
            including it into whitelisting policy. In fact, anything coming
            inbox will likely be whitelisted by default. And due to the way many
            whitelisting solutions work, that would automatically cover future
            versions.
        </p>
        <p>
            If you cannot ship your tool inbox, that's of course understood and
            is totally fine. However, in this case you need at least to comply
            with the following publishing requirements.
        </p>

        <p>
        <ol type="a">
            <li id="1a">
                All the files that comprise the tool itself should be
                digitally signed. (Preferably with Microsoft or other
                recognizable vendor certificate.)
            </li>
            <li id="1b">
                Ideally, the signing certificate should be the same for 
                all versions of your tool. If there's a necessity to change
                the certificate (e.g. it expired or was compromised), please
                <b>announce</b> the signature change in advance to your users.
            </li>
            <li id="1c">
                Note that even WSH or PowerShell scripts can (and should)
                be signed. Technically, a signature looks like a block
                of text at the end of the script. Hence, the script
                can still be distributed as a text file or embedded into
                other, more complex structures (e.g. XML)&mdash;without 
                invalidating the signature.
            </li>
            <li id="1d">
                For the above reason, the tool should not generate
                or modify scripts on the fly. If you need to provide
                variables dynamically, you can supply them as script
                parameters. (That would also not invalidate the signature
                of the script itself.)
            </li>
            <li id="1e">
                If the tool comes in the form of binary executable (.exe,
                .dll, etc.) the PE &ldquo;<a href="https://docs.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource">Version
                Info</a>&rdquo; should be present and populated with
                meaningful values for fields like &rdquo;Original File Name&rdquo;.
                These values should not be overly generic&mdash;i.e. the tool
                should be distinguishable from others.
                <blockquote>
                    You can casually examine the values of such fields in &ldquo;File
                    Properties&rdquo; box in Windows Explorer, or by exploring
                    &rdquo;Version Info&rdquo; property in PowerShell.
                </blockquote>
            </li>
            <li id="1f">
                There should be a clear way to differentiate between
                tool versions. Preferably, the version should be specified
                in respective fields of PE resource explained above, or in a custom
                script heading.
            </li>
            <li id="1g">
                The version should <b>not</b> be included into &ldquo;Original File
                Name&rdquo; field (e.g. &ldquo;MyTool v.1.2.3.4&rdquo;).
                Instead, the version should be specified in the designated
                fields (e.g. &ldquo;File Version&rdquo;). This is because
                every time value of &ldquo;Original File Name&rdquo; field is
                changed, it is typically distinguished by whitelisting
                mechanisms as a <b>new</b> binary. Hence such file needs to
                be added to the policy
                separately&mdash;which requires rebuilding and reapplying
                the policy. And this often means a reboot.
            </li>
            <li id="1h">
                The above requirements also apply to whatever packaging
                mechanism is used for delivery. I.e. if the tool comes with
                a self-extracting installer, that installer should be signed.
                If it comes in an archive, you should use cab format or other
                that supports embedded digital signature. (Zip format is known
                to <b>not</b> support signatures, so it should be avoided.)
                <blockquote>
                    In the other hand, cab format is known to not support
                    <i>folders.</i> So, if your tool relies on a complex directory
                    structure, please
                    evaluate other options&mdash;such as MSI or MSIX.
                </blockquote>
            </li>
        </ol>
        </p>

        <p>
            All of the above requirements are imposed by whitelisting solutions
            such as AppLocker or Windows Defender Application Control (WDAC), also
            known as &ldquo;Device Guard User Mode Code Integirty&rdquo; (UMCI).
        </p>
        <p>
            If you are not familiar with WDAC, you can learn about it
            <a href="https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-defender-application-control/windows-defender-application-control">
            here</a>. However, you don't have to become a WDAC expert. If you
            comply with the requirements listed above, you can rest assured
            that your tool does its best to comply with WDAC.<br />
        </p>

        <blockquote>
            If you happen to ship a kernel mode driver (which is very uncommon
            for troubleshooting tools) please make sure you comply with
            <a href="https://techcommunity.microsoft.com/t5/Windows-Hardware-Certification/Driver-compatibility-with-Device-Guard-in-Windows-10/ba-p/364865">
            HVCI requirements</a> as well.
        </blockquote>

        <p>
            The above requirements can be skipped only if the tool is a very
            simple PowerShell script which can be executed in
            <a href="https://devblogs.microsoft.com/powershell/powershell-constrained-language-mode/">
            Constrained Language Mode</a>. If your script is fully functional
            under constrained
            language mode, the above requirements are &ldquo;nice to
            have&rdquo;&mdash;but not strictly necessary.<br />
        </p>
        <p>
            If you are not sure whether your script supports Constrained
            Language Mode, please do not assume it does.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="2. Core">2.&nbsp;Server Core support</h2>
        <p>
            The tool has to support
            Server Core installation option. This typically
            means no GUI assumptions or dependencies. Or, more precisely, no
            explorer.exe (Windows shell), no WPF, no hardware graphic acceleration,
            etc.
        </p>

        <blockquote>
            You can learn more about Server Core <a href="https://docs.microsoft.com/windows-server/administration/server-core/what-is-server-core">
            here</a>. Please note that it's not a separate edition or &ldquo;SKU&rdquo;
            of Windows. It is an installation option available in <b>all</b> editions
            of Windows Server (Standard and Datacenter), and it is <i>recommended by
            default</i> for most use cases.<br />
            Unfortunately, as of today, there's not an easy way to check
            whether your tool is compatible with Server Core (other than
            <i>testing</i> it.)
            But we're working on it and appreciate your diligence.
        </blockquote>

        <p>
            The above does not mean your tool has to be command line <b>only.</b>
            You are very welcome to provide a GUI&mdash;as long as it's not the
            <b>only</b> mode of operation.
            However, all functionality should be <i>also</i> available in command line.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="3. Credentials">3.&nbsp;Current user credentials</h2>
        <p>
            It is very beneficial if the tool supports connecting to remote systems.
            In fact, that's one of the recommendations <a href="#7a. Remote">provided
            below</a>. However, you should
            not assume that fresh credentials are <b>always required</b> for that
            (or for any other operations.)
        </p>
        <p>
            There may be scenarios where current user cannot supply credentials on
            demand, especially if you prompt for user name and password and do not
            support smart cards. Some users are &ldquo;smartcard-only&rdquo; which
            means they have no password whatsoever. (Well, technically, they still
            do have a password&mdash;however, they do not know it.) Another somewhat
            similar scenario is Remote Credential Guard, where the user is not
            <b>supposed</b> to enter any credentials interactively, even though they
            technically can.
        </p>
        <p>
            For these reasons, the tool should always try using current user credentials
            first. (That is typically Kerberos ticket obtained from current logon
            session&mdash;however, normally you do not need to code anything special
            for this.) Only prompt for credentials optionally, or if everything else
            fails.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="4. Internet">4.&nbsp;Internet connectivity</h2>
        <p>
            Some customers operate fully air-gapped (that is, isolated) networks.
            For this reason, the tool should not assume it always can download
            components from the Internet, or upload diagnostic data directly to
            the vendor. This behavior can be optional, and of course many customers
            would appreciate that. However, there should be also an option to supply
            all the required components (e.g. baselines, metadata, schemas, updates,
            etc.) offline, and to emit results as a set of files which can be
            transferred to the vendor manually, if needed.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="5. Dependencies external">5.&nbsp;External dependencies</h2>
        <p>
            Generally, the number of dependencies should be as small as possible.
            One approach to achieve that is making the tool modular. E.g. it's
            certainly helpful
            if you support gathering data using several external tools (e.g.
            Sysinternals.) But it <i>also</i> helps if these operations are optional
            and can
            be omitted. Of course, running in reduced functionality will gather
            fewer data&mdash;but it might be still enough for some cases, or at least
            at the beginning. So, please do not assume you can only do &ldquo;all
            or nothing&rdquo;.
        </p>
        <p>
            The reason for this requirement is also application whitelisting. If your
            tool is a simple script (which, let's say, grabs some logs), it might not
            need being included into the
            whitelisting policy at all. And hence it would not require policy changes.
            However, if any dependency is a binary, it will certainly require a policy
            modification. Any such occasion imposes additional operational overhead
            and might slowdown or complicate troubleshooting efforts (which may be
            time-sensitive.)
        </p>
        <p>
            Even if your tool needs to be whitelisted by itself, adding fewer external
            items to the policy is often faster and easier to justify than adding a large
            number of dependencies with opaque scope.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="6. Dependencies inbox">6.&nbsp;Inbox dependencies</h2>
        <p>
            Any dependency should not be assumed as granted. It needs to be explicitly
            documented (in tool's readme, accompanying email template, etc.) This
            even applies to dependencies on any inbox components, such as PowerShell
            modules or 32-bit subsystem (WoW64.) Even though these components are
            included <i>by default,</i> if they can be uninstalled&mdash;there will
            be customers who remove them for one reason or another.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="7. Remote">7.&nbsp;Remote operations</h2>
        <p>
            The tool needs to support at least one of the following. (Bonus points
            if you manage to satisfy both requirements.)
        </p>
        <p>
            <ol type="a">
                <li id="7a. Remote">
                    Remote execution. That is, running on machine <i>A</i> while
                    analyzing or configuring machines <i>B</i> and <i>C</i>. You
                    typically use protocols such as WinRM (WS-MAN), WMI (CIM), WCF
                    (Indigo), RPC/DCOM or others to connect to a remote system.<br />
                    Whatever
                    protocol you use, it should be explicitly documented. If any
                    custom (non-standard) protocol is used, you absolutely need
                    to document its network requirements in detail (direction,
                    transport and ports being used, as well as authentication
                    mechanisms supported.)
                </li>
                <li id="7b. PS Remoting">
                    Support running in PowerShell remoting sessions. That typically
                    means no interactive prompts such as &ldquo;y/n&rdquo; and no
                    pop-up modal boxes (including errors or warnings.)<br />
                    This also includes uncommon cases such
                    as not assuming to be able to talk to winlogon process or
                    other session infrastructure or user profile artifacts. For
                    instance, it is known that if a user has never logged on
                    interactively (e.g. via Remote Desktop services), the profile
                    is not fully created&mdash;even though their <i>user folders</i>
                    are accessible and usable upon logging on via PowerShell remoting.
                    <blockquote>
                        Todo: add more technical details on what's available and
                        what's not.
                    </blockquote>
                    Such dependencies should be generally avoided&mdash;unless clearly
                    necessary and unavoidable. (In which case requirements should be
                    documented).<br />
                    That said, most of existing command-line tools are known to
                    operate in PowerShell remoting sessions just fine without any
                    noticeable limitations.
                </li>
            </ol>
        </p>

        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="8. Permissions">8.&nbsp;Permissions and privileges (local Administrator)</h2>
        <p>
            It is understood if you tool requires local administrator permission
            (or equivalent) to configure something on the machine. However, please
            be mindful about your requirements. If there are some operations that
            do not actually require those permissions (e.g. just analyzing the
            current configuration) you better allow your tool to run as standard user.
            In
            either case, please document the requirements explicitly&mdash;especially
            if you require some special privileges (such as &ldquo;Debug Programs&rdquo;
            or &ldquo;Generate Secuity Audits&rdquo;.) Do not assume every user who
            runs your tool can elevate their privileges on the fly.
        </p>
        <p>
            One specific example is when your tool runs on a machine to configure
            remote systems (such as explained in <a href="#7a. Remote">7a</a> above.)
            In this case it's
            almost certain that your tool <i>does not need</i> Administrator
            permissions on the &ldquo;management&rdquo; machine. And if some of your
            customers adopted certain Credential Theft Mitigation (CTM)
            best practices (such as &ldquo;<a href="https://docs.microsoft.com/en-us/windows-server/identity/securing-privileged-access/privileged-access-workstations">Privileged
            Access Workstations</a>&rdquo; or PAWs) this means that they
            definitely won't have
            Administrator permission on their <i>local</i> machines (even though they
            <b>are</b> administrators on the <i>remote</i> machines being managed.)
        </p>
        <p>
            It is fine if your tool requires Administrative permissions for 
            installation and updating. Just do not assume the same permissions 
            are available all the time even for running the tool, and do not 
            require process elevation unless actually needed.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>

        <h2 id="9. Distribution">9.&nbsp;Distribution mechanisms</h2>
        <p>
            If possible, offer automated distribution mechanism for your tool
            which is native for the operating system (e.g. Microsoft Store) 
            and/or the runtime environment (e.g. 
            <a href="https://www.powershellgallery.com/">PowerShell Gallery</a>.)
        </p>
        <p>
            Depending on the nature and target environments of your tool, this may
            or may not be not the <i>only</i> way to obtain it. E.g. Microsoft
            Store is not available on Windows Server, and it also cannot be easily
            used by the customers who run in air-gapped networks. However, it is the
            perfect solution for many users running Windows 10. So, it's perfectly
            fine if you provide multiple alternative installation vehicles. Having
            more options is generally better than fewer.
        </p>
        <p>
            Besides obvious benefits such as automatic updating offered by the Store,
            packaging for it (or PowerShell Gallery) implicitly makes you comply with
            digital signature requirements listed above.
        </p>
        <p>
            <a href="#top">back to top</a>
        </p>
    </body>
</html>