Tweak spider parser to ignore/strip matched parenthesis around URLs #2898
Comments
thc202
changed the title
thc202
added a commit
to thc202/zaproxy
that referenced
this issue
Sep 29, 2016
Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude parenthesis (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs
thc202
added a commit
to thc202/zaproxy
that referenced
this issue
Sep 30, 2016
Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs
Harinus
added a commit
to Harinus/zaproxy
that referenced
this issue
Jan 12, 2017
* Show the cause why a script was not loaded Change ExtensionScript to provide more details why a script was not successfully loaded (e.g. missing script type, invalid character sequence, other unexpected causes). * Latest files from Crowdin * Show white space chars of matches in Search panel Change SearchResultTableEntry to replace the white space characters in the string found with visible equivalent characters so that the matches in the Search panel are like: Content-Length:·453¤¶Connection:·close¤¶Content-Type: instead of: Content-Length: 453Connection: closeContent-Type: which does not reflect the actual match. * Address JavaDoc issues Address JavaDoc issues in some classes, tidy up class JavaDoc, add missing docs on methods and parameters, replace closing HTML tags with starting tags and remove empty docs of overridden methods. * Correct offset calculation in text header views Extract the calculation of offsets for view to header and header to view into a class (HttpTextViewUtils) and changed the text views to use it, reduces code duplication and uses the correct calculations in all cases (some calculations were already correct). Add tests to assert the expected behaviour of HttpTextViewUtils. Fix zaproxy#2793 - Wrong highlight in combined view with last part of request header * Fix typos in API endpoint descriptions Change descriptions of core API endpoints sendRequest and sendHarRequest from "now allowed" to "not allowed" (to send the requests in Safe mode). * Use dev version of API and support -w wiki_report * Upgrade to use ubuntu:16.04 * Exclude brackets in URLs in spider parsers Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs * Ignore incorrect start/end positions on highlight Change HttpTextViewUtils to return invalid position, instead of throwing an exception, if the start or end positions are greater than the length of the view/header/body, otherwise it would result in an exception when the combined view is showing a custom message in the body (when body is too large to display per configurations), moreover it is more resilient to future changes in the content shown. Update the tests to assert the new behaviour. * Remove "debug" code in SpiderTextParser Remove "debug" code accidentally committed in a previous change. * Tweak JavaDoc in StandardFieldsDialog Add missing parameter descriptions and add docs for the constructors and some methods. * Tweak JavaDocs in class TabbedPanel2 Fix typo, add descriptions to parameters of documented methods and other minor tweaks. * Latest files from Crowdin * Do not use null string literal in ApiResponseSet Change ApiResponseSet to not use null string literal, in XML and HTML formats, when the values are null otherwise it might seem that the null values of the set have a value (i.e. "null"), instead of nothing. For example, an alert with no evidence/attack (null) would shown "null" instead of nothing. * Differentiate the source of alerts Change the Alert class to have a source "unknown", "active", "manual", "passive" and "tool" which is set just before the alert is raised for active, manual and passive alerts, for remaining alerts it's considered as raised by a tool (e.g. custom scripts). Old alerts of existing sessions will default to "unknown" since its exact source is not immediately known. Change Alert tab to shown the new field of the alert, "Source:". Change CoreAPI to return the ID of the source of the alerts. Change databases and corresponding tables to have the source. Change HTML, XML and MD reports to include the ID of the source of the alerts. Fix zaproxy#2592 - Differentiate the source of alerts * Do not access the EDT in daemon mode Change class ExtensionSearch to not access the EDT (and view classes) if the view is not initialised when the session changes by adding a "view" SessionChangedListener when there's a View. * Include the base URL in SpiderTextParser Change SpiderTextParser to include the base URL when processing the URLs found in the response so if there's an error in the processing of those URLs it's included the page were the problematic URL was found. Without the change it would be logged: Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://example.com) which does not give any information where the problematic URL was found. The inclusion of base URL does not affect how the URLs are resolved, the processed URLs are already absolute. Tweak the class URLCanonicalizer to include the base URL when debug logging URLs with no authority, for same reason. * Added filterchain to parse out new line characters Many editors automatically append new lines to the end of files. Without this change, a trailing new line in version.txt causes the jar created to attempt to have a newline character in the name. * Include the "faulty" URI in exception message Change class HttpMethodHelper to include the "faulty" URI in the exception message, to give more information about the problem when being handled by caller code. * Latest files from Crowdin * Added setup as a dependency for dist The dist build target requires ${zap.jar} to be set. When it is not, the build does not actually execute successfully. Adding setup as a dependency for dist fixes this problem, and fixes zaproxy#1921 Also removes zap.jar property from day-stamped-release to avoid confusion with setting the property twice. * Do not access EDT in daemon mode in Context class Change Context class to not access the EDT if the view is not initialised, when restructuring the sites tree. * Fixed incorrect String comparisons * Always set Java mem to 1/4 available (over 512Mb) * Change ZAP API to read/use the request body Change API class to read/use the request body as that might be required for some API endpoints (e.g. "other" which might use the whole HTTP request). * Attempt to determine (String) body's charset Change HttpBody and HttpResponseBody to attempt to determine the charset of the contents (String) being set if the charset is unknown (that is, it was not previously set before the contents are set). Update tests to reflect the change in the behaviour. Related to zaproxy#2487 - Wrong charset used in HTTP body Fix zaproxy#2935 - Wrong charset used in response body if no charset set * Minor tidy up in context related panels Add JavaDoc to constructors and other undocumented parameters. Merge "initialize" methods into the constructors (and remove commented statement). Correct the name of a parameter. * Move HTML parser's test files into its own dir Move the files used by SpiderHtmlParserUnitTest to its own directory to be more clear what the files are used for and by what parser. Update SpiderHtmlParserUnitTest to use the new directory. * Reuse test files of HTML form spider parser Change the files to have the method as a variable so that it can be used for both GET and POST forms. Update SpiderHtmlFormParserUnitTest to reflect the changes. * Set a name to spider threads Initialise the spider threads with custom name as it makes it easier to identify that are threads created by ZAP, know it's purpose and to know to which spider scan they belong. Also, correct the site/name show when starting the spider. * Create first Root CA certificate synchronously Change class ExtensionDynSSL to create the Root CA certificate synchronously to have the certificate ready for when the Local Proxy is started, otherwise it could fail to process immediate SSL/TLS requests. * Use non absolute URI base HTML element Change SpiderHtmlParser and SpiderHtmlFormParser to properly handle non absolute URI base HTML element. Update tests to reflect the change in the behaviour. Change form HTML base tests to reuse the same file (have HTML base and form action as variables). Fix zaproxy#2939 - Use non absolute URI base HTML element in spider * Delay addition of the context being imported Changed Session to only add the context being imported if no errors occurred while importing it, otherwise the context could be left in a potentially inconsistent state which could cause issues in other parts of the code (for example, if it had no name (i.e. null) it would no longer be possible to add a new context, delete the one imported or create a new session). * Allow to export a Context through the context menu Add a pop up menu item to the context menu of the contexts tree to allow to export the selected context. * Correct charset determination in HttpResponseBody Remove use of platform's default charset when determining if the charset of the string is UTF-8, which was leading to wrong results if the platform's default charset was not UTF-8. Related to: - zaproxy#2935 - Wrong charset used in response body if no charset set - zaproxy#2941 - Attempt to determine (String) body's charset * Initialise panels when added to session dialogue Initialise the panels when added to session dialogue if it's shown, to ensure that the panels are in a consistent state. Also, ensure the session dialogue has a "UI shared context" when adding the panels of the newly added context. The change prevents exceptions (caused by the inconsistent state of the panels) when changes are done to the contexts (e.g. via ZAP API) while the dialogue is shown. Change to initialise the "regular" (i.e. non context) panels only once, when initParam(Object) is called (already done by base class). * Show correct header when selected panel is removed Change AbstractParamContainerPanel to (explicitly) show the first available panel when the selected panel is removed, to show the correct information in the panel header, title and help button. Also, do not show the panel if already shown (skip notifications that the same panel is hidden and then shown, header setup and re-setting the panel in the layout). * Do not allow Contexts with same name Change GUI/API to not allow to: - Import or create a context with no name, with an empty name or with name that already exists; - Change the name of the context to be null, empty or that duplicates an existing name. Fix zaproxy#1952 - Do not allow Contexts with same name * Export context's session management data Change ExtensionSessionManagement to also export session management data when exporting the context (not a problem for core implementations which do not have any data). * Remove WAVSEP spider tests The WAVSEP spider tests are no longer maintained, also the (HTML) spider parsers have now good unit test coverage and the spider is regularly tested with WIVET (through zapbot scans). * Support POST requests for API actions. Fixes zaproxy#2723 * Use L&F specified through JVM args Change GuiBootstrap to use the look and feel specified through the JVM arguments if able to find/set it, otherwise fallback to previous/current behaviour. Related to zaproxy#2964 - Allow to select the look and feel * Increase page size when accessing alerts It turns out that the paging is not implemented very efficiently, and choosing too small a page size can take a very long time. * Support break functionality in the API * Do not initialise dev logger if there's no view Change ExtensionLog4j to not initialise the "logger" if there's no view, it was only used if the view was initialised. Change ZapOutputWriter to require the view initialised and that the scan status label is provided (and remove view and null checks when logging, no longer needed per previous changes). Also, remove unused constructor. * Init status label in attack scanner only with view Change AttackModeScanner to not initialise the scan status label if there's no view, it's not needed in daemon mode. Also, change to use long to track elapsed time, instead of Date, to not create the Date objects unnecessarily. * Include date/time when logging that ZAP started The date/time allows to correlate the output logging with other logs and events more easily. * Modifications to Enableable Within org.zaproxy.zap.utils: * Add interface EnableableInterface (Extracted from Enableable). * Enableable now implements EnableableInterface. Within org.zaproxy.zap.view: * AbstractMultipleOptionsTableModel now leverages EnableableInterface. * AbstractMultipleOptionsTablePanel now leverages EnableableInterface. * Change ScriptType to define if enabled by default Change ScriptType to allow to define if the scripts of the script type should be enabled by default (e.g. when added/loaded via GUI). Related to zaproxy#2970 - Allow to configure, by script type, the enabled state of new/loaded scripts * Delay init of attack mode scanner to prevent NPE Change ExtensionActiveScan to delay the initialisation of AttackModeScanner to allow it to properly check if the view is initialised, using the extension. Caused by zaproxy#2972 - Init status label in attack scanner only with view * Change attack mode thread to daemon Change the thread used for the attack mode to be a daemon thread, to not prevent ZAP from terminating normally. For example, if the attack mode was enabled while starting ZAP (in daemon mode) and ZAP was not able to bind to the address/port it would be kept running instead of terminating. * Fix exception when getting sessions through ZAP API Change HttpSessionsAPI to obtain the optional parameter "session" with a default value, otherwise it would lead to a JSONException if it was not present in the API request. Fix zaproxy#2977 - HTTP500 from JSON/httpSessions/view/sessions/?site=FOO * Allow to disable default standard output logging Add a command line flag to disable the default standard output logging, allowing to configure/override it using the log4j.properties file. Add tests to assert the expected behaviour. * Change API JS script to check if method is defined Change CoreAPI JavaScript script to check if the formMethod field is defined before using it as not all the API calls (e.g. views) use/define it, leading to errors. * Fix typo in resource message key Change the name of the resource message key to match the name of the package of the extension ("uiutils"). * Tweak error message checks in ProxyServer Change how the exception's message is checked as newer versions of Java might return different messages, e.g.: - Java 7, Address already in use - Java 8, Address already in use (Bind failed) to keep showing a specific error/info message to the user. * Do not warn about non active attack mode scans Change active scanner extension to not warn/show as active actions the attack mode scans that are not active (i.e. either already stopped or still running but not scanning any message). * Latest files from Crowdin * Restore HostProcess/Scanner constructors Restore and deprecate HostProcess/Scanner constructors to keep binary compatibility with current/previous version, eases migration to newer version as some (add-on) tests use those constructors. * Correctly render all nodes in checkbox tree Change JCheckBoxTree to correctly render the top level nodes, the renderer will not show the checkbox if the node has no checkbox state moreover set the node's text to the label wherever it has or not a checkbox. Also, change to create the checkbox state of the tree nodes before the model is set to the base class (as it might be used by base class for painting calculations, using the custom renderer). Update test to reflect the change in behaviour (no longer throws a NullPointerException when setting a null model). * Latest files from Crowdin * Allow to passive scan just HTTP messages in scope Add an option disabled by default, to GUI and API, that allows to set the passive scanner to scan only messages that are in scope. Fix zaproxy#3004 - Allow to passive scan just HTTP messages in scope * Clarify passive scanner's enabled state (API) Change the description of API endpoint "setEnabled" to clarify that the enabled state is not persisted (i.e. defaults to passive scan always). * Added jenkins plugin and bug bounty links * Restore PassiveScanThread constructor Restore and deprecate PassiveScanThread constructor to keep binary compatibility with current/previous version, currently being used in add-on tests (passive scanners). * Support Factory Reset Fixes zaproxy#2701 * Call postInit when starting an extension Change ExtensionLoader to call the method Extension.postInit() when starting an extension (i.e. installed by an add-on). The change ensures the extension is properly/fully initialised when it is started/installed (e.g. sequence extension which adds a custom scan panel on postInit()). * Update dependencies and license * Allow to active scan a Context through the ZAP API Change ActiveScanAPI to: - Allow to specify a context for the "scan" action; - Not require the URL, in the actions "scan" and "scanAsUser", if the context is specified (for the latter action it is always). Add helper method to ApiImplementor that validates that an API parameter exists. Fix zaproxy#1853 - Allow to active scan a Context through ZAP API * Restore API generator methods Restore (and deprecate) methods of the API generators to keep binary compatibility with current/previous version (they are in use by zap-extensions project). * Correct proxy errors' Content-Length value Change ProxyThread to use the byte length of the error message instead of the number of characters for the Content-Length header, they might not be the same. Also, reorder the statements that set the headers to not need to guess the charset of the body being set. * Remove alerts.xml file Remove alerts.xml file, its contents (i.e. alerts' data) are not used nor maintained. Move the registry of the scanners IDs to a new file, scanners.md, which was previously in the alerts.xml (as XML comment). * Return request's type through the ZAP API Change ZAP API actions/views to include the type ID of the request (e.g. proxy, manual, spider, active) when returning the data of the HTTP message(s). Add JavaDoc to ApiResponseConversionUtils and made other minor changes (change logger variable to a constant and made class final). Update tests to check that the type is being set/used. * Add Spider URIs, to the UI, in the EDT Change the SpiderThread to add the URIs found to the UI in the EDT, to prevent concurrency issues between other threads and the EDT, e.g.: java.lang.NullPointerException at JTable.sortedTableChanged(JTable.java:4129) at JTable.tableChanged(JTable.java:4395) at JXTable.tableChanged(JXTable.java:1561) at AbstractTableModel.fireTableChanged(AbstractTableModel.java:296) at AbstractTableModel.fireTableRowsInserted(...) at o.z.z.extension.spider.SpiderPanelTableModel.addScanResult(...) at o.z.z.extension.spider.SpiderThread.foundURI(Unknown Source) at o.z.z.spider.Spider.notifyListenersFoundURI(Unknown Source) at o.z.z.spider.SpiderController.addSeed(Unknown Source) at o.z.z.spider.Spider.start(Unknown Source) at o.z.z.extension.spider.SpiderThread.startSpider(Unknown Source) at o.z.z.extension.spider.SpiderThread.runScan(Unknown Source) at o.z.z.extension.spider.SpiderThread.run(Unknown Source) (packages reduced/omitted to keep the lines short) Also, do not create the SpiderPanelTableModel if there's no view. Remove the synchronisation in SpiderPanelTableModel as that's not required, the model is accessed only through the EDT. * Fix concurrency issues when publishing ZAP events Change SimpleEventBus to control the read/write accesses to the publishers and consumers to prevent concurrency issues. For example, when a consumer is unregistered while publishing events, which could lead to exceptions, e.g.: java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(...) at java.util.ArrayList$Itr.next(...) at o.z.z.eventBus.SimpleEventBus.publishSyncEvent(...) at o.z.z.extension.alert.ExtensionAlert.publishAlertEvent(...) at o.z.z.extension.alert.ExtensionAlert.alertFound(...) at o.z.z.extension.pscan.PassiveScanThread.raiseAlert(...) (packages reduced/omitted to keep the lines short) Only one thread (write access) is allowed to manage the publishers and consumers while multiple threads can publish events (read access), as long no thread is managing the publishers or the consumers. Change the classes RegisteredConsumer and RegisteredPublisher to be static as they don't need to access the state of SimpleEventBus class. * Allow to select multiple parameters in Params tab Change ParamsPanel to allow to select multiple parameters (rows). Change Params tab pop up menus to be enabled only when one of the parameters is selected (to keep the same behaviour). Related to zaproxy#3040 - Export param tab contents * Add Spider scans to GUI in the EDT Change ExtensionSpider to add the spider scans to the GUI in the EDT, to prevent inconsistencies between EDT and other threads, which could lead to exceptions (and a freeze of GUI caused by inconsistent internal state of UI components). Change SpiderScan to not create the model when adding messages if the scan was already cleared, to prevent a leak of AlertEventConsumer(s). * Expose constants of core rule configurations Change RuleConfigParam to expose constants to access the core rule configurations when active/passive scanning. * Unit tests for the UsernamePasswordAuthenticationCredentials class * Return requests' timestamp/RTT through the ZAP API Change ApiResponseConversionUtils to also return the timestamp and RTT of the HTTP message. Update test to assert the returned data. * Add (some) JavaDoc to ScripType Add JavaDoc to the class and to capability related constant/methods. * Correct the loading of extensions' enabled state Change ExtensionFactory to use the ExtensionParam to obtain the enabled state of the extensions (which uses the new/correct configuration keys). Change ExtensionParam to allow to query the enabled state of an extension and change to use a map to keep the enabled states. Move ExtensionParam to OptionsParam as it needs to be early available for core code to use (i.e. ExtensionFactory). Update tests to assert the new behaviour. Issue introduced in zaproxy#2245 - Convert options to not use extensions' names as XML element names * Tweak log message in URLCanonicalizer Change a log message to include the URL that is being processed, also change to return immediately if the the URL is not valid after logging the problem (instead of throwing an exception, which would be caught (and logged) in the same method). * Clear old contexts, always, when loading a session Change Session to remove all the contexts before refreshing the UI when discarding the contexts, otherwise the contexts tree would have the contexts of the previous session if the loaded session had none. * Add initiator constant for AJAX spider requests Add a constant to HttpSender class for requests sent by the AJAX spider. Update the JavaScript HTTP Sender template script with the new constant. * Allow to extend ProxyThread Change ProxyThread to allow to be extended (from other packages) and use a custom HttpSender, required for the AJAX Spider to use a custom initiator ID. * Add tests for OptionsParamApi Add tests for OptionsParamApi to assert the expected behaviour. Also, do other tweaks to OptionsParamApi: - Remove commented code and related constant (unimplemented option); - Remove initialisations with default value and initialise the enabled instance variable as true (default value used when loading from file); - Properly handle malformed values in the configuration file; - Do not attempt to set and save the API key if the configurations was not set. * Support active scan rule and scan max duration Fixes zaproxy#2951 * Stop the spider scan if failed to properly start Change SpiderThread to stop the spider scan on exceptions during the starting process, to prevent the spider scan from becoming in undefined state (that is, not fully started nor stopped). Related to issues like zaproxy#3039. * Add initiator constant for Forced Browse requests Add a constant to HttpSender class for requests sent by the Forced Browse add-on. Related to zaproxy#3060 - Send Forced Browse requests through ZAP * Allow to deprecate ZAP API endpoints Change the ZAP API to allow to set its endpoints as deprecated (and add a description why they are). Change ZAP API UI to show a note when the endpoints are deprecated. Change JAVA API generator to annotate and add JavaDoc tag to deprecated endpoints. Fix zaproxy#3061 - Allow to deprecate API endpoints * Skip process automated msgs for HTTP Sessions tab Change class ExtensionHttpSessions to skip/ignore the responses of AJAX Spider and Forced Browse, as with other automated responses they should not be processed (would end up creating a lot of unnecessary sessions). Related to zaproxy#2674 - Automated authentication requests shown in HTTP Sessions tab * Expose add-on's file extension Change AddOn class to expose a constant for the file extension. Replace the literal string, in AddOn and ExtensionAutoUpdate, with the constant created. * Added cookie ignore list rule and inc sleep default to 20 to reduce FPs * Allow to show only bytes in HTTP message tables Add a check menu item to the context menu of the tables that show HTTP messages to allow to switch between just showing bytes (the new default) and other byte units (e.g. KiB, MiB). Fix zaproxy#2994 - show column 'Size Resp. Body' of history in bytes * Latest files from Crowdin * Log the name of the user of the active scan Change HostProcess to include the name of the user (if any) when logging the information of the scan being started. That information is useful when reviewing what the scan was doing (or, expected to do). * Latest files from Crowdin * Latest files from Crowdin * Update test_zap.config typo fixes * Latest files from Crowdin * ProxyThread SocketTimeoutException Verbosity Only log full exception if debug is enabled. Fixes zaproxy#3095 * Change policy's threshold/strength with ZAP API Add 2 optional params to AddScanPolicy api, default to medium level for AlertThreshold and AttackStrength. Add UpdateScanPolicy api, could change AttackStrength and AlertThreshold for a policy. * Log to file even if ZAP is run 'inline' Change CommandLineBootstrap to not disable the logging (to log to file by default), also, log when ZAP is started. It's useful to know what ZAP is doing or did. * fix mojibake HTML Report * Latest files from Crowdin * Do not set the "in scope" state in Contexts panel Change ContextListPanel to not set the "in scope" state to the contexts as that might (depending on the internal order of the panels) override the value set by/in ContextGeneralPanel. The ContextListPanel does not allow to change the "in scope" state so it should not be setting it. Fix zaproxy#3100 - Context's in scope change might not be applied * Add "Max children to crawl" to main spider options Add the (advanced) option "Maximum children to crawl" to main Spider options panel. The option is now available in both places (as the other advanced options). Fix zaproxy#3066 - Spidering options in the doc in two place * Do not require status/version in add-on file name Change AddOn class to not require the status and/or the version in the file name of the add-on. The add-on file name just needs to have the ID and have a ZAP extension. Also, deprecate old constructor/methods that require the file name to have the status/version and introduce new constructor/methods where applicable. Change BaseZapAddOnXmlData to read the status from the manifest file of the add-on (ZapAddOn.xml). Remove hardcoded manifest file name (ZapAddOn.xml) from JavaDoc and code (by using the constant from AddOn). Change AddOnCollection to iterate just ZAP add-on files and use the new AddOn constructor. Change ExtensionAutoUpdate to use the new constructors/methods and to copy the file from manual add-on installations using a normalised file name. Add some tests to assert the expected behaviour of AddOn class. Fix zaproxy#3090 - Be more lenient on add-on's file name format * Change default time to 15 and make publicly accessible * Latest files from Crowdin * Deprecate unused Spider menu items Deprecate unused Spider menu items (replaced by the Spider dialogue menu item). * Latest files from Crowdin * Correct location of i18n messages Swap the contents of two i18n messages, the title was being used as message and the message as title of the dialogue. * include SubjectAlternativeName extension in generated certificates * Add description to some Spider API endpoints Add descriptions to some of the Spider API endpoints and correct one that was wrong (it was for the action not view). * Log during start up the add-ons that are installed Change ExtensionFactory to log (as info) the IDs and version of the add-ons that are in installed state (all dependencies/requirements are fulfilled). * Add description to some core/ascan API endpoints Add description to some core and ascan API endpoints. * Latest files from Crowdin * Added security annotations for forms that dont need anti CSRF tokens * Latest files from Crowdin * Add description to active scan ZAP API option Add description to the active scan ZAP API option "Inject plugin ID in header for all active scan requests". Related to zaproxy#3133 - how disable send X-ZAP-Scan-ID header * Latest files from Crowdin
Harinus
added a commit
to Harinus/zaproxy
that referenced
this issue
Jan 12, 2017
* Show the cause why a script was not loaded Change ExtensionScript to provide more details why a script was not successfully loaded (e.g. missing script type, invalid character sequence, other unexpected causes). * Latest files from Crowdin * Show white space chars of matches in Search panel Change SearchResultTableEntry to replace the white space characters in the string found with visible equivalent characters so that the matches in the Search panel are like: Content-Length:·453¤¶Connection:·close¤¶Content-Type: instead of: Content-Length: 453Connection: closeContent-Type: which does not reflect the actual match. * Address JavaDoc issues Address JavaDoc issues in some classes, tidy up class JavaDoc, add missing docs on methods and parameters, replace closing HTML tags with starting tags and remove empty docs of overridden methods. * Correct offset calculation in text header views Extract the calculation of offsets for view to header and header to view into a class (HttpTextViewUtils) and changed the text views to use it, reduces code duplication and uses the correct calculations in all cases (some calculations were already correct). Add tests to assert the expected behaviour of HttpTextViewUtils. Fix zaproxy#2793 - Wrong highlight in combined view with last part of request header * Fix typos in API endpoint descriptions Change descriptions of core API endpoints sendRequest and sendHarRequest from "now allowed" to "not allowed" (to send the requests in Safe mode). * Use dev version of API and support -w wiki_report * Upgrade to use ubuntu:16.04 * Exclude brackets in URLs in spider parsers Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs * Ignore incorrect start/end positions on highlight Change HttpTextViewUtils to return invalid position, instead of throwing an exception, if the start or end positions are greater than the length of the view/header/body, otherwise it would result in an exception when the combined view is showing a custom message in the body (when body is too large to display per configurations), moreover it is more resilient to future changes in the content shown. Update the tests to assert the new behaviour. * Remove "debug" code in SpiderTextParser Remove "debug" code accidentally committed in a previous change. * Tweak JavaDoc in StandardFieldsDialog Add missing parameter descriptions and add docs for the constructors and some methods. * Tweak JavaDocs in class TabbedPanel2 Fix typo, add descriptions to parameters of documented methods and other minor tweaks. * Latest files from Crowdin * Do not use null string literal in ApiResponseSet Change ApiResponseSet to not use null string literal, in XML and HTML formats, when the values are null otherwise it might seem that the null values of the set have a value (i.e. "null"), instead of nothing. For example, an alert with no evidence/attack (null) would shown "null" instead of nothing. * Differentiate the source of alerts Change the Alert class to have a source "unknown", "active", "manual", "passive" and "tool" which is set just before the alert is raised for active, manual and passive alerts, for remaining alerts it's considered as raised by a tool (e.g. custom scripts). Old alerts of existing sessions will default to "unknown" since its exact source is not immediately known. Change Alert tab to shown the new field of the alert, "Source:". Change CoreAPI to return the ID of the source of the alerts. Change databases and corresponding tables to have the source. Change HTML, XML and MD reports to include the ID of the source of the alerts. Fix zaproxy#2592 - Differentiate the source of alerts * Do not access the EDT in daemon mode Change class ExtensionSearch to not access the EDT (and view classes) if the view is not initialised when the session changes by adding a "view" SessionChangedListener when there's a View. * Include the base URL in SpiderTextParser Change SpiderTextParser to include the base URL when processing the URLs found in the response so if there's an error in the processing of those URLs it's included the page were the problematic URL was found. Without the change it would be logged: Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://example.com) which does not give any information where the problematic URL was found. The inclusion of base URL does not affect how the URLs are resolved, the processed URLs are already absolute. Tweak the class URLCanonicalizer to include the base URL when debug logging URLs with no authority, for same reason. * Added filterchain to parse out new line characters Many editors automatically append new lines to the end of files. Without this change, a trailing new line in version.txt causes the jar created to attempt to have a newline character in the name. * Include the "faulty" URI in exception message Change class HttpMethodHelper to include the "faulty" URI in the exception message, to give more information about the problem when being handled by caller code. * Latest files from Crowdin * Added setup as a dependency for dist The dist build target requires ${zap.jar} to be set. When it is not, the build does not actually execute successfully. Adding setup as a dependency for dist fixes this problem, and fixes zaproxy#1921 Also removes zap.jar property from day-stamped-release to avoid confusion with setting the property twice. * Do not access EDT in daemon mode in Context class Change Context class to not access the EDT if the view is not initialised, when restructuring the sites tree. * Fixed incorrect String comparisons * Always set Java mem to 1/4 available (over 512Mb) * Change ZAP API to read/use the request body Change API class to read/use the request body as that might be required for some API endpoints (e.g. "other" which might use the whole HTTP request). * Attempt to determine (String) body's charset Change HttpBody and HttpResponseBody to attempt to determine the charset of the contents (String) being set if the charset is unknown (that is, it was not previously set before the contents are set). Update tests to reflect the change in the behaviour. Related to zaproxy#2487 - Wrong charset used in HTTP body Fix zaproxy#2935 - Wrong charset used in response body if no charset set * Minor tidy up in context related panels Add JavaDoc to constructors and other undocumented parameters. Merge "initialize" methods into the constructors (and remove commented statement). Correct the name of a parameter. * Move HTML parser's test files into its own dir Move the files used by SpiderHtmlParserUnitTest to its own directory to be more clear what the files are used for and by what parser. Update SpiderHtmlParserUnitTest to use the new directory. * Reuse test files of HTML form spider parser Change the files to have the method as a variable so that it can be used for both GET and POST forms. Update SpiderHtmlFormParserUnitTest to reflect the changes. * Set a name to spider threads Initialise the spider threads with custom name as it makes it easier to identify that are threads created by ZAP, know it's purpose and to know to which spider scan they belong. Also, correct the site/name show when starting the spider. * Create first Root CA certificate synchronously Change class ExtensionDynSSL to create the Root CA certificate synchronously to have the certificate ready for when the Local Proxy is started, otherwise it could fail to process immediate SSL/TLS requests. * Use non absolute URI base HTML element Change SpiderHtmlParser and SpiderHtmlFormParser to properly handle non absolute URI base HTML element. Update tests to reflect the change in the behaviour. Change form HTML base tests to reuse the same file (have HTML base and form action as variables). Fix zaproxy#2939 - Use non absolute URI base HTML element in spider * Delay addition of the context being imported Changed Session to only add the context being imported if no errors occurred while importing it, otherwise the context could be left in a potentially inconsistent state which could cause issues in other parts of the code (for example, if it had no name (i.e. null) it would no longer be possible to add a new context, delete the one imported or create a new session). * Allow to export a Context through the context menu Add a pop up menu item to the context menu of the contexts tree to allow to export the selected context. * Correct charset determination in HttpResponseBody Remove use of platform's default charset when determining if the charset of the string is UTF-8, which was leading to wrong results if the platform's default charset was not UTF-8. Related to: - zaproxy#2935 - Wrong charset used in response body if no charset set - zaproxy#2941 - Attempt to determine (String) body's charset * Initialise panels when added to session dialogue Initialise the panels when added to session dialogue if it's shown, to ensure that the panels are in a consistent state. Also, ensure the session dialogue has a "UI shared context" when adding the panels of the newly added context. The change prevents exceptions (caused by the inconsistent state of the panels) when changes are done to the contexts (e.g. via ZAP API) while the dialogue is shown. Change to initialise the "regular" (i.e. non context) panels only once, when initParam(Object) is called (already done by base class). * Show correct header when selected panel is removed Change AbstractParamContainerPanel to (explicitly) show the first available panel when the selected panel is removed, to show the correct information in the panel header, title and help button. Also, do not show the panel if already shown (skip notifications that the same panel is hidden and then shown, header setup and re-setting the panel in the layout). * Do not allow Contexts with same name Change GUI/API to not allow to: - Import or create a context with no name, with an empty name or with name that already exists; - Change the name of the context to be null, empty or that duplicates an existing name. Fix zaproxy#1952 - Do not allow Contexts with same name * Export context's session management data Change ExtensionSessionManagement to also export session management data when exporting the context (not a problem for core implementations which do not have any data). * Remove WAVSEP spider tests The WAVSEP spider tests are no longer maintained, also the (HTML) spider parsers have now good unit test coverage and the spider is regularly tested with WIVET (through zapbot scans). * Support POST requests for API actions. Fixes zaproxy#2723 * Use L&F specified through JVM args Change GuiBootstrap to use the look and feel specified through the JVM arguments if able to find/set it, otherwise fallback to previous/current behaviour. Related to zaproxy#2964 - Allow to select the look and feel * Increase page size when accessing alerts It turns out that the paging is not implemented very efficiently, and choosing too small a page size can take a very long time. * Support break functionality in the API * Do not initialise dev logger if there's no view Change ExtensionLog4j to not initialise the "logger" if there's no view, it was only used if the view was initialised. Change ZapOutputWriter to require the view initialised and that the scan status label is provided (and remove view and null checks when logging, no longer needed per previous changes). Also, remove unused constructor. * Init status label in attack scanner only with view Change AttackModeScanner to not initialise the scan status label if there's no view, it's not needed in daemon mode. Also, change to use long to track elapsed time, instead of Date, to not create the Date objects unnecessarily. * Include date/time when logging that ZAP started The date/time allows to correlate the output logging with other logs and events more easily. * Modifications to Enableable Within org.zaproxy.zap.utils: * Add interface EnableableInterface (Extracted from Enableable). * Enableable now implements EnableableInterface. Within org.zaproxy.zap.view: * AbstractMultipleOptionsTableModel now leverages EnableableInterface. * AbstractMultipleOptionsTablePanel now leverages EnableableInterface. * Change ScriptType to define if enabled by default Change ScriptType to allow to define if the scripts of the script type should be enabled by default (e.g. when added/loaded via GUI). Related to zaproxy#2970 - Allow to configure, by script type, the enabled state of new/loaded scripts * Delay init of attack mode scanner to prevent NPE Change ExtensionActiveScan to delay the initialisation of AttackModeScanner to allow it to properly check if the view is initialised, using the extension. Caused by zaproxy#2972 - Init status label in attack scanner only with view * Change attack mode thread to daemon Change the thread used for the attack mode to be a daemon thread, to not prevent ZAP from terminating normally. For example, if the attack mode was enabled while starting ZAP (in daemon mode) and ZAP was not able to bind to the address/port it would be kept running instead of terminating. * Fix exception when getting sessions through ZAP API Change HttpSessionsAPI to obtain the optional parameter "session" with a default value, otherwise it would lead to a JSONException if it was not present in the API request. Fix zaproxy#2977 - HTTP500 from JSON/httpSessions/view/sessions/?site=FOO * Allow to disable default standard output logging Add a command line flag to disable the default standard output logging, allowing to configure/override it using the log4j.properties file. Add tests to assert the expected behaviour. * Change API JS script to check if method is defined Change CoreAPI JavaScript script to check if the formMethod field is defined before using it as not all the API calls (e.g. views) use/define it, leading to errors. * Fix typo in resource message key Change the name of the resource message key to match the name of the package of the extension ("uiutils"). * Tweak error message checks in ProxyServer Change how the exception's message is checked as newer versions of Java might return different messages, e.g.: - Java 7, Address already in use - Java 8, Address already in use (Bind failed) to keep showing a specific error/info message to the user. * Do not warn about non active attack mode scans Change active scanner extension to not warn/show as active actions the attack mode scans that are not active (i.e. either already stopped or still running but not scanning any message). * Latest files from Crowdin * Restore HostProcess/Scanner constructors Restore and deprecate HostProcess/Scanner constructors to keep binary compatibility with current/previous version, eases migration to newer version as some (add-on) tests use those constructors. * Correctly render all nodes in checkbox tree Change JCheckBoxTree to correctly render the top level nodes, the renderer will not show the checkbox if the node has no checkbox state moreover set the node's text to the label wherever it has or not a checkbox. Also, change to create the checkbox state of the tree nodes before the model is set to the base class (as it might be used by base class for painting calculations, using the custom renderer). Update test to reflect the change in behaviour (no longer throws a NullPointerException when setting a null model). * Latest files from Crowdin * Allow to passive scan just HTTP messages in scope Add an option disabled by default, to GUI and API, that allows to set the passive scanner to scan only messages that are in scope. Fix zaproxy#3004 - Allow to passive scan just HTTP messages in scope * Clarify passive scanner's enabled state (API) Change the description of API endpoint "setEnabled" to clarify that the enabled state is not persisted (i.e. defaults to passive scan always). * Added jenkins plugin and bug bounty links * Restore PassiveScanThread constructor Restore and deprecate PassiveScanThread constructor to keep binary compatibility with current/previous version, currently being used in add-on tests (passive scanners). * Support Factory Reset Fixes zaproxy#2701 * Call postInit when starting an extension Change ExtensionLoader to call the method Extension.postInit() when starting an extension (i.e. installed by an add-on). The change ensures the extension is properly/fully initialised when it is started/installed (e.g. sequence extension which adds a custom scan panel on postInit()). * Update dependencies and license * Allow to active scan a Context through the ZAP API Change ActiveScanAPI to: - Allow to specify a context for the "scan" action; - Not require the URL, in the actions "scan" and "scanAsUser", if the context is specified (for the latter action it is always). Add helper method to ApiImplementor that validates that an API parameter exists. Fix zaproxy#1853 - Allow to active scan a Context through ZAP API * Restore API generator methods Restore (and deprecate) methods of the API generators to keep binary compatibility with current/previous version (they are in use by zap-extensions project). * Correct proxy errors' Content-Length value Change ProxyThread to use the byte length of the error message instead of the number of characters for the Content-Length header, they might not be the same. Also, reorder the statements that set the headers to not need to guess the charset of the body being set. * Remove alerts.xml file Remove alerts.xml file, its contents (i.e. alerts' data) are not used nor maintained. Move the registry of the scanners IDs to a new file, scanners.md, which was previously in the alerts.xml (as XML comment). * Return request's type through the ZAP API Change ZAP API actions/views to include the type ID of the request (e.g. proxy, manual, spider, active) when returning the data of the HTTP message(s). Add JavaDoc to ApiResponseConversionUtils and made other minor changes (change logger variable to a constant and made class final). Update tests to check that the type is being set/used. * Add Spider URIs, to the UI, in the EDT Change the SpiderThread to add the URIs found to the UI in the EDT, to prevent concurrency issues between other threads and the EDT, e.g.: java.lang.NullPointerException at JTable.sortedTableChanged(JTable.java:4129) at JTable.tableChanged(JTable.java:4395) at JXTable.tableChanged(JXTable.java:1561) at AbstractTableModel.fireTableChanged(AbstractTableModel.java:296) at AbstractTableModel.fireTableRowsInserted(...) at o.z.z.extension.spider.SpiderPanelTableModel.addScanResult(...) at o.z.z.extension.spider.SpiderThread.foundURI(Unknown Source) at o.z.z.spider.Spider.notifyListenersFoundURI(Unknown Source) at o.z.z.spider.SpiderController.addSeed(Unknown Source) at o.z.z.spider.Spider.start(Unknown Source) at o.z.z.extension.spider.SpiderThread.startSpider(Unknown Source) at o.z.z.extension.spider.SpiderThread.runScan(Unknown Source) at o.z.z.extension.spider.SpiderThread.run(Unknown Source) (packages reduced/omitted to keep the lines short) Also, do not create the SpiderPanelTableModel if there's no view. Remove the synchronisation in SpiderPanelTableModel as that's not required, the model is accessed only through the EDT. * Fix concurrency issues when publishing ZAP events Change SimpleEventBus to control the read/write accesses to the publishers and consumers to prevent concurrency issues. For example, when a consumer is unregistered while publishing events, which could lead to exceptions, e.g.: java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(...) at java.util.ArrayList$Itr.next(...) at o.z.z.eventBus.SimpleEventBus.publishSyncEvent(...) at o.z.z.extension.alert.ExtensionAlert.publishAlertEvent(...) at o.z.z.extension.alert.ExtensionAlert.alertFound(...) at o.z.z.extension.pscan.PassiveScanThread.raiseAlert(...) (packages reduced/omitted to keep the lines short) Only one thread (write access) is allowed to manage the publishers and consumers while multiple threads can publish events (read access), as long no thread is managing the publishers or the consumers. Change the classes RegisteredConsumer and RegisteredPublisher to be static as they don't need to access the state of SimpleEventBus class. * Allow to select multiple parameters in Params tab Change ParamsPanel to allow to select multiple parameters (rows). Change Params tab pop up menus to be enabled only when one of the parameters is selected (to keep the same behaviour). Related to zaproxy#3040 - Export param tab contents * Add Spider scans to GUI in the EDT Change ExtensionSpider to add the spider scans to the GUI in the EDT, to prevent inconsistencies between EDT and other threads, which could lead to exceptions (and a freeze of GUI caused by inconsistent internal state of UI components). Change SpiderScan to not create the model when adding messages if the scan was already cleared, to prevent a leak of AlertEventConsumer(s). * Expose constants of core rule configurations Change RuleConfigParam to expose constants to access the core rule configurations when active/passive scanning. * Unit tests for the UsernamePasswordAuthenticationCredentials class * Return requests' timestamp/RTT through the ZAP API Change ApiResponseConversionUtils to also return the timestamp and RTT of the HTTP message. Update test to assert the returned data. * Add (some) JavaDoc to ScripType Add JavaDoc to the class and to capability related constant/methods. * Correct the loading of extensions' enabled state Change ExtensionFactory to use the ExtensionParam to obtain the enabled state of the extensions (which uses the new/correct configuration keys). Change ExtensionParam to allow to query the enabled state of an extension and change to use a map to keep the enabled states. Move ExtensionParam to OptionsParam as it needs to be early available for core code to use (i.e. ExtensionFactory). Update tests to assert the new behaviour. Issue introduced in zaproxy#2245 - Convert options to not use extensions' names as XML element names * Tweak log message in URLCanonicalizer Change a log message to include the URL that is being processed, also change to return immediately if the the URL is not valid after logging the problem (instead of throwing an exception, which would be caught (and logged) in the same method). * Clear old contexts, always, when loading a session Change Session to remove all the contexts before refreshing the UI when discarding the contexts, otherwise the contexts tree would have the contexts of the previous session if the loaded session had none. * Add initiator constant for AJAX spider requests Add a constant to HttpSender class for requests sent by the AJAX spider. Update the JavaScript HTTP Sender template script with the new constant. * Allow to extend ProxyThread Change ProxyThread to allow to be extended (from other packages) and use a custom HttpSender, required for the AJAX Spider to use a custom initiator ID. * Add tests for OptionsParamApi Add tests for OptionsParamApi to assert the expected behaviour. Also, do other tweaks to OptionsParamApi: - Remove commented code and related constant (unimplemented option); - Remove initialisations with default value and initialise the enabled instance variable as true (default value used when loading from file); - Properly handle malformed values in the configuration file; - Do not attempt to set and save the API key if the configurations was not set. * Support active scan rule and scan max duration Fixes zaproxy#2951 * Stop the spider scan if failed to properly start Change SpiderThread to stop the spider scan on exceptions during the starting process, to prevent the spider scan from becoming in undefined state (that is, not fully started nor stopped). Related to issues like zaproxy#3039. * Add initiator constant for Forced Browse requests Add a constant to HttpSender class for requests sent by the Forced Browse add-on. Related to zaproxy#3060 - Send Forced Browse requests through ZAP * Allow to deprecate ZAP API endpoints Change the ZAP API to allow to set its endpoints as deprecated (and add a description why they are). Change ZAP API UI to show a note when the endpoints are deprecated. Change JAVA API generator to annotate and add JavaDoc tag to deprecated endpoints. Fix zaproxy#3061 - Allow to deprecate API endpoints * Skip process automated msgs for HTTP Sessions tab Change class ExtensionHttpSessions to skip/ignore the responses of AJAX Spider and Forced Browse, as with other automated responses they should not be processed (would end up creating a lot of unnecessary sessions). Related to zaproxy#2674 - Automated authentication requests shown in HTTP Sessions tab * Expose add-on's file extension Change AddOn class to expose a constant for the file extension. Replace the literal string, in AddOn and ExtensionAutoUpdate, with the constant created. * Added cookie ignore list rule and inc sleep default to 20 to reduce FPs * Allow to show only bytes in HTTP message tables Add a check menu item to the context menu of the tables that show HTTP messages to allow to switch between just showing bytes (the new default) and other byte units (e.g. KiB, MiB). Fix zaproxy#2994 - show column 'Size Resp. Body' of history in bytes * Latest files from Crowdin * Log the name of the user of the active scan Change HostProcess to include the name of the user (if any) when logging the information of the scan being started. That information is useful when reviewing what the scan was doing (or, expected to do). * Latest files from Crowdin * Latest files from Crowdin * Update test_zap.config typo fixes * Latest files from Crowdin * ProxyThread SocketTimeoutException Verbosity Only log full exception if debug is enabled. Fixes zaproxy#3095 * Change policy's threshold/strength with ZAP API Add 2 optional params to AddScanPolicy api, default to medium level for AlertThreshold and AttackStrength. Add UpdateScanPolicy api, could change AttackStrength and AlertThreshold for a policy. * Log to file even if ZAP is run 'inline' Change CommandLineBootstrap to not disable the logging (to log to file by default), also, log when ZAP is started. It's useful to know what ZAP is doing or did. * fix mojibake HTML Report * Latest files from Crowdin * Do not set the "in scope" state in Contexts panel Change ContextListPanel to not set the "in scope" state to the contexts as that might (depending on the internal order of the panels) override the value set by/in ContextGeneralPanel. The ContextListPanel does not allow to change the "in scope" state so it should not be setting it. Fix zaproxy#3100 - Context's in scope change might not be applied * Add "Max children to crawl" to main spider options Add the (advanced) option "Maximum children to crawl" to main Spider options panel. The option is now available in both places (as the other advanced options). Fix zaproxy#3066 - Spidering options in the doc in two place * Do not require status/version in add-on file name Change AddOn class to not require the status and/or the version in the file name of the add-on. The add-on file name just needs to have the ID and have a ZAP extension. Also, deprecate old constructor/methods that require the file name to have the status/version and introduce new constructor/methods where applicable. Change BaseZapAddOnXmlData to read the status from the manifest file of the add-on (ZapAddOn.xml). Remove hardcoded manifest file name (ZapAddOn.xml) from JavaDoc and code (by using the constant from AddOn). Change AddOnCollection to iterate just ZAP add-on files and use the new AddOn constructor. Change ExtensionAutoUpdate to use the new constructors/methods and to copy the file from manual add-on installations using a normalised file name. Add some tests to assert the expected behaviour of AddOn class. Fix zaproxy#3090 - Be more lenient on add-on's file name format * Change default time to 15 and make publicly accessible * Latest files from Crowdin * Deprecate unused Spider menu items Deprecate unused Spider menu items (replaced by the Spider dialogue menu item). * Latest files from Crowdin * Correct location of i18n messages Swap the contents of two i18n messages, the title was being used as message and the message as title of the dialogue. * include SubjectAlternativeName extension in generated certificates * Add description to some Spider API endpoints Add descriptions to some of the Spider API endpoints and correct one that was wrong (it was for the action not view). * Log during start up the add-ons that are installed Change ExtensionFactory to log (as info) the IDs and version of the add-ons that are in installed state (all dependencies/requirements are fulfilled). * Add description to some core/ascan API endpoints Add description to some core and ascan API endpoints. * Latest files from Crowdin * Added security annotations for forms that dont need anti CSRF tokens * Latest files from Crowdin * Add description to active scan ZAP API option Add description to the active scan ZAP API option "Inject plugin ID in header for all active scan requests". Related to zaproxy#3133 - how disable send X-ZAP-Scan-ID header * Latest files from Crowdin
martinkalina
pushed a commit
to martinkalina/zaproxy
that referenced
this issue
Mar 1, 2017
Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Using https://github.com/stephendonner/docker-zap/blob/a6e591ebbdfe8be78112a5f8e41bfb396388cd20/run-docker.sh I did a scan of https://treeherder.allizom.org, and it apparently found a comment:
Bootstrap v3.3.5 (http://getbootstrap.com)
And then tried to extract the host with the parenthesis:
3390 [pool-1-thread-1] WARN org.zaproxy.zap.spider.URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)
"@thc202: we should tweak the parser to ignore the parenthesis"
Full log below:
`Started by upstream project "docker-zap" build number 44
originally caused by:
Started by user Stephen Donner
Building in workspace /var/lib/jenkins/workspace/docker-zap/default
git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url https://github.com/stephendonner/docker-zap.git # timeout=10
Fetching upstream changes from https://github.com/stephendonner/docker-zap.git
git --version # timeout=10
git fetch --tags --progress https://github.com/stephendonner/docker-zap.git +refs/heads/:refs/remotes/origin/
Checking out Revision a6e591ebbdfe8be78112a5f8e41bfb396388cd20 (refs/remotes/origin/master)
git config core.sparsecheckout # timeout=10
git checkout -f a6e591ebbdfe8be78112a5f8e41bfb396388cd20
git rev-list a6e591ebbdfe8be78112a5f8e41bfb396388cd20 # timeout=10
[default] $ /usr/bin/env bash /tmp/hudson1902304993759994340.sh
https://treeherder.allizom.org
[INFO] ZAP is running
[INFO] Accessing URL https://treeherder.allizom.org
[INFO] Running spider...
[INFO] Running an active scan...
[INFO] Issues found: 0
=================================================================ZAP-daemon log output follows=================================================================
Found Java version 1.8.0_45-internal
Available memory: 1839 MB
Setting jvm heap size: -Xmx512m
449 [main] INFO org.zaproxy.zap.DaemonBootstrap - OWASP ZAP D-2016-09-05 started.
550 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config view.mode = attack was null
551 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config api.disablekey = true was null
551 [main] INFO org.parosproxy.paros.common.AbstractParam - Setting config scanner.attackOnStart = true was null
567 [main] INFO org.parosproxy.paros.network.SSLConnector - Reading supported SSL/TLS protocols...
568 [main] INFO org.parosproxy.paros.network.SSLConnector - Using a SSLEngine...
1093 [main] INFO org.parosproxy.paros.network.SSLConnector - Done reading supported SSL/TLS protocols: [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
1108 [main] INFO org.parosproxy.paros.extension.option.OptionsParamCertificate - Unsafe SSL renegotiation disabled.
2026 [main] INFO hsqldb.db..ENGINE - open start - state not modified
2370 [main] INFO hsqldb.db..ENGINE - dataFileCache open start
2393 [main] INFO hsqldb.db..ENGINE - dataFileCache open end
2556 [ZAP-daemon] INFO org.zaproxy.zap.control.ExtensionFactory - Loading extensions
5992 [ZAP-daemon] INFO org.zaproxy.zap.control.ExtensionFactory - Extensions loaded
6511 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Change user agent to other browsers.
6511 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Detect insecure or potentially malicious content in HTTP responses.
6511 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Detect and alert 'Set-cookie' attempt in HTTP response for modification.
6511 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Avoid browser cache (strip off IfModifiedSince)
6511 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Log cookies sent by browser.
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Log unique GET queries into file:filter/get.xls
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Log unique POST queries into file: filter/post.xls
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Log request and response into file: filter/message.txt
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Replace HTTP request body using defined pattern.
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Replace HTTP request header using defined pattern.
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Replace HTTP response body using defined pattern.
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Replace HTTP response header using defined pattern.
6512 [ZAP-daemon] INFO org.parosproxy.paros.extension.filter.FilterFactory - loaded filter Send ZAP session request ID
Sep 28, 2016 8:22:47 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
6670 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows ZAP to check for updates
6677 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionViewOption
6677 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionEdit
6677 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionFilter
6677 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Provides a rest based API for controlling and accessing ZAP
6755 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionState
6756 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionReport
6756 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHistory
6759 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Show hidden fields and enable disabled fields
6761 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Search messages for strings and regular expressions
6762 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Encode/Decode/Hash...
6762 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows you to intercept and modify requests and responses
6762 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Passive scanner
6878 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Script Passive Scan Rules
6878 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Stats Passive Scan Rule
6879 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Application Error Disclosure
6879 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Incomplete or No Cache-control and Pragma HTTP Header Set
6879 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Content-Type Header Missing
6879 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Cookie No HttpOnly Flag
6879 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Cookie Without Secure Flag
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Cross-Domain JavaScript Source File Inclusion
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Web Browser XSS Protection Not Enabled
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Secure Pages Include Mixed Content
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Password Autocomplete in Browser
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Private IP Disclosure
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Session ID in URL Rewrite
6880 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: X-Content-Type-Options Header Missing
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: X-Frame-Options Header Scanner
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Charset Mismatch
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Loosely Scoped Cookie
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Absence of Anti-CSRF Tokens
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Information Disclosure - Debug Error Messages
6881 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Information Disclosure - Sensitive Informations in URL
6882 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Information Disclosure - Sensitive Information in HTTP Referrer Header
6882 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Information Disclosure - Suspicious Comments
6882 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Weak Authentication Method
6882 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Insecure JSF ViewState
6882 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: HTTP Parameter Override
6883 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: Viewstate Scanner
6883 [ZAP-daemon] INFO org.zaproxy.zap.extension.pscan.ExtensionPassiveScan - loaded passive scan rule: WSDL File Passive Scanner
6901 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows you to view and manage alerts
6901 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Active scanner, heavily based on the original Paros active scanner, but with additional tests added
6907 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionSequence
6910 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Spider used for automatically finding URIs on a site
6917 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing A set of common popup menus for miscellaneous tasks
6918 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Forced browsing of files and directories using code from the OWASP DirBuster tool
6920 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Simple but effective port scanner
6921 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionManualRequest
6922 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Compares 2 sessions and generates an HTML file showing the differences
6922 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Invoke external applications passing context related information such as URLs and parameters
6923 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Handles anti cross site request forgery (CSRF) tokens
6926 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionAuthentication
6955 [ZAP-daemon] INFO org.zaproxy.zap.extension.authentication.ExtensionAuthentication - Loaded authentication method types: [Form-based Authentication, HTTP/NTLM Authentication, Manual Authentication, Script-based Authentication]
6957 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Creates a dynamic SSL certificate to allow SSL communications to be intercepted without warnings being generated by the browser
6957 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Logs errors to the Output tab in development mode only
6957 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionUserManagement
6960 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Summarise and analyse FORM and URL parameters as well as cookies
6961 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Script integration
6978 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Scripting console, supports all JSR 223 scripting languages
6978 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionForcedUser
6979 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Extension handling HTTP sessions
6981 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Zest is a specialized scripting language from Mozilla specifically designed to be used in security tools
7786 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionDiff
7786 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionRequestPostTableView
7786 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Simple browser configuration
7787 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionSessionManagement
7794 [ZAP-daemon] INFO org.zaproxy.zap.extension.sessions.ExtensionSessionManagement - Loaded session management method types: [Cookie-based Session Management, HTTP Authentication Session Management]
7795 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelRequestFormTableView
7795 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Capture messages from WebSockets with the ability to set breakpoints.
7807 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows you to import a WSDL file containing operations which ZAP will access, adding them to the Sites tree.
7808 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Core UI related functionality.
7808 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionAuthorization
7809 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing AJAX Spider, uses Crawljax
7813 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Add-on that adds a set of tools for testing access control in web applications.
7814 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Handles adding Global Excluded URLs
7814 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Adds menu item to refresh the Sites tree
7814 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Adds support for configurable keyboard shortcuts for all of the ZAP menus.
7814 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing OWASP ZAP User Guide
7815 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows you to configure which extensions are loaded when ZAP starts
7815 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelComponentonentAll
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelHexView
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelImageView
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelLargeRequestView
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelLargeResponseView
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelRequestQueryCookieTableView
7816 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionHttpPanelSyntaxHighlightTextView
7817 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Active and passive rule configuration
7820 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Statistics
7824 [ZAP-daemon] INFO org.zaproxy.zap.extension.stats.ExtensionStats - Start recording in memory stats
7826 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Context alert rules filter
7829 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Active Scan Rules
7831 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Active Scan Rules - beta
7831 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Translations of the core language files
7831 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Provides the foundation for concrete message types (for example, HTTP, WebSockets) expose fuzzer implementations.
7835 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows to fuzz HTTP messages.
7836 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing The ZAP Getting Started Guide
7836 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing The Online menu links
7836 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Passive Scan Rules
7839 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Passive Scan Rules - beta
7840 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Quick Start panel
7840 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionSaveRawHttpMessage
7840 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Provides WebDrivers to control several browsers using Selenium and includes HtmlUnit browser.
7843 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Tips and Tricks
7844 [ZAP-daemon] INFO org.parosproxy.paros.extension.ExtensionLoader - Initializing Allows to fuzz WebSocket messages.
7888 [Thread-6] INFO org.zaproxy.zap.extension.dynssl.ExtensionDynSSL - Creating new root CA certificate
8751 [ZAP-daemon] INFO org.zaproxy.zap.DaemonBootstrap - ZAP is now listening on 127.0.0.1:2375
12360 [Thread-9] INFO org.zaproxy.zap.extension.spider.SpiderThread - Starting spidering scan on SpiderApi-0 at Wed Sep 28 20:22:52 UTC 2016
12371 [Thread-9] INFO org.zaproxy.zap.spider.Spider - Spider initializing...
12510 [Thread-9] INFO org.zaproxy.zap.spider.Spider - Starting spider...
13390 [pool-1-thread-1] WARN org.zaproxy.zap.spider.URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)
13924 [pool-1-thread-2] INFO org.zaproxy.zap.spider.Spider - Spidering process is complete. Shutting down...
13931 [Thread-10] INFO org.zaproxy.zap.extension.spider.SpiderThread - Spider scanning complete: true
22697 [ZAP-ProxyThread-7] INFO org.parosproxy.paros.core.scanner.Scanner - scanner started
22800 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - Scanning 8 node(s) from https://treeherder.allizom.org
22804 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestPathTraversal strength MEDIUM threshold MEDIUM
23425 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestPathTraversal in 0.621s
23426 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestRemoteFileInclude strength MEDIUM threshold MEDIUM
23869 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestRemoteFileInclude in 0.443s
23870 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestServerSideInclude strength MEDIUM threshold MEDIUM
24303 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestServerSideInclude in 0.434s
24303 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestCrossSiteScriptV2 strength MEDIUM threshold MEDIUM
24337 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestCrossSiteScriptV2 in 0.034s
24338 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestSQLInjection strength MEDIUM threshold MEDIUM
24377 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestSQLInjection in 0.039s
24377 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | CodeInjectionPlugin strength MEDIUM threshold MEDIUM
24809 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | CodeInjectionPlugin in 0.432s
24810 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | CommandInjectionPlugin strength MEDIUM threshold MEDIUM
25074 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | CommandInjectionPlugin in 0.264s
25075 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestDirectoryBrowsing strength MEDIUM threshold MEDIUM
25748 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestDirectoryBrowsing in 0.673s
25749 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestExternalRedirect strength MEDIUM threshold MEDIUM
25763 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestExternalRedirect in 0.014s
25763 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | BufferOverflow strength MEDIUM threshold MEDIUM
25783 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | BufferOverflow in 0.02s
25783 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | FormatString strength MEDIUM threshold MEDIUM
26001 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | FormatString in 0.218s
26002 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestInjectionCRLF strength MEDIUM threshold MEDIUM
26219 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestInjectionCRLF in 0.217s
26220 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestParameterTamper strength MEDIUM threshold MEDIUM
26234 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestParameterTamper in 0.014s
26234 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestPersistentXSSPrime strength MEDIUM threshold MEDIUM
26255 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestPersistentXSSPrime in 0.021s
26255 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestPersistentXSSSpider strength MEDIUM threshold MEDIUM
26952 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestPersistentXSSSpider in 0.696s
26952 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | TestPersistentXSSAttack strength MEDIUM threshold MEDIUM
26973 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | TestPersistentXSSAttack in 0.021s
26973 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | ScriptsActiveScanner strength MEDIUM threshold MEDIUM
26975 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - skipped plugin [no scripts enabled] https://treeherder.allizom.org | ScriptsActiveScanner in 0.002s
26975 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SourceCodeDisclosureSVN strength MEDIUM threshold MEDIUM
27216 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SourceCodeDisclosureSVN in 0.24s
27216 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SourceCodeDisclosureWEBINF strength MEDIUM threshold MEDIUM
27218 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | ShellShockScanner strength MEDIUM threshold MEDIUM
27277 [ZAP-ActiveScanner-0] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SourceCodeDisclosureWEBINF in 0.059s
27433 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | ShellShockScanner in 0.215s
27434 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | Csrftokenscan strength MEDIUM threshold MEDIUM
27450 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | Csrftokenscan in 0.017s
27450 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | HeartBleedActiveScanner strength MEDIUM threshold MEDIUM
27452 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | CrossDomainScanner strength MEDIUM threshold MEDIUM
27454 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SourceCodeDisclosureCVE20121823 strength MEDIUM threshold MEDIUM
27557 [ZAP-ActiveScanner-1] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | CrossDomainScanner in 0.105s
27665 [ZAP-ActiveScanner-0] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | HeartBleedActiveScanner in 0.215s
28420 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SourceCodeDisclosureCVE20121823 in 0.966s
28420 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | RemoteCodeExecutionCVE20121823 strength MEDIUM threshold MEDIUM
28632 [ZAP-ActiveScanner-1] ERROR org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823 - Error scanning a URL for Remote Code Execution via CVE-2012-1823: escaped absolute path not valid
org.apache.commons.httpclient.URIException: escaped absolute path not valid
at org.apache.commons.httpclient.URI.setRawPath(URI.java:2837)
at org.apache.commons.httpclient.URI.parseUriReference(URI.java:2023)
at org.apache.commons.httpclient.URI.(URI.java:167)
at org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823.scan(RemoteCodeExecutionCVE20121823.java:145)
at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:391)
at java.lang.Thread.run(Thread.java:745)
28834 [ZAP-ActiveScanner-1] ERROR org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823 - Error scanning a URL for Remote Code Execution via CVE-2012-1823: escaped absolute path not valid
org.apache.commons.httpclient.URIException: escaped absolute path not valid
at org.apache.commons.httpclient.URI.setRawPath(URI.java:2837)
at org.apache.commons.httpclient.URI.parseUriReference(URI.java:2023)
at org.apache.commons.httpclient.URI.(URI.java:167)
at org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823.scan(RemoteCodeExecutionCVE20121823.java:145)
at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:391)
at java.lang.Thread.run(Thread.java:745)
28843 [ZAP-ActiveScanner-0] ERROR org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823 - Error scanning a URL for Remote Code Execution via CVE-2012-1823: escaped absolute path not valid
org.apache.commons.httpclient.URIException: escaped absolute path not valid
at org.apache.commons.httpclient.URI.setRawPath(URI.java:2837)
at org.apache.commons.httpclient.URI.parseUriReference(URI.java:2023)
at org.apache.commons.httpclient.URI.(URI.java:167)
at org.zaproxy.zap.extension.ascanrulesBeta.RemoteCodeExecutionCVE20121823.scan(RemoteCodeExecutionCVE20121823.java:145)
at org.parosproxy.paros.core.scanner.AbstractPlugin.run(AbstractPlugin.java:391)
at java.lang.Thread.run(Thread.java:745)
29103 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | RemoteCodeExecutionCVE20121823 in 0.683s
29104 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SessionFixation strength MEDIUM threshold MEDIUM
29125 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SessionFixation in 0.021s
29126 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SQLInjectionMySQL strength MEDIUM threshold MEDIUM
29354 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SQLInjectionMySQL in 0.229s
29355 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SQLInjectionHypersonic strength MEDIUM threshold MEDIUM
29599 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SQLInjectionHypersonic in 0.245s
29600 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SQLInjectionOracle strength MEDIUM threshold MEDIUM
29647 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SQLInjectionOracle in 0.047s
29648 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SQLInjectionPostgresql strength MEDIUM threshold MEDIUM
29701 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SQLInjectionPostgresql in 0.053s
29702 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | XpathInjectionPlugin strength MEDIUM threshold MEDIUM
29760 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | XpathInjectionPlugin in 0.058s
29760 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | XXEPlugin strength MEDIUM threshold MEDIUM
30414 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | XXEPlugin in 0.654s
30415 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | PaddingOraclePlugin strength MEDIUM threshold MEDIUM
30438 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | PaddingOraclePlugin in 0.023s
30439 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | ExpressionLanguageInjectionPlugin strength MEDIUM threshold MEDIUM
30683 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | ExpressionLanguageInjectionPlugin in 0.244s
30683 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | BackupFileDisclosure strength MEDIUM threshold MEDIUM
31959 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | BackupFileDisclosure in 1.276s
31960 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | IntegerOverflow strength MEDIUM threshold MEDIUM
31984 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | IntegerOverflow in 0.025s
31984 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | InsecureHTTPMethod strength MEDIUM threshold MEDIUM
32645 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | InsecureHTTPMethod in 0.661s
32645 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | HPP strength MEDIUM threshold MEDIUM
32857 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | HPP in 0.212s
32858 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | UsernameEnumeration strength MEDIUM threshold MEDIUM
32859 [Thread-12] INFO org.zaproxy.zap.extension.ascanrulesBeta.UsernameEnumeration - There does not appear to be any configured contexts using Form-based Authentication. Further attempts during the current scan will be skipped.
32860 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - skipped plugin https://treeherder.allizom.org | UsernameEnumeration in 0.002s
32860 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SOAPActionSpoofingActiveScanner strength MEDIUM threshold MEDIUM
32869 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SOAPActionSpoofingActiveScanner in 0.009s
32869 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - start host https://treeherder.allizom.org | SOAPXMLInjectionActiveScanner strength MEDIUM threshold MEDIUM
32890 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host/plugin https://treeherder.allizom.org | SOAPXMLInjectionActiveScanner in 0.021s
32890 [Thread-12] INFO org.parosproxy.paros.core.scanner.HostProcess - completed host https://treeherder.allizom.org in 10.161s
32891 [Thread-11] INFO org.parosproxy.paros.core.scanner.Scanner - scanner completed in 10.194s
37347 [Thread-6] INFO org.zaproxy.zap.extension.dynssl.ExtensionDynSSL - New root CA certificate created
e5181426208a9e3f92b710443b29fb16ef213ee52c81e318474a975d25de2707
Finished: SUCCESS`
The text was updated successfully, but these errors were encountered: