New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support POST requests for API actions #2723
Comments
We actually have a TODO in the code for this ;) |
Worth noting that the parameters still need to be properly encoded even when using POST method (e.g. |
What would be the best way to implement it for the ZAP API clients? Should I create an optional parameter 'post' for each method or can I just change all actions from GET to POST directly? |
If you changed all actions to POST then we wouldnt be able to release the clients until a new version of ZAP was released. So I think the best approach is a non default option, or maybe implement but disable in the code? |
Support POST requests for API actions. Fixes #2723
* Show the cause why a script was not loaded Change ExtensionScript to provide more details why a script was not successfully loaded (e.g. missing script type, invalid character sequence, other unexpected causes). * Latest files from Crowdin * Show white space chars of matches in Search panel Change SearchResultTableEntry to replace the white space characters in the string found with visible equivalent characters so that the matches in the Search panel are like: Content-Length:·453¤¶Connection:·close¤¶Content-Type: instead of: Content-Length: 453Connection: closeContent-Type: which does not reflect the actual match. * Address JavaDoc issues Address JavaDoc issues in some classes, tidy up class JavaDoc, add missing docs on methods and parameters, replace closing HTML tags with starting tags and remove empty docs of overridden methods. * Correct offset calculation in text header views Extract the calculation of offsets for view to header and header to view into a class (HttpTextViewUtils) and changed the text views to use it, reduces code duplication and uses the correct calculations in all cases (some calculations were already correct). Add tests to assert the expected behaviour of HttpTextViewUtils. Fix zaproxy#2793 - Wrong highlight in combined view with last part of request header * Fix typos in API endpoint descriptions Change descriptions of core API endpoints sendRequest and sendHarRequest from "now allowed" to "not allowed" (to send the requests in Safe mode). * Use dev version of API and support -w wiki_report * Upgrade to use ubuntu:16.04 * Exclude brackets in URLs in spider parsers Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs * Ignore incorrect start/end positions on highlight Change HttpTextViewUtils to return invalid position, instead of throwing an exception, if the start or end positions are greater than the length of the view/header/body, otherwise it would result in an exception when the combined view is showing a custom message in the body (when body is too large to display per configurations), moreover it is more resilient to future changes in the content shown. Update the tests to assert the new behaviour. * Remove "debug" code in SpiderTextParser Remove "debug" code accidentally committed in a previous change. * Tweak JavaDoc in StandardFieldsDialog Add missing parameter descriptions and add docs for the constructors and some methods. * Tweak JavaDocs in class TabbedPanel2 Fix typo, add descriptions to parameters of documented methods and other minor tweaks. * Latest files from Crowdin * Do not use null string literal in ApiResponseSet Change ApiResponseSet to not use null string literal, in XML and HTML formats, when the values are null otherwise it might seem that the null values of the set have a value (i.e. "null"), instead of nothing. For example, an alert with no evidence/attack (null) would shown "null" instead of nothing. * Differentiate the source of alerts Change the Alert class to have a source "unknown", "active", "manual", "passive" and "tool" which is set just before the alert is raised for active, manual and passive alerts, for remaining alerts it's considered as raised by a tool (e.g. custom scripts). Old alerts of existing sessions will default to "unknown" since its exact source is not immediately known. Change Alert tab to shown the new field of the alert, "Source:". Change CoreAPI to return the ID of the source of the alerts. Change databases and corresponding tables to have the source. Change HTML, XML and MD reports to include the ID of the source of the alerts. Fix zaproxy#2592 - Differentiate the source of alerts * Do not access the EDT in daemon mode Change class ExtensionSearch to not access the EDT (and view classes) if the view is not initialised when the session changes by adding a "view" SessionChangedListener when there's a View. * Include the base URL in SpiderTextParser Change SpiderTextParser to include the base URL when processing the URLs found in the response so if there's an error in the processing of those URLs it's included the page were the problematic URL was found. Without the change it would be logged: Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://example.com) which does not give any information where the problematic URL was found. The inclusion of base URL does not affect how the URLs are resolved, the processed URLs are already absolute. Tweak the class URLCanonicalizer to include the base URL when debug logging URLs with no authority, for same reason. * Added filterchain to parse out new line characters Many editors automatically append new lines to the end of files. Without this change, a trailing new line in version.txt causes the jar created to attempt to have a newline character in the name. * Include the "faulty" URI in exception message Change class HttpMethodHelper to include the "faulty" URI in the exception message, to give more information about the problem when being handled by caller code. * Latest files from Crowdin * Added setup as a dependency for dist The dist build target requires ${zap.jar} to be set. When it is not, the build does not actually execute successfully. Adding setup as a dependency for dist fixes this problem, and fixes zaproxy#1921 Also removes zap.jar property from day-stamped-release to avoid confusion with setting the property twice. * Do not access EDT in daemon mode in Context class Change Context class to not access the EDT if the view is not initialised, when restructuring the sites tree. * Fixed incorrect String comparisons * Always set Java mem to 1/4 available (over 512Mb) * Change ZAP API to read/use the request body Change API class to read/use the request body as that might be required for some API endpoints (e.g. "other" which might use the whole HTTP request). * Attempt to determine (String) body's charset Change HttpBody and HttpResponseBody to attempt to determine the charset of the contents (String) being set if the charset is unknown (that is, it was not previously set before the contents are set). Update tests to reflect the change in the behaviour. Related to zaproxy#2487 - Wrong charset used in HTTP body Fix zaproxy#2935 - Wrong charset used in response body if no charset set * Minor tidy up in context related panels Add JavaDoc to constructors and other undocumented parameters. Merge "initialize" methods into the constructors (and remove commented statement). Correct the name of a parameter. * Move HTML parser's test files into its own dir Move the files used by SpiderHtmlParserUnitTest to its own directory to be more clear what the files are used for and by what parser. Update SpiderHtmlParserUnitTest to use the new directory. * Reuse test files of HTML form spider parser Change the files to have the method as a variable so that it can be used for both GET and POST forms. Update SpiderHtmlFormParserUnitTest to reflect the changes. * Set a name to spider threads Initialise the spider threads with custom name as it makes it easier to identify that are threads created by ZAP, know it's purpose and to know to which spider scan they belong. Also, correct the site/name show when starting the spider. * Create first Root CA certificate synchronously Change class ExtensionDynSSL to create the Root CA certificate synchronously to have the certificate ready for when the Local Proxy is started, otherwise it could fail to process immediate SSL/TLS requests. * Use non absolute URI base HTML element Change SpiderHtmlParser and SpiderHtmlFormParser to properly handle non absolute URI base HTML element. Update tests to reflect the change in the behaviour. Change form HTML base tests to reuse the same file (have HTML base and form action as variables). Fix zaproxy#2939 - Use non absolute URI base HTML element in spider * Delay addition of the context being imported Changed Session to only add the context being imported if no errors occurred while importing it, otherwise the context could be left in a potentially inconsistent state which could cause issues in other parts of the code (for example, if it had no name (i.e. null) it would no longer be possible to add a new context, delete the one imported or create a new session). * Allow to export a Context through the context menu Add a pop up menu item to the context menu of the contexts tree to allow to export the selected context. * Correct charset determination in HttpResponseBody Remove use of platform's default charset when determining if the charset of the string is UTF-8, which was leading to wrong results if the platform's default charset was not UTF-8. Related to: - zaproxy#2935 - Wrong charset used in response body if no charset set - zaproxy#2941 - Attempt to determine (String) body's charset * Initialise panels when added to session dialogue Initialise the panels when added to session dialogue if it's shown, to ensure that the panels are in a consistent state. Also, ensure the session dialogue has a "UI shared context" when adding the panels of the newly added context. The change prevents exceptions (caused by the inconsistent state of the panels) when changes are done to the contexts (e.g. via ZAP API) while the dialogue is shown. Change to initialise the "regular" (i.e. non context) panels only once, when initParam(Object) is called (already done by base class). * Show correct header when selected panel is removed Change AbstractParamContainerPanel to (explicitly) show the first available panel when the selected panel is removed, to show the correct information in the panel header, title and help button. Also, do not show the panel if already shown (skip notifications that the same panel is hidden and then shown, header setup and re-setting the panel in the layout). * Do not allow Contexts with same name Change GUI/API to not allow to: - Import or create a context with no name, with an empty name or with name that already exists; - Change the name of the context to be null, empty or that duplicates an existing name. Fix zaproxy#1952 - Do not allow Contexts with same name * Export context's session management data Change ExtensionSessionManagement to also export session management data when exporting the context (not a problem for core implementations which do not have any data). * Remove WAVSEP spider tests The WAVSEP spider tests are no longer maintained, also the (HTML) spider parsers have now good unit test coverage and the spider is regularly tested with WIVET (through zapbot scans). * Support POST requests for API actions. Fixes zaproxy#2723 * Use L&F specified through JVM args Change GuiBootstrap to use the look and feel specified through the JVM arguments if able to find/set it, otherwise fallback to previous/current behaviour. Related to zaproxy#2964 - Allow to select the look and feel * Increase page size when accessing alerts It turns out that the paging is not implemented very efficiently, and choosing too small a page size can take a very long time. * Support break functionality in the API * Do not initialise dev logger if there's no view Change ExtensionLog4j to not initialise the "logger" if there's no view, it was only used if the view was initialised. Change ZapOutputWriter to require the view initialised and that the scan status label is provided (and remove view and null checks when logging, no longer needed per previous changes). Also, remove unused constructor. * Init status label in attack scanner only with view Change AttackModeScanner to not initialise the scan status label if there's no view, it's not needed in daemon mode. Also, change to use long to track elapsed time, instead of Date, to not create the Date objects unnecessarily. * Include date/time when logging that ZAP started The date/time allows to correlate the output logging with other logs and events more easily. * Modifications to Enableable Within org.zaproxy.zap.utils: * Add interface EnableableInterface (Extracted from Enableable). * Enableable now implements EnableableInterface. Within org.zaproxy.zap.view: * AbstractMultipleOptionsTableModel now leverages EnableableInterface. * AbstractMultipleOptionsTablePanel now leverages EnableableInterface. * Change ScriptType to define if enabled by default Change ScriptType to allow to define if the scripts of the script type should be enabled by default (e.g. when added/loaded via GUI). Related to zaproxy#2970 - Allow to configure, by script type, the enabled state of new/loaded scripts * Delay init of attack mode scanner to prevent NPE Change ExtensionActiveScan to delay the initialisation of AttackModeScanner to allow it to properly check if the view is initialised, using the extension. Caused by zaproxy#2972 - Init status label in attack scanner only with view * Change attack mode thread to daemon Change the thread used for the attack mode to be a daemon thread, to not prevent ZAP from terminating normally. For example, if the attack mode was enabled while starting ZAP (in daemon mode) and ZAP was not able to bind to the address/port it would be kept running instead of terminating. * Fix exception when getting sessions through ZAP API Change HttpSessionsAPI to obtain the optional parameter "session" with a default value, otherwise it would lead to a JSONException if it was not present in the API request. Fix zaproxy#2977 - HTTP500 from JSON/httpSessions/view/sessions/?site=FOO * Allow to disable default standard output logging Add a command line flag to disable the default standard output logging, allowing to configure/override it using the log4j.properties file. Add tests to assert the expected behaviour. * Change API JS script to check if method is defined Change CoreAPI JavaScript script to check if the formMethod field is defined before using it as not all the API calls (e.g. views) use/define it, leading to errors. * Fix typo in resource message key Change the name of the resource message key to match the name of the package of the extension ("uiutils"). * Tweak error message checks in ProxyServer Change how the exception's message is checked as newer versions of Java might return different messages, e.g.: - Java 7, Address already in use - Java 8, Address already in use (Bind failed) to keep showing a specific error/info message to the user. * Do not warn about non active attack mode scans Change active scanner extension to not warn/show as active actions the attack mode scans that are not active (i.e. either already stopped or still running but not scanning any message). * Latest files from Crowdin * Restore HostProcess/Scanner constructors Restore and deprecate HostProcess/Scanner constructors to keep binary compatibility with current/previous version, eases migration to newer version as some (add-on) tests use those constructors. * Correctly render all nodes in checkbox tree Change JCheckBoxTree to correctly render the top level nodes, the renderer will not show the checkbox if the node has no checkbox state moreover set the node's text to the label wherever it has or not a checkbox. Also, change to create the checkbox state of the tree nodes before the model is set to the base class (as it might be used by base class for painting calculations, using the custom renderer). Update test to reflect the change in behaviour (no longer throws a NullPointerException when setting a null model). * Latest files from Crowdin * Allow to passive scan just HTTP messages in scope Add an option disabled by default, to GUI and API, that allows to set the passive scanner to scan only messages that are in scope. Fix zaproxy#3004 - Allow to passive scan just HTTP messages in scope * Clarify passive scanner's enabled state (API) Change the description of API endpoint "setEnabled" to clarify that the enabled state is not persisted (i.e. defaults to passive scan always). * Added jenkins plugin and bug bounty links * Restore PassiveScanThread constructor Restore and deprecate PassiveScanThread constructor to keep binary compatibility with current/previous version, currently being used in add-on tests (passive scanners). * Support Factory Reset Fixes zaproxy#2701 * Call postInit when starting an extension Change ExtensionLoader to call the method Extension.postInit() when starting an extension (i.e. installed by an add-on). The change ensures the extension is properly/fully initialised when it is started/installed (e.g. sequence extension which adds a custom scan panel on postInit()). * Update dependencies and license * Allow to active scan a Context through the ZAP API Change ActiveScanAPI to: - Allow to specify a context for the "scan" action; - Not require the URL, in the actions "scan" and "scanAsUser", if the context is specified (for the latter action it is always). Add helper method to ApiImplementor that validates that an API parameter exists. Fix zaproxy#1853 - Allow to active scan a Context through ZAP API * Restore API generator methods Restore (and deprecate) methods of the API generators to keep binary compatibility with current/previous version (they are in use by zap-extensions project). * Correct proxy errors' Content-Length value Change ProxyThread to use the byte length of the error message instead of the number of characters for the Content-Length header, they might not be the same. Also, reorder the statements that set the headers to not need to guess the charset of the body being set. * Remove alerts.xml file Remove alerts.xml file, its contents (i.e. alerts' data) are not used nor maintained. Move the registry of the scanners IDs to a new file, scanners.md, which was previously in the alerts.xml (as XML comment). * Return request's type through the ZAP API Change ZAP API actions/views to include the type ID of the request (e.g. proxy, manual, spider, active) when returning the data of the HTTP message(s). Add JavaDoc to ApiResponseConversionUtils and made other minor changes (change logger variable to a constant and made class final). Update tests to check that the type is being set/used. * Add Spider URIs, to the UI, in the EDT Change the SpiderThread to add the URIs found to the UI in the EDT, to prevent concurrency issues between other threads and the EDT, e.g.: java.lang.NullPointerException at JTable.sortedTableChanged(JTable.java:4129) at JTable.tableChanged(JTable.java:4395) at JXTable.tableChanged(JXTable.java:1561) at AbstractTableModel.fireTableChanged(AbstractTableModel.java:296) at AbstractTableModel.fireTableRowsInserted(...) at o.z.z.extension.spider.SpiderPanelTableModel.addScanResult(...) at o.z.z.extension.spider.SpiderThread.foundURI(Unknown Source) at o.z.z.spider.Spider.notifyListenersFoundURI(Unknown Source) at o.z.z.spider.SpiderController.addSeed(Unknown Source) at o.z.z.spider.Spider.start(Unknown Source) at o.z.z.extension.spider.SpiderThread.startSpider(Unknown Source) at o.z.z.extension.spider.SpiderThread.runScan(Unknown Source) at o.z.z.extension.spider.SpiderThread.run(Unknown Source) (packages reduced/omitted to keep the lines short) Also, do not create the SpiderPanelTableModel if there's no view. Remove the synchronisation in SpiderPanelTableModel as that's not required, the model is accessed only through the EDT. * Fix concurrency issues when publishing ZAP events Change SimpleEventBus to control the read/write accesses to the publishers and consumers to prevent concurrency issues. For example, when a consumer is unregistered while publishing events, which could lead to exceptions, e.g.: java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(...) at java.util.ArrayList$Itr.next(...) at o.z.z.eventBus.SimpleEventBus.publishSyncEvent(...) at o.z.z.extension.alert.ExtensionAlert.publishAlertEvent(...) at o.z.z.extension.alert.ExtensionAlert.alertFound(...) at o.z.z.extension.pscan.PassiveScanThread.raiseAlert(...) (packages reduced/omitted to keep the lines short) Only one thread (write access) is allowed to manage the publishers and consumers while multiple threads can publish events (read access), as long no thread is managing the publishers or the consumers. Change the classes RegisteredConsumer and RegisteredPublisher to be static as they don't need to access the state of SimpleEventBus class. * Allow to select multiple parameters in Params tab Change ParamsPanel to allow to select multiple parameters (rows). Change Params tab pop up menus to be enabled only when one of the parameters is selected (to keep the same behaviour). Related to zaproxy#3040 - Export param tab contents * Add Spider scans to GUI in the EDT Change ExtensionSpider to add the spider scans to the GUI in the EDT, to prevent inconsistencies between EDT and other threads, which could lead to exceptions (and a freeze of GUI caused by inconsistent internal state of UI components). Change SpiderScan to not create the model when adding messages if the scan was already cleared, to prevent a leak of AlertEventConsumer(s). * Expose constants of core rule configurations Change RuleConfigParam to expose constants to access the core rule configurations when active/passive scanning. * Unit tests for the UsernamePasswordAuthenticationCredentials class * Return requests' timestamp/RTT through the ZAP API Change ApiResponseConversionUtils to also return the timestamp and RTT of the HTTP message. Update test to assert the returned data. * Add (some) JavaDoc to ScripType Add JavaDoc to the class and to capability related constant/methods. * Correct the loading of extensions' enabled state Change ExtensionFactory to use the ExtensionParam to obtain the enabled state of the extensions (which uses the new/correct configuration keys). Change ExtensionParam to allow to query the enabled state of an extension and change to use a map to keep the enabled states. Move ExtensionParam to OptionsParam as it needs to be early available for core code to use (i.e. ExtensionFactory). Update tests to assert the new behaviour. Issue introduced in zaproxy#2245 - Convert options to not use extensions' names as XML element names * Tweak log message in URLCanonicalizer Change a log message to include the URL that is being processed, also change to return immediately if the the URL is not valid after logging the problem (instead of throwing an exception, which would be caught (and logged) in the same method). * Clear old contexts, always, when loading a session Change Session to remove all the contexts before refreshing the UI when discarding the contexts, otherwise the contexts tree would have the contexts of the previous session if the loaded session had none. * Add initiator constant for AJAX spider requests Add a constant to HttpSender class for requests sent by the AJAX spider. Update the JavaScript HTTP Sender template script with the new constant. * Allow to extend ProxyThread Change ProxyThread to allow to be extended (from other packages) and use a custom HttpSender, required for the AJAX Spider to use a custom initiator ID. * Add tests for OptionsParamApi Add tests for OptionsParamApi to assert the expected behaviour. Also, do other tweaks to OptionsParamApi: - Remove commented code and related constant (unimplemented option); - Remove initialisations with default value and initialise the enabled instance variable as true (default value used when loading from file); - Properly handle malformed values in the configuration file; - Do not attempt to set and save the API key if the configurations was not set. * Support active scan rule and scan max duration Fixes zaproxy#2951 * Stop the spider scan if failed to properly start Change SpiderThread to stop the spider scan on exceptions during the starting process, to prevent the spider scan from becoming in undefined state (that is, not fully started nor stopped). Related to issues like zaproxy#3039. * Add initiator constant for Forced Browse requests Add a constant to HttpSender class for requests sent by the Forced Browse add-on. Related to zaproxy#3060 - Send Forced Browse requests through ZAP * Allow to deprecate ZAP API endpoints Change the ZAP API to allow to set its endpoints as deprecated (and add a description why they are). Change ZAP API UI to show a note when the endpoints are deprecated. Change JAVA API generator to annotate and add JavaDoc tag to deprecated endpoints. Fix zaproxy#3061 - Allow to deprecate API endpoints * Skip process automated msgs for HTTP Sessions tab Change class ExtensionHttpSessions to skip/ignore the responses of AJAX Spider and Forced Browse, as with other automated responses they should not be processed (would end up creating a lot of unnecessary sessions). Related to zaproxy#2674 - Automated authentication requests shown in HTTP Sessions tab * Expose add-on's file extension Change AddOn class to expose a constant for the file extension. Replace the literal string, in AddOn and ExtensionAutoUpdate, with the constant created. * Added cookie ignore list rule and inc sleep default to 20 to reduce FPs * Allow to show only bytes in HTTP message tables Add a check menu item to the context menu of the tables that show HTTP messages to allow to switch between just showing bytes (the new default) and other byte units (e.g. KiB, MiB). Fix zaproxy#2994 - show column 'Size Resp. Body' of history in bytes * Latest files from Crowdin * Log the name of the user of the active scan Change HostProcess to include the name of the user (if any) when logging the information of the scan being started. That information is useful when reviewing what the scan was doing (or, expected to do). * Latest files from Crowdin * Latest files from Crowdin * Update test_zap.config typo fixes * Latest files from Crowdin * ProxyThread SocketTimeoutException Verbosity Only log full exception if debug is enabled. Fixes zaproxy#3095 * Change policy's threshold/strength with ZAP API Add 2 optional params to AddScanPolicy api, default to medium level for AlertThreshold and AttackStrength. Add UpdateScanPolicy api, could change AttackStrength and AlertThreshold for a policy. * Log to file even if ZAP is run 'inline' Change CommandLineBootstrap to not disable the logging (to log to file by default), also, log when ZAP is started. It's useful to know what ZAP is doing or did. * fix mojibake HTML Report * Latest files from Crowdin * Do not set the "in scope" state in Contexts panel Change ContextListPanel to not set the "in scope" state to the contexts as that might (depending on the internal order of the panels) override the value set by/in ContextGeneralPanel. The ContextListPanel does not allow to change the "in scope" state so it should not be setting it. Fix zaproxy#3100 - Context's in scope change might not be applied * Add "Max children to crawl" to main spider options Add the (advanced) option "Maximum children to crawl" to main Spider options panel. The option is now available in both places (as the other advanced options). Fix zaproxy#3066 - Spidering options in the doc in two place * Do not require status/version in add-on file name Change AddOn class to not require the status and/or the version in the file name of the add-on. The add-on file name just needs to have the ID and have a ZAP extension. Also, deprecate old constructor/methods that require the file name to have the status/version and introduce new constructor/methods where applicable. Change BaseZapAddOnXmlData to read the status from the manifest file of the add-on (ZapAddOn.xml). Remove hardcoded manifest file name (ZapAddOn.xml) from JavaDoc and code (by using the constant from AddOn). Change AddOnCollection to iterate just ZAP add-on files and use the new AddOn constructor. Change ExtensionAutoUpdate to use the new constructors/methods and to copy the file from manual add-on installations using a normalised file name. Add some tests to assert the expected behaviour of AddOn class. Fix zaproxy#3090 - Be more lenient on add-on's file name format * Change default time to 15 and make publicly accessible * Latest files from Crowdin * Deprecate unused Spider menu items Deprecate unused Spider menu items (replaced by the Spider dialogue menu item). * Latest files from Crowdin * Correct location of i18n messages Swap the contents of two i18n messages, the title was being used as message and the message as title of the dialogue. * include SubjectAlternativeName extension in generated certificates * Add description to some Spider API endpoints Add descriptions to some of the Spider API endpoints and correct one that was wrong (it was for the action not view). * Log during start up the add-ons that are installed Change ExtensionFactory to log (as info) the IDs and version of the add-ons that are in installed state (all dependencies/requirements are fulfilled). * Add description to some core/ascan API endpoints Add description to some core and ascan API endpoints. * Latest files from Crowdin * Added security annotations for forms that dont need anti CSRF tokens * Latest files from Crowdin * Add description to active scan ZAP API option Add description to the active scan ZAP API option "Inject plugin ID in header for all active scan requests". Related to zaproxy#3133 - how disable send X-ZAP-Scan-ID header * Latest files from Crowdin
* Show the cause why a script was not loaded Change ExtensionScript to provide more details why a script was not successfully loaded (e.g. missing script type, invalid character sequence, other unexpected causes). * Latest files from Crowdin * Show white space chars of matches in Search panel Change SearchResultTableEntry to replace the white space characters in the string found with visible equivalent characters so that the matches in the Search panel are like: Content-Length:·453¤¶Connection:·close¤¶Content-Type: instead of: Content-Length: 453Connection: closeContent-Type: which does not reflect the actual match. * Address JavaDoc issues Address JavaDoc issues in some classes, tidy up class JavaDoc, add missing docs on methods and parameters, replace closing HTML tags with starting tags and remove empty docs of overridden methods. * Correct offset calculation in text header views Extract the calculation of offsets for view to header and header to view into a class (HttpTextViewUtils) and changed the text views to use it, reduces code duplication and uses the correct calculations in all cases (some calculations were already correct). Add tests to assert the expected behaviour of HttpTextViewUtils. Fix zaproxy#2793 - Wrong highlight in combined view with last part of request header * Fix typos in API endpoint descriptions Change descriptions of core API endpoints sendRequest and sendHarRequest from "now allowed" to "not allowed" (to send the requests in Safe mode). * Use dev version of API and support -w wiki_report * Upgrade to use ubuntu:16.04 * Exclude brackets in URLs in spider parsers Change the regular expressions for HTML comments, in SpiderHtmlParser, and for text, in SpiderTextParser, to exclude brackets (which are not expected to be (decoded) in the URLs). Also, change the regular expression of SpiderTextParser to be case insensitive. Add tests to assert the expected behaviour of SpiderTextParser. Update test of SpiderHtmlParser. Fix zaproxy#2898 - Tweak spider parser to ignore/strip matched parenthesis around URLs * Ignore incorrect start/end positions on highlight Change HttpTextViewUtils to return invalid position, instead of throwing an exception, if the start or end positions are greater than the length of the view/header/body, otherwise it would result in an exception when the combined view is showing a custom message in the body (when body is too large to display per configurations), moreover it is more resilient to future changes in the content shown. Update the tests to assert the new behaviour. * Remove "debug" code in SpiderTextParser Remove "debug" code accidentally committed in a previous change. * Tweak JavaDoc in StandardFieldsDialog Add missing parameter descriptions and add docs for the constructors and some methods. * Tweak JavaDocs in class TabbedPanel2 Fix typo, add descriptions to parameters of documented methods and other minor tweaks. * Latest files from Crowdin * Do not use null string literal in ApiResponseSet Change ApiResponseSet to not use null string literal, in XML and HTML formats, when the values are null otherwise it might seem that the null values of the set have a value (i.e. "null"), instead of nothing. For example, an alert with no evidence/attack (null) would shown "null" instead of nothing. * Differentiate the source of alerts Change the Alert class to have a source "unknown", "active", "manual", "passive" and "tool" which is set just before the alert is raised for active, manual and passive alerts, for remaining alerts it's considered as raised by a tool (e.g. custom scripts). Old alerts of existing sessions will default to "unknown" since its exact source is not immediately known. Change Alert tab to shown the new field of the alert, "Source:". Change CoreAPI to return the ID of the source of the alerts. Change databases and corresponding tables to have the source. Change HTML, XML and MD reports to include the ID of the source of the alerts. Fix zaproxy#2592 - Differentiate the source of alerts * Do not access the EDT in daemon mode Change class ExtensionSearch to not access the EDT (and view classes) if the view is not initialised when the session changes by adding a "view" SessionChangedListener when there's a View. * Include the base URL in SpiderTextParser Change SpiderTextParser to include the base URL when processing the URLs found in the response so if there's an error in the processing of those URLs it's included the page were the problematic URL was found. Without the change it would be logged: Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://example.com) which does not give any information where the problematic URL was found. The inclusion of base URL does not affect how the URLs are resolved, the processed URLs are already absolute. Tweak the class URLCanonicalizer to include the base URL when debug logging URLs with no authority, for same reason. * Added filterchain to parse out new line characters Many editors automatically append new lines to the end of files. Without this change, a trailing new line in version.txt causes the jar created to attempt to have a newline character in the name. * Include the "faulty" URI in exception message Change class HttpMethodHelper to include the "faulty" URI in the exception message, to give more information about the problem when being handled by caller code. * Latest files from Crowdin * Added setup as a dependency for dist The dist build target requires ${zap.jar} to be set. When it is not, the build does not actually execute successfully. Adding setup as a dependency for dist fixes this problem, and fixes zaproxy#1921 Also removes zap.jar property from day-stamped-release to avoid confusion with setting the property twice. * Do not access EDT in daemon mode in Context class Change Context class to not access the EDT if the view is not initialised, when restructuring the sites tree. * Fixed incorrect String comparisons * Always set Java mem to 1/4 available (over 512Mb) * Change ZAP API to read/use the request body Change API class to read/use the request body as that might be required for some API endpoints (e.g. "other" which might use the whole HTTP request). * Attempt to determine (String) body's charset Change HttpBody and HttpResponseBody to attempt to determine the charset of the contents (String) being set if the charset is unknown (that is, it was not previously set before the contents are set). Update tests to reflect the change in the behaviour. Related to zaproxy#2487 - Wrong charset used in HTTP body Fix zaproxy#2935 - Wrong charset used in response body if no charset set * Minor tidy up in context related panels Add JavaDoc to constructors and other undocumented parameters. Merge "initialize" methods into the constructors (and remove commented statement). Correct the name of a parameter. * Move HTML parser's test files into its own dir Move the files used by SpiderHtmlParserUnitTest to its own directory to be more clear what the files are used for and by what parser. Update SpiderHtmlParserUnitTest to use the new directory. * Reuse test files of HTML form spider parser Change the files to have the method as a variable so that it can be used for both GET and POST forms. Update SpiderHtmlFormParserUnitTest to reflect the changes. * Set a name to spider threads Initialise the spider threads with custom name as it makes it easier to identify that are threads created by ZAP, know it's purpose and to know to which spider scan they belong. Also, correct the site/name show when starting the spider. * Create first Root CA certificate synchronously Change class ExtensionDynSSL to create the Root CA certificate synchronously to have the certificate ready for when the Local Proxy is started, otherwise it could fail to process immediate SSL/TLS requests. * Use non absolute URI base HTML element Change SpiderHtmlParser and SpiderHtmlFormParser to properly handle non absolute URI base HTML element. Update tests to reflect the change in the behaviour. Change form HTML base tests to reuse the same file (have HTML base and form action as variables). Fix zaproxy#2939 - Use non absolute URI base HTML element in spider * Delay addition of the context being imported Changed Session to only add the context being imported if no errors occurred while importing it, otherwise the context could be left in a potentially inconsistent state which could cause issues in other parts of the code (for example, if it had no name (i.e. null) it would no longer be possible to add a new context, delete the one imported or create a new session). * Allow to export a Context through the context menu Add a pop up menu item to the context menu of the contexts tree to allow to export the selected context. * Correct charset determination in HttpResponseBody Remove use of platform's default charset when determining if the charset of the string is UTF-8, which was leading to wrong results if the platform's default charset was not UTF-8. Related to: - zaproxy#2935 - Wrong charset used in response body if no charset set - zaproxy#2941 - Attempt to determine (String) body's charset * Initialise panels when added to session dialogue Initialise the panels when added to session dialogue if it's shown, to ensure that the panels are in a consistent state. Also, ensure the session dialogue has a "UI shared context" when adding the panels of the newly added context. The change prevents exceptions (caused by the inconsistent state of the panels) when changes are done to the contexts (e.g. via ZAP API) while the dialogue is shown. Change to initialise the "regular" (i.e. non context) panels only once, when initParam(Object) is called (already done by base class). * Show correct header when selected panel is removed Change AbstractParamContainerPanel to (explicitly) show the first available panel when the selected panel is removed, to show the correct information in the panel header, title and help button. Also, do not show the panel if already shown (skip notifications that the same panel is hidden and then shown, header setup and re-setting the panel in the layout). * Do not allow Contexts with same name Change GUI/API to not allow to: - Import or create a context with no name, with an empty name or with name that already exists; - Change the name of the context to be null, empty or that duplicates an existing name. Fix zaproxy#1952 - Do not allow Contexts with same name * Export context's session management data Change ExtensionSessionManagement to also export session management data when exporting the context (not a problem for core implementations which do not have any data). * Remove WAVSEP spider tests The WAVSEP spider tests are no longer maintained, also the (HTML) spider parsers have now good unit test coverage and the spider is regularly tested with WIVET (through zapbot scans). * Support POST requests for API actions. Fixes zaproxy#2723 * Use L&F specified through JVM args Change GuiBootstrap to use the look and feel specified through the JVM arguments if able to find/set it, otherwise fallback to previous/current behaviour. Related to zaproxy#2964 - Allow to select the look and feel * Increase page size when accessing alerts It turns out that the paging is not implemented very efficiently, and choosing too small a page size can take a very long time. * Support break functionality in the API * Do not initialise dev logger if there's no view Change ExtensionLog4j to not initialise the "logger" if there's no view, it was only used if the view was initialised. Change ZapOutputWriter to require the view initialised and that the scan status label is provided (and remove view and null checks when logging, no longer needed per previous changes). Also, remove unused constructor. * Init status label in attack scanner only with view Change AttackModeScanner to not initialise the scan status label if there's no view, it's not needed in daemon mode. Also, change to use long to track elapsed time, instead of Date, to not create the Date objects unnecessarily. * Include date/time when logging that ZAP started The date/time allows to correlate the output logging with other logs and events more easily. * Modifications to Enableable Within org.zaproxy.zap.utils: * Add interface EnableableInterface (Extracted from Enableable). * Enableable now implements EnableableInterface. Within org.zaproxy.zap.view: * AbstractMultipleOptionsTableModel now leverages EnableableInterface. * AbstractMultipleOptionsTablePanel now leverages EnableableInterface. * Change ScriptType to define if enabled by default Change ScriptType to allow to define if the scripts of the script type should be enabled by default (e.g. when added/loaded via GUI). Related to zaproxy#2970 - Allow to configure, by script type, the enabled state of new/loaded scripts * Delay init of attack mode scanner to prevent NPE Change ExtensionActiveScan to delay the initialisation of AttackModeScanner to allow it to properly check if the view is initialised, using the extension. Caused by zaproxy#2972 - Init status label in attack scanner only with view * Change attack mode thread to daemon Change the thread used for the attack mode to be a daemon thread, to not prevent ZAP from terminating normally. For example, if the attack mode was enabled while starting ZAP (in daemon mode) and ZAP was not able to bind to the address/port it would be kept running instead of terminating. * Fix exception when getting sessions through ZAP API Change HttpSessionsAPI to obtain the optional parameter "session" with a default value, otherwise it would lead to a JSONException if it was not present in the API request. Fix zaproxy#2977 - HTTP500 from JSON/httpSessions/view/sessions/?site=FOO * Allow to disable default standard output logging Add a command line flag to disable the default standard output logging, allowing to configure/override it using the log4j.properties file. Add tests to assert the expected behaviour. * Change API JS script to check if method is defined Change CoreAPI JavaScript script to check if the formMethod field is defined before using it as not all the API calls (e.g. views) use/define it, leading to errors. * Fix typo in resource message key Change the name of the resource message key to match the name of the package of the extension ("uiutils"). * Tweak error message checks in ProxyServer Change how the exception's message is checked as newer versions of Java might return different messages, e.g.: - Java 7, Address already in use - Java 8, Address already in use (Bind failed) to keep showing a specific error/info message to the user. * Do not warn about non active attack mode scans Change active scanner extension to not warn/show as active actions the attack mode scans that are not active (i.e. either already stopped or still running but not scanning any message). * Latest files from Crowdin * Restore HostProcess/Scanner constructors Restore and deprecate HostProcess/Scanner constructors to keep binary compatibility with current/previous version, eases migration to newer version as some (add-on) tests use those constructors. * Correctly render all nodes in checkbox tree Change JCheckBoxTree to correctly render the top level nodes, the renderer will not show the checkbox if the node has no checkbox state moreover set the node's text to the label wherever it has or not a checkbox. Also, change to create the checkbox state of the tree nodes before the model is set to the base class (as it might be used by base class for painting calculations, using the custom renderer). Update test to reflect the change in behaviour (no longer throws a NullPointerException when setting a null model). * Latest files from Crowdin * Allow to passive scan just HTTP messages in scope Add an option disabled by default, to GUI and API, that allows to set the passive scanner to scan only messages that are in scope. Fix zaproxy#3004 - Allow to passive scan just HTTP messages in scope * Clarify passive scanner's enabled state (API) Change the description of API endpoint "setEnabled" to clarify that the enabled state is not persisted (i.e. defaults to passive scan always). * Added jenkins plugin and bug bounty links * Restore PassiveScanThread constructor Restore and deprecate PassiveScanThread constructor to keep binary compatibility with current/previous version, currently being used in add-on tests (passive scanners). * Support Factory Reset Fixes zaproxy#2701 * Call postInit when starting an extension Change ExtensionLoader to call the method Extension.postInit() when starting an extension (i.e. installed by an add-on). The change ensures the extension is properly/fully initialised when it is started/installed (e.g. sequence extension which adds a custom scan panel on postInit()). * Update dependencies and license * Allow to active scan a Context through the ZAP API Change ActiveScanAPI to: - Allow to specify a context for the "scan" action; - Not require the URL, in the actions "scan" and "scanAsUser", if the context is specified (for the latter action it is always). Add helper method to ApiImplementor that validates that an API parameter exists. Fix zaproxy#1853 - Allow to active scan a Context through ZAP API * Restore API generator methods Restore (and deprecate) methods of the API generators to keep binary compatibility with current/previous version (they are in use by zap-extensions project). * Correct proxy errors' Content-Length value Change ProxyThread to use the byte length of the error message instead of the number of characters for the Content-Length header, they might not be the same. Also, reorder the statements that set the headers to not need to guess the charset of the body being set. * Remove alerts.xml file Remove alerts.xml file, its contents (i.e. alerts' data) are not used nor maintained. Move the registry of the scanners IDs to a new file, scanners.md, which was previously in the alerts.xml (as XML comment). * Return request's type through the ZAP API Change ZAP API actions/views to include the type ID of the request (e.g. proxy, manual, spider, active) when returning the data of the HTTP message(s). Add JavaDoc to ApiResponseConversionUtils and made other minor changes (change logger variable to a constant and made class final). Update tests to check that the type is being set/used. * Add Spider URIs, to the UI, in the EDT Change the SpiderThread to add the URIs found to the UI in the EDT, to prevent concurrency issues between other threads and the EDT, e.g.: java.lang.NullPointerException at JTable.sortedTableChanged(JTable.java:4129) at JTable.tableChanged(JTable.java:4395) at JXTable.tableChanged(JXTable.java:1561) at AbstractTableModel.fireTableChanged(AbstractTableModel.java:296) at AbstractTableModel.fireTableRowsInserted(...) at o.z.z.extension.spider.SpiderPanelTableModel.addScanResult(...) at o.z.z.extension.spider.SpiderThread.foundURI(Unknown Source) at o.z.z.spider.Spider.notifyListenersFoundURI(Unknown Source) at o.z.z.spider.SpiderController.addSeed(Unknown Source) at o.z.z.spider.Spider.start(Unknown Source) at o.z.z.extension.spider.SpiderThread.startSpider(Unknown Source) at o.z.z.extension.spider.SpiderThread.runScan(Unknown Source) at o.z.z.extension.spider.SpiderThread.run(Unknown Source) (packages reduced/omitted to keep the lines short) Also, do not create the SpiderPanelTableModel if there's no view. Remove the synchronisation in SpiderPanelTableModel as that's not required, the model is accessed only through the EDT. * Fix concurrency issues when publishing ZAP events Change SimpleEventBus to control the read/write accesses to the publishers and consumers to prevent concurrency issues. For example, when a consumer is unregistered while publishing events, which could lead to exceptions, e.g.: java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(...) at java.util.ArrayList$Itr.next(...) at o.z.z.eventBus.SimpleEventBus.publishSyncEvent(...) at o.z.z.extension.alert.ExtensionAlert.publishAlertEvent(...) at o.z.z.extension.alert.ExtensionAlert.alertFound(...) at o.z.z.extension.pscan.PassiveScanThread.raiseAlert(...) (packages reduced/omitted to keep the lines short) Only one thread (write access) is allowed to manage the publishers and consumers while multiple threads can publish events (read access), as long no thread is managing the publishers or the consumers. Change the classes RegisteredConsumer and RegisteredPublisher to be static as they don't need to access the state of SimpleEventBus class. * Allow to select multiple parameters in Params tab Change ParamsPanel to allow to select multiple parameters (rows). Change Params tab pop up menus to be enabled only when one of the parameters is selected (to keep the same behaviour). Related to zaproxy#3040 - Export param tab contents * Add Spider scans to GUI in the EDT Change ExtensionSpider to add the spider scans to the GUI in the EDT, to prevent inconsistencies between EDT and other threads, which could lead to exceptions (and a freeze of GUI caused by inconsistent internal state of UI components). Change SpiderScan to not create the model when adding messages if the scan was already cleared, to prevent a leak of AlertEventConsumer(s). * Expose constants of core rule configurations Change RuleConfigParam to expose constants to access the core rule configurations when active/passive scanning. * Unit tests for the UsernamePasswordAuthenticationCredentials class * Return requests' timestamp/RTT through the ZAP API Change ApiResponseConversionUtils to also return the timestamp and RTT of the HTTP message. Update test to assert the returned data. * Add (some) JavaDoc to ScripType Add JavaDoc to the class and to capability related constant/methods. * Correct the loading of extensions' enabled state Change ExtensionFactory to use the ExtensionParam to obtain the enabled state of the extensions (which uses the new/correct configuration keys). Change ExtensionParam to allow to query the enabled state of an extension and change to use a map to keep the enabled states. Move ExtensionParam to OptionsParam as it needs to be early available for core code to use (i.e. ExtensionFactory). Update tests to assert the new behaviour. Issue introduced in zaproxy#2245 - Convert options to not use extensions' names as XML element names * Tweak log message in URLCanonicalizer Change a log message to include the URL that is being processed, also change to return immediately if the the URL is not valid after logging the problem (instead of throwing an exception, which would be caught (and logged) in the same method). * Clear old contexts, always, when loading a session Change Session to remove all the contexts before refreshing the UI when discarding the contexts, otherwise the contexts tree would have the contexts of the previous session if the loaded session had none. * Add initiator constant for AJAX spider requests Add a constant to HttpSender class for requests sent by the AJAX spider. Update the JavaScript HTTP Sender template script with the new constant. * Allow to extend ProxyThread Change ProxyThread to allow to be extended (from other packages) and use a custom HttpSender, required for the AJAX Spider to use a custom initiator ID. * Add tests for OptionsParamApi Add tests for OptionsParamApi to assert the expected behaviour. Also, do other tweaks to OptionsParamApi: - Remove commented code and related constant (unimplemented option); - Remove initialisations with default value and initialise the enabled instance variable as true (default value used when loading from file); - Properly handle malformed values in the configuration file; - Do not attempt to set and save the API key if the configurations was not set. * Support active scan rule and scan max duration Fixes zaproxy#2951 * Stop the spider scan if failed to properly start Change SpiderThread to stop the spider scan on exceptions during the starting process, to prevent the spider scan from becoming in undefined state (that is, not fully started nor stopped). Related to issues like zaproxy#3039. * Add initiator constant for Forced Browse requests Add a constant to HttpSender class for requests sent by the Forced Browse add-on. Related to zaproxy#3060 - Send Forced Browse requests through ZAP * Allow to deprecate ZAP API endpoints Change the ZAP API to allow to set its endpoints as deprecated (and add a description why they are). Change ZAP API UI to show a note when the endpoints are deprecated. Change JAVA API generator to annotate and add JavaDoc tag to deprecated endpoints. Fix zaproxy#3061 - Allow to deprecate API endpoints * Skip process automated msgs for HTTP Sessions tab Change class ExtensionHttpSessions to skip/ignore the responses of AJAX Spider and Forced Browse, as with other automated responses they should not be processed (would end up creating a lot of unnecessary sessions). Related to zaproxy#2674 - Automated authentication requests shown in HTTP Sessions tab * Expose add-on's file extension Change AddOn class to expose a constant for the file extension. Replace the literal string, in AddOn and ExtensionAutoUpdate, with the constant created. * Added cookie ignore list rule and inc sleep default to 20 to reduce FPs * Allow to show only bytes in HTTP message tables Add a check menu item to the context menu of the tables that show HTTP messages to allow to switch between just showing bytes (the new default) and other byte units (e.g. KiB, MiB). Fix zaproxy#2994 - show column 'Size Resp. Body' of history in bytes * Latest files from Crowdin * Log the name of the user of the active scan Change HostProcess to include the name of the user (if any) when logging the information of the scan being started. That information is useful when reviewing what the scan was doing (or, expected to do). * Latest files from Crowdin * Latest files from Crowdin * Update test_zap.config typo fixes * Latest files from Crowdin * ProxyThread SocketTimeoutException Verbosity Only log full exception if debug is enabled. Fixes zaproxy#3095 * Change policy's threshold/strength with ZAP API Add 2 optional params to AddScanPolicy api, default to medium level for AlertThreshold and AttackStrength. Add UpdateScanPolicy api, could change AttackStrength and AlertThreshold for a policy. * Log to file even if ZAP is run 'inline' Change CommandLineBootstrap to not disable the logging (to log to file by default), also, log when ZAP is started. It's useful to know what ZAP is doing or did. * fix mojibake HTML Report * Latest files from Crowdin * Do not set the "in scope" state in Contexts panel Change ContextListPanel to not set the "in scope" state to the contexts as that might (depending on the internal order of the panels) override the value set by/in ContextGeneralPanel. The ContextListPanel does not allow to change the "in scope" state so it should not be setting it. Fix zaproxy#3100 - Context's in scope change might not be applied * Add "Max children to crawl" to main spider options Add the (advanced) option "Maximum children to crawl" to main Spider options panel. The option is now available in both places (as the other advanced options). Fix zaproxy#3066 - Spidering options in the doc in two place * Do not require status/version in add-on file name Change AddOn class to not require the status and/or the version in the file name of the add-on. The add-on file name just needs to have the ID and have a ZAP extension. Also, deprecate old constructor/methods that require the file name to have the status/version and introduce new constructor/methods where applicable. Change BaseZapAddOnXmlData to read the status from the manifest file of the add-on (ZapAddOn.xml). Remove hardcoded manifest file name (ZapAddOn.xml) from JavaDoc and code (by using the constant from AddOn). Change AddOnCollection to iterate just ZAP add-on files and use the new AddOn constructor. Change ExtensionAutoUpdate to use the new constructors/methods and to copy the file from manual add-on installations using a normalised file name. Add some tests to assert the expected behaviour of AddOn class. Fix zaproxy#3090 - Be more lenient on add-on's file name format * Change default time to 15 and make publicly accessible * Latest files from Crowdin * Deprecate unused Spider menu items Deprecate unused Spider menu items (replaced by the Spider dialogue menu item). * Latest files from Crowdin * Correct location of i18n messages Swap the contents of two i18n messages, the title was being used as message and the message as title of the dialogue. * include SubjectAlternativeName extension in generated certificates * Add description to some Spider API endpoints Add descriptions to some of the Spider API endpoints and correct one that was wrong (it was for the action not view). * Log during start up the add-ons that are installed Change ExtensionFactory to log (as info) the IDs and version of the add-ons that are in installed state (all dependencies/requirements are fulfilled). * Add description to some core/ascan API endpoints Add description to some core and ascan API endpoints. * Latest files from Crowdin * Added security annotations for forms that dont need anti CSRF tokens * Latest files from Crowdin * Add description to active scan ZAP API option Add description to the active scan ZAP API option "Inject plugin ID in header for all active scan requests". Related to zaproxy#3133 - how disable send X-ZAP-Scan-ID header * Latest files from Crowdin
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Using GET request everywhere (instead of POSTs for actions) makes certain API calls quite tricky to get right because one has to remember to url-encode any special characters - e.g. (python API):
(Notice the required & -> %26 and % -> %25 escaping)
loginRequestData
is a url query parameter and therefore it's value cannot contain any special characters (like an ampersand)Changing API GET requests to POST would most likely solve the problem, but changing the way authentication methods are set might suffice.
The text was updated successfully, but these errors were encountered: