This repository has been archived by the owner on Nov 2, 2023. It is now read-only.
Release v0.1.0 beta.3 #43
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Julio-Guerra
commented
Mar 21, 2019
•
Julio-Guerra
commented
| # | x | Content | Record |
|---|---|---|---|
| 0 | x | Ensure that you are respecting the release schedule (only release every other week in normal circumstances) | |
| 1 | x | Check the pull requests on the agent. Any left open should have a good reason to be | |
| 2 | N/A | If the release already exist in Jira, make sure every projected ticket for this release has been resolved. Create it otherwise. | |
| 3 | N/A | Check pull requests on ShieldRules and other actions on Rules. | |
| 4 | x | Launch integrations tests on the PR. Meanwhile, perform manual tests against current production backend. | |
| 5 | x | Perform performance tests | |
| 6 | x | Update Changelog in the agent GIT repo | |
| 7 | x | Add potential changes of agent behavior to the agents documentation | |
| 8 | x | Update Agent feature matrix | |
| 9 | N/A | Merge PR (expect for Go) | |
| 10 | N/A | If a rule need to be deployed to enable a new agent feature, it should be deployed then. Ensure rule is versioned accordingly to this agent's version. | |
| 11 | x | Organize a dogfooding session with your buddy (1h+). Target the common architectures of our clients. | |
| 12 | N/A | Trigger one click deploy on Jenkins or merge to master for Go. | |
| 13 | N/A | For PHP only, create new version for extension (and / or daemon) in Admin interface | |
| 14 | X | Create PR with release notes in documentation repository | |
| 15 | N/A | Update agents of Sqreen services to use latest agent versions (e.g. Python: Back-End, Java: Jenkins, ...). | |
| 16 | X | Send an email to the team regarding the release. |
New feature: - sdk: explicit user identification method (#26).
Move the content of the existing `agent` package into a new sub-package named `internal` so that it is no longer possible to import it from other packages. Only the parent package `agent` will be able to import it.
Now the agent internals are strictly private and cannot be imported anymore, we need to define the interface the SDK expects so that it no longer imports the agent but rather its interface. This interface is defined in `agent/types` which will be from now the package exporting the public types of the agent. It is therefore possible to implement a "disabled agent" in the SDK, implementing this interface as noops, so that it is disabled until the agent is initialized.
Now that we have an abstract interface, it is now simple to reach the 100% statement coverage of the `sdk` layer, by simply implementing the agent interface as a mockup.
Fix the SDK so that it is possible to use it with nil pointer receivers. Note that this is unlikely to happen because it is only possible because of a programming mistake.
Simply adapt the existing internal agent API to the new one. Only renaming here.
Move the agent internals into a package named `internal`, which makes it importable from parent packages only, and thus impossible to import from outside packages. It is therefore required to separate the SDK dependency to the agent internals by defining a Go interface including everything it expects from the agent. The agent automatically "sets itself" on initialization. When the agent is not included, the SDK uses a disabled agent, which no-ops everything. A consequence is that it is easy now to unit-test the SDK with 100% coverage by simply mocking the agent and checking everything. Related to SQR-5192, SQR-5204, SQR-5242 and SQR-5262.
In order to be able to properly quit, restart or stop the agent, remove every global variable. Now store them as structure members, whose life-cycle is derived from the agent's: when the agent is initialized/deinitialization, he is now responsible of each data initialization/deinitialization, and the same applies to sub-layers of the agent. Another benefit will be better testing, as every dependency is now exposed in the interfaces.
Refactor the agent so that it no longer relies on Go globals whose life-cycles are managed by the Go runtime, while we would like to be able te manage them ourselves to restart and stop the agent. In order to be able to properly quit, restart or stop the agent, replace every global variable by structure members, whose life-cycle depends on the agent's: when the agent is initialized/deinitialization, it is now responsible of each data initialization/deinitialization, and the same applies recursively to sub-objects. Another benefit will be better testing, as every dependency is now exposed in the interfaces, so they will be mockable as we did for the SDK to reach 100% coverage. Related to SQR-5262 and SQR-5262.
Implement the remote command manager which simply associates command names to methods calling back agent entrypoints.
Add remote commands management in the agent loop: 1. Read them from the login response. 2. Send results in the heartbeat loop. 3. Receive new ones in the heartbeat response.
Implement instrumentation enable/disable agent methods, expected by the command manager. For now, it only enables/disables the SDK as nothing else is yet supported. The SDK is therefore no longer enabled by default, but rather enabled by the applogin response which commands to enable instrumentation and thus the SDK events.
Do not ignore unknown commands but rather create a result whose status is `false` and stating it in the result output field.
- add support for remote commands. - add support for `instrumentation_enable` and `instrumentation_remove` commands. - 100% coverage of the new command.go file. Closes SQR-5262.
Remove protobuf methods for now as they are not used nor tested, which kills the average coverage of the agent.
Remove protobuf methods for now as they are not used nor tested, which degraded the average coverage of the agent because the `api/` folder was the biggest.
Add a new configuration key `strip_http_referer` (env key being `SQREEN_STRIP_HTTP_REFERER`) whose boolean value true when set to any non-empty value, false otherwise.
Do not send the referer header when explicitly configured by the user.
Remove `omitempty` json tags as the backend expects them, even if the value is the zero value.
Remove `omitempty` json tags as the backend expects them, even if the value is the zero value.
- Add a new configuration key `strip_http_referer` (corresponding to the env key `SQREEN_STRIP_HTTP_REFERER`), whose boolean value is true when set to a non-empty value, false otherwise. - Do not send the referer header when explicitly configured by the user. Note that it is intentionally named after the `referer` HTTP header, including the missing `r` letter. Closes SQR-6195 and SQR-6196.
Add simple logic to add and send the request record only when necessary, ie. only when there are some events other than a single identify. To do so, add a simple boolean value in the record to keep track of when the record should be sent of not.
Early logs happening before the agent actual starting do not show up because the loggers are not yet enabled. Move the enabling in the agent initialization so that we can see early logs.
Add simple logic to add and send the request record only when necessary, ie. only when there are some events other than a single identify. To do so, add a simple boolean value in the record to keep track of when the record should be sent of not. Closes SQR-6245.
Early logs happening before the agent actual starting do not show up because the loggers are not yet enabled. Move the enabling in the agent initialization so that we can see early logs.
The length of an IP address cannot be read using `len(ip)` because some `net` package functions return a 16-byte slice even for IPv4 addresses. The only way to safely check if an IP address is a v4 or v6 is to use `ip.To4()` which checks the internals and returns nil if it's not.
The length of an IP address cannot be read using `len(ip)` because some `net` package functions return a 16-byte slice even for IPv4 addresses. The only way to safely check if an IP address is a v4 or v6 is to use `ip.To4()` which checks the internals and returns nil if it's not. Closes SQR-6259
The HA Proxy header `X-Unique-Id` can be added using a user-configured format that may include the IP address. To be able to properly parse it, the agent needs to be given the format string that is used to generate the header value so that it can parse it back and find the IP address in it. The configuration should be: ``` ip_header: X-Unique-Id ip_header_format: <your format> ``` For now, `ip_header_format` value is only used as a boolean value to activate this behaviour, but with a hard-coded format: the client IP address first, in hexadecimal number, followed by a semicolon (port number separator).
The User-Agent is already included in the request record, so remove it from the list of tracked headers to avoid sending it twice in the same request record.
The User-Agent is already included in the request record, so remove it from the list of tracked headers to avoid sending it twice in the same request record. Closes SQR-6261.
The HA Proxy header `X-Unique-Id` can be added using a user-configured format that may include the IP address. To be able to properly parse it, the agent needs to be given the format string that is used to generate the header value so that it can parse it back and find the IP address in it. The configuration should be: ``` ip_header: X-Unique-Id ip_header_format: <your format> ``` For now, `ip_header_format` value is only used as a boolean value to activate this behaviour, but with a hard-coded format: the client IP address first, in hexadecimal number, followed by a semicolon (port number separator). Closes SQR-6191
|
vdeturckheim
approved these changes
Mar 21, 2019
Codecov Report
@@ Coverage Diff @@
## master #43 +/- ##
===========================================
+ Coverage 17.14% 29.86% +12.71%
===========================================
Files 12 18 +6
Lines 1913 1480 -433
===========================================
+ Hits 328 442 +114
+ Misses 1567 1014 -553
- Partials 18 24 +6
|
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.