Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ticket21315 #3

Merged
merged 12 commits into from Jun 28, 2019
Merged

Ticket21315 #3

merged 12 commits into from Jun 28, 2019

Conversation

Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
@cohosh
Copy link
Owner

@cohosh cohosh commented Jun 12, 2019

No description provided.

cohosh added 8 commits Jun 11, 2019
Set the resolution of metrics data collection to be every 24 hours
Added three new metrics:
- proxyIdleCount counts the number of times a proxy polls and receives
no snowflakes
- clientDeniedCount counts the number of times a client requested a
snowflake but none were available
- clientProxyMatchCount counts the number of times a client successfully
received a snowflake
Output is now printed out in Tor Directory Protocol Format, as specified
in https://trac.torproject.org/projects/tor/ticket/21315#comment:19.
Change it so that we log the geoip country code of proxies if they poll
within the current metrics epoch. We make sure we log by unique IP
address
Added unit tests for metrics logging. Refactored the logMetrics()
function to allow for easier testing
@cohosh
Copy link
Owner Author

@cohosh cohosh commented Jun 12, 2019

Looks like Travis doesn't like the convey package. The branch is passing CI, even though this PR isn't.

Copy link
Contributor

@NullHypothesis NullHypothesis left a comment

The patch looks generally good to me! I only have a number of nitpicks.

broker/metrics.go Outdated Show resolved Hide resolved
type CountryStats struct {
ips map[string]bool
Copy link
Contributor

@NullHypothesis NullHypothesis Jun 13, 2019

Nitpick: we are using both ip and addr to refer to IP addresses. I think we should be consistent in our naming. (Personally, I like addr more.)

broker/metrics.go Outdated Show resolved Hide resolved
//update map of countries and counts
m.countryStats.counts[country]++
//update map of unique ips and counts
if !m.countryStats.ips[addr] {
Copy link
Contributor

@NullHypothesis NullHypothesis Jun 13, 2019

To save some CPU cycles, I think we could move this check to the beginning of the function, so we don't have to find the country code if we already have the IP address.

broker/metrics.go Show resolved Hide resolved
<-done

ctx.metrics.printMetrics()
So(buf.String(), ShouldResemble, "snowflake-stats-end "+time.Now().UTC().Format("2006-01-02 15:04:05")+" ( 86400 s)\nsnowflake-ips CA=1,\nsnowflake-idle-count 8\nclient-denied-count 0\nclient-snowflake-match-count 0\n")
Copy link
Contributor

@NullHypothesis NullHypothesis Jun 13, 2019

Huh, where's the extra space in ( 86400 s) coming from? Our metrics format requires (NSEC s), so we should fix this.

@@ -289,7 +292,7 @@ func main() {
metricsFile = os.Stdout
}

metricsLogger := log.New(metricsFile, "", log.LstdFlags|log.LUTC)
metricsLogger := log.New(metricsFile, "", 0)
Copy link
Contributor

@NullHypothesis NullHypothesis Jun 13, 2019

Why are we getting rid of the flags here? I'm not saying we shouldn't; I'm just curious.

Copy link
Owner Author

@cohosh cohosh Jun 14, 2019

The flags for loggers are mostly about formatting timestamps and prefixes for log messages. We don't want any prefixes for the log file, only the information in the specification.

cohosh added 2 commits Jun 14, 2019
Also moved the geoip check to occur after we've make sure the proxy IP
hasn't yet been recorded. This is will cut down on unecessary
computation.
- removed trailing ","s
- removed unecessary space before seconds
Copy link
Contributor

@NullHypothesis NullHypothesis left a comment

Looks good to me!

@@ -82,6 +82,10 @@ func (m *Metrics) UpdateCountryStats(addr string) {
var country string
var ok bool

if m.countryStats.addrs[addr] {
Copy link
Contributor

@NullHypothesis NullHypothesis Jun 14, 2019

Go suggests the use of two-value assignments to test for the existence of a key, but I think relying on the zero value of a bool should also be fine: https://blog.golang.org/go-maps-in-action#TOC_3.

cohosh added 2 commits Jun 25, 2019
Added another metrics item that counts the total availabel snowflakes
(unique by IP address)
Updated the tests to pass with our new snowflake-ips-total stat
@cohosh cohosh merged commit 908cf3f into master Jun 28, 2019
2 checks passed
@cohosh cohosh deleted the ticket21315 branch Mar 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment