Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: transition into new terminology #1188

Merged
merged 10 commits into from Jun 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/script-samples.md
Expand Up @@ -141,7 +141,7 @@ $ orchestrator-client -c api -path instance/$master_host/3306 | jq .
"Binlog_format": "ROW",
"BinlogRowImage": "FULL",
"LogBinEnabled": true,
"LogSlaveUpdatesEnabled": true,
"LogReplicationUpdatesEnabled": true,
"SelfBinlogCoordinates": {
"LogFile": "mysql-bin.000002",
"LogPos": 333006336,
Expand All @@ -152,8 +152,8 @@ $ orchestrator-client -c api -path instance/$master_host/3306 | jq .
"Port": 0
},
"IsDetachedMaster": false,
"Slave_SQL_Running": false,
"Slave_IO_Running": false,
"ReplicationSQLThreadRuning": false,
"ReplicationIOThreadRuning": false,
"HasReplicationFilters": false,
"GTIDMode": "OFF",
"SupportsOracleGTID": false,
Expand Down Expand Up @@ -185,11 +185,11 @@ $ orchestrator-client -c api -path instance/$master_host/3306 | jq .
"SQLDelay": 0,
"ExecutedGtidSet": "",
"GtidPurged": "",
"SlaveLagSeconds": {
"ReplicationLagSeconds": {
"Int64": 0,
"Valid": true
},
"SlaveHosts": [
"Replicas": [
{
"Hostname": "mysql-2222.dc1.domain.net",
"Port": 3306
Expand Down Expand Up @@ -286,14 +286,14 @@ mysql-00ff.dc1.domain.net
#### How many replicas to a specific instance?

```shell
$ orchestrator-client -c api -path instance/$master_host/3306 | jq '.SlaveHosts | length'
$ orchestrator-client -c api -path instance/$master_host/3306 | jq '.Replicas | length'
3
```

#### How many replicas to each of a cluster's members?

```shell
$ orchestrator-client -c api -path cluster/alias/mycluster | jq '.[].SlaveHosts | length'
$ orchestrator-client -c api -path cluster/alias/mycluster | jq '.[].Replicas | length'
3
0
2
Expand Down
22 changes: 11 additions & 11 deletions docs/using-the-web-api.md
Expand Up @@ -46,7 +46,7 @@ This sample is followed by a field breakdown:
"ReadOnly": false,
"Binlog_format": "ROW",
"LogBinEnabled": true,
"LogSlaveUpdatesEnabled": true,
"LogReplicationUpdatesEnabled": true,
"SelfBinlogCoordinates": {
"LogFile": "mysql-bin.015656",
"LogPos": 15082,
Expand All @@ -56,8 +56,8 @@ This sample is followed by a field breakdown:
"Hostname": "mysql.01.instance.com",
"Port": 3306
},
"Slave_SQL_Running": true,
"Slave_IO_Running": true,
"ReplicationSQLThreadRuning": true,
"ReplicationIOThreadRuning": true,
"HasReplicationFilters": false,
"SupportsOracleGTID": true,
"UsingOracleGTID": true,
Expand Down Expand Up @@ -86,11 +86,11 @@ This sample is followed by a field breakdown:
},
"SQLDelay": 0,
"ExecutedGtidSet": "230ea8ea-81e3-11e4-972a-e25ec4bd140a:1-49",
"SlaveLagSeconds": {
"ReplicationLagSeconds": {
"Int64": 0,
"Valid": true
},
"SlaveHosts": [ ],
"Replicas": [ ],
"ClusterName": "mysql.01.instance.com:3306",
"DataCenter": "",
"PhysicalEnvironment": "",
Expand All @@ -117,11 +117,11 @@ The structure of an Instance evolves and documentation will always fall behind.
* `ReadOnly`: the global `read_only` boolean value
* `Binlog_format`: the global `binlog_format` MySQL param
* `LogBinEnabled`: whether binary logs are enabled
* `LogSlaveUpdatesEnabled`: whether `log_slave_updates` MySQL param is enabled
* `LogReplicationUpdatesEnabled`: whether `log_slave_updates` MySQL param is enabled
* `SelfBinlogCoordinates`: binary log file & position this instance write to (as in `SHOW MASTER STATUS`)
* `MasterKey`: hostname & port of master, if any
* `Slave_SQL_Running`: direct mapping from `SHOW SLAVE STATUS`'s `Slave_SQL_Running`
* `Slave_IO_Running`: direct mapping from `SHOW SLAVE STATUS`'s `Slave_IO_Running`
* `ReplicationSQLThreadRuning`: direct mapping from `SHOW SLAVE STATUS`'s `Slave_SQL_Running`
* `ReplicationIOThreadRuning`: direct mapping from `SHOW SLAVE STATUS`'s `Slave_IO_Running`
* `HasReplicationFilters`: true if there's any replication filter
* `SupportsOracleGTID`: true if cnfigured with `gtid_mode` (Oracle MySQL >= 5.6)
* `UsingOracleGTID`: true if replica replicates via Oracle GTID
Expand All @@ -136,8 +136,8 @@ The structure of an Instance evolves and documentation will always fall behind.
`"Valid": false` indicates a `NULL`
* `SQLDelay`: the configured `MASTER_DELAY`
* `ExecutedGtidSet`: if using Oracle GTID, the executed GTID set
* `SlaveLagSeconds`: when `ReplicationLagQuery` provided, the computed replica lag; otherwise same as `SecondsBehindMaster`
* `SlaveHosts`: list of MySQL replicas _hostname & port_)
* `ReplicationLagSeconds`: when `ReplicationLagQuery` provided, the computed replica lag; otherwise same as `SecondsBehindMaster`
* `Replicas`: list of MySQL replicas _hostname & port_)
* `ClusterName`: name of cluster this instance is associated with; uniquely identifies cluster
* `DataCenter`: (metadata) name of data center, infered by `DataCenterPattern` config variable
* `PhysicalEnvironment`: (metadata) name of environment, infered by `PhysicalEnvironmentPattern` config variable
Expand Down Expand Up @@ -191,5 +191,5 @@ curl -s "http://my.orchestrator.service.com/api/instance-replicas/${master}" | j
- Find all intermediate masters in `my_cluster`:

```
curl -s "http://my.orchestrator.service.com/api/cluster/alias/my_cluster" | jq '.[] | select(.MasterKey.Hostname!="") | select(.SlaveHosts!=[]) .Key.Hostname'
curl -s "http://my.orchestrator.service.com/api/cluster/alias/my_cluster" | jq '.[] | select(.MasterKey.Hostname!="") | select(.Replicas!=[]) .Key.Hostname'
```
103 changes: 63 additions & 40 deletions go/inst/instance.go
Expand Up @@ -18,6 +18,7 @@ package inst

import (
"database/sql"
"encoding/json"
"fmt"
"strconv"
"strings"
Expand All @@ -32,50 +33,57 @@ const ReasonableDiscoveryLatency = 500 * time.Millisecond
// Instance represents a database instance, including its current configuration & status.
// It presents important replication configuration and detailed replication status.
type Instance struct {
Key InstanceKey
InstanceAlias string
Uptime uint
ServerID uint
ServerUUID string
Version string
VersionComment string
FlavorName string
ReadOnly bool
Binlog_format string
BinlogRowImage string
LogBinEnabled bool
LogSlaveUpdatesEnabled bool
SelfBinlogCoordinates BinlogCoordinates
MasterKey InstanceKey
MasterUUID string
AncestryUUID string
IsDetachedMaster bool
Slave_SQL_Running bool
Slave_IO_Running bool
ReplicationSQLThreadState ReplicationThreadState
ReplicationIOThreadState ReplicationThreadState
HasReplicationFilters bool
GTIDMode string
SupportsOracleGTID bool
UsingOracleGTID bool
UsingMariaDBGTID bool
UsingPseudoGTID bool
ReadBinlogCoordinates BinlogCoordinates
ExecBinlogCoordinates BinlogCoordinates
IsDetached bool
RelaylogCoordinates BinlogCoordinates
LastSQLError string
LastIOError string
SecondsBehindMaster sql.NullInt64
SQLDelay uint
ExecutedGtidSet string
GtidPurged string
GtidErrant string
Key InstanceKey
InstanceAlias string
Uptime uint
ServerID uint
ServerUUID string
Version string
VersionComment string
FlavorName string
ReadOnly bool
Binlog_format string
BinlogRowImage string
LogBinEnabled bool
LogSlaveUpdatesEnabled bool
LogReplicationUpdatesEnabled bool
SelfBinlogCoordinates BinlogCoordinates
MasterKey InstanceKey
MasterUUID string
AncestryUUID string
IsDetachedMaster bool

Slave_SQL_Running bool
ReplicationSQLThreadRuning bool
Slave_IO_Running bool
ReplicationIOThreadRuning bool
ReplicationSQLThreadState ReplicationThreadState
ReplicationIOThreadState ReplicationThreadState

HasReplicationFilters bool
GTIDMode string
SupportsOracleGTID bool
UsingOracleGTID bool
UsingMariaDBGTID bool
UsingPseudoGTID bool
ReadBinlogCoordinates BinlogCoordinates
ExecBinlogCoordinates BinlogCoordinates
IsDetached bool
RelaylogCoordinates BinlogCoordinates
LastSQLError string
LastIOError string
SecondsBehindMaster sql.NullInt64
SQLDelay uint
ExecutedGtidSet string
GtidPurged string
GtidErrant string

masterExecutedGtidSet string // Not exported

SlaveLagSeconds sql.NullInt64
ReplicationLagSeconds sql.NullInt64
SlaveHosts InstanceKeyMap
Replicas InstanceKeyMap
ClusterName string
SuggestedClusterAlias string
DataCenter string
Expand Down Expand Up @@ -131,6 +139,21 @@ func NewInstance() *Instance {
}
}

func (this *Instance) MarshalJSON() ([]byte, error) {
i := struct {
Instance
}{}
i.Instance = *this
// change terminology. Users of the orchestrator API can switch to new terminology and avoid using old terminology
i.Replicas = i.SlaveHosts
i.ReplicationLagSeconds = this.SlaveLagSeconds
i.ReplicationSQLThreadRuning = this.Slave_SQL_Running
i.ReplicationIOThreadRuning = this.Slave_IO_Running
i.LogReplicationUpdatesEnabled = this.LogSlaveUpdatesEnabled
//
return json.Marshal(i)
}

// Equals tests that this instance is the same instance as other. The function does not test
// configuration or status.
func (this *Instance) Equals(other *Instance) bool {
Expand Down Expand Up @@ -179,7 +202,7 @@ func (this *Instance) IsSmallerMajorVersion(other *Instance) bool {
return IsSmallerMajorVersion(this.Version, other.Version)
}

// IsSmallerMajorVersionByString cehcks if this instance has a smaller major version number than given one
// IsSmallerMajorVersionByString checks if this instance has a smaller major version number than given one
func (this *Instance) IsSmallerMajorVersionByString(otherVersion string) bool {
return IsSmallerMajorVersion(this.Version, otherVersion)
}
Expand Down
4 changes: 2 additions & 2 deletions resources/bin/orchestrator-client
Expand Up @@ -313,11 +313,11 @@ function filter_keys {
}

function filter_broken_replicas {
cat - | jq '.[] | select((.Slave_SQL_Running == false or .Slave_IO_Running == false) and (.LastSQLError != "" or .LastIOError != "")) | [.]'
cat - | jq '.[] | select((.ReplicationSQLThreadRuning == false or .ReplicationIOThreadRuning == false) and (.LastSQLError != "" or .LastIOError != "")) | [.]'
}

function filter_running_replicas {
cat - | jq '.[] | select(.Slave_SQL_Running == true and .Slave_IO_Running == true) | [.]'
cat - | jq '.[] | select(.ReplicationSQLThreadRuning == true and .ReplicationIOThreadRuning == true) | [.]'
}

function print_key {
Expand Down
4 changes: 2 additions & 2 deletions resources/public/js/audit-failure-detection.js
Expand Up @@ -52,9 +52,9 @@ $(document).ready(function() {

var moreInfo = "";
moreInfo += '<div>Detected: ' + audit.RecoveryStartTimestamp + '</div>';
if (audit.AnalysisEntry.SlaveHosts.length > 0) {
if (audit.AnalysisEntry.Replicas.length > 0) {
moreInfo += '<div>' + audit.AnalysisEntry.CountReplicas + ' replicating hosts :<ul>';
audit.AnalysisEntry.SlaveHosts.forEach(function(instanceKey) {
audit.AnalysisEntry.Replicas.forEach(function(instanceKey) {
moreInfo += "<li><code>" + getInstanceTitle(instanceKey.Hostname, instanceKey.Port) + "</code></li>";
});
moreInfo += "</ul></div>";
Expand Down
4 changes: 2 additions & 2 deletions resources/public/js/audit-recovery.js
Expand Up @@ -46,9 +46,9 @@ $(document).ready(function() {
});
moreInfo += "</ul></div>";
}
if (audit.AnalysisEntry.SlaveHosts.length > 0) {
if (audit.AnalysisEntry.Replicas.length > 0) {
moreInfo += '<div>' + audit.AnalysisEntry.CountReplicas + ' replicating hosts :<ul>';
audit.AnalysisEntry.SlaveHosts.forEach(function(instanceKey) {
audit.AnalysisEntry.Replicas.forEach(function(instanceKey) {
moreInfo += "<li><code>" + getInstanceTitle(instanceKey.Hostname, instanceKey.Port) + "</code></li>";
});
moreInfo += "</ul></div>";
Expand Down
16 changes: 8 additions & 8 deletions resources/public/js/cluster.js
Expand Up @@ -441,7 +441,7 @@ function Cluster() {
}
if (droppableNode.MasterKey.Hostname && droppableNode.MasterKey.Hostname != "_") {
// droppableNode has master
if (!droppableNode.LogSlaveUpdatesEnabled) {
if (!droppableNode.LogReplicationUpdatesEnabled) {
// Obviously can't handle.
return {
accept: false
Expand Down Expand Up @@ -518,7 +518,7 @@ function Cluster() {
}
if (droppableNode.MasterKey.Hostname && droppableNode.MasterKey.Hostname != "_") {
// droppableNode has master
if (!droppableNode.LogSlaveUpdatesEnabled) {
if (!droppableNode.LogReplicationUpdatesEnabled) {
// Obviously can't handle.
return {
accept: false
Expand Down Expand Up @@ -624,7 +624,7 @@ function Cluster() {
};
}
if (instancesAreSiblings(node, droppableNode)) {
if (node.hasProblem || droppableNode.hasProblem || droppableNode.isAggregate || !droppableNode.LogSlaveUpdatesEnabled) {
if (node.hasProblem || droppableNode.hasProblem || droppableNode.isAggregate || !droppableNode.LogReplicationUpdatesEnabled) {
return {
accept: false
};
Expand Down Expand Up @@ -787,7 +787,7 @@ function Cluster() {
}
if (droppableNode.MasterKey.Hostname && droppableNode.MasterKey.Hostname != "_") {
// droppableNode has master
if (!droppableNode.LogSlaveUpdatesEnabled) {
if (!droppableNode.LogReplicationUpdatesEnabled) {
// Obviously can't handle.
return {
accept: false
Expand Down Expand Up @@ -842,7 +842,7 @@ function Cluster() {
}
if (droppableNode.MasterKey.Hostname && droppableNode.MasterKey.Hostname != "_") {
// droppableNode has master
if (!droppableNode.LogSlaveUpdatesEnabled) {
if (!droppableNode.LogReplicationUpdatesEnabled) {
// Obviously can't handle.
return {
accept: false
Expand Down Expand Up @@ -1204,7 +1204,7 @@ function Cluster() {
if (isAnonymized()) {
instanceDescription = anonymizeInstanceId(instance.id);
}
instanceDescription += ", " + instance.SlaveLagSeconds.Int64 + "s lag";
instanceDescription += ", " + instance.ReplicationLagSeconds.Int64 + "s lag";
incrementProblems("", instanceDescription)
instanceFullNames.push(getInstanceTitle(instance.Key.Hostname, instance.Key.Port));
instance.Problems.forEach(function(problem) {
Expand Down Expand Up @@ -1553,7 +1553,7 @@ function Cluster() {
if (replica.SQLDelay > 0) {
return
}
if (!replica.LogSlaveUpdatesEnabled) {
if (!replica.LogReplicationUpdatesEnabled) {
return
}
if (replica.lastCheckInvalidProblem()) {
Expand All @@ -1580,7 +1580,7 @@ function Cluster() {
if (!sibling.LogBinEnabled) {
return
}
if (!sibling.LogSlaveUpdatesEnabled) {
if (!sibling.LogReplicationUpdatesEnabled) {
return
}
if (sibling.lastCheckInvalidProblem()) {
Expand Down
2 changes: 1 addition & 1 deletion resources/public/js/instance-problems.js
Expand Up @@ -25,7 +25,7 @@ $(document).ready(function() {
function SortByProblemOrder(instance0, instance1) {
var orderDiff = instance0.problemOrder - instance1.problemOrder;
if (orderDiff != 0) return orderDiff;
var orderDiff = instance1.SlaveLagSeconds.Int64 - instance0.SlaveLagSeconds.Int64;
var orderDiff = instance1.ReplicationLagSeconds.Int64 - instance0.ReplicationLagSeconds.Int64;
if (orderDiff != 0) return orderDiff;
orderDiff = instance0.title.localeCompare(instance1.title);
if (orderDiff != 0) return orderDiff;
Expand Down