Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes #18926: nodes API with include managementTechnologyDetails leads to error 500 response #3513

Merged
merged 1 commit into from
Feb 28, 2021
Merged

Fixes #18926: nodes API with include managementTechnologyDetails leads to error 500 response #3513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions ...dap-inventory/inventory-api/src/main/scala/com/normation/inventory/domain/DataTypes.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,11 @@ object InventoryProcessingLogger extends NamedZioLogger {
object timing extends NamedZioLogger(){ def loggerName = "inventory-logger.timing"}
}

object InventoryDataLogger extends NamedZioLogger {
override def loggerName: String = "inventory-data"
}


sealed trait InventoryError extends RudderError

object InventoryError {
Expand Down
19 changes: 15 additions & 4 deletions ...sitory/src/main/scala/com/normation/inventory/ldap/core/FullInventoryRepositoryImpl.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,18 +289,29 @@ class FullInventoryRepositoryImpl(
case None => LDAPRudderError.Consistancy(s"Could not find node inventories in ${dit.BASE_DN}").fail
case Some(tree) => tree.succeed
}
nodes <- ZIO.foreach(nodeTree.children.values) { tree => mapper.nodeFromTree(tree) }
// we don't want that one error somewhere breaks everything
nodes <- ZIO.foreach(nodeTree.children.values) { tree =>
mapper.nodeFromTree(tree).foldM(
err => InventoryDataLogger.error(s"Error when mapping inventory data for entry '${tree.root().rdn.map(_.toString).getOrElse("")}': ${err.fullMsg}") *> None.succeed
, ok => Some(ok).succeed
)
}

// Get into Machines subtree
machineTree <- tree.flatMap(_.children.get(dit.MACHINES.rdn)) match {
case None => LDAPRudderError.Consistancy(s"Could not find machine inventories in ${dit.BASE_DN}").fail
case Some(tree) => tree.succeed
}
machines <- ZIO.foreach(machineTree.children.values){ tree => mapper.machineFromTree(tree) }
machines <- ZIO.foreach(machineTree.children.values){ tree =>
mapper.machineFromTree(tree).foldM(
err => InventoryDataLogger.error(s"Error when mapping inventory data for entry '${tree.root().rdn.map(_.toString).getOrElse("")}': ${err.fullMsg}") *> None.succeed
, ok => Some(ok).succeed
)
}

} yield {
val machineMap = machines.map(m => (m.id, m)).toMap
nodes.map(
val machineMap = machines.flatten.map(m => (m.id, m)).toMap
nodes.flatten.map(
node => {
val machine = node.machineId.flatMap(mid => machineMap.get(mid._1))
val inventory = FullInventory(node,machine)
Expand Down
17 changes: 11 additions & 6 deletions ...ventory-repository/src/main/scala/com/normation/inventory/ldap/core/InventoryMapper.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -935,12 +935,17 @@ class InventoryMapper(
agentNames <- {
val agents = entry.valuesFor(A_AGENTS_NAME).toSeq.map(Some(_))
val agentWithKeys = agents.zipAll(publicKeys, None,None).filter(_._1.isDefined)
ZIO.foreach(agentWithKeys) {
case (Some(agent), key) =>
AgentInfoSerialisation.parseCompatNonJson(agent,key).chainError(s"Error when parsing agent security token '${agent}'")
case (None, key) =>
InventoryMappingRudderError.MissingMandatory("Error when parsing agent security token: agent is undefined").fail
}
ZIO.foreach(agentWithKeys) { case (opt, key) => ((opt, key) match {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need the extra case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we use a syntax shortcut in methods accepting function as argument: you can create a partial function with the following syntax { case x => ... } and so use it directly as argument, what we regularly do for map and alike function - and in that case, we are careful to use a partial function which is actually total by not imposing restriction on the x:

myUserList.map { case User(name, age) => name.capitalize }

But that really syntax for partial function, so you can put as many guard as you want in the pattern, use as many pattern as you want, and even don't make it total (and be sad when the pattern match exception happens):

myUserList.map { 
  case User(name, age) if age > 17 => 
  case User(name, age) if age > 5  => 
// and pattern match exception when age <= 6, but YOLO!
}

So in our case, since I wanted to have a different logic for the two cases of Option:

ZIO.foreach(agentWithKeys) { 
  case (Some(agent), key) => ...
  case (None, key)        =>
}

But then, I needed to apply a fold on result, and I don't know of a syntax that allow to do that with several pattern match in the partial function syntact, so I reverted to the usual pattern match syntax and only one branch for the partial function:

ZIO.foreach(agentWithKeys) { case (opt, key) =>  //that's for the function arg of `foreach`
  (opt, key) match { // that's a standard pattern matching 
    case (Some(agent), key) => ...
    case (None, key)        =>
  }
}

And with the fold:

ZIO.foreach(agentWithKeys) { case (opt, key) =>  //that's for the function arg of `foreach`
  ((opt, key) match { // that's a standard pattern matching 
    case (Some(agent), key) => ...
    case (None, key)        =>
  }).fold(....)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the time spent for the superb explanation of why adding the extra case, i learnt something

case (Some(agent), key) =>
AgentInfoSerialisation.parseCompatNonJson(agent,key).chainError(s"Error when parsing agent security token '${agent}'")
case (None, key) =>
InventoryMappingRudderError.MissingMandatory("Error when parsing agent security token: agent is undefined").fail
}).foldM(
err => InventoryDataLogger.error(s"Error when parsing agent information for node '${id.value}': that agent will be " +
s"ignored for the node, which will likely cause problem like the node being ignored: ${err.fullMsg}") *> None.succeed
, ok => Some(ok).succeed
)
}.map(_.flatten)
}
//now, look for the OS type
osDetails <- mapOsDetailsFromEntry(entry).toIO
Expand Down
4 changes: 2 additions & 2 deletions ...pp/sources/rudder/rudder-rest/src/main/scala/com/normation/rudder/rest/lift/NodeApi.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -661,7 +661,7 @@ class NodeApiService6 (
nodeIds = nodeFilter.getOrElse(nodeInfos.keySet).toSet
runs <- roAgentRunsRepository.getNodesLastRun(nodeIds)
inventories <- if(detailLevel.needFullInventory()) {
inventoryRepository.getAllInventories(state).toBox
inventoryRepository.getAllInventories(state).toBox ?~! "Error when looking for node inventories"
} else {
Full(Map[NodeId, FullInventory]())
}
Expand All @@ -684,7 +684,7 @@ class NodeApiService6 (
toJsonResponse(None, ( "nodes" -> JArray(nodes.toList)))
}
case eb: EmptyBox => {
val message = (eb ?~ (s"Could not fetch ${state.name} Nodes")).msg
val message = (eb ?~ (s"Could not fetch ${state.name} Nodes")).messageChain
toJsonError(None, message)
}
}
Expand Down
118 changes: 118 additions & 0 deletions webapp/sources/rudder/rudder-rest/src/test/resources/api/api_nodes.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
description: Get node (minimal)
method: GET
url: /api/nodes/node1?include=minimal
response:
code: 200
content: >-
{
"action":"nodeDetails",
"id":"node1",
"result":"success",
"data":{
"nodes":[
{
"id":"root",
"hostname":"server.rudder.local",
"status":"accepted"
}
]
}
}
---
description: Get node (full)
method: GET
url: /api/nodes/node1?include=full
response:
code: 200
content: >-
{
"action":"nodeDetails",
"id":"node1",
"result":"success",
"data":{
"nodes":[
{
"id":"root",
"hostname":"server.rudder.local",
"status":"accepted",
"os":{
"type":"Linux",
"name":"Debian",
"version":"7.0",
"fullName":"Jessie",
"kernelVersion":"3.2"
},
"machine":{
"id":"machine1",
"type":"Virtual",
"provider":"vbox"
},
"ipAddresses":["127.0.0.1","192.168.0.100"],
"description":"",
"lastInventoryDate":"2021-01-30 01:20",
"policyServerId":"root",
"managementTechnology":[{
"name":"Rudder",
"version":"4.0.0",
"capabilities":[],
"nodeKind":"root",
"rootComponents":[
"rudder-db",
"rudder-inventory-endpoint",
"rudder-inventory-ldap",
"rudder-jetty",
"rudder-ldap",
"rudder-reports",
"rudder-server-root",
"rudder-webapp"
]
}],
"properties":[],
"policyMode":"enforce",
"timezone":{"name":"UTC","offset":"+00"},
"environmentVariables":{"THE_VAR":"THE_VAR value!"},
"fileSystems":[{"name":"swap"}],
"managementTechnologyDetails":{"cfengineKeys":[],"cfengineUser":"root"},
"networkInterfaces":[],
"software":[
{"name":"s09","version":"1.0"},
{"name":"s06","version":"1.0"},
{"name":"s05","version":"1.0"},
{"name":"s13","version":"1.0"},
{"name":"s08","version":"1.0"},
{"name":"s02","version":"1.0"},
{"name":"s03","version":"1.0"},
{"name":"s04","version":"1.0"},
{"name":"s10","version":"1.0"},
{"name":"s11","version":"1.0"},
{"name":"s12","version":"1.0"},
{"name":"s01","version":"1.0"},
{"name":"s00","version":"1.0"},
{"name":"s07","version":"1.0"}
]
}
]
}
}
---
description: List node with select and include managementTechnologie
method: GET
url: /api/nodes?include=minimal,managementTechnologyDetails&select=nodeAndPolicyServer&where=[{"objectType":"node","attribute":"nodeHostname","comparator":"eq","value":"node1.localhost"}]
response:
code: 200
content: >-
{
"action":"listAcceptedNodes",
"result":"success",
"data":{
"nodes":[
{
"id":"node1",
"hostname":"node1.localhost",
"status":"accepted",
"managementTechnologyDetails":{"cfengineKeys":[],"cfengineUser":"root"}
}
]
}
}