Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bazel test fails because of OSFamily property not defined #576

Closed
steedmicro opened this issue Dec 22, 2023 · 5 comments · Fixed by #577
Closed

Bazel test fails because of OSFamily property not defined #576

steedmicro opened this issue Dec 22, 2023 · 5 comments · Fixed by #577

Comments

@steedmicro
Copy link
Contributor

I ran into this issue when I try to re-install this project on my Linux VPS - Ubuntu 22.0.4.
I installed Git, Rust, Docker, Cargo and Bazel on the pure VPS.
It successfully ran the nativelink server with cargo run command.
But when I ran bazel test command it shows this error message and fails.

`

ERROR: /home/project/turbo-cache/local-remote-execution/examples/BUILD.bazel:3:10: Compiling local-remote-execution/examples/hello.cpp failed: (Exit 34): INVALID_ARGUMENT: Unknown platform property 'OSFamily' : Failed to convert platform property in build_action_info : Failed on execute() command
`

It seems like because it couldn't get platform property of 'OSFamily' while implementing local remote execution feature and it has strict bazel build requirement.
I used to face error while I was running bazel test on my Windows machine also.
Is it possible to set our local remote execution module not to affect the whole project's bazel test command?
Or do we just need to refine our local remote execution module to match various OS family including Windows, MacOS and various machines including Cloud VPS?

Here is the screenshot of the error.


Capture

@steedmicro steedmicro changed the title Bazel test fails because of "OSFamily" property not defined Bazel test fails because of OSFamily property not defined Dec 22, 2023
@allada
Copy link
Member

allada commented Dec 22, 2023

Can you share the .json file you are using?

@steedmicro
Copy link
Contributor Author

Which JSON file you mean by?
The global config json file? I use the standard basic_cas.json file.

{
  "stores": {
    "AC_MAIN_STORE": {
      "memory": {
        "eviction_policy": {
          // 100mb.
          "max_bytes": 100000000,
        }
      }
    },
    "WORKER_FAST_SLOW_STORE": {
      "fast_slow": {
        // "fast" must be a "filesystem" store because the worker uses it to make
        // hardlinks on disk to a directory where the jobs are running.
        "fast": {
          "filesystem": {
            "content_path": "/tmp/nativelink/data-worker-test/content_path-cas",
            "temp_path": "/tmp/nativelink/data-worker-test/tmp_path-cas",
            "eviction_policy": {
              // 10gb.
              "max_bytes": 10000000000,
            }
          }
        },
        "slow": {
          /// Discard data.
          /// This example usage has the CAS and the Worker live in the same place,
          /// so they share the same underlying CAS. Since workers require a fast_slow
          /// store, we use the fast store as our primary data store, and the slow store
          /// is just a noop, since there's no shared storage in this config.
          "noop": {}
        }
      }
    }
  },
  "schedulers": {
    "MAIN_SCHEDULER": {
      "simple": {
        "supported_platform_properties": {
          "cpu_count": "minimum",
          "memory_kb": "minimum",
          "network_kbps": "minimum",
          "disk_read_iops": "minimum",
          "disk_read_bps": "minimum",
          "disk_write_iops": "minimum",
          "disk_write_bps": "minimum",
          "shm_size": "minimum",
          "gpu_count": "minimum",
          "gpu_model": "exact",
          "cpu_vendor": "exact",
          "cpu_arch": "exact",
          "cpu_model": "exact",
          "kernel_version": "exact",
          // Example of how to set which docker images are available and set
          // them in the platform properties.
          // "docker_image": "priority",
        }
      }
    }
  },
  "workers": [{
    "local": {
      "worker_api_endpoint": {
        "uri": "grpc://127.0.0.1:50061",
      },
      "cas_fast_slow_store": "WORKER_FAST_SLOW_STORE",
      "upload_action_result": {
        "ac_store": "AC_MAIN_STORE",
      },
      "work_directory": "/tmp/nativelink/work",
      "platform_properties": {
        "cpu_count": {
          "values": ["16"],
        },
        "memory_kb": {
          "values": ["500000"],
        },
        "network_kbps": {
          "values": ["100000"],
        },
        "cpu_arch": {
          "values": ["x86_64"],
        },
        // Example of how to set which docker images are available and set
        // them in the platform properties.
        // "docker_image": {
        //   "query_cmd": "docker images --format {{.Repository}}:{{.Tag}}",
        // }
      }
    }
  }],
  "servers": [{
    "name": "public",
    "listener": {
      "http": {
        "socket_address": "0.0.0.0:50051"
      }
    },
    "services": {
      "cas": {
        "main": {
          "cas_store": "WORKER_FAST_SLOW_STORE"
        }
      },
      "ac": {
        "main": {
          "ac_store": "AC_MAIN_STORE"
        }
      },
      "execution": {
        "main": {
          "cas_store": "WORKER_FAST_SLOW_STORE",
          "scheduler": "MAIN_SCHEDULER",
        }
      },
      "capabilities": {
        "main": {
          "remote_execution": {
            "scheduler": "MAIN_SCHEDULER",
          }
        }
      },
      "bytestream": {
        "cas_stores": {
          "main": "WORKER_FAST_SLOW_STORE",
        }
      }
    }
  }, {
    "name": "private_workers_servers",
    "listener": {
      "http": {
        "socket_address": "0.0.0.0:50061"
      }
    },
    "services": {
      "experimental_prometheus": {
        "path": "/metrics"
      },
      // Note: This should be served on a different port, because it has
      // a different permission set than the other services.
      // In other words, this service is a backend api. The ones above
      // are a frontend api.
      "worker_api": {
        "scheduler": "MAIN_SCHEDULER",
      },
      "admin": {}
    }
  }],
  "global": {
    "max_open_files": 512
  }
}

@allada
Copy link
Member

allada commented Dec 22, 2023

I see...

@aaronmondal , it looks like we missed some:

"kernel_version": "exact",

We need to add OsFamily similar to here:

@steedmicro
Copy link
Contributor Author

Thanks, it works. 👍

@aaronmondal
Copy link
Member

aaronmondal commented Dec 22, 2023

Yeah we must have missed this in #512. Thanks @steed924!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants