Skip to content

Commit

Permalink
Polish docs & messages
Browse files Browse the repository at this point in the history
  • Loading branch information
blokhin committed Oct 18, 2023
1 parent 9635156 commit ead4228
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 14 deletions.
23 changes: 13 additions & 10 deletions CLOUD.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,22 @@ az account subscription list

Create a dedicated Resource Group. See [documentation][az_manage_rg].
For example, consider `yascheduler-rg` in `westeurope` location.
Write down the resource group and location to the config.
Save the resource group and location to the cloud config.

```bash
az group create -l westeurope -g yascheduler-rg
```

Create a dedicated _Enterprise Application_ for service.
See [documentation][az_app_create].
Write down `appId` as `client_id` to config.
Save `appId` as `az_client_id` to the cloud config.

```bash
az ad app create --display-name yascheduler
```

Assign roles _Network Contributor_ and _Virtual Machine Contributor_
in the _Resource Group_. Use the correct `appId`.
in the _Resource Group_. Use the correct `appId`:

```bash
az role assignment create \
Expand All @@ -45,14 +45,15 @@ az role assignment create \
```

Create an _Application Registration_.
Add _Client Secret_ to the Application Registration. Use correct `appId`.
Write down `tenant` as `tenant_id` and `password` as `client_secret`.
Add the _Client Secret_ to this Application Registration. Use the correct `appId`:

```bash
az ad app credential reset --id 00000000-0000-0000-0000-000000000000 --append
```

Create virtual networks.
Use `tenant` as the `az_tenant_id` and `password` as the `az_client_secret` cloud settings.

Create virtual networks:

```bash
az network nsg create \
Expand All @@ -71,14 +72,16 @@ az network vnet create \
--subnet-prefix 10.0.0.0/22
```

It is possible to setup a _jump host_. It allows connections from the outside.
If `yascheduler` is installed in the internal network, it is optional.
According to our experience, while creating the nodes, the Azure allocates the new public
IP-addresses slowly and unwillingly, so we support **the internal IP-addresses** only.
This is no problem, if `yascheduler` is installed in the internal network.
If this is not the case, one has to setup a _jump host_, allowing connections from the outside:

```bash
az vm create \
-g yascheduler-rg -l westeurope \
--name yascheduler-jump-host \
--image Debian \
--image Debian11 \
--size Standard_B1s \
--nsg yascheduler-nsg \
--public-ip-address yascheduler-jump-host-ip \
Expand All @@ -90,7 +93,7 @@ az vm create \
--ssh-key-values "$(ssh-keygen -y -f path/to/private/key)"
```

Write down `publicIpAddress` as `jump_host`. `jump_user` will be `yascheduler`.
Save the `publicIpAddress` as `az_jump_host`, and `az_jump_user` will be `yascheduler`.

[az_cli_install]: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli
[az_manage_rg]: https://docs.microsoft.com/en-us/cli/azure/manage-azure-groups-azure-cli
Expand Down
18 changes: 16 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,20 @@ result = yac.queue_submit_task(
print(result)
```

Or run directly in console with `yascheduler` (use a key `-l DEBUG` to change the log level).

_Supervisor_ config reads e.g.:

```
[program:scheduler]
command=/usr/local/bin/yascheduler
user=root
autostart=true
autorestart=true
stderr_logfile=/data/yascheduler.log
stdout_logfile=/data/yascheduler.log
```

File paths can be set using the environment variables:

- `YASCHEDULER_CONF_PATH`
Expand Down Expand Up @@ -104,7 +118,7 @@ Connection to a PostgreSQL database.
Path to root directory of local data files.
Can be relative to the current working directory.

_Default_: `./data`
_Default_: `./data` (but it's always a good idea to set up explicitly!)

_Example_: `/srv/yadata`

Expand Down Expand Up @@ -362,7 +376,7 @@ Settings prefix is `upcloud`.

Password.

#### Engines `[engine.*]`
### Engines `[engine.*]`

Supported engines should be defined in the section(s) `[engine.name]`.
The name is alphanumeric string to represent the real engine name.
Expand Down
3 changes: 2 additions & 1 deletion yascheduler/clouds/cloud_api_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,9 @@ async def select_best_provider(

self.log.debug("Used providers: %s", used_providers)
if not suitable_providers:
self.log.debug("No suitable cloud provides")
self.log.debug("No suitable cloud providers")
return

ok_apis = filter(lambda x: x.name in suitable_providers, self.apis.values())
ok_apis_sorted = sorted(ok_apis, key=lambda x: x.config.priority, reverse=True)
api = ok_apis_sorted[0]
Expand Down
2 changes: 1 addition & 1 deletion yascheduler/scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -497,7 +497,7 @@ async def allocator_producer(
tasks = await self.db.get_tasks_by_status((TaskStatus.TO_DO,), tlim)
if tasks:
ids = [str(t.task_id) for t in tasks]
self.log.debug("Want allocate tasks: %s" % ", ".join(ids))
self.log.debug("Want to allocate tasks: %s" % ", ".join(ids))
for task in tasks:
yield UMessage(task.task_id, task)

Expand Down

0 comments on commit ead4228

Please sign in to comment.