Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

[VC update] rest-server #1831

Merged
merged 14 commits into from
Dec 26, 2018
Merged

[VC update] rest-server #1831

merged 14 commits into from
Dec 26, 2018

Conversation

mzmssg
Copy link
Member

@mzmssg mzmssg commented Dec 5, 2018

add 3 vc related api:

  1. PUT virtual-clusters/:vcName
    add or update a vc quota

  2. DELETE virtual-clusters/:vcName
    delete a vc

  3. PUT virtual-clusters/:vcName/status
    change vc status

@mzmssg mzmssg requested a review from Gerhut December 5, 2018 11:55
}
});
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fanyangCS

Shall we add a yarn dependency to rest server?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question. the desired behavior is: if yarn is available, it works. if yarn is unavailable, the system says it doesn't support the feature yet.


In reply to: 239478932 [](ancestors = 239478932)

Copy link
Member Author

@mzmssg mzmssg Dec 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gerhut @fanyangCS
I will move the validation somewhere to remove the dependency. But I think we need clarify vc firstly, it will affect our API design.

If we consider vc as the same concept of hadoop queue, moving the check to vc related API sounds reasonable, because only yarn provider this feature.
But if vc is designed as a abstract interface, hadoop queue is only one implementation. I think we need refactor our vc structure, to support other implementation. At least, some API coupling with hadoop queue shoulded be add to our doc.

function traverse(queueInfo, queueDict) {
if (queueInfo.type === 'capacitySchedulerLeafQueueInfo') {
queueDict[queueInfo.queueName] = {
capacity: queueInfo.absoluteCapacity,
maxCapacity: queueInfo.absoluteMaxCapacity,
capacity: parseInt(queueInfo.absoluteCapacity),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use Number() instead of parseInt() to avoid number base issues.

Copy link
Member Author

@mzmssg mzmssg Dec 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolve by math.round

Copy link
Member

@Gerhut Gerhut Dec 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built-in Math.round is enough

if (err) {
return callback(err);
} else {
let defaultQuotaIfUpdated = vcList['default']['capacity'] + vcList[vcName]['capacity'];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make sure about these 2 values will be Number? + will be treated as string concat when one of these is string.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this two value comes from YARN, they are both number.

@coveralls
Copy link

Coverage Status

Coverage increased (+1.2%) to 53.022% when pulling 9714940 on zimiao/queue_zookeeper into b7e8451 on vc_update.

2 similar comments
@coveralls
Copy link

Coverage Status

Coverage increased (+1.2%) to 53.022% when pulling 9714940 on zimiao/queue_zookeeper into b7e8451 on vc_update.

@coveralls
Copy link

Coverage Status

Coverage increased (+1.2%) to 53.022% when pulling 9714940 on zimiao/queue_zookeeper into b7e8451 on vc_update.

@coveralls
Copy link

coveralls commented Dec 10, 2018

Coverage Status

Coverage increased (+1.0%) to 52.734% when pulling 88037b2 on zimiao/queue_zookeeper into b7e8451 on vc_update.

@mzmssg
Copy link
Member Author

mzmssg commented Dec 26, 2018

Checkin firstly to help frontend test, will remove this deps in other pr

@mzmssg mzmssg merged commit ca93b41 into vc_update Dec 26, 2018
mzmssg added a commit that referenced this pull request Jan 10, 2019
* VC update API doc (#1816)

* refine api doc

* refine doc

* refine

* refine

* [VC update] rest-server (#1831)

* change scheduler storage to zookeeper

* add queue update

* fix

* add vc update api

* add yarn health check, refine error

* add api doc

* refine api doc

* refine api doc

* update yarn response

* add unit test

* unit test

* replace parseint with math.round, fix unit test

* replace mathjs with build-in Math

* vc update (#1974)

* Update yarnContainerScript.mustache

add entrypoint=""

* add design for vc update

* refine design of vc update

* add description of /api/v1/:username/virtualClusters

* remove design of vc_update to dir of webportal

* refine design of vc

* VC function development

* vc style modification

* vc list style modification

* Modify the prompt

* repair bug

* remove design of vc_update and refine annotation

* refine var name

* Change eslint error

* Optimize the code according to pr

* Modified read-only
@xudifsd xudifsd deleted the zimiao/queue_zookeeper branch May 24, 2019 02:14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants