[FLINK-20356][docs] Updated Mesos documentation. #14258

XComp · 2020-11-29T22:35:03Z

First version of the Mesos page. See a screenshot of the page below...

flinkbot · 2020-11-29T22:36:53Z

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit 299cc30 (Sun Nov 29 22:36:53 UTC 2020)

Warnings:

Documentation files were touched, but no .zh.md files: Update Chinese documentation or file Jira ticket.

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

❓ 1. The [description] looks good.
❓ 2. There is [consensus] that the contribution should go into to Flink.
❓ 3. Needs [attention] from.
❓ 4. The change fits into the overall [architecture].
❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
@flinkbot approve all to approve all aspects
@flinkbot approve-until architecture to approve everything until architecture
@flinkbot attention @username1 [@username2 ..] to require somebody's attention
@flinkbot disapprove architecture to remove an approval you gave earlier

flinkbot · 2020-11-29T22:57:41Z

CI report:

6231f2a Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run travis re-run the last Travis build
@flinkbot run azure re-run the last Azure build

rmetzger

I started reviewing, will continue later

docs/deployment/resource-providers/mesos.md

tillrohrmann

Thanks for creating this PR @XComp. I had a couple of comments. Please take a look.

docs/deployment/resource-providers/mesos.md

tillrohrmann · 2020-11-30T16:02:25Z

docs/deployment/resource-providers/mesos.md

+You may install Flink on all of your Mesos Master and Agent nodes. You can also pull the binaries 
+from the Flink website during deployment and apply your custom configuration before launching the 
+application master. A more convenient and easier-to-maintain approach is to use Docker containers 
+to manage the Flink binaries and configuration.


I'd say to go with the docker image approach.

docs/deployment/resource-providers/mesos.md

tillrohrmann · 2020-11-30T16:37:49Z

docs/deployment/resource-providers/mesos.md

+1. Another option is to create a `${FLINK_HOME}/usrlib/` directory and to put the user code jar 
+   in that directory. The user code jar will be loaded by the user code classloader after launching 
+   a Per-Job Cluster via `${FLINK_HOME}/bin/mesos-appmaster-job.sh ...`.


Have you tried it out whether it works?

tillrohrmann · 2020-11-30T16:41:32Z

docs/deployment/resource-providers/mesos.md

    {
        "id": "flink",
        "cmd": "$FLINK_HOME/bin/mesos-appmaster.sh -Djobmanager.memory.process.size=1472m -Djobmanager.rpc.port=6123 -Drest.port=8081 -Dtaskmanager.memory.process.size=1024m -Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2 -Dmesos.resourcemanager.tasks.cpus=1",
        "cpus": 1.0,
        "mem": 1024
    }


I think this yaml is not correct. We allocate mem: 1024 but set -Djobmanager.memory.process.size=1472m. Maybe we can put the simplest definition possible here (e.g. w/o specifying the resources and the ports, for example) or we completely remove it and refer to the documentation of Marathon.

…new deployment page style.

…o Flink on Mesos Reference. The Marathon section's example is not working, yet. I haven't found a solution for setting the RPC address in a multi-slave environment, yet.

tillrohrmann

Thanks for updating this PR @XComp. I had a couple for additional comments. Please take a look.

docs/deployment/resource-providers/mesos.md

tillrohrmann · 2020-12-07T14:04:00Z

docs/deployment/resource-providers/mesos.md

+{% highlight javascript %}
+{
+    "id": "flink",
+    "cmd": "/opt/flink-1.11.2/bin/mesos-appmaster.sh -Dmesos.resourcemanager.framework.user=root -Dmesos.resourcemanager.tasks.taskmanager-cmd=/opt/flink-1.11.2/bin/mesos-taskmanager.sh -Dmesos.master=master:5050 -Djobmanager.memory.process.size=1472m -Dtaskmanager.memory.process.size=3500m -Dtaskmanager.numberOfTaskSlots=2 -Dparallelism.default=2",


Do we have to specify all the memory configurations or can we use the default values?

Where comes master:5050 from?

docs/deployment/resource-providers/mesos.md

…er.sh can be called from.

…h version.

tillrohrmann

Thanks for updating the documentation @XComp. The changes look good to me. I had a single comment/question. Please take a look.

docs/deployment/resource-providers/mesos.md

…e Getting Started section.

tillrohrmann

Thanks for the update @XComp. The changes look good to me. Merging this PR now.

…new deployment page style. [FLINK-20356][docs] Added missing "Mode". [FLINK-20356][docs] Added missing "is". [FLINK-20356][docs] Fixed wrong number. [FLINK-20356][docs] Reorganized Mesos page moving advanced topics into Flink on Mesos Reference. The Marathon section's example is not working, yet. I haven't found a solution for setting the RPC address in a multi-slave environment, yet. [FLINK-20356][docs] Improved wording. [FLINK-20356][docs] Fixing the description on where bin/mesos-appmaster.sh can be called from. [FLINK-20356][docs] Made Flink version dynamic. [FLINK-20356][docs] Fixed highlight block. [FLINK-20356][docs] Shortened long line. [FLINK-20356][docs] Moved Flink on Mesos Architecture section down. [FLINK-20356][docs] Added Hadoop section. [FLINK-20352][docs] Updated Chinese CLI docs to align with the English version. [FLINK-20352][docs] Reorganized Marathon section. [FLINK-20352][docs] Fixed headline. [FLINK-20352][docs] Updated code blocks. [FLINK-20352][docs] Added remark on Marathon's environment variable. [FLINK-20352][docs] Aligned placeholder labels. [FLINK-20352][docs] Added job submission command. [FLINK-20352][docs] Replaced root by <flink-user> to align it with the Getting Started section. [FLINK-20352][docs] Aligned Chinese with English version. This closes #14258.

…new deployment page style. [FLINK-20356][docs] Added missing "Mode". [FLINK-20356][docs] Added missing "is". [FLINK-20356][docs] Fixed wrong number. [FLINK-20356][docs] Reorganized Mesos page moving advanced topics into Flink on Mesos Reference. The Marathon section's example is not working, yet. I haven't found a solution for setting the RPC address in a multi-slave environment, yet. [FLINK-20356][docs] Improved wording. [FLINK-20356][docs] Fixing the description on where bin/mesos-appmaster.sh can be called from. [FLINK-20356][docs] Made Flink version dynamic. [FLINK-20356][docs] Fixed highlight block. [FLINK-20356][docs] Shortened long line. [FLINK-20356][docs] Moved Flink on Mesos Architecture section down. [FLINK-20356][docs] Added Hadoop section. [FLINK-20352][docs] Updated Chinese CLI docs to align with the English version. [FLINK-20352][docs] Reorganized Marathon section. [FLINK-20352][docs] Fixed headline. [FLINK-20352][docs] Updated code blocks. [FLINK-20352][docs] Added remark on Marathon's environment variable. [FLINK-20352][docs] Aligned placeholder labels. [FLINK-20352][docs] Added job submission command. [FLINK-20352][docs] Replaced root by <flink-user> to align it with the Getting Started section. [FLINK-20352][docs] Aligned Chinese with English version. This closes apache#14258.

rmetzger added review=description? component=Documentation component=Runtime/Coordination labels Nov 29, 2020

rmetzger reviewed Nov 30, 2020

View reviewed changes

docs/deployment/resource-providers/mesos.md Show resolved Hide resolved

docs/deployment/resource-providers/mesos.md Outdated Show resolved Hide resolved

docs/deployment/resource-providers/mesos.md Outdated Show resolved Hide resolved

tillrohrmann requested changes Nov 30, 2020

View reviewed changes

XComp added 5 commits December 3, 2020 15:37

[FLINK-20356][docs] Updated Mesos documentation matching it with the …

2009c62

…new deployment page style.

[FLINK-20356][docs] Added missing "Mode".

ddcc57b

[FLINK-20356][docs] Added missing "is".

371cb4c

[FLINK-20356][docs] Fixed wrong number.

9c1444d

[FLINK-20356][docs] Reorganized Mesos page moving advanced topics int…

321e355

…o Flink on Mesos Reference. The Marathon section's example is not working, yet. I haven't found a solution for setting the RPC address in a multi-slave environment, yet.

XComp force-pushed the feature/FLINK-20356 branch from 299cc30 to 321e355 Compare December 6, 2020 23:55

tillrohrmann requested changes Dec 7, 2020

View reviewed changes

XComp added 13 commits December 8, 2020 15:22

[FLINK-20356][docs] Improved wording.

b1e3bd5

[FLINK-20356][docs] Fixing the description on where bin/mesos-appmast…

b20ff6f

…er.sh can be called from.

[FLINK-20356][docs] Made Flink version dynamic.

deb1efe

[FLINK-20356][docs] Fixed highlight block.

8e4823f

[FLINK-20356][docs] Shortened long line.

d5efb7a

[FLINK-20356][docs] Moved Flink on Mesos Architecture section down.

4ce0a03

[FLINK-20356][docs] Added Hadoop section.

f69f7b1

[FLINK-20352][docs] Updated Chinese CLI docs to align with the Englis…

1536314

…h version.

[FLINK-20352][docs] Reorganized Marathon section.

6030ef7

[FLINK-20352][docs] Fixed headline.

3dd111d

[FLINK-20352][docs] Updated code blocks.

f2e2346

[FLINK-20352][docs] Added remark on Marathon's environment variable.

8e7cfa1

[FLINK-20352][docs] Aligned placeholder labels.

60c22f6

tillrohrmann approved these changes Dec 9, 2020

View reviewed changes

docs/deployment/resource-providers/mesos.md Show resolved Hide resolved

docs/deployment/resource-providers/mesos.md Outdated Show resolved Hide resolved

XComp added 2 commits December 9, 2020 16:53

[FLINK-20352][docs] Added job submission command.

ba5d2bf

[FLINK-20352][docs] Replaced root by <flink-user> to align it with th…

490af70

…e Getting Started section.

[FLINK-20352][docs] Aligned Chinese with English version.

6231f2a

tillrohrmann approved these changes Dec 10, 2020

View reviewed changes

tillrohrmann closed this in 672d646 Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-20356][docs] Updated Mesos documentation. #14258

[FLINK-20356][docs] Updated Mesos documentation. #14258

XComp commented Nov 29, 2020

flinkbot commented Nov 29, 2020

flinkbot commented Nov 29, 2020 •

edited

rmetzger left a comment

tillrohrmann left a comment

tillrohrmann Nov 30, 2020

tillrohrmann Nov 30, 2020

tillrohrmann Nov 30, 2020

tillrohrmann left a comment

tillrohrmann Dec 7, 2020

tillrohrmann Dec 7, 2020

tillrohrmann left a comment

tillrohrmann left a comment

[FLINK-20356][docs] Updated Mesos documentation. #14258

[FLINK-20356][docs] Updated Mesos documentation. #14258

Conversation

XComp commented Nov 29, 2020

flinkbot commented Nov 29, 2020

Automated Checks

Review Progress

flinkbot commented Nov 29, 2020 • edited

CI report:

rmetzger left a comment

Choose a reason for hiding this comment

tillrohrmann left a comment

Choose a reason for hiding this comment

tillrohrmann Nov 30, 2020

Choose a reason for hiding this comment

tillrohrmann Nov 30, 2020

Choose a reason for hiding this comment

tillrohrmann Nov 30, 2020

Choose a reason for hiding this comment

tillrohrmann left a comment

Choose a reason for hiding this comment

tillrohrmann Dec 7, 2020

Choose a reason for hiding this comment

tillrohrmann Dec 7, 2020

Choose a reason for hiding this comment

tillrohrmann left a comment

Choose a reason for hiding this comment

tillrohrmann left a comment

Choose a reason for hiding this comment

flinkbot commented Nov 29, 2020 •

edited