Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add developer container #114

Merged
merged 1 commit into from
Aug 31, 2023
Merged

Conversation

vsoch
Copy link
Member

@vsoch vsoch commented Jun 17, 2023

Problem: it is hard to build this for use on a native solution
Solution: create a development container. We initially targeted a VSCode devcontainer, but ran into considerable issues (#112) that we have not figured out yet! A good fall-back is to use a basic container that can be provided via an automated build. I'm hoping to have this container (and automated build) so I can test more workflows. Some questions I have:

Building on Pull Requests

Should the container build run on pull requests? it's a fairly long build, however if we have a container provided here we can pull first (and essentially use that as a cache) it might be faster. I would say it's worth keeping the pull_request trigger, doing a build and deploy, and then seeing how long a subsequent build takes. If it's quick, we leave it. Otherwise, we can do something like a weekly build (and be alerted then if something fails).

pcituils

My tests failed originally because I didn't have pciutils installed - I didn't see it mentioned anywhere and I wonder if we should?

GPU required?

Is there a way to run tests without assuming I have a GPU? I do not, and it would be good to sanity check that stuffs are working in my container (even without GPU).

Thank you!

@vsoch
Copy link
Member Author

vsoch commented Jun 17, 2023

I'll need your help on this rst format tool - I followed the instructions to run it, and it gave me a fairly generic (not helpful) message because I don't see what it's talking about!

root@7fb8616869a3:/code# ./scripts/check-rst-format.sh 
:25: (WARNING/2) Inline literal start-string without end-string.
Traceback (most recent call last):
  File "/usr/local/bin/rstfmt", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/rstfmt/__main__.py", line 96, in main
    do_file(args, f, misformatted)
  File "/usr/local/lib/python3.8/dist-packages/rstfmt/__main__.py", line 31, in do_file
    doc = rstfmt.parse_string(inp)
  File "/usr/local/lib/python3.8/dist-packages/rstfmt/rstfmt.py", line 758, in parse_string
    parser.parse(s, doc)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/__init__.py", line 184, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 169, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 2785, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 2785, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 325, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 391, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 279, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 195, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 233, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.8/dist-packages/docutils/statemachine.py", line 445, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 2716, in blank
    paragraph, literalnext = self.paragraph(
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 416, in paragraph
    textnodes, messages = self.inline_text(text, lineno)
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 425, in inline_text
    nodes, messages = self.inliner.parse(text, lineno,
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 649, in parse
    before, inlines, remaining, sysmessages = method(self, match,
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 899, in literal
    before, inlines, remaining, sysmessages, endstring = self.inline_obj(
  File "/usr/local/lib/python3.8/dist-packages/docutils/parsers/rst/states.py", line 727, in inline_obj
    msg = self.reporter.warning(
  File "/usr/local/lib/python3.8/dist-packages/docutils/utils/__init__.py", line 224, in warning
    return self.system_message(self.WARNING_LEVEL, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/rstfmt/rstfmt.py", line 281, in system_message
    msg = super().system_message(level, message, *children, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docutils/utils/__init__.py", line 197, in system_message
    raise SystemMessage(msg, level)
docutils.utils.SystemMessage: :25: (WARNING/2) Inline literal start-string without end-string.

@slabasan
Copy link
Collaborator

slabasan commented Jun 19, 2023

Hi @vsoch, I didn't have permissions to push changes to your branch, but here is the formatted rst file per the formatting script. I agree, I don't know what to make of your error messages above, but rstfmt -w 80 developer.rst produced this file.

############################
 PerfFlowAspect Development
############################

This is a short developer guide for using the included development environment.
While we do not provide VSCode (there was `a bug we could not figure out
<https://github.com/flux-framework/PerfFlowAspect/issues/112>`_) we provide a
Dockerfile that makes it easy to build and use PerfFlow!

********************
 Building Container
********************

You can build the container from scratch:

.. code:: console

   $ docker build -t ghcr.io/flux-framework/perf-flow-aspect .

Note that for a faster build (or no build!), you can pull or use an existing
container:

.. code:: console

   $ docker pull ghcr.io/flux-framework/perf-flow-aspect

To shell into the container:

.. code:: console

   $ docker run -it ghcr.io/flux-framework/perf-flow-aspect

Once inside, you can cd to where the tests are, and run a few!

.. code:: console

   cd src/c/build/test
   ./t0001-cbinding-basic.t

Note that if you don't have a GPU, these probably will error (I do not)! Here is
how to run Python tests:

.. code:: console

   cd src/python/test
   ./t0001-pybinding-basic.t

You'll again have issues without an actual GPU. And that's it! Note that we will
update this documentation as we create more examples.

@vsoch
Copy link
Member Author

vsoch commented Jun 19, 2023

Thank you! There must be some dependency bug on my computer - I can only get a stack trace when I run that command directly.

Problem: it is hard to build this for use on a native solution
Solution: create a development container. We intiially targeted a VSCode
devcontainer, but ran into considerable issue that we have not figured out
yet! A good fall-back is to use a basic container that can be provided
via an automated build.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch vsoch mentioned this pull request Jun 20, 2023
@slabasan slabasan merged commit 5a4a7c1 into flux-framework:main Aug 31, 2023
5 checks passed
slabasan added a commit that referenced this pull request Aug 31, 2023
slabasan added a commit that referenced this pull request Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants