Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion .github/workflows/cdci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:

jobs:
test:
name: Test
name: Unittests+streamlit
runs-on: ubuntu-latest
strategy:
matrix:
Expand All @@ -30,4 +30,59 @@ jobs:
pip install -e .
- name: Run tests
run: python -m pytest tests
- name: Execute streamlit report
run: |
cd docs
vuegen --config example_data/MicW2Graph/report_config_micw2graph.yaml

other-reports:
name: Integration tests
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip' # caching pip dependencies
cache-dependency-path: '**/pyproject.toml'
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -e ".[quarto]"
- name: Execute streamlit report (to check)
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml
# repeat for easier inspection on GitHub:
- name: quarto html report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt html
- name: quarto pdf report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt pdf
- name: quarto docx report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt docx
- name: quarto odt report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt odt
- name: quarto revealjs report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt revealjs
- name: quarto pptx report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt pptx
- name: quarto jupyter report
run: |
cd docs
vuegen -c example_data/MicW2Graph/report_config_micw2graph.yaml -rt jupyter
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,8 @@ quarto_report/
.DS_Store

# Extra info
UML_diagrams/
Graphical_abstract/
docs/images/UML_diagrams/
docs/images/Graphical_abstract/
docs/images/Nfcore_module_figure
docs/presentations/
test.py
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,18 @@ quarto check
Run VueGen using a configuration file with the following command:

```bash
python vuegen/main.py --config docs/example_data/MicW2Graph/report_config_micw2graph.yaml --report_type streamlit
cd docs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! Why is it necessary to change the directory to docs? In my local environment, it's working when I execute the program from the base folder, not from docs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to check when I am back, but you could double check in a clean environment within a code space on that branch.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I will check that out. Thanks!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested in a code space clean environment and it works without changing the directory to docs. Also, I identified some errors with the docx, odt, and pdf formats when converting a df into an image:
OSError: Chrome executable not able to be found on your machine

Copy link
Collaborator Author

@enryH enryH Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well. Then this is another requirement. So which tool relies on chronium? Probably something for the interactive dataframes (itables)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why cd docs:

  1. I wanted to check that the import is not local. vuegen is on the main folder, not jet using the src layout, i.e. moving the packages into a separate src folder.
  2. For building the website I will need to have the example report relative to the docs/conf.py file. In the end someone who uses the report generator will have only a folder resembling the docs.

In short: yes we can remove the docs, but let's to it in a separate PR where we change the layout of the repo.

Copy link
Collaborator

@sayalaruano sayalaruano Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that dataframe_image is the one that relies on chromium.
It's ok to remove the cd docs in an independent PR. And should we merge the current PR into the meain with the cd docs?

Copy link
Collaborator Author

@enryH enryH Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will do it now.

vuegen --config example_data/MicW2Graph/report_config_micw2graph.yaml --report_type streamlit
```

> 💡 If `vuegen` does not work, try `python -m vuegen` instead.

By default, the `streamlit_autorun` argument is set to False, but you can use it in case you want to automatically run the streamlit app.

It's also possible to provide a directory instead of a configuration file:

```bash
python vuegen/main.py --directory docs/example_data/MicW2Graph/ --report_type streamlit
vuegen --directory docs/example_data/MicW2Graph/ --report_type streamlit
```

The current report types are streamlit, html, pdf, docx, odt, revealjs, pptx, and jupyter.
Expand Down
Binary file modified docs/images/vuegen_classdiagram_noattmeth.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 5 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ kaleido = "0.2.0"
vl-convert-python = "^1.7.0"
dataframe-image = "^0.2.6"
strenum = { version = "^0.4.15", python = "<3.11" }

# optional doc depencencies, follow approach as described here:
# https://github.com/python-poetry/poetry/issues/2567#issuecomment-646766059
sphinx = {version="*", optional=true}
Expand All @@ -35,13 +36,12 @@ myst-nb = {version="*", optional=true}
ipywidgets = {version="*", optional=true}
sphinx-new-tab-link = {version = "!=0.2.2", optional=true}
jupytext = {version="*", optional=true}
# quartro
# quarto
quarto-cli = {version="*", optional=true}

[tool.poetry.group.dev.dependencies]
ipykernel = {version="^6.29.5", optional=true}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Expand All @@ -52,5 +52,6 @@ streamlit = ["streamlit"]
quarto = ["quarto-cli", "ipykernel"]
docs = ["sphinx", "sphinx-book-theme", "myst-nb", "ipywidgets", "sphinx-new-tab-link", "jupytext"]

# [project.scripts]
# my-script = "vuegen.main:main"
[tool.poetry.scripts]
# https://python-poetry.org/docs/pyproject/#scripts
vuegen = "vuegen.__main__:main"
2 changes: 1 addition & 1 deletion tests/test_module_imports.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

def test_imports():
import vuegen
import vuegen.main
import vuegen.__main__
import vuegen.quarto_reportview
import vuegen.report
import vuegen.report_generator
Expand Down
49 changes: 49 additions & 0 deletions vuegen/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import sys
from pathlib import Path

from vuegen import report_generator
from vuegen.utils import get_logger, get_parser


def main():
# Parse command-line arguments
parser = get_parser(prog_name="VueGen")
args = parser.parse_args()

# Determine the vuegen arguments
config_path = args.config
dir_path = args.directory
report_type = args.report_type
streamlit_autorun = args.streamlit_autorun

# Determine the report name for logger suffix
if config_path:
report_name = Path(config_path).stem
elif dir_path:
report_name = Path(dir_path).name
else:
print("Please provide a configuration file or directory path:\n")
# https://docs.python.org/3/library/argparse.html#printing-help
parser.print_help()
sys.exit(1)

if config_path and dir_path:
print("Please provide only one of configuration file or directory path:\n")
parser.print_help()
sys.exit(1) # otherwise could resort to either or ?

# Define logger suffix based on report type and name
logger_suffix = f"{report_type}_report_{str(report_name)}"

# Initialize logger
logger = get_logger(f"{logger_suffix}")

# Generate the report
report_generator.get_report(report_type=report_type,
logger=logger,
config_path=config_path,
dir_path=dir_path,
streamlit_autorun=streamlit_autorun)

if __name__ == '__main__':
main()
33 changes: 0 additions & 33 deletions vuegen/main.py

This file was deleted.

10 changes: 8 additions & 2 deletions vuegen/quarto_reportview.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,8 @@ def run_report(self, output_dir: str = BASE_DIR) -> None:
"""
try:
subprocess.run(["quarto", "render", os.path.join(output_dir, f"{self.BASE_DIR}.qmd")], check=True)
if self.report_type == r.ReportType.JUPYTER:
subprocess.run(["quarto", "convert", os.path.join(output_dir, f"{self.BASE_DIR}.qmd")], check=True)
self.report.logger.info(f"'{self.report.title}' '{self.report_type}' report rendered")
except subprocess.CalledProcessError as e:
self.report.logger.error(f"Error running '{self.report.title}' {self.report_type} report: {str(e)}")
Expand Down Expand Up @@ -174,8 +176,12 @@ def _create_yaml_header(self) -> str:
toc: false
output: true""",
r.ReportType.JUPYTER: """
jupyter:
kernel: python3"""
html:
toc: true
toc-location: left
toc-depth: 3
page-layout: full
self-contained: true"""
}

# Create a key based on the report type and format
Expand Down
8 changes: 7 additions & 1 deletion vuegen/streamlit_reportview.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,13 @@ def run_report(self, output_dir: str = SECTIONS_DIR) -> None:
# If autorun is False, print instructions for manual execution
self.report.logger.info(f"All the scripts to build the Streamlit app are available at {output_dir}")
self.report.logger.info(f"To run the Streamlit app, use the following command:")
self.report.logger.info(f"streamlit run {os.path.join(output_dir, self.REPORT_MANAG_SCRIPT)}")
self.report.logger.info(f"streamlit run {os.path.join(output_dir, self.REPORT_MANAG_SCRIPT)}")
msg = (
f"\nAll the scripts to build the Streamlit app are available at: {output_dir}\n\n"
f"To run the Streamlit app, use the following command:\n\n"
f"\tstreamlit run {os.path.join(output_dir, self.REPORT_MANAG_SCRIPT)}"
)
print(msg)

def _format_text(self, text: str, type: str, level: int = 1, color: str = '#000000', text_align: str = 'center') -> str:
"""
Expand Down
8 changes: 4 additions & 4 deletions vuegen/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
from strenum import StrEnum

from io import StringIO
from pathlib import Path
from typing import Type
from urllib.parse import urlparse
from pathlib import Path

import networkx as nx
import requests
Expand Down Expand Up @@ -145,7 +145,7 @@ def create_folder(directory_path: str, is_nested: bool = False) -> bool:
except OSError as e:
raise OSError(f"Error creating directory '{directory_path}': {e}")

def get_args(prog_name: str, others: dict = {}) -> argparse.Namespace:
def get_parser(prog_name: str, others: dict = {}) -> argparse.Namespace:
"""
Initiates argparse.ArgumentParser() and adds common arguments.

Expand Down Expand Up @@ -193,7 +193,7 @@ def get_args(prog_name: str, others: dict = {}) -> argparse.Namespace:
"-rt",
"--report_type",
type = str,
default = None,
default = 'streamlit',
help = "Type of the report to generate (streamlit, html, pdf, docx, odt, revealjs, pptx, or jupyter)."
)
parser.add_argument(
Expand All @@ -205,7 +205,7 @@ def get_args(prog_name: str, others: dict = {}) -> argparse.Namespace:
)

# Parse arguments
return parser.parse_args()
return parser

def fetch_file_stream(file_path: str) -> StringIO:
"""
Expand Down
Loading