Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
6b021be
README update
LukasGold Oct 25, 2024
c7d7c2a
import_with_fallback implented and some reformatting
LukasGold Oct 25, 2024
81b3d66
Doc string reformatting
LukasGold Nov 6, 2024
d7824fc
Reformatting
LukasGold Nov 6, 2024
2cbac21
Bug fixes and doc improvements to express.py
LukasGold Nov 6, 2024
df5cecd
Fix import_with_fallback()
LukasGold Nov 6, 2024
22bbbaf
Improvements to usability and readability
LukasGold Nov 6, 2024
01f0b1d
Fix cases where the default_factory would not be present in the inher…
LukasGold Nov 7, 2024
c104e39
Fix: import_with_fallback() & Context Manager for osw_download_file()
LukasGold Nov 15, 2024
91bd5a2
Moving some functionality from functions in osw.utils.wiki to methods…
LukasGold Nov 15, 2024
8539b95
Making SearchParam available at WtSite.SearchParam
LukasGold Nov 15, 2024
b7222f3
Making openpyxl a requirement for dataimport
LukasGold Nov 15, 2024
5b5060c
Improvements to import_utility.py - potentially breaking some functio…
LukasGold Nov 15, 2024
13acb67
Renaming of casted -> transformed
LukasGold Nov 16, 2024
76bec5e
Fixing failing tests due to missing methods (on classes)
LukasGold Nov 17, 2024
f63df22
Moving NoneType to a separate module
LukasGold Nov 18, 2024
df680e0
Validate attributes of OswBaseModel on assignment
LukasGold Nov 18, 2024
57957a1
Fixing validation errors, causing tests to fail
LukasGold Nov 18, 2024
2967d43
A collection of the param objects defined in the package
LukasGold Nov 18, 2024
632a0a1
Introducing change id and StoreEntityResult
LukasGold Nov 18, 2024
a42fddd
Resolving circular import due to identical naming of osw.types
LukasGold Nov 18, 2024
ee3def9
Minor improvements to express functions
LukasGold Nov 18, 2024
b279ebf
Small fix to osw.core.OSW.StoreEntityParam
LukasGold Nov 18, 2024
beffddf
Still using pydantic.v1
LukasGold Nov 18, 2024
3f1d047
Fixes to QueryInstancesParam & some readability improvements
LukasGold Nov 19, 2024
d908a41
Fixing validation error (introduced earlier)
LukasGold Nov 19, 2024
81c51d2
Accompanying changes in the Category:Entity schema
LukasGold Nov 20, 2024
8d61b9c
Deleting empty strings from jsondata to avoid validation errors
LukasGold Nov 21, 2024
2af7054
Added method to delete a list of pages
LukasGold Nov 21, 2024
1b2bf1f
Fixing params.py: Enums can't be inherited, added new param object
LukasGold Nov 21, 2024
e22b0e4
Applying template for Entity.name & Introducing Meta.uuid
LukasGold Nov 21, 2024
665a189
Merge pull request #93 from OpenSemanticLab/feat-add-rdf-generation
LukasGold Nov 21, 2024
c434777
Fixing: length of dictionary changed during iteration
LukasGold Nov 22, 2024
f8959d2
apply_overwrite_policy() should apply the removal of empty strings
LukasGold Nov 22, 2024
0d9b87b
Fix: Validation error for Entity.label
LukasGold Nov 22, 2024
d316236
First tutorial draft
LukasGold Oct 16, 2024
db0d59e
Continued content creation
LukasGold Oct 25, 2024
5846338
Content added and reworked
LukasGold Nov 6, 2024
6606316
Renaming of casted -> transformed
LukasGold Nov 7, 2024
557866f
README.md update to account for tutorial
LukasGold Nov 7, 2024
0ae37ca
Rework tutorial notebook to incorporate tests and feedback, as well a…
LukasGold Nov 15, 2024
403adbc
Draft of start_tutorial.ipynb - a notebook to start from zero
LukasGold Nov 18, 2024
e093f41
More content: semantic search
LukasGold Nov 21, 2024
ac7f8f3
Added optional section to install & run tutorial in online notebook s…
LukasGold Nov 25, 2024
0bb9ec9
Moving Entity.__init__ to static.py to avoid overwrite by fetch_schema
LukasGold Nov 25, 2024
95fe87e
applying remove_empty instead of remove_empty_string
LukasGold Nov 25, 2024
aa7a238
Renaming custom FilePath to PossibleFilePath
LukasGold Nov 25, 2024
ed7f4de
Additional info on usage in Google Colab
LukasGold Nov 25, 2024
2edc49b
Fixing jsonschema in example
LukasGold Nov 25, 2024
02e6273
Deactivating test_statement_creation() for now
LukasGold Nov 25, 2024
80e7004
Some usability features
LukasGold Nov 25, 2024
48f1c81
Improved readability
LukasGold Nov 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
[![PyPI-Server](https://img.shields.io/pypi/v/osw.svg)](https://pypi.org/project/osw/)
[![DOI](https://zenodo.org/badge/458130867.svg)](https://zenodo.org/badge/latestdoi/458130867)
[![Coveralls](https://img.shields.io/coveralls/github/OpenSemanticLab/osw-python/main.svg)](https://coveralls.io/r/<USER>/osw)
[![Coveralls](https://img.shields.io/coveralls/github/OpenSemanticLab/osw-python/main.svg)](https://coveralls.io/r/OpenSemanticLab/osw)
[![docs](xx.xx)](https://opensemanticlab.github.io/osw-python/)
![license](https://img.shields.io/github/license/OpenSemanticLab/osw-python.svg)
<!-- [![Pydantic v2](https://img.shields.io/endpoint?url=https://raw.githubusercontent.
com/pydantic/pydantic/main/docs/badge/v2.json)](https://pydantic.dev) -->
[![Project generated with PyScaffold](https://img.shields.io/badge/-PyScaffold-005CA0?logo=pyscaffold)](https://pyscaffold.org/)

# osw

Python toolset for data processing, queries, wikicode generation and page manipulation within OpenSemanticLab.
General features for object oriented interaction with knowledge graphs are planned to be moved to a standalone package: [oold-python](https://github.com/OpenSemanticWorld/oold-python)

General features for object-oriented interaction with knowledge graphs are planned to be moved to a standalone package:
[oold-python](https://github.com/OpenSemanticWorld/oold-python)

## Installation
```
Expand Down Expand Up @@ -47,6 +53,10 @@ os.environ["PATH"] += os.pathsep + "/home/jovyan/.local/bin"

https://opensemanticlab.github.io/osw-python/

### Tutorials

Tutorials can be found at [tutorials](docs/tutorials/)
- [Basics](docs/tutorials/basics.md) - a description of the data model of OpenSemanticLab and how to interact with it

## Development

Expand Down
1,476 changes: 1,476 additions & 0 deletions docs/tutorials/basics.ipynb

Large diffs are not rendered by default.

Binary file added docs/tutorials/img/osw_intro_data_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
19 changes: 19 additions & 0 deletions examples/clean_demo_platform.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from pathlib import Path

import osw.params as prm
from osw.express import OswExpress

osw_obj = OswExpress(
domain="demo.open-semantic-lab.org", cred_filepath=Path("accounts.pwd.yaml")
)
orgs = osw_obj.site.semantic_search(
prm.SearchParam(
query="[[HasType::Category:OSW1969007d5acf40539642877659a02c23]]", debug=False
)
)
persons = osw_obj.site.semantic_search(
prm.SearchParam(
query="[[HasType::Category:OSW44deaa5b806d41a2a88594f562b110e9]]", debug=False
)
)
osw_obj.site.delete_page(orgs + persons)
1 change: 0 additions & 1 deletion examples/use_express_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
overwrite=True, # Required if file already exists
)
local_file_path = local_file.path
local_file.close() # required to release the file lock

# Open a file with context manager directly from an OSW instance
with osw_download_file(
Expand Down
2 changes: 1 addition & 1 deletion scripts/migration/file_page_migration.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
import osw.wiki_tools as wt
from osw.core import OSW
from osw.model.entity import Label, WikiFile
from osw.utils.regex import match_first_regex_pattern, test_regex_pattern
from osw.utils.regex_pattern import REGEX_PATTERN_LIB, REGEX_PATTERN_LIST
from osw.utils.strings import match_first_regex_pattern, test_regex_pattern
from osw.utils.util import parallelize
from osw.wtsite import WtPage, WtSite

Expand Down
3 changes: 3 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,11 @@ S3 =
dataimport =
geopy
deepl
openpyxl
UI =
pysimplegui
tutorial =
%(dataimport)s
all =
%(dataimport)s
%(DB)s
Expand Down
14 changes: 10 additions & 4 deletions src/osw/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
from warnings import warn

import yaml
from pydantic.v1 import FilePath, PrivateAttr
from pydantic.v1 import PrivateAttr

from osw.custom_types import PossibleFilePath
from osw.model.static import OswBaseModel

CREDENTIALS_FN_DEFAULT = "credentials.pwd.yaml"
Expand All @@ -17,9 +18,13 @@
class CredentialManager(OswBaseModel):
"""Handles credentials"""

cred_filepath: Optional[Union[Union[str, FilePath], List[Union[str, FilePath]]]]
cred_filepath: Optional[
Union[Union[str, PossibleFilePath], List[Union[str, PossibleFilePath]]]
]
"""Filepath to yaml file with credentials for osw and connected services"""
cert_filepath: Optional[Union[Union[str, FilePath], List[Union[str, FilePath]]]]
cert_filepath: Optional[
Union[Union[str, PossibleFilePath], List[Union[str, PossibleFilePath]]]
]
"""Filepath to the certificates for osw and connected services"""

_credentials: List[BaseCredential] = PrivateAttr([])
Expand Down Expand Up @@ -233,7 +238,7 @@ def iri_in_file(self, iri: str) -> bool:

def save_credentials_to_file(
self,
filepath: Union[str, FilePath] = None,
filepath: Union[str, PossibleFilePath] = None,
set_cred_filepath: bool = False,
):
"""Saves the in memory credentials to a file
Expand All @@ -254,6 +259,7 @@ def save_credentials_to_file(
if self.cred_filepath is None:
filepath_ = [Path.cwd() / CREDENTIALS_FN_DEFAULT]
if set_cred_filepath:
# Creates error if file does not exist -> Using custom FilePath
self.cred_filepath = filepath_
for fp in filepath_:
file = Path(fp)
Expand Down
3 changes: 2 additions & 1 deletion src/osw/controller/file/remote.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
from osw.controller.file.base import FileController
from osw.core import model

# TODO: add addional remove file with https://docs.prefect.io/2.11.4/concepts/filesystems/
# TODO: add additional remove file with
# https://docs.prefect.io/2.11.4/concepts/filesystems/


# Note: the order of the base classes is important
Expand Down
2 changes: 1 addition & 1 deletion src/osw/controller/page_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from osw.model import page_package as package
from osw.model.page_package import NAMESPACE_CONST_TO_NAMESPACE_MAPPING
from osw.model.static import OswBaseModel
from osw.utils.strings import RegExPatternExtended
from osw.utils.regex import RegExPatternExtended
from osw.wtsite import WtSite

# Definition of constants
Expand Down
Loading
Loading