Entity resolution #848
Merged
Entity resolution #848
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Added intValue to Signal * Update Boeing.sadl * Added intValue * update to accomodate Boeing IDD and Tests * corrected Test_Execution to Test_Procedure (not Test) * obverses moved to IDD_Test * SBVT_Test_Procedure only to SubDD_Req or SRS_Req * RACK UI: display package description if available from manifest. Remove unused imports. * data export and count --model-graph should be optional * using latest semtk and semtk-python3 * add manifest for static turnstile data * Use latest semtk & python with SPARQLgraph URL param: reportId * Update rack-box doc rack-box/README.md: We resolved the issues blocking us from bumping Fuseki to its latest version a while ago so stop mentioning 3.16.0. * RACK UI: add capability to load Turnstile data, run ASSIST data verification (#777) * Compile SADL files Running "cli/setup-owl.sh -b" on RACK's SADL files produces the following changes in addition to generating OWL files (which aren't checked into git): modified: GE-Ontology/OwlModels/ont-policy.rdf new file: sadl-examples/OwlModels/TurnstileSecurity.rules new file: sadl-examples/OwlModels/configuration.rdf new file: sadl-examples/OwlModels/ont-policy.rdf We might as well check these files into git so that anyone else running setup-owl.sh in build mode doesn't end up with modified or untracked files. * Bump Ubuntu (ISO) and Fuseki (tar.gz) Link to Ubuntu iso breaks after every point release, so update link again. Also make sure rack-box uses latest Fuseki version. action.yml: Bump Fuseki tar.gz from 4.5.0 to 4.6.1. rack-box-virtualbox.json: Bump Ubuntu iso from 20.04.4 to 20.04.5. * Prepare for practice v10.9 release * Made compatible with local Windows environment: remove Dash Pages, remove use of callback_context in @dash.callback * Notify user that the 'Verify' feature is not yet supported on Windows * Display ingestion warnings in RACK UI * Set loading log level to ERROR and changed Verify on windows error message * RACK UI provide nice error message if SemTK Services are not reachable * Add example nodegroup * Update README.md * initial manifest documentation * Fix ASSIST-DV to handle owl:allValuesFrom restriction. Previously handled owl:someValuesFrom, but it did not identify a property_extra value_from constraint for owl:allValuesFrom. Also added some convenience functions to help/simplify the higher level check. * sync cli/README with RACK CLI wiki * Turnstile uses rack manifest only; also don't need ingest nodegroups * Hopefully improve virtualbox build rack-box/README.md: Update explanation of steps needed to create files for packer builds. Delete old instructions for using 'act' to run CI locally since it's easier to run real GitHub CI and then examine the logs closely. rack-box/http/user-data: Add sleep 60 in case it helps. rack-box/rack-box-hyperv.json: Update ISO url and checksum (use https instead of http too). Replace boot_command with same new boot_command below. rack-box/rack-box-virtualbox.json: Tweak ISO url (use https instead of http). Add disk_size variable and let it default to usual setting. Replace boot_command with new command verified to work in bento's package_templates/ubuntu. * [ASSIST-DV] ensure property_range_type is exported from rack model. * [ASSIST-DV] ensure blank nodes are not checked for valid target type. * [ASSIST-DV] add check for property constraints with invalid domains. * Fix domain of BDU_CONFIDENCE_ASSESSMENT properties. * Change CONFIDENCE_ASSESSMENT from a THING to an ENTITY. A confidence assessment is something which exists, and is therefore reasonable to be an entity. In addition, it has a wasGeneratedBy property that is not defined for THING but which is defined for ENTITY. This issue was detected by ASSIST-DV with the enhancements in PR#782 with the following report: > ERROR: Property http://arcos.rack/PROV-S#wasGeneratedBy was referenced on class http://arcos.rack/CONFIDENCE#CONFIDENCE_ASSESSMENT, but that property is only defined for the unrelated class http://arcos.rack/PROV-S#ENTITY * [ASSIST-DV] extend invalid property domain checking to derived properties. If a property is derived from a parent property, ensure that the class for the property is valid for the domain of the parent property. * [ASSIST-DV] Refine messaging for property domain reporting. * [cli] Update README "how-to-use" section with new manifest command. * [cli] additional rewording of manifest command preeminence. * [ASSIST-DV] do not return blank nodes as direct properties. * changed property name to avoid overloading * updated changelog * add last minute updates to changelog * Update versions for next release * [ASSIST-DV] no type checking of properties with literal targets. * Use semtk oct-11 and latest semtk-python3 * Update data graphs in turnstile manifests * semtk oct-12 Firefox fix * Use semtk oct-14 with improved get_graph_names() * suppress warnings on arcos setup (#796) * Add SPARQLgraph report verification to RACK UI (#807) * Build VM image with virtualbox-ovf instead of virtualbox-iso The latest Ubuntu 20.04.05 LTS ISO hangs while trying to install Ubuntu, so we are forced to build our VM image from an existing vagrant box instead of building a new Ubuntu VM directly from the ISO. Modify the Packer template and the release workflow accordingly. .github/workflows/release.yml: Add a new step to download the vagrant box in order to build the virtualbox image. rack-box/.gitignore: Ignore the temporary focal64 directory where the vagrant box is unpacked. rack-box/files/GE_External_Root_CA_2_1.crt: Add GE's man-in-the-middle proxy certificate so we can test the Packer template inside the intranet. rack-box/rack-box-docker.json: Reorder some lines alphabetically like the other Packer templates. rack-box/rack-box-hyperv.json: Reorder some lines alphabetically like the other Packer templates. rack-box/rack-box-virtualbox.json: Build the VM using virtualbox-ovf instead of virtualbox-iso. Delete or update some lines to specify the necessary builder parameters. Reorder some lines alphabetically like the other Packer templates. rack-box/scripts/install.sh: Always install some packages from now on since we can't assume that an ISO installer will install them for us anymore. rack-box/vagrant: Add a private ssh key to allow Packer to connect to the vagrant box's VM. Co-authored-by: AbhaMoitra <moitraa@ge.com> Co-authored-by: kityansiu <siu@ge.com> Co-authored-by: Jenny Williams <weisenje@research.ge.com> Co-authored-by: Eric Mertens <emertens@galois.com> Co-authored-by: Paul Cuddihy <cuddihy@research.ge.com> Co-authored-by: John Interrante <interran@research.ge.com> Co-authored-by: Kevin Quick <kquick@galois.com> Co-authored-by: Valentin Robert <val@galois.com> Co-authored-by: Baoluo Meng <baoluo.meng@ge.com>
… allows the SAME_AS to have a dataInsertedBy property without having to have an identifier.
…ion:SameAs (copied in from SemTK). Note mulitple super-classes of SAME_AS. This triggers SemTK to know how to do entity resolution
… allows the SAME_AS to have a dataInsertedBy property without having to have an identifier.
…ion:SameAs (copied in from SemTK). Note mulitple super-classes of SAME_AS. This triggers SemTK to know how to do entity resolution
…ce/RACK into EntityResolution
removing unnecessary file
deleting file as it is no longer in the master branch
removed un used lines
removed unused variable
removed unused variable
removed unused variable
removed unused variable
Get CI working again on EntityResolution branch by replacing the broken files in .github with the master branch's .github files: ~/ARCOS/RACK$ git switch master ~/ARCOS/RACK$ cp -a .github master.github ~/ARCOS/RACK$ git switch EntityResolution ~/ARCOS/RACK$ rm -rf .github ~/ARCOS/RACK$ mv master.github/ .github ~/ARCOS/RACK$ git add .github/ ~/ARCOS/RACK$ git commit -m <this message>
* Added intValue to Signal * Update Boeing.sadl * Added intValue * update to accomodate Boeing IDD and Tests * corrected Test_Execution to Test_Procedure (not Test) * obverses moved to IDD_Test * SBVT_Test_Procedure only to SubDD_Req or SRS_Req * RACK UI: display package description if available from manifest. Remove unused imports. * data export and count --model-graph should be optional * using latest semtk and semtk-python3 * add manifest for static turnstile data * Use latest semtk & python with SPARQLgraph URL param: reportId * Update rack-box doc rack-box/README.md: We resolved the issues blocking us from bumping Fuseki to its latest version a while ago so stop mentioning 3.16.0. * RACK UI: add capability to load Turnstile data, run ASSIST data verification (#777) * Compile SADL files Running "cli/setup-owl.sh -b" on RACK's SADL files produces the following changes in addition to generating OWL files (which aren't checked into git): modified: GE-Ontology/OwlModels/ont-policy.rdf new file: sadl-examples/OwlModels/TurnstileSecurity.rules new file: sadl-examples/OwlModels/configuration.rdf new file: sadl-examples/OwlModels/ont-policy.rdf We might as well check these files into git so that anyone else running setup-owl.sh in build mode doesn't end up with modified or untracked files. * Bump Ubuntu (ISO) and Fuseki (tar.gz) Link to Ubuntu iso breaks after every point release, so update link again. Also make sure rack-box uses latest Fuseki version. action.yml: Bump Fuseki tar.gz from 4.5.0 to 4.6.1. rack-box-virtualbox.json: Bump Ubuntu iso from 20.04.4 to 20.04.5. * Prepare for practice v10.9 release * Made compatible with local Windows environment: remove Dash Pages, remove use of callback_context in @dash.callback * Notify user that the 'Verify' feature is not yet supported on Windows * Display ingestion warnings in RACK UI * Set loading log level to ERROR and changed Verify on windows error message * RACK UI provide nice error message if SemTK Services are not reachable * Add example nodegroup * Update README.md * initial manifest documentation * Fix ASSIST-DV to handle owl:allValuesFrom restriction. Previously handled owl:someValuesFrom, but it did not identify a property_extra value_from constraint for owl:allValuesFrom. Also added some convenience functions to help/simplify the higher level check. * sync cli/README with RACK CLI wiki * Turnstile uses rack manifest only; also don't need ingest nodegroups * Hopefully improve virtualbox build rack-box/README.md: Update explanation of steps needed to create files for packer builds. Delete old instructions for using 'act' to run CI locally since it's easier to run real GitHub CI and then examine the logs closely. rack-box/http/user-data: Add sleep 60 in case it helps. rack-box/rack-box-hyperv.json: Update ISO url and checksum (use https instead of http too). Replace boot_command with same new boot_command below. rack-box/rack-box-virtualbox.json: Tweak ISO url (use https instead of http). Add disk_size variable and let it default to usual setting. Replace boot_command with new command verified to work in bento's package_templates/ubuntu. * [ASSIST-DV] ensure property_range_type is exported from rack model. * [ASSIST-DV] ensure blank nodes are not checked for valid target type. * [ASSIST-DV] add check for property constraints with invalid domains. * Fix domain of BDU_CONFIDENCE_ASSESSMENT properties. * Change CONFIDENCE_ASSESSMENT from a THING to an ENTITY. A confidence assessment is something which exists, and is therefore reasonable to be an entity. In addition, it has a wasGeneratedBy property that is not defined for THING but which is defined for ENTITY. This issue was detected by ASSIST-DV with the enhancements in PR#782 with the following report: > ERROR: Property http://arcos.rack/PROV-S#wasGeneratedBy was referenced on class http://arcos.rack/CONFIDENCE#CONFIDENCE_ASSESSMENT, but that property is only defined for the unrelated class http://arcos.rack/PROV-S#ENTITY * [ASSIST-DV] extend invalid property domain checking to derived properties. If a property is derived from a parent property, ensure that the class for the property is valid for the domain of the parent property. * [ASSIST-DV] Refine messaging for property domain reporting. * [cli] Update README "how-to-use" section with new manifest command. * [cli] additional rewording of manifest command preeminence. * [ASSIST-DV] do not return blank nodes as direct properties. * changed property name to avoid overloading * updated changelog * add last minute updates to changelog * Update versions for next release * [ASSIST-DV] no type checking of properties with literal targets. * Use semtk oct-11 and latest semtk-python3 * Update data graphs in turnstile manifests * semtk oct-12 Firefox fix * Use semtk oct-14 with improved get_graph_names() * suppress warnings on arcos setup (#796) * Add SPARQLgraph report verification to RACK UI (#807) * Build VM image with virtualbox-ovf instead of virtualbox-iso The latest Ubuntu 20.04.05 LTS ISO hangs while trying to install Ubuntu, so we are forced to build our VM image from an existing vagrant box instead of building a new Ubuntu VM directly from the ISO. Modify the Packer template and the release workflow accordingly. .github/workflows/release.yml: Add a new step to download the vagrant box in order to build the virtualbox image. rack-box/.gitignore: Ignore the temporary focal64 directory where the vagrant box is unpacked. rack-box/files/GE_External_Root_CA_2_1.crt: Add GE's man-in-the-middle proxy certificate so we can test the Packer template inside the intranet. rack-box/rack-box-docker.json: Reorder some lines alphabetically like the other Packer templates. rack-box/rack-box-hyperv.json: Reorder some lines alphabetically like the other Packer templates. rack-box/rack-box-virtualbox.json: Build the VM using virtualbox-ovf instead of virtualbox-iso. Delete or update some lines to specify the necessary builder parameters. Reorder some lines alphabetically like the other Packer templates. rack-box/scripts/install.sh: Always install some packages from now on since we can't assume that an ISO installer will install them for us anymore. rack-box/vagrant: Add a private ssh key to allow Packer to connect to the vagrant box's VM. Co-authored-by: AbhaMoitra <moitraa@ge.com> Co-authored-by: kityansiu <siu@ge.com> Co-authored-by: Jenny Williams <weisenje@research.ge.com> Co-authored-by: Eric Mertens <emertens@galois.com> Co-authored-by: Paul Cuddihy <cuddihy@research.ge.com> Co-authored-by: John Interrante <interran@research.ge.com> Co-authored-by: Kevin Quick <kquick@galois.com> Co-authored-by: Valentin Robert <val@galois.com> Co-authored-by: Baoluo Meng <baoluo.meng@ge.com>
removing unnecessary file
deleting file as it is no longer in the master branch
removed un used lines
removed unused variable
removed unused variable
removed unused variable
removed unused variable
Get CI working again on EntityResolution branch by replacing the broken files in .github with the master branch's .github files: ~/ARCOS/RACK$ git switch master ~/ARCOS/RACK$ cp -a .github master.github ~/ARCOS/RACK$ git switch EntityResolution ~/ARCOS/RACK$ rm -rf .github ~/ARCOS/RACK$ mv master.github/ .github ~/ARCOS/RACK$ git add .github/ ~/ARCOS/RACK$ git commit -m <this message>
…ce/RACK into EntityResolution
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.