Skip to content

Ruby: add XML tables to dbscheme #10863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Ruby: add XML tables to dbscheme #10863

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
77b0d7d
Ruby: add XML tables to dbscheme
aibaars Oct 17, 2022
f10c936
Ruby: add upgrade/downgrade scripts
aibaars Oct 17, 2022
6fc8d04
Ruby: add XML library
aibaars Oct 17, 2022
a13c848
Ruby: changenote
aibaars Oct 17, 2022
a085feb
Ruby: basic XML test
aibaars Oct 17, 2022
c38de11
Ruby: extract xml.erb files when measuring db stats
aibaars Oct 17, 2022
d1d2ca5
Ruby: update stats
aibaars Oct 18, 2022
73d79b4
CI: update actions/cache to v3
aibaars Oct 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/ruby-dataset-measure.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,14 @@ jobs:
strategy:
fail-fast: false
matrix:
repo: [rails/rails, discourse/discourse, spree/spree, ruby/ruby]
repo:
[
rails/rails,
discourse/discourse,
spree/spree,
ruby/ruby,
wikimedia/puppet,
]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand All @@ -40,11 +47,13 @@ jobs:
path: ${{ github.workspace }}/repo
- name: Create database
run: |
codeql database create \
codeql database init \
--search-path "${{ github.workspace }}/ruby/extractor-pack" \
--threads 4 \
--language ruby --source-root "${{ github.workspace }}/repo" \
"${{ runner.temp }}/database"
codeql database trace-command --threads 4 --index-traceless-dbs "${{ runner.temp }}/database"
codeql database index-files -l xml --include "**/*.xml.erb" "${{ runner.temp }}/database"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit surprised that our xml parser handles xml.erb files in the general case. Do you think there could be spurious syntax errors, or will it be guaranteed to work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I don't think it would work in the general case, but it might be alright for all the .xml.erb files in the five projects above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be syntax errors especially on <% token, but it mostly works. Note that this workflow is just for generating stats. Real XML files are pretty rare in Ruby projects, so this was the closest thing I could find.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I don't think it would work in the general case, but it might be alright for all the .xml.erb files in the five projects above.

There are plenty of XML syntax errors in the xml.erb files in the 5 projects we currently use. However, these errors are non-fatal. The purpose is just to gather some stats about XML-like data. For that purpose it works fine.

codeql database finalize "${{ runner.temp }}/database"
- name: Measure database
run: |
mkdir -p "stats/${{ matrix.repo }}"
Expand Down
2 changes: 1 addition & 1 deletion ruby/actions/create-extractor-pack/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ description: Builds the Ruby CodeQL pack
runs:
using: composite
steps:
- uses: actions/cache@v2
- uses: actions/cache@v3
with:
path: |
~/.cargo/registry
Expand Down
Loading