Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8af57d8
CU-869aa22g2: Add ES requirement
mart-r Aug 29, 2025
64f95ad
CU-869aa22g2: Add initial README
mart-r Aug 29, 2025
fcf4d96
CU-869aa22g2: Add (slightly) converted credentials
mart-r Aug 29, 2025
bfdcf82
CU-869aa22g2: Add cogstack module ported from WWC
mart-r Aug 29, 2025
2332a63
CU-869aa22g2: Add search template notebook
mart-r Aug 29, 2025
4dc92aa
CU-869aa22g2: Add ipython dependency (for cogstack and notebook)
mart-r Aug 29, 2025
4f2d463
CU-869aa22g2: Add tqdm dependency
mart-r Aug 29, 2025
61b20f0
CU-869aa22g2: Add pandas dependency
mart-r Aug 29, 2025
ec2a780
CU-869aa22g2: Add a few initial tests
mart-r Aug 29, 2025
0322164
CU-869aa22g2: Remove a bunch of extra whitespace
mart-r Aug 29, 2025
696e669
CU-869aa22g2: Add ruff dependecny
mart-r Aug 29, 2025
90d339b
CU-869aa22g2: Run ruff on cogstack module
mart-r Aug 29, 2025
77b3f3d
CU-869aa22g2: Move to primitives collections for type hinting
mart-r Aug 29, 2025
fe1656f
CU-869aa22g2: Some further linting changes
mart-r Aug 29, 2025
780a3fd
CU-869aa22g2: Refactor cogstack module to make a little more sense
mart-r Aug 29, 2025
403a2e8
CU-869aa22g2: Rename some methods for better descriptions
mart-r Aug 29, 2025
2fd8507
CU-869aa22g2: Rename a method name for better descriptions
mart-r Aug 29, 2025
7e04913
CU-869aa22g2: Minor whitespace fix
mart-r Aug 29, 2025
62b60ca
CU-869aa22g2: Remove path add in notebook
mart-r Aug 29, 2025
92aaabd
CU-869aa22g2: Remove unused import from notebook
mart-r Aug 29, 2025
651a55b
CU-869aa22g2: Import username and password from credentials in case t…
mart-r Aug 29, 2025
e98f647
CU-869aa22g2: Add nbconvert dev-dependency
mart-r Aug 29, 2025
4bd6246
CU-869aa22g2: Add default indices to get fields for
mart-r Aug 29, 2025
ab5dee8
CU-869aa22g2: Improve error handling (avoid hiding stack trace)
mart-r Aug 29, 2025
ab8da2c
CU-869aa22g2: Add default indices in notebook example
mart-r Aug 29, 2025
4b77182
CU-869aa22g2: Update progress bar handling during exception handling
mart-r Aug 29, 2025
3a604cf
CU-869aa22g2: Add default indices in notebook examples
mart-r Aug 29, 2025
921d682
CU-869aa22g2: Add data folder
mart-r Aug 29, 2025
bd4f5bd
CU-869aa22g2 Fix data folder in notebook
mart-r Aug 29, 2025
b1790e0
CU-869aa22g2: Add initial notebook tests
mart-r Aug 29, 2025
42eb024
CU-869aa22g2: Simplify test slightly
mart-r Aug 29, 2025
e570521
CU-869aa22g2: Remove test-time debug output
mart-r Aug 29, 2025
35cce29
CU-869aa22g2: Add assertion and removal of data file created by notebook
mart-r Aug 29, 2025
a0c6fa3
CU-869aa22g2: Add initial workflow
mart-r Aug 29, 2025
95fffd1
CU-869aa22g2: Fix workflow working directory
mart-r Aug 29, 2025
fdea8d8
CU-869aa22g2: Add OpenSearch dependency
mart-r Sep 23, 2025
329f047
CU-869aa22g2: Allow OpenSearch to be used instead of ES
mart-r Sep 23, 2025
379eb06
CU-869aa22g2: Add missing ES/OS helpers import
mart-r Sep 23, 2025
349b7f3
CU-869aa22g2: Fix typo in variable name
mart-r Sep 23, 2025
96857f0
CU-869aa22g2: Fix test time mocking
mart-r Sep 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/cogstack-es_main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: cogstack-es - Test

on:
push:
branches: [ main ]
pull_request:
paths:
- 'cogstack-es/**'
- '.github/workflows/cogstack-es**'

defaults:
run:
working-directory: ./cogstack-es

jobs:
types-lint-tests:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ '3.9', '3.10', '3.11', '3.12' ]
max-parallel: 4

steps:
- uses: actions/checkout@v5
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Check types
run: |
python -m mypy --follow-imports=normal *.py
- name: Lint
run: |
ruff check *.py
- name: Test
run: |
pytest tests
Comment on lines +17 to +42

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI about 2 months ago

To fix the issue, add a permissions block specifying the least necessary privileges for the job. In this workflow, the job does not push changes, create issues, comment on pull requests, or modify repository content—it only checks out code and runs tests/linting. Therefore, limiting permissions to contents: read at the job or workflow root is most appropriate. You can add this block either at the top (outside jobs:) so it applies to all jobs, or directly under the types-lint-tests: job section if you only want to affect that job. The best approach is adding to the root so that future jobs inherit safe defaults.

Edit .github/workflows/cogstack-es_main.yml and insert the block:

permissions:
  contents: read

immediately after the workflow name: and before the on: key.

Suggested changeset 1
.github/workflows/cogstack-es_main.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/cogstack-es_main.yml b/.github/workflows/cogstack-es_main.yml
--- a/.github/workflows/cogstack-es_main.yml
+++ b/.github/workflows/cogstack-es_main.yml
@@ -1,4 +1,6 @@
 name: cogstack-es - Test
+permissions:
+  contents: read
 
 on:
   push:
EOF
@@ -1,4 +1,6 @@
name: cogstack-es - Test
permissions:
contents: read

on:
push:
Copilot is powered by AI and may make mistakes. Always verify output.
27 changes: 27 additions & 0 deletions cogstack-es/ReadMe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@

# Login and search
This project is responsible for logging in and performing a search.

## Login details
1. Create a [credentials.py](credentials.py)
2. Populate it with your cogstack instance and login details
An example template can be seen below:
```
hosts = [] # This is a list of your cogstack elasticsearch instances.

# These are your login details (either via http_auth or API)
username = None
password = None
```

__Note__: If these fields are left blank then the user will be prompted to enter the details themselves.

If you are unsure about the above information please contact your CogStack system administrator.

## How to build a Search query

A core component of cogstack is Elasticsearch which is a search engine built on top of Apache Lucene.

Lucene has a custom query syntax for querying its indexes (Lucene Query Syntax). This query syntax allows for features such as Keyword matching, Wildcard matching, Regular expression, Proximity matching, Range searches.

Full documentation for this syntax is available as part of Elasticsearch [query string syntax](https://www.elastic.co/guide/en/elasticsearch/reference/8.5/query-dsl-query-string-query.html#query-string-syntax).
Loading