-
Notifications
You must be signed in to change notification settings - Fork 0
Update all the docs #133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update all the docs #133
Conversation
Add two new reference documents: - eql-functions.md: Complete API reference for all 40+ EQL functions organized by category (configuration, query, JSONB, array, helpers, aggregates). Includes both operator and function forms with examples. - database-indexes.md: PostgreSQL B-tree index guide covering creation, usage requirements, troubleshooting, and best practices. Based on operator_class_test.sql implementation. These documents provide the missing comprehensive reference material for EQL's public API and PostgreSQL index integration.
Complete rewrite of json-support.md to match actual EQL API: - Remove references to non-existent functions (ste_vec_value, ste_vec_term, ste_vec_terms) - Document actual functions: jsonb_path_query, jsonb_array_elements, jsonb_path_exists, grouped_value - Show correct operator usage (@>, <@, ->, ->>) - Add proper selector-based querying examples - Reduce from 887 to 297 lines, removing outdated content - Add explanation of how ste_vec indexing works All examples now verified against test files and source code.
Fix multiple documentation issues verified against implementation: - index-config.md: Correct function names from cs_* to eql_v2.*, add missing cast types (real, double) - proxy-configuration.md: Change from wrapped function calls to direct operator usage (= instead of hmac_256() wrappers), add function alternatives for Supabase - docs/README.md: Fix broken links, add references to new documentation (database-indexes.md, eql-functions.md) All changes verified against test files to ensure accuracy.
Add comprehensive user guidance to main README.md: - Add step-by-step getting started example with SQL code showing table creation, column configuration, and index setup - Add troubleshooting section with common errors, causes, and solutions - Clarify that CipherStash Proxy/Protect.js are required for encryption - Update TOC: change "Developing" to "Contributing" - Improve installation instructions and upgrade process Makes the README more actionable for new users with concrete examples and solutions to common setup issues.
Add "Adding SQL" section to DEVELOPMENT.md with guidelines for: - Modular SQL file organization - Operator wrappers vs implementation separation - Dependency declarations using REQUIRE comments - Test file naming conventions - Build system usage with tsort Provides clear guidance for contributors developing new SQL features.
Fix critical documentation issues and add missing function documentation: - Fix return type names: change eql_v2_* to eql_v2.* for index term extraction functions (hmac_256, blake3, bloom_filter, ore_block_u64_8_256) - Add missing function equivalents for Supabase compatibility: ilike(), lt(), lte(), gt(), gte() - Document configuration lifecycle functions: migrate_config(), activate_config(), discard(), reload_config() - Add aggregate functions: min(), max() - Document jsonb overloads for JSONB path and array functions - Add new Utility Functions section with version(), to_encrypted(), to_jsonb(), check_encrypted() - Standardize terminology: change "context" to "prefix" for ste_vec index configuration - Update table of contents to include new sections This update brings the documentation in line with actual implementation and test behavior, addressing 15+ previously undocumented functions.
| - Recreate the index, OR | ||
| - Truncate and repopulate the table | ||
|
|
||
| **Correct order:** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens when we are using Protect? Do those functions do something special?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch.
The add_search_config function is just how the config is added when using EQL/Proxy.
For protect.js, assuming the config is setup and EQL is installed, just needs to be:
CREATE INDEX ON users (encrypted_email eql_v2.encrypted_operator_class);
Maybe these docs should mention the config in more general terms, and explicitly reference other config options?
Definitely another step here to take this reference documentation and turn into reference for protect and support content for website.
freshtonic
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given the prior state of the docs this is a massive improvement.
Added non-blocker comment about improving code examples.
| ``` | ||
| -- Range comparison - use comparison operators directly | ||
| SELECT * FROM events | ||
| WHERE encrypted_date < '{"v":2,"k":"pt","p":"2023-10-05","i":{"t":"events","c":"encrypted_date"}}'::eql_v2_encrypted; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a blocker but more of a seed to think about how we can improve this later. The raw payload is very noisy and grabs attention when what we really care about here is drawing attention to the <.
Same for the other examples.
- Fix check_encrypted() description to reflect actual validation (v,c,i fields not v,k,i) - Update check_encrypted() behavior: raises exceptions instead of returning false - Fix version() example to not show hardcoded version string - Enhance JSON path operator examples with specific selector types (text, encrypted, integer) - Add array index access example for -> operator
U̶p̶d̶a̶t̶e̶ Rewrite all the docs
I've completed a thorough and intensive documentation review of the Encrypt Query Language (EQL) project.
Major Changes
jsonb_array_elements, operators)
Cross-References Established
All documentation files now have proper cross-references:
Key Corrections Made
Verification Process
All documentation has been verified against: