Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo database server with REST API #2182

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from

Commits on Apr 9, 2024

  1. Added the first cut of the db_demo. REST api to insert, update, delet…

    …e, query key-values. Data is currently held in a map.
    Alexander Mihail committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    62e92f7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3fc909d View commit details
    Browse the repository at this point in the history
  3. Added a Disk back-end to Cache. It currently simply creates/opens two…

    … text files for Index and Data. Keys go to Index, whereas Values go to the Data file.
    Alexander Mihail committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    d02fba8 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. Better Lock. Cache renamed to Table. Added /?op=list, /?op=invalidate…

    …, /?op=purge, /?op=compact. Repaired the overall logic of the server and disk persistence. More REST commands to: /quit, /config?trace_level=1111, /config?max_cache=1. Enhanced the demos/rest.cmd with a more complex workflow to demonstrate fragmentation and defragmentation of keys and values files.
    Alexander Mihail committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    83f811f View commit details
    Browse the repository at this point in the history
  2. Added handler_base class-based route handlers in preparation for long…

    … responses. Test batch rest.cmd exits after 100 iterations.
    Alexander Mihail committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    4b5f815 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2024

  1. A much more serious db server. REST api reworked to handle the master…

    … table as well as any other user-table. The root of the web server is a generated HTML page with navigation for all the system features. There is now a map of tables. Selecting the current table is with the /tablename/?use rest command. Tables can also be dropped. Table::compact was reworked to shift-up deleted keys in the index. An index_revolver coroutine was introduced to return one index per invocation. This logic also has index cleanup capabilities. The Table::list() command uses this coroutine to produce the list of all keys. Table::get/set are now boolean. The set prevents overinserting and updating of unknown keys. The get returns false if the key is unknown. This allows now for inserting or updating "null" values. Better statistics. Better formatted output. All REST logic is in one DefaultHandle where the table list lock is maneaged. Everything the DB server does is listed in the / page which is built by DefaultHandle::build_help. Added the new Table::self_test method and the corresponding /?selftest=0 REST command. This system of self-testing obsoletes the rest.bat file while also aiming at raw performance.
    
    This version has problems with deleting keys that are in cache. TODO.
    This version closes issues: #1, #2, #3, #4, #5, #6, #8. It advances on issues #13 and #11.
    Alexander Mihail committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    4df6de3 View commit details
    Browse the repository at this point in the history
  2. Added REST API to read the keys and values files. Rethinking locking.…

    … Closes issue: #14.
    Alexander Mihail committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    a241280 View commit details
    Browse the repository at this point in the history
  3. Removed files that shouldn't be in the sources of SeaStar.

    Alexander Mihail committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    3315c01 View commit details
    Browse the repository at this point in the history
  4. Removed files that shouldn't be in the sources of SeaStar.

    Alexander Mihail committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    c034324 View commit details
    Browse the repository at this point in the history
  5. Removed files that shouldn't be in the sources of SeaStar.

    Alexander Mihail committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    fab1bd3 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2024

  1. Repaired the key deletion code path. Closes bug #13. Added loops to s…

    …elf_test.
    Alexander Mihail committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    86d4823 View commit details
    Browse the repository at this point in the history
  2. Table::Index::access is the new, low-level, file IO method. All file …

    …IO code should be reduced to this. Much better trace messages. Self tests. Corrections so that the self tests pass.
    Alexander Mihail committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    54dbf69 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Reworked Table::set to use Index::access for low-level file IO. Close…

    …s issue #16 regarding the encapsulation of IO operations. This version has a self-test as per issue #11.
    Alexander Mihail committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    277dfe2 View commit details
    Browse the repository at this point in the history
  2. Reworked Table::set to use Index::access for low-level file IO. Close…

    …s issue #16 regarding the encapsulation of IO operations. This version has a self-test as per issue #11.
    Alexander Mihail committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    91b9834 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Using a future to quit the process but have the browser navigate back…

    … to the main page before that. This is to avoid quit-restarting in a loop. Perparing to async-ify. A global and configurable mode_async. Added a rest provision to read the console log, if available. Reporting the sub-log of a Failed test when running in a loop. Git-ignoring Visual Studio temporary files and user-files of cmake. Working on issue #17.
    Alexander Mihail committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    e7df9dc View commit details
    Browse the repository at this point in the history
  2. Using a timer future to quit the process but have the browser navigat…

    …e back to the main page before that. This is to avoid quit-restarting in a loop. Perparing to async-ify. A global and configurable mode_async.
    
    Added the static Table::co_self_test coroutine to wrap around run Table::self_test. The HTTP DefaultHandler calls either the self_test or the co_self_test depending on mode_async. In async mode, the self-test, which is a huge time penalty, is coroutined in the hope that it would split-up into individual, smaller, batches for the SHARD executor. This is to increase parallelism of the HTTP server when multiple users are concurrently active. Table::self_test is recursive in loop mode, and potentially endless. Merged the Prometheus sats server into the Database Demo server so only one port is exposed from the demo container. Added a rest provision to read the console log, if available. Reporting the sub-log of a Failed test when running in a loop. Git-ignoring Visual Studio temporary files and user-files of cmake. Working on issue #17.
    Alexander Mihail committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    16dcc78 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Avoid re-logging the console log.

    Alexander Mihail committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    3b80cd1 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Resolves #15. Got rid of my own non-concrete Generator template for t…

    …he index revolver, in favor of co_await table.co_index_locked() which is used by future Table::list and future Table::rownext.
    Alexander Mihail committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    cb2eb71 View commit details
    Browse the repository at this point in the history
  2. Turned the most important logic of class Table to async. Dropped the …

    …configurable sync/async table mode, as now there's just the async mode. Simplified get to return bool, internally use _get that returns iterator, which is also used by set. And it seems that I'm nolonger getting SIG35, so perhaps this closes #17.
    Alexander Mihail committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    763d01c View commit details
    Browse the repository at this point in the history