Refactor Client package to use shared Lectern types

[joneubank]

This is a major refactor of the lectern client code. The starting point was the code as copied from js-lectern-client. The objective of the refactor is to:

• Use Lectern Dictionary types throughout the Lectern Client
• Remove type assertions and use of any types
• Organize code into logical modules
• Remove unused or out of scope modules (parallelization solution)
• Keep the package exports as unchanged as possible (may be changed in future refactoring steps)

Code Reorganization

Actions Taken on Original Code

• src/change-analyzer.ts: Provides a different structure for reviewing changes retrieved from the lectern server diff between schemas
  • Code is kept, moved into directory src/changeAnalyzer
  • Appears to be built for ARGO project, specific ARGO dictionary meta properties were removed from this work
  • This whole module should be considered for removal in the future
• src/schema-entities.ts: Types to support client, most were duplicate of the shared lectern dictionary types
  • Removed duplicate lectern dictionary types
  • Unique client types were about the data submitted for validation, these were moved to src/types/dataRecord.ts
  • Schema validation error types were relocated alongside the validation code
  • Change Analysis types were moved alongside the changeAnalyzer code
• src/schema-functions.ts: This included code for processing raw TSV data, organizing validation pipelines, and all the validation logic for each restriction type
  • the main exports here were the raw data processing functions, these were moved to a module src/processing
  • the remaining code is mostly related to validating data and utility functions to assist in this, this was reorganized into several sub-modules in src/validation
• src/schema-error-messages.ts: Code used for generating the error content for validation and processing failures
  • Error message formats were strictly typed for each validation type in src/validation/types/validationErrorTypes.ts
  • Functions for generating each type of error message now live alongside the validation code
• src/schema-rest-client.ts: Data fetching from a Lectern server
  • Remained mostly unchanged, move to src/rest/index.ts
• src/utils.ts: Type checking functions and a few other formatting utilities
  • Left mostly unchanged in the same location
  • Checks namespace and functions deleted, all references to this were removed
• src/parallel.ts and src/schema-worker.ts: Node server solution for running the process function in a worker thread
  • These were completely removed as out of scope for the client. Server's are free to write their own functionality for running processing in parallel or in the main thread, as they need. The client has the objective of being useful in any server (not just NodeJS) and also in the browser.

New Code Structure

src/
├── changeAnalysis
├── processing
├── rest
├── types
└── validation/
    ├── dataRecordValidation
    ├── fieldRestrictions
    ├── schemaRestrictions
    ├── types
    └── utils

• src/changeAnalysis: Processing for schema diffs retrieved from Lectern Server. Appears to be added for ICGC ARGO. Consider for removal from Lectern Client unless a general need for this is identified.
• src/processing: Raw data processing. Exports the functions that raw TSV data is passed to to be checked vs a Lectern dictionary. Relies heavily on the validation module.
• src/rest: Rest client to fetch dictionaries and diffs from Lectern. This can be heavily expanded in future work to interact with all of Lectern's REST features.
• src/types: TS types to represent the TSV data records provided for validation.
• src/validation: Functionality to validate data records.
  • src/validation/dataRecordValidation: Validation code for unprocessed data records. Includes checks such as confirming the all columns are provided, no unrecognized columns are provided, and that the data provided is the expected type (string/number/boolean, an array, etc.)
  • src/validation/fieldRestrictions: Validation code for a single field in a data record. Includes checks such as Regex, Range, Scripts etc.
  • src/validation/schemaRestrictions: Validation code for an entire schema or data dictionary. Includes the UniqueKey and ForeignKey checks
  • src/validation/types: Validation function structure and error response types
  • src/validation/utils: Utilities for interacting with data records used by several validation functions