Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] KCL watch system #1212

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions kclvm/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions kclvm/tools/src/LSP/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ im-rc = "15.0.0"
rustc_lexer = "0.1.0"
clap = "4.3.0"
maplit = "1.0.2"
walkdir = "2"

kclvm-tools = { path = "../../../tools" }
kclvm-error = { path = "../../../error" }
Expand Down
56 changes: 56 additions & 0 deletions kclvm/tools/src/LSP/src/config_manager.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
use serde::{Deserialize, Serialize};
use std::path::PathBuf;

#[derive(Serialize, Deserialize, Clone)]
struct JsonFile {
watch: PathBuf,
recursive: Option<bool>,
patterns: Option<Vec<String>>,
}

#[derive(Debug, Clone)]
pub struct Config {
path: PathBuf,
recursive: bool,
patterns: Vec<String>,
}

impl Config {
/// Load configuration from file
pub fn load_from_file(file_path: &PathBuf) -> Result<Self, Box<dyn std::error::Error>> {
let file_content = std::fs::read_to_string(file_path)?;
let config: JsonFile = serde_json::from_str(&file_content)?;
Ok(Config {
path: config.watch,
recursive: config.recursive.unwrap_or(false),
patterns: config.patterns.unwrap_or_default(),
})
}

/// Get the path from configuration
pub fn path(&self) -> &PathBuf {
&self.path
}

/// Check if the configuration is recursive
pub fn is_recursive(&self) -> bool {
self.recursive
}

/// Get the file patterns from configuration
pub fn patterns(&self) -> &Vec<String> {
&self.patterns
}
}

/// Get the configuration file path
pub fn get_config_file() -> Option<PathBuf> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid reading from this configuration file, we only need to maintain scalability at the code level, without the need to open configuration content to users.

let current_dir = std::env::current_dir().ok()?;
let config_path = current_dir.join("observer.json");

if config_path.exists() {
Some(config_path)
} else {
None
}
}
174 changes: 174 additions & 0 deletions kclvm/tools/src/LSP/src/file.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
use crate::config_manager::Config;
use std::collections::HashMap;
use std::fs::Metadata;
use std::path::PathBuf;
use std::sync::Arc;
use std::time::Duration;
use std::time::SystemTime;
use walkdir::DirEntry;
use walkdir::WalkDir;

/// Define a trait for file handlers
pub trait FileHandler: Send + Sync {
fn handle(&self, file: &File);
}

/// File structure to hold file metadata and data
#[derive(Debug, Clone, PartialEq)]
pub struct File {
name: String,
path: PathBuf,
data: FileData,
}

/// Data structure for file metadata
#[derive(Debug, Clone, PartialEq)]
struct FileData {
last_accesed: SystemTime,
last_modified: SystemTime,
}

impl File {
/// Create a new File instance from a DirEntry
pub fn new(file: &DirEntry) -> Self {
let metadata = file.metadata().unwrap();
File {
name: file.file_name().to_str().unwrap().to_string(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoiding the unwrap() function calling.

path: file.path().to_path_buf(),
data: FileData::new(metadata),
}
}

/// Get the file name
pub fn name(&self) -> String {
Arc::new(&self.name).to_string()
}

/// Get the file extension
pub fn extension(&self) -> Option<String> {
self.path
.extension()
.map(|ext| ext.to_string_lossy().to_string())
}

/// Get the display path of the file
pub fn ds_path(&self) -> String {
Arc::new(&self.path)
.to_path_buf()
.to_str()
.unwrap()
.to_string()
}

/// Check if the file was deleted
pub fn was_deleted(&self) -> bool {
!self.path.exists()
}

/// Get the last modification time of the file
pub fn last_modification(&self) -> SystemTime {
self.data.last_modified
}

/// Set the last modification time of the file
pub fn set_modification(&mut self, time: SystemTime) {
self.data.last_modified = time;
}

/// Detect file type based on extension
pub fn detect_file_type(&self) -> Option<String> {
self.extension().map(|ext| {
match ext.as_str() {
"k" => "K File",
"mod" => "Mod File",
"JSON" | "json" => "JSON File",
"YAML" | "yaml" => "YAML File",
_ => "Unknown File Type",
}
.to_string()
})
}
}

impl FileData {
/// Create a new FileData instance from Metadata
pub fn new(metadata: Metadata) -> Self {
FileData {
last_accesed: metadata.accessed().unwrap(),
last_modified: metadata.modified().unwrap(),
}
}
}

/// Define file events
#[derive(Debug)]
pub enum FileEvent {
Modified(File),
}

/// Observer structure to watch files
#[derive(Debug)]
pub struct Observer {
config: Config,
files: HashMap<String, File>,
}

impl Observer {
/// Initialize a new Observer instance with a configuration
pub fn new(config: Config) -> Self {
Observer {
files: get_files(&config),
config,
}
}

/// Iterator for file events
pub fn iter_events(&mut self) -> impl Iterator<Item = FileEvent> + '_ {
let interval = Duration::from_millis(500);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use 500 here?

let last_files = self.files.clone();
std::iter::from_fn(move || {
let current_files = get_files(&self.config);

let mut events = Vec::new();
for (name, file) in current_files.iter() {
if let Some(last_file) = last_files.get(name) {
if file.last_modification() > last_file.last_modification() {
events.push(FileEvent::Modified(file.clone()));
}
}
}
std::thread::sleep(interval);
if !events.is_empty() {
self.files = current_files;
Some(events.remove(0))
} else {
None
}
})
}
}

/// Get files based on configuration
fn get_files(config: &Config) -> HashMap<String, File> {
let files = match config.is_recursive() {
true => WalkDir::new(config.path()).min_depth(1),
false => WalkDir::new(config.path()).min_depth(1).max_depth(1),
}
.into_iter()
.filter(|x| x.as_ref().unwrap().metadata().unwrap().is_file())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoiding the unwrap() function calling. Need to carefully check other places where the unwrap() function is called.

.map(|x| File::new(&x.unwrap()))
.map(|f| (f.name(), f))
.collect::<HashMap<_, _>>();

if config.patterns().is_empty() {
return files;
}
let mut filtered_files = HashMap::new();
for (name, file) in files {
let ext = file.extension().unwrap_or_default();
if config.patterns().contains(&ext) {
filtered_files.insert(name, file);
}
}
filtered_files
}
73 changes: 73 additions & 0 deletions kclvm/tools/src/LSP/src/kcl_watch_system.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
use crate::config_manager::Config;
use crate::file::{FileEvent, FileHandler, Observer};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use std::thread;

/// HandlerRegistry to register and manage file handlers
pub struct HandlerRegistry {
handlers: HashMap<String, Box<dyn FileHandler>>,
}

impl HandlerRegistry {
/// Create a new HandlerRegistry instance
pub fn new() -> Self {
HandlerRegistry {
handlers: HashMap::new(),
}
}

/// Register a handler for a file type
pub fn register_handler(&mut self, file_type: &str, handler: Box<dyn FileHandler>) {
self.handlers.insert(file_type.to_string(), handler);
}

/// Get a handler for a file type
pub fn get_handler(&self, file_type: &str) -> Option<&Box<dyn FileHandler>> {
self.handlers.get(file_type)
}

/// Handle file event
pub fn handle_event(&self, event: &FileEvent) {
match event {
FileEvent::Modified(file) => {
if let Some(handler) =
self.get_handler(&file.detect_file_type().unwrap_or_default())
{
handler.handle(file);
}
}
}
}
}

/// KCL Watch System structure to manage the observer and handler registry
pub struct KCLWatchSystem {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub struct KCLWatchSystem {
pub struct WatchSystem {

observer: Arc<Mutex<Observer>>,
handler_registry: Arc<Mutex<HandlerRegistry>>,
}

impl KCLWatchSystem {
/// Create a new KCL Watch System instance with a configuration
pub fn new(config: Config) -> Self {
let observer = Arc::new(Mutex::new(Observer::new(config.clone())));
let handler_registry = Arc::new(Mutex::new(HandlerRegistry::new()));
KCLWatchSystem {
observer,
handler_registry,
}
}

/// Start the observer
pub fn start_observer(&self) {
let observer = self.observer.clone();
let handler_registry = self.handler_registry.clone();
thread::spawn(move || loop {
let mut observer_lock = observer.lock().unwrap();
let event_opt = observer_lock.iter_events().next();
if let Some(event) = event_opt {
handler_registry.lock().unwrap().handle_event(&event);
}
});
}
}
3 changes: 3 additions & 0 deletions kclvm/tools/src/LSP/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,18 @@ mod analysis;
mod capabilities;
mod completion;
mod config;
mod config_manager;
mod db;
mod dispatcher;
mod document_symbol;
mod error;
mod file;
mod find_refs;
mod formatting;
mod from_lsp;
mod goto_def;
mod hover;
mod kcl_watch_system;
mod main_loop;
mod notification;
mod quick_fix;
Expand Down
24 changes: 21 additions & 3 deletions kclvm/tools/src/LSP/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
use crate::main_loop::main_loop;
use config::Config;
use kcl_watch_system::KCLWatchSystem;
use main_loop::app;

mod analysis;
mod capabilities;
mod completion;
mod config;
mod config_manager;
mod db;
mod dispatcher;
mod document_symbol;
Expand All @@ -28,6 +30,9 @@ mod formatting;
#[cfg(test)]
mod tests;

mod file;
mod kcl_watch_system; // Import the new module
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mod kcl_watch_system; // Import the new module
mod watch_system; // Import the new module


/// Main entry point for the `kcl-language-server` executable.
fn main() -> Result<(), anyhow::Error> {
let args: Vec<String> = std::env::args().collect();
Expand Down Expand Up @@ -60,9 +65,9 @@ fn main() -> Result<(), anyhow::Error> {
#[allow(dead_code)]
/// Main entry point for the language server
fn run_server() -> anyhow::Result<()> {
// Setup IO connections
/// Setup IO connections
let (connection, io_threads) = lsp_server::Connection::stdio();
// Wait for a client to connect
/// Wait for a client to connect
let (initialize_id, initialize_params) = connection.initialize_start()?;

let initialize_params =
Expand All @@ -82,8 +87,21 @@ fn run_server() -> anyhow::Result<()> {
.map_err(|_| anyhow::anyhow!("Initialize result error"))?;

connection.initialize_finish(initialize_id, initialize_result)?;
let config = Config::default();

/// Load configuration from file
let config_file =
config_manager::config::get_config_file().expect("Failed to find configuration file");
let config =
config_manager::Config::load_from_file(&config_file).expect("Failed to load configuration");

/// Create a new KCL Watch System instance
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the normal comment //

let kcl_watch_system = KCLWatchSystem::new(config.clone());

// Start the observer
kcl_watch_system.start_observer();
octonawish-akcodes marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not start the watch system in the language server. Just use the watcher in the kcl language server.


main_loop(connection, config, initialize_params)?;

io_threads.join()?;
Ok(())
}
Expand Down