Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
52 changes: 52 additions & 0 deletions LICENSE-binary
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,10 @@ software.amazon.awssdk:bundle:2.35.4
software.amazon.s3.analyticsaccelerator:analyticsaccelerator-s3:1.3.1
net.jodah:failsafe:2.4.4

atlaskit/pragmatic-drag-and-drop 1.7.4
atlaskit/pragmatic-drag-and-drop-auto-scroll 2.1.1
atlaskit/pragmatic-drag-and-drop-hitbox 1.1.0
typescript 5.8.3
--------------------------------------------------------------------------------
This product bundles various third-party components under other open source
licenses. This section summarizes those components and their licenses.
Expand Down Expand Up @@ -551,6 +555,47 @@ select2 v4.0.0
snippet-ss v1.11.0
spin.js v2.3.2

hookform/resolvers 5.1.1
radix-ui/react-accordion 1.2.11
radix-ui/react-checkbox 1.3.2
radix-ui/react-collapsible 1.1.11
radix-ui/react-context-menu 2.2.15
radix-ui/react-dialog 1.1.14
radix-ui/react-dropdown-menu 2.1.15
radix-ui/react-label 2.1.7
radix-ui/react-popover 1.1.14
radix-ui/react-progress 1.1.7
radix-ui/react-scroll-area 1.2.9
radix-ui/react-select 2.2.5
radix-ui/react-separator 1.1.7
radix-ui/react-slot 1.2.3
radix-ui/react-switch 1.2.5
radix-ui/react-tabs 1.1.12
radix-ui/react-toggle 1.1.9
radix-ui/react-toggle-group 1.1.10
radix-ui/react-tooltip 1.2.7
react-router/node 7.5.3
react-router/serve 7.5.3
xyflow/react 12.8.1
clsx 2.1.1
cmdk 1.1.1
dagre 0.8.5
es-toolkit 1.39.6
eslint 9.18.0
immer 10.0.0
nanoid 5.1.5
prettier 3.5.0
react 19.1.0
react-resizable-panels 3.0.3
sonner 2.0.5
tailwind-merge 3.3.1
tailwindcss 4.1.4
vaul 1.1.2
vite 6.4.1
vitest 3.2.4
zod 3.25.71
zustand 5.0.6

com.microsoft.azure:azure-cosmosdb:2.4.5
com.microsoft.azure:azure-cosmosdb-commons:2.4.5
com.microsoft.azure:azure-cosmosdb-direct:2.4.5
Expand Down Expand Up @@ -633,10 +678,17 @@ JDOM License
org.jdom:jdom2:2.0.6.1


ISC License
-----------

lucide-react 0.525.0


Public Domain
-------------

aopalliance:aopalliance:1.0
isbot 5.1.27

Dom4J license
-------------
Expand Down
4 changes: 2 additions & 2 deletions dev-support/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ pipeline {
environment {
YETUS='yetus'
// Branch or tag name. Yetus release tags are 'rel/X.Y.Z'
YETUS_VERSION='rel/0.14.0'
YETUS_VERSION='a7d29a6a72750a0c5c39512f33945e773e69303e'
}

parameters {
Expand All @@ -71,7 +71,7 @@ pipeline {
checkout([
$class: 'GitSCM',
branches: [[name: "${env.YETUS_VERSION}"]],
userRemoteConfigs: [[ url: 'https://github.com/apache/yetus.git']]]
userRemoteConfigs: [[ url: 'https://github.com/ayushtkn/yetus.git']]]
)
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
# Apache Hadoop YARN Capacity Scheduler UI

A modern React-based web interface for managing and configuring the YARN Capacity Scheduler.

## Overview

The YARN Capacity Scheduler UI provides an intuitive graphical interface for:
- **Queue Management**: Visual queue hierarchy with drag-and-drop organization
- **Capacity Configuration**: Interactive capacity allocation and resource management
- **Node Labels**: Partition management and node-to-label assignments
- **Placement Rules**: Configure application placement policies
- **Global Settings**: Manage scheduler-wide configuration parameters
- **Configuration Validation**: Real-time validation with helpful error messages
- **Change Staging**: Review and apply configuration changes in batches

## Technology Stack

- **Framework**: React 19 with TypeScript (strict mode)
- **Build Tool**: Vite 6.4 with React Router v7 (SPA mode)
- **UI Components**: shadcn/ui, Radix UI, Tailwind CSS
- **State Management**: Zustand with Immer
- **Visualization**: XYFlow (React Flow) for queue hierarchy
- **Testing**: Vitest + React Testing Library

## Building

### Prerequisites

- **Maven**: 3.6.0 or later
- **Node.js**: 22.16.0+ (automatically installed by Maven during build)
- **Java**: JDK 8 or later

### Maven Build

Build the UI as part of the YARN build using the `yarn-ui` profile:

```bash
# From hadoop-yarn directory
cd hadoop-yarn-project/hadoop-yarn
mvn clean package -Pyarn-ui

# From hadoop-yarn-capacity-scheduler-ui directory
cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui
mvn clean package -Pyarn-ui
```

This will:
1. Install Node.js 22.16.0 and npm locally in `target/webapp/node/`
2. Install npm dependencies
3. Build the React application
4. Package everything into a WAR file at `target/hadoop-yarn-capacity-scheduler-ui-*.war`

### Build Output

The build creates:
- `target/webapp/build/client/` - Built React application (static files)
- `target/hadoop-yarn-capacity-scheduler-ui-*.war` - Deployable WAR file

## Development

### Local Development Setup

```bash
cd src/main/webapp

# Install dependencies
npm install

# Start development server
npm run dev
```

The development server runs at `http://localhost:5173` with hot module replacement.

### Environment Variables

Create a `.env` file in `src/main/webapp/` based on `.env.example`:

```bash
# Mock mode: "static" (use mock data), "cluster" (proxy to real cluster), "off" (no mocking)
VITE_API_MOCK_MODE=static

# YARN ResourceManager URL (required when VITE_API_MOCK_MODE=cluster)
VITE_CLUSTER_PROXY_TARGET=http://localhost:8088

# Development flags
VITE_READONLY_MODE=false
VITE_YARN_USER_NAME=admin
```

### Available Scripts

```bash
npm run dev # Start development server
npm run build # Production build
npm run start # Preview production build
npm run test # Run tests
npm run test:ui # Run tests with UI
npm run test:coverage # Generate test coverage report
npm run lint # Lint code
npm run lint:fix # Fix linting issues
npm run format # Format code with Prettier
```

### Running Tests

```bash
# Run all tests
npm test

# Run tests in watch mode
npm run test

# Run tests with coverage
npm run test:coverage

# Run tests with UI
npm run test:ui
```

## Deployment

### WAR Deployment

The built WAR file can be deployed to a servlet container (Tomcat, Jetty, etc.) or integrated into the YARN ResourceManager web application.

### API Endpoints

The UI requires access to the following YARN ResourceManager REST API endpoints:

- `GET /ws/v1/cluster/scheduler` - Get scheduler configuration
- `PUT /ws/v1/cluster/scheduler-conf` - Update scheduler configuration
- `GET /ws/v1/cluster/scheduler-conf` - Get mutable configuration
- `GET /ws/v1/cluster/nodes` - Get cluster nodes
- `GET /conf` - Get Hadoop configuration

### Integration with ResourceManager

When deployed alongside the ResourceManager, the UI can access these endpoints directly. The `web.xml` includes SPA routing configuration to handle client-side routing.

## Architecture

### Directory Structure

```
src/main/webapp/
├── src/
│ ├── app/ # React Router application setup
│ ├── components/ # Reusable UI components
│ ├── config/ # Scheduler configuration schemas
│ ├── contexts/ # React contexts
│ ├── features/ # Feature modules
│ │ ├── global-settings/
│ │ ├── node-labels/
│ │ ├── placement-rules/
│ │ ├── property-editor/
│ │ ├── queue-management/
│ │ ├── staged-changes/
│ │ ├── template-config/
│ │ └── validation/
│ ├── hooks/ # Custom React hooks
│ ├── lib/ # API client, utilities
│ ├── stores/ # Zustand stores
│ ├── types/ # TypeScript type definitions
│ └── utils/ # Helper functions
├── public/ # Static assets
└── WEB-INF/ # Web application descriptor
```

### State Management

The application uses Zustand for global state management with the following stores:

- **SchedulerStore**: Manages scheduler configuration and queue hierarchy
- **NodeLabelStore**: Handles node labels and partitions
- **StagedChangesStore**: Tracks pending configuration changes
- **ValidationStore**: Manages validation state and error messages

## Configuration

### Build Profiles

- **yarn-ui**: Production build (default profile in this module)
- **yarn-ui-dev**: Development build (includes mock data in WAR)

### Maven Properties

- `packagingType`: WAR when profile is active, POM otherwise
- `webappDir`: Build directory (`${basedir}/target/webapp`)
- `nodeExecutable`: Path to locally installed Node.js
- `keepUIBuildCache`: Set to `true` to preserve node_modules between builds

## Browser Support

- Chrome/Edge: Latest 2 versions
- Firefox: Latest 2 versions
- Safari: Latest 2 versions

## Contributing

This module follows Apache Hadoop's contribution guidelines. All source files must include the Apache License header.

### Code Style

- TypeScript strict mode enabled
- ESLint for code quality
- Prettier for code formatting
- Conventional commit messages

## Developer Documentation

Detailed guides for extending and customizing the UI:

- **[Design Document](src/main/webapp/docs/design_doc.md)** - Architecture, design decisions, and technical specifications
- **[Adding Validation Rules](src/main/webapp/docs/development/adding-validation-rules.md)** - Guide for implementing custom validation rules
- **[Extending Scheduler Properties](src/main/webapp/docs/development/extending-scheduler-properties.md)** - Instructions for adding new configuration properties

## License

Licensed under the Apache License, Version 2.0. See the LICENSE file in the Hadoop root directory.

## Related Documentation

- [YARN Capacity Scheduler Documentation](../hadoop-yarn-site/src/site/markdown/CapacityScheduler.md)
- [YARN REST API Documentation](../hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md)
- [Hadoop Main Documentation](https://hadoop.apache.org/docs/current/)
Loading