-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable tiered category hierarchy/dropdowns for docs #17495
Conversation
### What changes are proposed in this pull request? mentioned in Alluxio#16629 ### Why are the changes needed? The `Cluster.ActiveRpcReadCount` metric could get an impossible value like a negative number. The main reason is that the request handling doesn't cover all situations. ### Does this PR introduce any user facing changes? Nope pr-link: Alluxio#17234 change-id: cid-f7d28cbf57ef2212e4b8d300f8a18300b593b213
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#17390 change-id: cid-30d303cb559a315f334cdf880d4dde99ed7779c6
### What changes are proposed in this pull request? This is the doc update from a few recent commits: 1. Alluxio@aee3c5c 2. Alluxio@9a4e154 3. Alluxio@dbac084 4. Alluxio@8da5953 5. Alluxio@141ee0e 6. Alluxio@26257e6 In short, a few improvements have been added that the admin can now see the version/revision of the running Alluxio components in the cluster. A few commands were also added to help with gracefully decommissioning/restarting workers in the cluster. ### Why are the changes needed? This doc change helps users understand how to utilize those features. pr-link: Alluxio#17381 change-id: cid-1d45c1750f84ac94d3fa79b9bbe0f0f328f7cb45
### What changes are proposed in this pull request? Avoid the NPE exception in copy object with x-amz-tagging-directive ### Why are the changes needed? Avoid the NPE exception pr-link: Alluxio#17400 change-id: cid-db0f6ac83b1f6de7937c17f82911b153fe63155d
A workaround to fix the master health check issue, related to Alluxio#17382 which is caused by a recent change that starts grpc servers on standby masters. Follower masters have a gRPC server always active. When the `MasterHealthCheckClient` connects to the gRPC server, it may find that some services aren't registered (NotFound) rather than unavailable. This manifests in a bug where `bin/alluxio-start.sh master` performs a health check after starting a master and incorrectly reports the master as unhealthy. pr-link: Alluxio#17383 change-id: cid-38f5babf01a3597099a821ec3be61f325630c098
### What changes are proposed in this pull request? Remove s3 handler map in s3 proxy. ### Why are the changes needed? Alluxio#17403 ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs NO 2. addition or removal of property keys NO 3. webui NO pr-link: Alluxio#17406 change-id: cid-ddf598601ab15f9d9ecb5a22901c191b54935d85
…nfig deprecation warning Don't print a warning in log if the deprecated config is not set by the user explicitly. Even if a deprecated config is not set, when a client or a worker loads cluster default config from master, the config is considered to be `CLUSTER_DEFUALT` level, and triggers a deprecation warning. This PR changes the condition for a deprecation message to be logged to be that the level is higher than `CLUSTER_DEFAULT`, which is the same condition used by `isSetByUser`. After this change, if a deprecated config does get set in `alluxio-site.properties`, the config source level would be `SITE_PROPERTIES` and get logged at the master's log. No. pr-link: Alluxio#17333 change-id: cid-584c3b66bbb701e7fc70d6e27d91dbd3072851f7
… and timestamp in web page Display the config hash and config updated time <img width="550" alt="image" src="https://user-images.githubusercontent.com/17329931/207004719-8c684f3d-24c7-4fce-bd95-32ff795b6172.png"> <img width="534" alt="image" src="https://user-images.githubusercontent.com/17329931/207027205-865183d3-c288-4c99-bbee-a53c6d4ba579.png"> Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#16642 change-id: cid-075d4f9ea1cee81f23706781dcf5cc5ca155e416
### What changes are proposed in this pull request? Support print xattr for stat command. ### Why are the changes needed? Needs to show the xattr for user. ### Does this PR introduce any user facing changes? User can see the xattr leverage this PR. pr-link: Alluxio#17395 change-id: cid-edcb428ee08c8d460c61ae5ce9d555cc6453f02e
…ration from mount option Support config obs configuration from mount option For now, we cannot specify other obs client configurations through any way. After this PR, we can config any obs configuration. pr-link: Alluxio#17191 change-id: cid-a7f9ac3929d6deceb9ee71a02a2dc5f6c44bb489
…ncurrently Make the client poll the master concurrently. Currently, we find the active master node by polling each node in series. If a node did not respond due to network or other issues, this may take a lot of time and affect service availability. Make the client poll the master concurrently can alleviate this problem. pr-link: Alluxio#17186 change-id: cid-d1ef20b596deda02375324978adb23e6f8ce8dce
This PR change replace the `registerGaugeIfAbsent` in `DefaultBlockWorker:Metrics:registerGauges` with `registerCachedGaugeIfAbsent`, and made a container `BlockWorkerMetrics` for `BlockStoreMeta` of `DefaultBlockWorker` Previously the MetricSystem will copy a full BlcokStoreMeta everytime it's trying to update the metric, which cost is expensive and can be avoid. No pr-link: Alluxio#17357 change-id: cid-0e05812a75e51e9b74dbb802b82abb8c56d19b27
### What changes are proposed in this pull request? When the filesystem establishes a connection with the master, the configuration of the filesystem will be overridden by the master's configuration, resulting in changes. Therefore, we should not use the configuration of the filesystem anymore, but instead use the configuration of the local client. ### Why are the changes needed? It is very common for the master configuration and client configuration to be different. We should avoid the newly created filesystem from being influenced by the server. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#17439 change-id: cid-1af4fd37ee8c52cd32552818989dce557256b228
the rolling upgrade feature is for limited experimental use currently with many limitations. we may bring back the docs in a future version when it is more widely applicable to various user scenarios. pr-link: Alluxio#17455 change-id: cid-9a65f7f7b1fa55373594f7ca3ddcc33a979addec
Regarding Alluxio#16940 (comment). Add the substring "Xmx" that's supposed to be checked on. pr-link: Alluxio#17459 change-id: cid-b4cc0afb6d84e7198a9fe245e994c9eb2100d0ac
### What changes are proposed in this pull request? Check sleep timer interval change when update conf ### Why are the changes needed? We currently generate these logs every time we execute the updateConf command, even if the value has not been modified. ``` 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Cluster Metrics Updater interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Lost Worker Detection interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Worker register stream session cleaner interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Block Integrity Check interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master TTL Check interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Lost Files Detection interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Replication Check interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Persistence Scheduler interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Persistence Checker interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Metrics Time Series interval supplier. 2023-04-24 11:37:27,445 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Lost Master Detection interval supplier. 2023-04-24 11:37:27,446 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Log Config Report Scheduling interval supplier. 2023-04-24 11:37:27,446 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Journal Space Monitor interval supplier. 2023-04-24 11:37:27,446 INFO [config-hash-master-heartbeat-0](SleepingTimer.java:88) - update Master Throttle interval supplier. ``` ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#17309 change-id: cid-4352f517ca1f1b37cab9784e1f89b104f09426d8
### What changes are proposed in this pull request? Support changing log level at runtime for alluxio-fuse. Now with the configuration "alluxio.fuse.web.enabled=true", the user can change the logLevel for alluxio-fuse with the following ways 1. With `logLevel` command ``` $ ./bin/alluxio logLevel --logName=alluxio.fuse.AlluxioJniFuseFileSystem --target=localhost:49999 --level=INFO Role inferred from port: localhost:49999[fuse] localhost:49999[fuse]LogInfo{INFO, alluxio.fuse.AlluxioJniFuseFileSystem, Setting Level to INFO} ``` 2. With `curl` ``` $ curl -X POST "http://localhost:49999/api/v1/fuse/logLevel?logName=alluxio.fuse.AlluxioJniFuseFileSystem&level=DEBUG" {"logName":"alluxio.fuse.AlluxioJniFuseFileSystem","message":"Setting Level to DEBUG","level":"DEBUG"} ``` ### Why are the changes needed? Please refer Alluxio#17174 ### Does this PR introduce any user facing changes? NO pr-link: Alluxio#17175 change-id: cid-4130150f647217da8fd396dc542753400d6b568f
This reverts commit 27d7479. There's permission issue with the Dockerfile while doing `ln`. pr-link: Alluxio#17500 change-id: cid-ff42b4edacf8fbe3345f3d85182237d6da8a2a56
### What changes are proposed in this pull request? Modify the visibility and inheritance of some interfaces ### Why are the changes needed? Supports better scalability pr-link: Alluxio#17499 change-id: cid-c53c83f377b46bb4509d9aa10a556ed3cb9bfc37
### What changes are proposed in this pull request? Add toModeBits for extension ### Why are the changes needed? Add toModeBits for extension ### Does this PR introduce any user facing changes? NA pr-link: Alluxio#17501 change-id: cid-196e47cd2e77cae8143b3a287b2ec96df1e96cc4
### What changes are proposed in this pull request? - generate `libexec/version.sh` for using `--dryRun` ### Why are the changes needed? For using dry run flag for dev purposes ### Does this PR introduce any user facing changes? pr-link: Alluxio#17498 change-id: cid-c5f26e5a2afb9db1c7d87e979bc63d84a4db7674
### What changes are proposed in this pull request? make scheduler universally work for copyjob + doraloadjob ### Why are the changes needed? resolve conflict of copyjob to use the new scheduler ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#17368 change-id: cid-e67f2b0a931944e56d35b06f421223e7d8eb2c4f
### What changes are proposed in this pull request? Fix a deadlock in `LocalCacheManager.getAndLoad`. To reproduce the deadlock: dbw9580@ff87c73 ![image](https://github.com/Alluxio/alluxio/assets/6999708/d604f39b-e409-4c24-8fb1-b55893825599) ### Why are the changes needed? Deadlock detected by jstack: <details> <summary>jstack output</summary> ``` Found one Java-level deadlock: ============================= "pool-4-thread-1": waiting for ownable synchronizer 0x000000070eb3b9f0, (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync), which is held by "pool-4-thread-6" "pool-4-thread-6": waiting for ownable synchronizer 0x000000070eb25728, (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync), which is held by "pool-4-thread-3" "pool-4-thread-3": waiting for ownable synchronizer 0x000000070eb3b9f0, (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync), which is held by "pool-4-thread-6" Java stack information for the threads listed above: =================================================== "pool-4-thread-1": at jdk.internal.misc.Unsafe.park(java.base@11.0.19/Native Method) - parking to wait for <0x000000070eb3b9f0> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync) at java.util.concurrent.locks.LockSupport.park(java.base@11.0.19/LockSupport.java:194) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.19/AbstractQueuedSynchronizer.java:885) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.19/AbstractQueuedSynchronizer.java:917) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.19/AbstractQueuedSynchronizer.java:1240) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.19/ReentrantReadWriteLock.java:959) at alluxio.resource.LockResource.<init>(LockResource.java:102) at alluxio.resource.LockResource.<init>(LockResource.java:73) at alluxio.resource.LockResource.<init>(LockResource.java:57) at alluxio.client.file.cache.LocalCacheManager.getAndLoad(LocalCacheManager.java:525) at alluxio.client.file.cache.LocalCacheManagerTest.lambda$concurrentGetAndLoad$9(LocalCacheManagerTest.java:1133) at alluxio.client.file.cache.LocalCacheManagerTest$$Lambda$167/0x00000008001c8c40.call(Unknown Source) at java.util.concurrent.FutureTask.run(java.base@11.0.19/FutureTask.java:264) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.19/ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.19/ThreadPoolExecutor.java:628) at java.lang.Thread.run(java.base@11.0.19/Thread.java:829) "pool-4-thread-6": at jdk.internal.misc.Unsafe.park(java.base@11.0.19/Native Method) - parking to wait for <0x000000070eb25728> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync) at java.util.concurrent.locks.LockSupport.park(java.base@11.0.19/LockSupport.java:194) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.19/AbstractQueuedSynchronizer.java:885) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.19/AbstractQueuedSynchronizer.java:917) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.19/AbstractQueuedSynchronizer.java:1240) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.19/ReentrantReadWriteLock.java:959) at alluxio.resource.LockResource.<init>(LockResource.java:102) at alluxio.resource.LockResource.<init>(LockResource.java:73) at alluxio.resource.LockResource.<init>(LockResource.java:57) at alluxio.client.file.cache.LocalCacheManager.putAttempt(LocalCacheManager.java:374) at alluxio.client.file.cache.LocalCacheManager.putInternal(LocalCacheManager.java:278) at alluxio.client.file.cache.LocalCacheManager.put(LocalCacheManager.java:238) at alluxio.client.file.cache.CacheManager.put(CacheManager.java:206) at alluxio.client.file.cache.LocalCacheManager.getAndLoad(LocalCacheManager.java:546) at alluxio.client.file.cache.LocalCacheManagerTest.lambda$concurrentGetAndLoad$9(LocalCacheManagerTest.java:1133) at alluxio.client.file.cache.LocalCacheManagerTest$$Lambda$167/0x00000008001c8c40.call(Unknown Source) at java.util.concurrent.FutureTask.run(java.base@11.0.19/FutureTask.java:264) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.19/ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.19/ThreadPoolExecutor.java:628) at java.lang.Thread.run(java.base@11.0.19/Thread.java:829) "pool-4-thread-3": at jdk.internal.misc.Unsafe.park(java.base@11.0.19/Native Method) - parking to wait for <0x000000070eb3b9f0> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync) at java.util.concurrent.locks.LockSupport.park(java.base@11.0.19/LockSupport.java:194) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.19/AbstractQueuedSynchronizer.java:885) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.19/AbstractQueuedSynchronizer.java:917) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.19/AbstractQueuedSynchronizer.java:1240) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.19/ReentrantReadWriteLock.java:959) at alluxio.resource.LockResource.<init>(LockResource.java:102) at alluxio.resource.LockResource.<init>(LockResource.java:73) at alluxio.resource.LockResource.<init>(LockResource.java:57) at alluxio.client.file.cache.LocalCacheManager.putAttempt(LocalCacheManager.java:375) at alluxio.client.file.cache.LocalCacheManager.putInternal(LocalCacheManager.java:278) at alluxio.client.file.cache.LocalCacheManager.put(LocalCacheManager.java:238) at alluxio.client.file.cache.CacheManager.put(CacheManager.java:206) at alluxio.client.file.cache.LocalCacheManager.getAndLoad(LocalCacheManager.java:546) at alluxio.client.file.cache.LocalCacheManagerTest.lambda$concurrentGetAndLoad$9(LocalCacheManagerTest.java:1133) at alluxio.client.file.cache.LocalCacheManagerTest$$Lambda$167/0x00000008001c8c40.call(Unknown Source) at java.util.concurrent.FutureTask.run(java.base@11.0.19/FutureTask.java:264) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.19/ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.19/ThreadPoolExecutor.java:628) at java.lang.Thread.run(java.base@11.0.19/Thread.java:829) ``` </details> ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#17504 change-id: cid-c5154832c855027011b89b7af17731eec9eddc3a
@Zijian-Zhu Thanks for the docs structure! Some small suggestion about the format style: |
- CreateFile from client to worker - CompleteFile from client to worker - OpenFileHandle in worker - OpenFileHandles in worker pr-link: Alluxio#17505 change-id: cid-8cf626c579dec84284265354ad7026430e8ecb95
### What changes are proposed in this pull request? fixed directory deletion failure when ufs is cos ### Why are the changes needed? 1. cos bucket name is bucket-appid 2. COSUnderFileSystem.deleteObjects request is not bucket-appid case `Failed to delete 1 paths from the under file system: /xxx (UFS delete dir failed)` 3. ObjectUnderFileSystem.submitBatch exception should print ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. when ufs is cos can normal exec alluxio rm -r xxx pr-link: Alluxio#17493 change-id: cid-1dc02fa2781254b85f0c7dc58c582469cfc19781
Cherry-pick of existing commit. orig-pr: Alluxio#17469 orig-commit: Alluxio/alluxio@9d2f9cf orig-commit-author: maobaolong <307499405@qq.com> pr-link: Alluxio#17507 change-id: cid-1dde372d4a0aa0604b44ddbbe90762293f31c20c
### What changes are proposed in this pull request? Modify the visibility and inheritance of some interfaces ### Why are the changes needed? Modify the visibility and inheritance of some interfaces pr-link: Alluxio#17601 change-id: cid-11e1459ed874f67c2cb17fcf31114c8d5e54786c
### What changes are proposed in this pull request? Make mount command work in dora ### Why are the changes needed? na ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17606 change-id: cid-e8f88e0ae0f0b14dda70d6ca7db4e8935fd4e6f9
### What changes are proposed in this pull request? Fix move job tasks didn't picking up active workers. ### Why are the changes needed? Fix move job bugs ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17604 change-id: cid-e668e9d50017a6a6c37f15daaec3ac348ab8033d
### What changes are proposed in this pull request? change task exceptions to catch throwable ### Why are the changes needed? otherwise when throwable happens it would report task success ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17608 change-id: cid-018ac3d186e337f0fdf777e8065fd7fbd7900247
### What changes are proposed in this pull request? Fixed an error in `UnderStorageReadIntegrationTest. java`. Added some mReadNoCache cases in the test cases. ### Why are the changes needed? Fix a test bug and add more cases. ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17602 change-id: cid-97b66e220c0eb8db4f2c1e3187c43abdadd92db1
### What changes are proposed in this pull request? Modify for GroupMappingService extension ### Why are the changes needed? Modify for GroupMappingService extension ### Does this PR introduce any user facing changes? NA pr-link: Alluxio#17613 change-id: cid-a9bfff652b55f638250fe9472bde163032fca235
Remove dora unrelated docs pr-link: Alluxio#17584 change-id: cid-e2ff76843387b6b6b5a5d6c6454865d63a2d038d
pr-link: Alluxio#17614 change-id: cid-73f795e1035e665071ac1db3629dc929968ee6af
need to fix the doc check. it used to ensure the categories on each page is an existing category listed in the config yml, but now that mechanism is removed, so we could also remove the check. in a future PR, we should add back a similar check to ensure that config yml file list and the list of all doc md files are exactly the same. |
### What changes are proposed in this pull request? Add and implement functions to invalidate all cached pages by fileId. ### Why are the changes needed? When we know that some file is to be removed, or renamed, or overwritten, we need to invalidate the cached pages in PagingStore. Please note, at this moment, it is possible that the target file is already removed from UFS. So, getStatus() of this file will fail with exception. But we still need to invalidate all the cached pages. ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#17582 change-id: cid-0ce28e1593308a3b429162d86792249b50d3b4c9
### What changes are proposed in this pull request? Remove mount check when starting alluxio if users do not intend to mount ### Why are the changes needed? When the user does not want to mount and starts alluxio directly, the user can enter `./bin/alluxio-start.sh local` or `./bin/alluxio-start.sh all`. The old code will not be able to start alluxio successfully, it will refuse to start or the AlluxioWorker will fail to start. This PR fixes the issue. ![截屏2023-06-08 17 25 44](https://github.com/Alluxio/alluxio/assets/132879614/53cd6cb6-19e0-424b-8ff2-8ba49c8f0cb6) The reason for deleting the mount check part of the code is: We found that the mount check is only started when the user enters `./bin/alluxio-start.sh worker` or `./bin/alluxio-start.sh local`. However, if the user intends to use alluxio after mounting, the user will use `alluxio-mount.sh` for the mount operation. If the mount fails, an error will be reported, and `alluxio-start.sh` will not be executed. If alluxio is started directly without mounting, there is no need to perform mount check. Therefore, under the logic of our current `main` branch, this mount check is redundant and will cause alluxio to fail to start. This problem can be corrected after deletion. As for `alluxio-start.sh local SudoMount` involves mount check, execute `alluxio-start.sh local SudoMount` to check whether it is mounted before. If it has been mounted before, the mount command will not be executed. If it has not been mounted before, mount it load disk. But the check mount here is redundant. This function is to force mounting before using alluxio, which does not meet the purpose of "use alluxio directly without mounting". After removing the check mount function, it does not affect the use of this command. ![截屏2023-06-14 11 57 35](https://github.com/Alluxio/alluxio/assets/132879614/4cc56dfa-8674-499f-a263-c0d672de5b21) ### Does this PR introduce any user facing changes? No pr-link: Alluxio#17603 change-id: cid-e67fe1697e4c75fbf0b565a00675f57f6136cc31
### What changes are proposed in this pull request? Hookup permission checks in dora worker for scheduler job related endpoints ### Why are the changes needed? To handle permission checking correctly ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#17620 change-id: cid-7473717a206e25011d66552a09658e83f1910e67
### What changes are proposed in this pull request? Implement http server for providing the get page RESTful API. Now we can get the page by sending a HTTP request: `http://<HOST>:<PORT>/page?fileId=<FILE_ID>&pageIndex=<PAGE_INDEX>` An example snapshot is shown as follows: <img width="985" alt="image" src="https://github.com/Alluxio/alluxio/assets/6129818/b079c763-cc86-4df7-b49d-5d6cd822a85b"> ### Why are the changes needed? It can provide a more convenient API for AI scenarios and improve reading performance. ### Does this PR introduce any user facing changes? No, it doesn't. pr-link: Alluxio#17579 change-id: cid-5d0cbd8311081444fc46e1ad2799e9de04d16108
### What changes are proposed in this pull request? Add getOpenFileHandleContainer method ### Why are the changes needed? Add getOpenFileHandleContainer method pr-link: Alluxio#17618 change-id: cid-2d29ea8c51f0c219c21b308296bfdde6410c632c
### What changes are proposed in this pull request? Add a maven export action in `pom.xml` of `dora/tests` to export all the test classes for other classes, because the tests module provides utilities for local integration testing. ### Why are the changes needed? For other modules to reuse the code ### Does this PR introduce any user facing changes? No pr-link: Alluxio#17619 change-id: cid-e6147b62f62464b3211134b6286f0d3c1fbb1e3e
remove the doc check in pr build since the config.yml file changes its structure. pr-link: Alluxio#17624 change-id: cid-ed9092e74b1cf8677930523fc795174e44274f51
### What changes are proposed in this pull request? Add get mount id to worker master rpc ### Why are the changes needed? support get mount id from worker ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17625 change-id: cid-b9e179fad02234bc345532bd70fdd3ec1e61dc6d
### What changes are proposed in this pull request? Expose the getJobState API to scheduler. ### Why are the changes needed? This is needed for EE updates policy status. ### Does this PR introduce any user facing changes? na pr-link: Alluxio#17626 change-id: cid-1c5b8bab24b76ed1a524b79b1105ae20d0aa838c
### What changes are proposed in this pull request? - delete recognized page file, including TEMP pages - handle empty result from getPageInfo(), which returns Optional.empty() if page is not recognized ### Why are the changes needed? When Worker (re-)starts, worker scans the StoreDir to establish the metadata for PagingStore. If some page files are not recognized, the original solution is to delete all metadata and all existing page files. One special case is page files in TEMP dir. These TEMP page files are not recognized as normal page files. And also, we don't know if these TEMP page files are valid or complete. So, we have to delete those unrecognized page files (including TEMP page files). When this happens, we also need to handle the Optional.empty result from getPageInfo(). ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#17593 change-id: cid-8889577f492e46470411eedeed80dfaa613a28aa
update with a clear example update the style updated based on comments small fix remove not used directory update for better style and url change the margin of url refactor style relate code to golluxio change the config file to match the docs files change
I create a directory named
overview-example-level-1-dir
underdocs/en
to make an example of this tiered docs.The following screenshot is the example of the actually docs and directory under
docs/en/overview-example-level-1-dir
As you can see
overview.md
andGetting-Started.md
are level 1 docs.Architecture.md
andFAQ.md
are level 2 docs.JobService.md
is level 3 docs andUse-Cases.md
is level 4 docs.The following screenshot is example of content in the new Yaml file.
All the sub-directories need to be listed in the
subitems
with theirtitle
which does not need to be the same as their directory name and theirbuttonTitle
which is thetitle
without white space since white space will bug in the HTML.All the docs located in each directory need to be listed in the
subfiles
with docstitle
andurl
which is the path of the doc after jekyll bulid (this path always the same of the path of the files except the file type is.html
not.md
The following screenshot is the output of the example.
This is the full expansion of the example list. The blue string is the URL of the corresponding docs. And the button is collapse type and use the
buttonTitle
as name