代码重构与化简 #141

qinzuoyan · 2018-07-18T06:07:54Z

完成

layer2 app model: d1df52f
c api of perf counter: a55bdca
c api of CLI: 87194db
c api of layer1/global checker; dmodule: d8ac3cf
thread pool code
error code
task code
gpid
buid process
rpc address: rpc_address: remove c interface of dsn_address_t #4
memory allocator: utility: refactor tls memory allocator #25
rpc: rpc: several refactors for rpc #27
task tracker: task_tracker: refactor on task_tracker #41
task: remove task's c interface #48
future task: task: reimplement safe_late_task with future task #53
configuration: configuration: refactor configuration related APIs #54
rename: replication: some rename refactors #57
memory provider: core: remove useless memory provider #60
io-per-queue: core: remove io-per-queue's io mode & fastrun's running mode #62 core: remove io-per-queue's io mode & fastrun's running mode #93
rpc session: rpc_session: do some refactor #116
fastrun: Remove fastrun #117
perf counter: perf_counter: code refactor #119
clientlet: clientlet: remove class clientlet, add new class thread_access_checker #121
test suites: core: remove useless test suites #127
perf counter: perf-counter: bug fix & refactor #134
nfs: nfs: remove nfs from service engine, refactor nfs to a single module #142
file system: filesystem: bug fix & code refactor #148
rpc message: rpc: remove c interface of dsn message #151
partition resolver: rpc: remove uri resolver; move partition resolver to replication/client #205

TBD

The text was updated successfully, but these errors were encountered:

neverchanje · 2018-11-19T05:46:34Z

不管目录重整多不多，我们可以先合议一个最终目标出来，后面一步一步改。本意是改的目录结构大家提前有一个共识（参考 #195 的讨论）

整体项目分为 core 和 dist 两大部分，按照讨论，所有 client/server 相关的模块放在 dist，dist 在 core 之上构建。

- src/core/utils: core 里的 utilities
- src/core/concurrent 或者 src/core/task
- src/core/aio
- src/core/rpc
- src/core/perf_counter
============
- src/dist/http
- src/dist/nfs
- src/dist/block_service
- src/dist/failure_detector
- src/dist/cli
- src/dist/replica_server => 
    - src/dist/replica_server/store
    - src/dist/replica_server/store/simple_kv
    - src/dist/replica_server/store/rocksdb
- src/dist/meta_server
- src/dist/zookeeper
- src/dist/duplication
- src/dist/security
- src/dist/backup
- src/dist/client 这里放我们的 ddl client，后面两个项目整合可以一起放 pegasus client

shengofsun · 2018-11-19T05:57:11Z

把rserver和mserver改名成replica_server和meta_server就好，别的没问题。

acelyc111 · 2020-06-29T04:01:11Z

为什么还要保持两个目录(dist,core)呢？为什么不去掉dist，core并把目录平摊到一层呢？

neverchanje · 2020-06-29T07:28:51Z

为什么还要保持两个目录(dist,core)呢？为什么不去掉dist，core并把目录平摊到一层呢？

core 目录下为 dsn.runtime 相关代码, 因为当时许多组件的代码还和 dsn.runtime 绑定, 所以我们在结构上分为 core 和 dist, 分别代表 dsn.runtime 与 基于runtime的上层实现.

而目前许多模块已经从 dsn.runtime 剥离出来, 并且目前距上次讨论已经间隔2年, 确实我们应该再次整理代码的结构.

- src/runtime/task
- src/runtime/rpc # rpc,task和部分零碎代码在剥离后可以移出runtime
- src/runtime
============
- src/utils
- src/aio
- src/perf_counter
- src/http
- src/nfs
- src/block_service
- src/failure_detector
- src/remote_cmd
- src/replica => 
    - src/replica/store
    - src/replica/store/simple_kv
    - src/replica/store/rocksdb
- src/meta
- src/zookeeper
- src/duplication
- src/security
- src/backup
- src/client 这里放我们的 ddl client，后面两个项目整合可以一起放 pegasus client

@acelyc111 @levy5307

levy5307 · 2020-06-29T07:51:34Z

还要有个common吧？存放meta server和replica server共同使用的部分，比如replication_options等
另外还有那些tool/tool-api/toolet/tool_lib/utils，很容易让人产生困惑，无法分清有什么区别

acelyc111 · 2020-06-29T09:49:33Z

可以参考下kudu的目录结构和划分逻辑：
common：存放跟rdsn/Pegasus项目紧密相关的通用文件
util：最基础的工具类，比如time、string、lock等最基础的组件，他们甚至可以移植到其他项目。跟现在rdsn的include/dsn/utility基本一致
tools：存放真正用户可使用的“工具”相关的类。跟我们的ddl_client，remote command相似
其他的tool-api/toolet/tool_lib真的无法从名字看出它是干什么的了，尽量就不保留了

levy5307 · 2020-06-29T11:27:46Z

可以参考下kudu的目录结构和划分逻辑：
common：存放跟rdsn/Pegasus项目紧密相关的通用文件
util：最基础的工具类，比如time、string、lock等最基础的组件，他们甚至可以移植到其他项目。跟现在rdsn的include/dsn/utility基本一致
tools：存放真正用户可使用的“工具”相关的类。跟我们的ddl_client，remote command相似
其他的tool-api/toolet/tool_lib真的无法从名字看出它是干什么的了，尽量就不保留了

所以，我们是不是也要把remote command和ddl_client这些也放到tool里，而不是单独放在一个路径下？

neverchanje · 2020-06-30T05:29:35Z

还要有个common吧？存放meta server和replica server共同使用的部分，比如replication_options等

可以多加个 common，虽然我不太喜欢这个名字，因为很难理解这个 common 指的是什么。

所以，我们是不是也要把remote command和ddl_client这些也放到tool里，而不是单独放在一个路径下？

kudu 的 tools 本质上是 command-line tool, 是无法跟 ddl_client 和 remote_command 相提并论。

如果后续将 Pegasus 与 rdsn 合并，或许可以把 remote_command 和 ddl_client 放到 shell 的目录下。

- src/runtime/task
- src/runtime/rpc # rpc,task和部分零碎代码在剥离后可以移出runtime
- src/runtime
============
- src/utils
- src/aio
- src/perf_counter
- src/http
- src/nfs
- src/block_service
- src/failure_detector
- src/remote_cmd # remote command
- src/common # replica/meta 公共代码
- src/replica => 
    - src/replica/store
    - src/replica/store/simple_kv
    - src/replica/store/rocksdb
- src/meta
- src/zookeeper
- src/duplication
- src/security
- src/backup
- src/client 这里放我们的 ddl client，后面两个项目整合可以一起放 pegasus client

levy5307 · 2020-07-02T03:20:31Z

另外还有个tools，所以应该是这样的：

src/runtime/task
src/runtime/rpc # rpc,task和部分零碎代码在剥离后可以移出runtime
src/runtime
============
src/tools
src/utils
src/aio
src/perf_counter
src/http
src/nfs
src/block_service
src/failure_detector
src/remote_cmd # remote command
src/common # replica/meta 公共代码
src/replica =>
- src/replica/store
- src/replica/store/simple_kv
- src/replica/store/rocksdb
src/meta
src/zookeeper
src/duplication
src/security
src/backup
src/client 这里放我们的 ddl client，后面两个项目整合可以一起放 pegasus client

neverchanje · 2020-07-02T03:27:24Z

另外还有个tools

tools 包括哪些组件？

levy5307 · 2020-07-02T03:29:00Z

另外还有个tools

tools 包括哪些组件？

比如profiler,tracer这些，具体需要我后面整理一下

acelyc111 · 2020-07-02T03:41:59Z

另外还有个tools

tools 包括哪些组件？

比如profiler,tracer这些，具体需要我后面整理一下

profiler,tracer 应该属于util

levy5307 · 2020-07-02T03:50:09Z

另外还有个tools

tools 包括哪些组件？

比如profiler,tracer这些，具体需要我后面整理一下

profiler,tracer 应该属于util

我理解utils应该是最基础的工具类，应该是即使是跨项目也可以直接用的东西。但是profiler和tracer依赖了很多task和rpc的东西

acelyc111 · 2020-07-02T05:38:10Z

另外还有个tools

tools 包括哪些组件？

比如profiler,tracer这些，具体需要我后面整理一下

profiler,tracer 应该属于util

我理解utils应该是最基础的工具类，应该是即使是跨项目也可以直接用的东西。但是profiler和tracer依赖了很多task和rpc的东西

我以为你说的是#510 这个tracer。依赖task，rpc的那个是不是放到common好点？

levy5307 · 2020-07-02T05:51:14Z

另外还有个tools

tools 包括哪些组件？

比如profiler,tracer这些，具体需要我后面整理一下

profiler,tracer 应该属于util

我理解utils应该是最基础的工具类，应该是即使是跨项目也可以直接用的东西。但是profiler和tracer依赖了很多task和rpc的东西

我以为你说的是#510 这个tracer。依赖task，rpc的那个是不是放到common好点？

如吴涛所说，其实common这个路径看名字很难理解是做什么的。所以我在想要不要common改成common_define，里面只保留一些公共的变量定义等等，不向里面放入太多东西

qinzuoyan added the type/enhancement Indicates new feature requests label Jul 18, 2018

qinzuoyan assigned shengofsun Jul 18, 2018

shengofsun mentioned this issue Aug 25, 2018

rpc message: remove dsn_message_t apache/incubator-pegasus#168

Merged

neverchanje mentioned this issue Sep 20, 2018

util: make rand a standalone util decoupled from rdsn runtime #163

Merged

neverchanje added type/bug-fix This PR fixes a bug. type/refactoring and removed type/enhancement Indicates new feature requests type/bug-fix This PR fixes a bug. labels Sep 20, 2018

neverchanje mentioned this issue Nov 19, 2018

apps/simple_kv: some refactor #195

Merged

neverchanje mentioned this issue Dec 7, 2018

replica-server: support table level write throttling #204

Merged

neverchanje mentioned this issue Feb 28, 2019

utility: refactor logging module #224

Closed

qinzuoyan assigned neverchanje Feb 28, 2019

neverchanje assigned acelyc111 Apr 19, 2019

neverchanje mentioned this issue Jul 6, 2019

http: move http as a submodule of dist #266

Merged

neverchanje removed the type/refactoring label Dec 26, 2019

neverchanje mentioned this issue Feb 3, 2020

feat: add check for app envs #353

Merged

levy5307 mentioned this issue Feb 14, 2020

refactor: make clock decoupled from rdsn runtime #398

Merged

neverchanje added this to In progress in Refactoring Feb 17, 2020

levy5307 mentioned this issue Feb 24, 2020

refactor(utils): make logger decoupled from dsn runtime #403

Merged

neverchanje moved this from In progress to To do in Refactoring Feb 25, 2020

This was referenced Jun 24, 2020

refactor: move task to a seperated directory #515

Merged

refactor: make utils decoupled from dsn_runtime #517

Merged

This was referenced Jul 14, 2020

refactor: change tests module name to test #557

Merged

refactor: move tools module from src/dist/replication/tool_lib to src/tools #558

Merged

refactor: move runtime module from src/core/core to src/runtime #566

Merged

hycdong mentioned this issue Sep 3, 2020

Release 2.1.0 apache/incubator-pegasus#577

Closed

levy5307 mentioned this issue Jun 30, 2021

feat: add partition guardian to keep one primary and two secondaries #847

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

代码重构与化简 #141

代码重构与化简 #141

qinzuoyan commented Jul 18, 2018 •

edited by neverchanje

neverchanje commented Nov 19, 2018 •

edited

shengofsun commented Nov 19, 2018

acelyc111 commented Jun 29, 2020

neverchanje commented Jun 29, 2020

levy5307 commented Jun 29, 2020 •

edited

acelyc111 commented Jun 29, 2020

levy5307 commented Jun 29, 2020

neverchanje commented Jun 30, 2020 •

edited

levy5307 commented Jul 2, 2020

neverchanje commented Jul 2, 2020

levy5307 commented Jul 2, 2020

acelyc111 commented Jul 2, 2020

levy5307 commented Jul 2, 2020 •

edited

acelyc111 commented Jul 2, 2020

levy5307 commented Jul 2, 2020

代码重构与化简 #141

代码重构与化简 #141

Comments

qinzuoyan commented Jul 18, 2018 • edited by neverchanje

完成

TBD

neverchanje commented Nov 19, 2018 • edited

shengofsun commented Nov 19, 2018

acelyc111 commented Jun 29, 2020

neverchanje commented Jun 29, 2020

levy5307 commented Jun 29, 2020 • edited

acelyc111 commented Jun 29, 2020

levy5307 commented Jun 29, 2020

neverchanje commented Jun 30, 2020 • edited

levy5307 commented Jul 2, 2020

neverchanje commented Jul 2, 2020

levy5307 commented Jul 2, 2020

acelyc111 commented Jul 2, 2020

levy5307 commented Jul 2, 2020 • edited

acelyc111 commented Jul 2, 2020

levy5307 commented Jul 2, 2020

qinzuoyan commented Jul 18, 2018 •

edited by neverchanje

neverchanje commented Nov 19, 2018 •

edited

levy5307 commented Jun 29, 2020 •

edited

neverchanje commented Jun 30, 2020 •

edited

levy5307 commented Jul 2, 2020 •

edited