lib refector and add lifelong learning #72

JoeyHwong-gk · 2021-05-31T07:24:39Z

Issue #46 by JoeyHwong

Lib structural decoupling
Interface unification
add global log and fileops
Independent algorithm module
add lifelong learning

llhuii · 2021-06-01T02:11:49Z

examples/federated-learning-surface-defect-detection-aggregation.Dockerfile

 COPY ./lib/requirements.txt /home
 RUN pip install -r /home/requirements.txt
+RUN pip install git+https://github.com/joeyhwong-gk/uvicorn


why using personal pip package?

Federated learning usually sends a large weight data to the aggregation service, but uvicorn does not support change the limit size. A patch has been added.

We have contacted the official to fix the problem and I will checkout to the official version later.

Good. In the present can we patch it with some command instead of personal link?

llhuii · 2021-06-01T02:15:33Z

examples/federated_learning/surface_defect_detection/training_worker/inference.py

@@ -11,21 +11,38 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+import os


better to add new empty line after copyright declaration

llhuii · 2021-06-01T02:56:46Z

examples/incremental_learning/helmet_detection/training/eval.py

+    # use original dataset url,
+    # see https://github.com/kubeedge/sedna/issues/35
+    original_dataset_url = Context.get_parameters('original_dataset_url')
+    return original_dataset_url + os.path.sep + dataset_url


why does example code need to know this path handling?

kubeedge/sedna/issues/35

I mean this original_dataset_url should not appear in user code (here example code)

llhuii · 2021-06-01T03:41:40Z

examples/incremental_learning/helmet_detection/training/train.py

+    # use original dataset url,
+    # see https://github.com/kubeedge/sedna/issues/35
+    original_dataset_url = Context.get_parameters('original_dataset_url')
+    return original_dataset_url + os.path.sep + dataset_url


same question here: why example code need to know this original_dataset_url

llhuii · 2021-06-01T03:43:24Z

examples/joint_inference/helmet_detection_inference/README.md

@@ -76,7 +76,7 @@ EOF
 Note the setting of the following parameters, which have to same as the script [little_model.py](/examples/joint_inference/helmet_detection_inference/little_model/little_model.py):
 - hardExampleMining: set hard example algorithm from {IBT, CrossEntropy} for inferring in edge side.
 - video_url: set the url for video streaming. 
- all_examples_inference_output: set your output path for the inference results.
+- all_examples_inference_output: set your output path for the inference results, and note that the root path has to be /home/data.


why has to be /home/data?

llhuii · 2021-06-01T03:53:35Z

lib/sedna/algorithms/aggregation/__init__.py

+        self.weights = None
+
+    @abc.abstractmethod
+    def aggregate(self, weights, size=0):


what does size parameter of aggregate mean?

sample number of each loop

What does sample number of each loop mean? Does size equal sizeof(weights)?
pls add some comment about this.

fedavg algorithms average a new weights based on the number of samples in the batch and the weights generated by the training.

llhuii · 2021-06-01T03:54:56Z

lib/sedna/algorithms/aggregation/__init__.py

+        """
+
+
+@ClassFactory.register(ClassType.FLAGG)


better to rename FLAGG into FL_AGG. easy to misunderstand FLAGG with FLAG and G

llhuii · 2021-06-01T03:57:33Z

lib/sedna/algorithms/aggregation/__init__.py

+        for inx, weight in enumerate(weights):
+            old_weight = self.weights[inx]
+            row_weight = (np.array(weight) - old_weight) * (size / total_sample) + old_weight
+        self.weights = deepcopy(updates)


deepcopy here?

Prevents developer-defined post-processing from modifying the original weights.

deepcopy is expensive, especially the weights.
In what case, post-processing modify it? I think rare case, even so let the copy to developer

llhuii · 2021-06-01T03:58:17Z

lib/sedna/algorithms/aggregation/__init__.py

+        updates = []
+        for inx, weight in enumerate(weights):
+            old_weight = self.weights[inx]
+            row_weight = (np.array(weight) - old_weight) * (size / total_sample) + old_weight


row_weight is unused in anywhere

updates always empty

llhuii · 2021-06-01T04:00:21Z

lib/sedna/algorithms/hard_example_mining/__init__.py

+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Hard Example Mining Algorithms for Cloud Migration Judgment"""


what does Cloud Migration mean here?

llhuii · 2021-06-01T06:58:43Z

lib/sedna/core/base.py

+        except Exception as err:
+            self.log.error(err)
+            with open(FileOps.join_path(self.config.model_url, "lc_msg.json"), 'w') as f_out:
+                json.dump(message, f_out)


just log enough, no need to dump into model url

llhuii · 2021-06-01T06:59:53Z

lib/sedna/core/base.py

+                    }
+                )
+        else:
+            ckpt_model_url = FileOps.remove_path_prefix(self.config.model_url,


ckpt/pb are tensorflow backend specified

Adapting to CRD

then they should appear in backend directory, not in core/base.py

llhuii · 2021-06-01T07:11:18Z

lib/sedna/common/utils.py

+        s.connect(("8.8.8.8", 80))
+        ip = s.getsockname()[0]
+    except:
+        pass


when except, need to s.close()

llhuii · 2021-06-01T07:15:04Z

lib/sedna/common/log.py

+
+log_config = {
+    'DEBUG': {
+        'level': 10,


we could just use the python logging level number.

llhuii · 2021-06-01T07:16:17Z

lib/sedna/common/log.py

+
+        self.logger.addHandler(self.handler)
+        self.logLevel = 'DEBUG'
+        self.logger.setLevel(logging.DEBUG)


default logLevel should be INFO or WARN not DEBUG.

llhuii · 2021-06-01T08:17:30Z

lib/sedna/service/server/base.py

+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import uuid


import order for clarity:

standard packages empty new line third packages empty new line self packages

llhuii · 2021-06-01T08:20:36Z

lib/sedna/service/client.py

+        json_data.update({"data": x})
+        _url = "{}/{}".format(self.endpoint, "predict")
+        try:
+            res = http_request(url=_url, method="POST", json=json_data)


just

return http_request(url=_url, method="POST", json=json_data)

llhuii · 2021-06-01T08:21:39Z

lib/sedna/service/client.py

+
+    def inference(self, x, **kwargs):
+        """Use the remote big model server to inference."""
+        json_data = deepcopy(kwargs)


same question: why deepcopy?

if will change below

llhuii · 2021-06-01T08:22:42Z

lib/sedna/service/client.py

+        """Use the remote big model server to inference."""
+        json_data = deepcopy(kwargs)
+        json_data.update({"data": x})
+        _url = "{}/{}".format(self.endpoint, "predict")


this _url is fixed, can be initialized once

llhuii · 2021-06-01T08:23:28Z

lib/sedna/service/client.py

+
+    def check_server_status(self):
+        try:
+            http_request(url=self.endpoint, method="GET", json={})


json={} can be omitted

llhuii · 2021-06-01T08:36:38Z

please fix ci first

MooreZheng · 2021-06-03T07:00:09Z

docs/proposals/lifelong-learning.md

+## Motivation
+
+
+The edge-cloud collaborative lifelong learning combines the attributes of multi-task learning and incremental learning at the edge to deal with the problem of heterogeneous data and small samples in complex scenarios. Rely on the memory of the knowledge base to adapt to new situations, thus effectively solving the small sample problem of traditional machine learning.


Motivation is important and thus careful polishing would be needed. My suggestion is to modify the writing a little bit like "At present, edge-cloud synergy machine learning is confronted with the challenge of heterogeneous data distributions in complex scenarios and small samples on the edge. The edge-cloud synergy lifelong learning is accordingly proposed: 1) In order to learn with shared knowledge between historical scenarios, the scheme is essentially the combination of another two learning schemes, i.e., multi-task learning and incremental learning; 2) The cloud knowledge base in lifelong learning empowers the scheme with memory ability, which helps to adapt historical knowledge to new and unseen situations on the edge. Joining the forces of multi-task learning, incremental learning and the knowledge base, the lifelong learning scheme seeks to fundamentally overcome the above challenges of edge-cloud synergy machine learning."

llhuii · 2021-06-03T07:29:07Z

lib/sedna/common/class_factory.py

+class ClassFactory(object):
+    """
+        A Factory Class to manage all class need to register with config.
+        通过装饰器的方式来注册模块--只需要在上面加上一行ClassFactory.register(<class type>)即可完成注册。


pls fix these chinese comments

llhuii · 2021-06-03T08:17:26Z

lib/sedna/common/log.py

+        self.logger.log(log_level, msg, *args, **kwargs)
+
+
+sednaLogger = Logger().logger


sednaLogger rename into LOGGER or logger, without sedna word.

And here only use Logger.logger, then looks the other function of Logger is useless.

llhuii · 2021-06-03T09:06:23Z

lib/sedna/algorithms/hard_example_mining/__init__.py

+                    log_sum += class_data * math.log(class_data)
+                confidence_score = 1 + 1.0 * log_sum / math.log(
+                    len(infer_result))
+                return confidence_score >= self.threshold_cross_entropy


fix it done by #68

llhuii · 2021-06-03T09:15:13Z

docs/proposals/lifelong-learning.md

+|Lifelong-learning-job Reported State Updated    |  The controller appends the reported status of the job by LC in the cloud. |
+
+### Details of api between GM(cloud) and LC(edge)
+1. GM(downstream controller) syncs the job info to LC:


this section is same with the section of incremental learning job, please use a link instead of copy

jaypume · 2021-06-03T09:41:04Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+                 task_mining=None,
+                 task_remodeling=None,
+                 inference_integrate=None,
+


why there exist an empty line?

jaypume · 2021-06-03T09:56:24Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+            res = model_obj.train(train_data=task.samples, **kwargs)
+            if callback:
+                res = callback(model_obj, res)
+            model_path = model_obj.save(model_name=f"{task.entry}.pkl")


why the model end with fix string "pkl", other model format should be considered.

jaypume · 2021-06-03T09:58:04Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+            task.result = pred
+            task.model = m
+            tasks.append(task)
+        res = self.inference_integrate(tasks)


it is not a good name for me to understand the function.

jaypume · 2021-06-03T10:00:24Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+            metrics_param = {"average": "micro"}
+
+        data.x['pred'] = result
+        data.x['y'] = data.y


it maybe better understood that you use keys named "pred_y" and "real_y"

jaypume · 2021-06-03T10:03:40Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+        if metrics:
+            if callable(metrics):
+                m_name = getattr(metrics, '__name__', "mtl_eval")
+                m_dict = {
+                    m_name: metrics
+                }
+            elif isinstance(metrics, (set, list)):
+                for inx, m in enumerate(metrics):
+                    m_name = getattr(m, '__name__', f"mtl_eval_{inx}")
+                    if isinstance(m, str):
+                        m = getattr(sk_metrics, m)
+                    if not callable(m):
+                        continue
+                    m_dict[m_name] = m
+            elif isinstance(metrics, str):
+                m_dict = {
+                    metrics: getattr(sk_metrics, metrics, sk_metrics.log_loss)
+                }
+            elif isinstance(metrics, dict):
+                for k, v in metrics.items():
+                    if isinstance(v, str):
+                        v = getattr(sk_metrics, v)
+                    if not callable(v):
+                        continue
+                    m_dict[k] = v


there are so many "if else", and it is not intuitive what these lines do? you should add some comment or make the code more clearer.

jaypume · 2021-06-03T10:05:00Z

lib/sedna/algorithms/multi_task_learning/multi_task_learning.py

+        return {
+                   name: metric(data.y, result, **metrics_param)
+                   for name, metric in m_dict.items()
+               }, tasks_detail


in this long function, either you add comments in segments, or you break them down into smaller functions.

jaypume · 2021-06-03T10:06:59Z

lib/sedna/algorithms/multi_task_learning/task_jobs/task_definition.py

+        samples = BaseDataSource(data_type=d_type)
+        samples.x = x_data
+        samples.y = y_data
+        return tasks, task_index, samples


It's best to write down the data structures of these objects in comments.

jaypume · 2021-06-03T10:09:25Z

lib/sedna/algorithms/multi_task_learning/task_jobs/task_mining.py

+        allocations = meta_attr.apply(
+            lambda x: self.task_extractor.get(
+                "_".join(
+                        map(lambda y: str(x[y]).replace("_", "-").replace(" ", ""),
+                            self.attr_filed)
+                    ),
+                0),
+            axis=1).values.tolist()


these line looks strange, why you use so hard code "replace" function to do something we don't know.

Avoid errors caused by special characters in some metadata.

llhuii · 2021-06-03T10:57:21Z

please squash/rebase these commits into clean state

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

llhuii · 2021-06-04T03:08:50Z

examples/lifelong_learning/atcii/README.md

+      spec:
+        nodeName: "edge-node"
+        containers:
+          - image: kubeedge/sedna-example-lifelong-learning-atcii-classifier:v0.1.0


tag => v0.3.0

llhuii · 2021-06-04T03:09:26Z

examples/lifelong_learning/atcii/README.md

+  trainSpec:
+    template:
+      spec:
+        nodeName: "edge-node"


pls use $WORKER_NODE instead of edge-node

llhuii · 2021-06-04T03:10:15Z

examples/lifelong_learning/atcii/README.md

+      spec:
+        nodeName: "edge-node"
+        containers:
+          - image: kubeedge/sedna-example-lifelong-learning-atcii-classifier:v0.1.0


ditto, tag v0.1. => v0.3.0

llhuii · 2021-06-04T03:10:42Z

examples/lifelong_learning/atcii/README.md

+  evalSpec:
+    template:
+      spec:
+        nodeName: "edge-node"


ditto, fix nodeName

llhuii · 2021-06-04T03:13:34Z

examples/lifelong_learning/atcii/README.md

+
+
+### Effect Display  
+in this example, false and failed detections occur at stage of inference before lifelong learning, after lifelong learning, 


Suggested change

in this example, false and failed detections occur at stage of inference before lifelong learning, after lifelong learning,

In this example, **false** and **failed** detections occur at stage of inference before lifelong learning.

llhuii · 2021-06-04T03:16:06Z

examples/lifelong_learning/atcii/README.md

+
+### Effect Display  
+in this example, false and failed detections occur at stage of inference before lifelong learning, after lifelong learning, 
+Greatly improve the precision and accuracy of the dataset.


fill the xx%

Suggested change

Greatly improve the precision and accuracy of the dataset.

After lifelong learning, the precision and accuracy of the dataset have been improved by xx%.

llhuii · 2021-06-04T03:32:48Z

lib/sedna/core/incremental_learning/incremental_learning.py

+                              ) and len(task_info_res):
+                    task_info_res = list(task_info_res)[0]
+                final_res.append(task_info_res)
+        else:


this else branch can be merge into if branch

llhuii · 2021-06-04T03:36:06Z

lib/sedna/core/joint_inference/joint_inference.py

+              valid_data=None,
+              post_process=None,
+              **kwargs):
+        self.log.warn("Joint inference Experiment should offer model file")


Why does joint-inference have train?

Experiment means ?

Why does warn at the very firstly of train

llhuii · 2021-06-04T03:43:17Z

lib/sedna/core/joint_inference/joint_inference.py

+
+class JointInference(JobBase):
+    """
+   Joint inference Experiment


ditto, fix Experiment word

llhuii · 2021-06-04T03:45:11Z

lib/sedna/core/joint_inference/joint_inference.py

+        self.cloud = ModelClient(service_name=self.job_name,
+                                 host=self.remote_ip, port=self.port)
+
+    def train(self, train_data,


many duplicate codes with TSBigModelService.train

ugvddm · 2021-06-03T13:12:44Z

examples/federated_learning/surface_defect_detection/training_worker/interface.py

+os.environ['BACKEND_TYPE'] = 'KERAS'
+
+
+class Estimator:


Please add a comment to describe the function of Estimator

Estimator As an input parameter of the core of sedna, it represents a functional object that has a training callback function.
Thease parameters should be described in the readme of the lib, not in the example.

OK，so it will be better to call it as XXXEstimator?

This is a great piece of advice and I hope to fix it in the next version.

llhuii · 2021-06-04T04:06:52Z

lib/sedna/service/server/knowledgeBase/server.py

+
+
+if __name__ == '__main__':
+    KBServer(host="127.0.0.1").start()


Is 127.0.0.1 right? listening to 127.0.0.1 will forbid train-worker access KB server

llhuii · 2021-06-04T07:05:42Z

/lgtm

Let's improve this code with addressing the following but not limited issues in next one or two months:

KB server need its standalone code directory instead of placing into lib directory
KB API need to be reviewed again(such as /file/download, /file/upload)
In multi-task-learning of lib code, need to add docs and code comment
Remove scikit-learn requirement for federated-learning surface-defect-detection example
Reduce lifelong example image size
Code bugs, doc issues

llhuii · 2021-06-04T07:26:02Z

examples/lifelong_learning/atcii/README.md

+Follow the [Sedna installation document](/docs/setup/install.md) to install Sedna.
+
+### Prepare Dataset
+In this example, you can use [ASHRAE Global Thermal Comfort Database II](https://datadryad.org/stash/dataset/doi:10.6078/D1F671) to initial lifelong learning Knowledgebase .


Suggested change

In this example, you can use [ASHRAE Global Thermal Comfort Database II](https://datadryad.org/stash/dataset/doi:10.6078/D1F671) to initial lifelong learning Knowledgebase .

In this example, you can use [ASHRAE Global Thermal Comfort Database II](https://datadryad.org/stash/dataset/doi:10.6078/D1F671) to initial lifelong learning job.

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

llhuii · 2021-06-04T07:51:53Z

/lgtm
/approve

kubeedge-bot · 2021-06-04T07:52:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: llhuii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~docs/OWNERS~~ [llhuii]
~~examples/OWNERS~~ [llhuii]
~~lib/OWNERS~~ [llhuii]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kubeedge-bot requested review from JimmyYang20 and TymonXie May 31, 2021 07:24

kubeedge-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label May 31, 2021

JoeyHwong-gk mentioned this pull request Jun 1, 2021

REQUEST: New membership for <JoeyHwong-gk> kubeedge/community#51

Closed

6 tasks

llhuii reviewed Jun 1, 2021

View reviewed changes

JoeyHwong-gk changed the title ~~lib refector~~ lib refector and add lifelong learning Jun 1, 2021

JoeyHwong-gk marked this pull request as draft June 2, 2021 01:27

kubeedge-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jun 2, 2021

MooreZheng reviewed Jun 3, 2021

View reviewed changes

llhuii reviewed Jun 3, 2021

View reviewed changes

jaypume reviewed Jun 3, 2021

View reviewed changes

JoeyHwong-gk added 4 commits June 3, 2021 19:02

Lib Refector

0c7f132

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

Add Lifelong Learning

684aa03

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

Add Knowledgebase service script

64ba37d

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

Add Lifelong Learning doc

8a33c29

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

JoeyHwong-gk force-pushed the lib-refactor branch from 122bae8 to 8a33c29 Compare June 3, 2021 11:03

llhuii reviewed Jun 4, 2021

View reviewed changes

examples/lifelong_learning/atcii/README.md Outdated

evalSpec:

template:

spec:

nodeName: "edge-node"

Copy link

llhuii Jun 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, fix nodeName

llhuii reviewed Jun 4, 2021

View reviewed changes

JoeyHwong-gk force-pushed the lib-refactor branch from 4547dfd to 75509a9 Compare June 4, 2021 03:28

llhuii reviewed Jun 4, 2021

View reviewed changes

JoeyHwong-gk force-pushed the lib-refactor branch 2 times, most recently from fc1f7d8 to 1edef5f Compare June 4, 2021 03:54

ugvddm reviewed Jun 4, 2021

View reviewed changes

llhuii reviewed Jun 4, 2021

View reviewed changes

JoeyHwong-gk force-pushed the lib-refactor branch from 1edef5f to a3440eb Compare June 4, 2021 06:45

kubeedge-bot assigned llhuii Jun 4, 2021

kubeedge-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 4, 2021

llhuii reviewed Jun 4, 2021

View reviewed changes

fix pr comment

97a77d0

Signed-off-by: JoeyHwong <joeyhwong@gknow.cn>

JoeyHwong-gk force-pushed the lib-refactor branch from a3440eb to 97a77d0 Compare June 4, 2021 07:41

kubeedge-bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 4, 2021

kubeedge-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 4, 2021

kubeedge-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2021

kubeedge-bot merged commit 9411a2c into kubeedge:main Jun 4, 2021

This was referenced Jun 4, 2021

lifelong learning enhancements tracking issue #85

Open

Lib Refactoring #46

Closed

MooreZheng mentioned this pull request May 5, 2022

REQUEST: New membership for MooreZheng kubeedge/community#76

Closed

7 tasks

		## Motivation


		The edge-cloud collaborative lifelong learning combines the attributes of multi-task learning and incremental learning at the edge to deal with the problem of heterogeneous data and small samples in complex scenarios. Rely on the memory of the knowledge base to adapt to new situations, thus effectively solving the small sample problem of traditional machine learning.

		self.logger.log(log_level, msg, args, *kwargs)


		sednaLogger = Logger().logger



		### Effect Display
		in this example, false and failed detections occur at stage of inference before lifelong learning, after lifelong learning,

	in this example, false and failed detections occur at stage of inference before lifelong learning, after lifelong learning,
	In this example, false and failed detections occur at stage of inference before lifelong learning.

	Greatly improve the precision and accuracy of the dataset.
	After lifelong learning, the precision and accuracy of the dataset have been improved by xx%.



		if __name__ == '__main__':
		KBServer(host="127.0.0.1").start()

	In this example, you can use [ASHRAE Global Thermal Comfort Database II](https://datadryad.org/stash/dataset/doi:10.6078/D1F671) to initial lifelong learning Knowledgebase .
	In this example, you can use [ASHRAE Global Thermal Comfort Database II](https://datadryad.org/stash/dataset/doi:10.6078/D1F671) to initial lifelong learning job.

		"""


		@ClassFactory.register(ClassType.FLAGG)

lib refector and add lifelong learning #72

lib refector and add lifelong learning #72

Conversation

JoeyHwong-gk commented May 31, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llhuii Jun 2, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llhuii commented Jun 1, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llhuii Jun 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llhuii Jun 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JoeyHwong-gk commented May 31, 2021 •

edited

Loading

llhuii Jun 2, 2021 •

edited

Loading

llhuii Jun 3, 2021 •

edited

Loading

llhuii Jun 3, 2021 •

edited

Loading