Search feature improvement to-dos

[KnightChaser]

Currently, the search tab doesn't provide any kind of pagination. If users search for some conditions in a large-size EVTX dataset collection, the current code will just return lots of rows at once like the picture below. Also, the current status performs sorting via _id property after receiving all data from MongoDB, so it might be bad practice in case of large data handling

Improvement suggestions

• Pagination (e.g. Showing 30 results at once. Use MongoDB's aggregation to limit the number of results) ▶ Using infinite scroll? ▶ abolished
• Showing pagination bar (use Bootstrap components) both on and below the table.
• Sorting the result on the MongoDB side, not the client side may reduce the speed
• Printing how many rows are searched (e.g. 1,920 results) on the table
• Forcibly fix the table width (design)
• Possible improvements in Go backend related to search features /webRequestHandler/api.go

Adopting pagination may require additional calculations for querying a specific range of data from MongoDB.

[KnightChaser]

The user may often receive over 10,000 rows while searching the log data with certain conditions. In this case, pagination won't be useless. However, trying to render every row from the database will be extremely inefficient and even cause lagging on the interface. So, I guess adopting infinite scroll instead of pagination that is clicking-interaction-based would be better for Aesir project. (example: showing the first 300 rows for the first time. If the user scrolls to the end, additional 300 rows will appear, and go on.)

[KnightChaser]

Be aware that you're using MongoDB while implementing infinite scroll or something similar alternatives in the future. Sort according to _id value attached to every document in the MongoDB collection, and use skip() and limit() methods provided from MongoDB.

For example, the following showPage() function will return 2 documents according to its integer-type parameter page from the collection orders applied to sort based on the _id field in descending order(-1).

var showPage = function(page){
... return db.orders.find().sort( { "_id": -1 } ).skip((page-1)*2).limit(2);
... }

[KnightChaser]

I changed my thoughts, it'd be better if I just skipped the pagination or infinite scroll feature for now because loading approx. Searching and filtering conditions 10,000 rows (~10MiB) is taken in less than 1 second(kinda acceptable I think). Instead, it'd be better if I show the progress, or just skip this feature for it.

[KnightChaser]

Successfully merged. Closing the issues and may be reopened in the future in case of requiring additional new features related to the search tab.