Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Total index memory in _cat/indices #7008

Closed
bobrik opened this issue Jul 24, 2014 · 3 comments

Comments

@bobrik
Copy link
Contributor

commented Jul 24, 2014

Currently memory stats are available separately (which is good), but it not in accumulated metric like total memory used by index. Some awk could fix that, but having built-in ability would be great.

# curl -s http://es:9200/_cat/indices?h=index,totalMemory
statistics-20140620 45.2mb

Also, bloom filter memory usage is not available in _cat/indices. Not sure if something else is missing. Is there a hope to see full memory usage per index?

@s1monw

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2014

sounds useful - do you wanna come up with a PR?

@s1monw s1monw self-assigned this Jul 24, 2014

@s1monw s1monw added the enhancement label Jul 24, 2014

@s1monw s1monw removed their assignment Jul 24, 2014

@s1monw s1monw added v2.0.0 labels Jul 24, 2014

@bobrik

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2014

I'm not a java programmer, I can only come up with something like this:

[~] % curl -s http://127.0.0.1:9200/_cat/indices?h=index,fm,fcm,im,pm,sm,siwm,svvm,tm
whatever 0b 0b 0b -1b 6.6kb 0b 6.6kb
diff --git a/src/main/java/org/elasticsearch/action/admin/indices/stats/CommonStats.java b/src/main/java/org/elasticsearch/action/admin/indices/stats/CommonStats.java
index a4dd483..744209d 100644
--- a/src/main/java/org/elasticsearch/action/admin/indices/stats/CommonStats.java
+++ b/src/main/java/org/elasticsearch/action/admin/indices/stats/CommonStats.java
@@ -24,6 +24,7 @@ import org.elasticsearch.common.Nullable;
 import org.elasticsearch.common.io.stream.StreamInput;
 import org.elasticsearch.common.io.stream.StreamOutput;
 import org.elasticsearch.common.io.stream.Streamable;
+import org.elasticsearch.common.unit.ByteSizeValue;
 import org.elasticsearch.common.xcontent.ToXContent;
 import org.elasticsearch.common.xcontent.XContentBuilder;
 import org.elasticsearch.index.cache.filter.FilterCacheStats;
@@ -463,6 +464,18 @@ public class CommonStats implements Streamable, ToXContent {
         return stats;
     }

+    public ByteSizeValue getTotalMemory() {
+        long size = this.getFieldData().getMemorySizeInBytes() +
+                this.getFilterCache().getMemorySizeInBytes() +
+                this.getIdCache().getMemorySizeInBytes() +
+                this.getPercolate().getMemorySizeInBytes() +
+                this.getSegments().getMemoryInBytes() +
+                this.getSegments().getIndexWriterMemoryInBytes() +
+                this.getSegments().getVersionMapMemoryInBytes();
+
+        return new ByteSizeValue(size);
+    }
+
     @Override
     public void readFrom(StreamInput in) throws IOException {
         if (in.readBoolean()) {
diff --git a/src/main/java/org/elasticsearch/rest/action/cat/RestIndicesAction.java b/src/main/java/org/elasticsearch/rest/action/cat/RestIndicesAction.java
index 2001a4b..c771787 100644
--- a/src/main/java/org/elasticsearch/rest/action/cat/RestIndicesAction.java
+++ b/src/main/java/org/elasticsearch/rest/action/cat/RestIndicesAction.java
@@ -264,6 +264,8 @@ public class RestIndicesAction extends AbstractCatAction {
         table.addCell("suggest.total", "sibling:pri;alias:suto,suggestTotal;default:false;text-align:right;desc:number of suggest ops");
         table.addCell("pri.suggest.total", "default:false;text-align:right;desc:number of suggest ops");

+        table.addCell("memory.total", "sibling:pri;alias:tm,memoryTotal;default:false;text-align:right;desc:total used memory");
+        table.addCell("pri.memory.total", "default:false;text-align:right;desc:total user memory");

         table.endHeaders();
         return table;
@@ -443,6 +445,9 @@ public class RestIndicesAction extends AbstractCatAction {
             table.addCell(indexStats == null ? null : indexStats.getTotal().getSuggest().getCount());
             table.addCell(indexStats == null ? null : indexStats.getPrimaries().getSuggest().getCount());

+            table.addCell(indexStats == null ? null : indexStats.getTotal().getTotalMemory());
+            table.addCell(indexStats == null ? null : indexStats.getPrimaries().getTotalMemory());
+
             table.endRow();
         }

Is that ok?

@jpountz jpountz removed the adoptme label Aug 1, 2014

@clintongormley clintongormley added v1.5.0 and removed v1.4.0.Beta1 labels Sep 8, 2014

dadoonet added a commit to dadoonet/elasticsearch that referenced this issue Oct 7, 2014
Admin: add total index memory in `_cat/indices`
This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map).

```
% curl 'localhost:9200/_cat/indices?v&h=i,tm'
i     tm
wiki  8.1gb
test  30.5kb
user  1.9mb
```

Closes elastic#7008

@dadoonet dadoonet closed this in 80ca8e5 Oct 8, 2014

dadoonet added a commit that referenced this issue Oct 8, 2014
Admin: add total index memory in `_cat/indices`
This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map).

```
% curl 'localhost:9200/_cat/indices?v&h=i,tm'
i     tm
wiki  8.1gb
test  30.5kb
user  1.9mb
```

Closes #7008

(cherry picked from commit 80ca8e5)
@bobrik

This comment has been minimized.

Copy link
Contributor Author

commented Oct 8, 2014

Awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.