Put limits on local and inline counts for inlining

We have some non-linear algorithms as part of inlining - of note, in the
deopt tracking. If the frame gets too big in various ways, the cost of
that can become very large. Put some limits in place to catch such
cases, so we won't end up spending a vast amount of time trying to win
performance, but end up blocking the application for multiple seconds
due to the spesh thread getting stuck in a highly costly loop.

Author: jnthn

src/spesh/inline.c

@@ -83,6 +83,16 @@ static int is_static_frame_inlineable(MVMThreadContext *tc, MVMSpeshGraph *inlin
         return 0;
     }
 
+    /* Ensure that we haven't hit size limits. */
+    if (inliner->num_locals > MVM_SPESH_INLINE_MAX_LOCALS) {
+        *no_inline_reason = "inliner has too many locals";
+        return 0;
+    }
+    if (inliner->num_inlines > MVM_SPESH_INLINE_MAX_INLINES) {
+        *no_inline_reason = "inliner has too many inlines";
+        return 0;
+    }
+
     return 1;
 }
 

src/spesh/inline.h

@@ -1,6 +1,12 @@
 /* Default maximum size of bytecode we'll inline. */
 #define MVM_SPESH_DEFAULT_MAX_INLINE_SIZE 192
 
+/* The maximum number of locals an inliner can reach, and maximum number of
+ * inlines we can reach, before we stop inlining; this is to prevent us
+ * reaching sizes where the analysis becomes hugely costly. */
+#define MVM_SPESH_INLINE_MAX_LOCALS     512
+#define MVM_SPESH_INLINE_MAX_INLINES    128
+
 /* Inline table entry. The data is primarily used in deopt. */
 struct MVMSpeshInline {
     /* Start and end position in the bytecode where we're inside of this