Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8264954: unified handling for VectorMask object re-materialization during de-optimization #3408

Closed
wants to merge 3 commits into from
Closed
Changes from 1 commit
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
@@ -267,11 +267,7 @@ void PhaseVector::scalarize_vbox_node(VectorBoxNode* vec_box) {
// logic for both predicated and non-predicated targets.
bool is_mask = is_vector_mask(iklass);
if (is_mask) {
if (vec_value->Opcode() == Op_VectorLoadMask) {
const Type* in1_ty = vec_value->in(1)->bottom_type();
assert(in1_ty->isa_vect() && in1_ty->is_vect()->element_basic_type() == T_BOOLEAN, "");
vec_value = vec_value->in(1);
} else if (vec_value->Opcode() != Op_VectorStoreMask) {
if (vec_value->Opcode() != Op_VectorStoreMask) {

This comment has been minimized.

Loading
@iwanowww

iwanowww Apr 9, 2021
Outdated

It's better to do the check against the type of vec_value: VectorStoreMask produces a vector of booleans and you can guard gvn.transform(VectorStoreMaskNode::make(...)) call with bt != T_BOOLEAN check instead.

This comment has been minimized.

Loading
@jatin-bhateja

jatin-bhateja Apr 12, 2021
Author Member Outdated

In addition to bt != T_BOOLEAN check, we also need to check if type is a vectormask (explicit type added for mask generating nodes on targets supporting predicate registers).

A similar check needs to be added at following location
https://github.com/openjdk/jdk/blob/master/src/hotspot/share/opto/vector.cpp#L329

Keeping this as it is i.e. an Opcode based check.

This comment has been minimized.

Loading
@iwanowww

iwanowww Apr 12, 2021
Outdated

In addition to bt != T_BOOLEAN check, we also need to check if type is a vectormask (explicit type added for mask generating nodes on targets supporting predicate registers).

You check here that the mask is in canonical representation (which is a vector of booleans) before applying the canonicalizing conversion. Alternatively, you could check that the mask is in "native"/platform-dependent representation (vec_value has TypeVectMask type).

So, my reading of your response is that bt != T_BOOLEAN check is not enough to reliably distinguish between the representations: a mask for vector of booleans (in native representation) will have the same type as the vector of booleans produced by the canonicalizing cast of a mask value. I agree with that.

But until TypeVectMask is used uniformly and independently of predicate register support, TypeVect vs TypeVectMask doesn't cut the problem as well. And adding bt != T_BOOLEAN doesn't help.

So, I'm fine with leaving the Opcode check for now.

In the longer term, we need to come up with a uniform representation in type system for masks in native format to be able to reliably distinguish between masks and vectors in cross-platform manner.

const TypeVect* vt = vec_value->bottom_type()->is_vect();
BasicType bt = vt->element_basic_type();
vec_value = gvn.transform(VectorStoreMaskNode::make(gvn, vec_value, bt, vt->length()));

This comment has been minimized.

Loading
@iwanowww

iwanowww Apr 9, 2021
Outdated

VectorStoreMaskNode::Identity() already handles VectorStoreMask (VectorLoadMask bv) elem_size ==> bv transformation, doesn't it? So, you can just unconditionally instantiate VectorStoreMaskNode and let IGVN handle it.

Also, an observation on naming: VectorLoadMask and VectorStoreMask names are misleading. On the surface, they look like memory nodes, but in fact, are cast nodes (Vector <-> Mask). Please, file an RFE to address that eventually.

@@ -65,6 +65,8 @@ BasicType VectorSupport::klass2bt(InstanceKlass* ik) {

if (is_vector_shuffle(ik)) {
return T_BYTE;
} else if (is_vector_mask(ik)) {
return T_BOOLEAN;
} else { // vector and mask
oop value = ik->java_mirror()->obj_field(fd.offset());
BasicType elem_bt = java_lang_Class::as_BasicType(value);
@@ -86,45 +88,36 @@ jint VectorSupport::klass2length(InstanceKlass* ik) {
return vlen;
}

void VectorSupport::init_payload_element(typeArrayOop arr, bool is_mask, BasicType elem_bt, int index, address addr) {
if (is_mask) {
switch (elem_bt) {
case T_BYTE:
case T_SHORT:
case T_INT:
case T_LONG:
case T_FLOAT:
case T_DOUBLE: arr->bool_at_put(index, (*(jbyte*)addr) != 0); break;

default: fatal("unsupported: %s", type2name(elem_bt));
}
} else {
switch (elem_bt) {
case T_BYTE: arr-> byte_at_put(index, *(jbyte*)addr); break;
case T_SHORT: arr-> short_at_put(index, *(jshort*)addr); break;
case T_INT: arr-> int_at_put(index, *(jint*)addr); break;
case T_FLOAT: arr-> float_at_put(index, *(jfloat*)addr); break;
case T_LONG: arr-> long_at_put(index, *(jlong*)addr); break;
case T_DOUBLE: arr->double_at_put(index, *(jdouble*)addr); break;

default: fatal("unsupported: %s", type2name(elem_bt));
}
// Masks require special handling: when boxed they are packed and stored in boolean
// arrays, but in scalarized form they have the same size as corresponding vectors.
// For example, Int512Mask is represented in memory as boolean[16], but
// occupies the whole 512-bit vector register when scalarized.
// During scalarization inserting a VectorStoreMask node between mask
// and safepoint node always ensures the existence of masks in a boolean array.
//
// TODO: revisit when predicate registers are fully supported.

This comment has been minimized.

Loading
@iwanowww

iwanowww Apr 9, 2021
Outdated

TODO line can be removed now.

//
void VectorSupport::init_payload_element(typeArrayOop arr, BasicType elem_bt, int index, address addr) {
switch (elem_bt) {
case T_BOOLEAN:

This comment has been minimized.

Loading
@iwanowww

iwanowww Apr 9, 2021
Outdated

Please, use bool_at_put() accessor here (just for clarity):

case T_BOOLEAN: arr->bool_at_put(index, *(jboolean*)addr); break;
case T_BYTE: arr-> byte_at_put(index, *(jbyte*)addr); break;
case T_SHORT: arr-> short_at_put(index, *(jshort*)addr); break;
case T_INT: arr-> int_at_put(index, *(jint*)addr); break;
case T_FLOAT: arr-> float_at_put(index, *(jfloat*)addr); break;
case T_LONG: arr-> long_at_put(index, *(jlong*)addr); break;
case T_DOUBLE: arr->double_at_put(index, *(jdouble*)addr); break;

default: fatal("unsupported: %s", type2name(elem_bt));
}
}

Handle VectorSupport::allocate_vector_payload_helper(InstanceKlass* ik, frame* fr, RegisterMap* reg_map, Location location, TRAPS) {
bool is_mask = is_vector_mask(ik);

int num_elem = klass2length(ik);
BasicType elem_bt = klass2bt(ik);
// Inserting a VectorStoreMask before stitching the mask
// to SafePointNode will ensure packing the mask into a
// byte array for masks present in both predicated register
// or vector registers.
int elem_size = is_mask ? 1 : type2aelembytes(elem_bt);
int elem_size = type2aelembytes(elem_bt);

// On-heap vector values are represented as primitive arrays.
TypeArrayKlass* tak = TypeArrayKlass::cast(Universe::typeArrayKlassObj(is_mask ? T_BOOLEAN : elem_bt));
TypeArrayKlass* tak = TypeArrayKlass::cast(Universe::typeArrayKlassObj(elem_bt));

typeArrayOop arr = tak->allocate(num_elem, CHECK_NH); // safepoint

@@ -137,13 +130,13 @@ Handle VectorSupport::allocate_vector_payload_helper(InstanceKlass* ik, frame* f
int off = (i * elem_size) % VMRegImpl::stack_slot_size;

address elem_addr = reg_map->location(vreg, vslot) + off; // assumes little endian element order
init_payload_element(arr, is_mask, elem_bt, i, elem_addr);
init_payload_element(arr, elem_bt, i, elem_addr);
}
} else {
// Value was directly saved on the stack.
address base_addr = ((address)fr->unextended_sp()) + location.stack_offset();
for (int i = 0; i < num_elem; i++) {
init_payload_element(arr, is_mask, elem_bt, i, base_addr + i * elem_size);
init_payload_element(arr, elem_bt, i, base_addr + i * elem_size);
}
}
return Handle(THREAD, arr);
@@ -42,7 +42,7 @@ class VectorSupport : AllStatic {
static Handle allocate_vector_payload(InstanceKlass* ik, frame* fr, RegisterMap* reg_map, ScopeValue* payload, TRAPS);
static Handle allocate_vector_payload_helper(InstanceKlass* ik, frame* fr, RegisterMap* reg_map, Location location, TRAPS);

static void init_payload_element(typeArrayOop arr, bool is_mask, BasicType elem_bt, int index, address addr);
static void init_payload_element(typeArrayOop arr, BasicType elem_bt, int index, address addr);

static BasicType klass2bt(InstanceKlass* ik);
static jint klass2length(InstanceKlass* ik);