Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support serializable lambdas for up-to-date checks #17327

Closed
wolfs opened this issue May 31, 2021 · 6 comments · Fixed by #21054
Closed

Support serializable lambdas for up-to-date checks #17327

wolfs opened this issue May 31, 2021 · 6 comments · Fixed by #21054
Assignees
Labels
a:feature A new functionality in:execution-engine incremental, up-to-date, overlapping outputs
Milestone

Comments

@wolfs
Copy link
Member

wolfs commented May 31, 2021

Gradle currently doesn't support Java lambdas (aka invoke dynamic implementations of classes) since it is unable to track the implementation for the classes the JVM generates at runtime. For serializable lambdas, Gradle could extract enough information to track the implementation and therefore support those as task actions. This is especially interesting, since Kotlin 1.5 by default generates invoke dynamic calls when a lambda implements a SAM type.

As a next step we could automatically make the lambdas we use for doLast etc. serializable.

Design spec


cc: @gradle/execution

@wolfs wolfs added a:feature A new functionality @execution in:execution-engine incremental, up-to-date, overlapping outputs labels May 31, 2021
@wolfs
Copy link
Member Author

wolfs commented May 31, 2021

See #10751 for the deprecation of Java lambdas as task actions.

@wolfs
Copy link
Member Author

wolfs commented May 31, 2021

I did an investigation what we need to do to support serializable lambdas. This snippet allows you to obtain a SerializedLambda object from something implemented by a serializable lambda:

private static Optional<SerializedLambda> serializedLambdaFor(Object lambda) {
    if (!(lambda instanceof Serializable)) {
        return Optional.empty();
    }
    for (Class<?> lambdaClass = lambda.getClass(); lambdaClass != null; lambdaClass = lambdaClass.getSuperclass()) {
        try {
            Method replaceMethod = lambdaClass.getDeclaredMethod("writeReplace");
            replaceMethod.setAccessible(true);
            Object serializedForm = replaceMethod.invoke(lambda);
            if (serializedForm instanceof SerializedLambda) {
                return Optional.of((SerializedLambda) serializedForm);
            } else {
                return Optional.empty();
            }
        } catch (NoSuchMethodException e) {
            // continue
        } catch (InvocationTargetException | IllegalAccessException e) {
            return Optional.empty();
        }
    }
    return Optional.empty();
}

Running this on a task action implemented by a serializable lambda gives something like this:
image

I suppose we need to capture the fields of this object and the implementation snapshot of the capturing class. We will also need to capture the capturedArgs.

All of this seems to be doable, though we'll need to change some part of our infrastructure since passing around the Class<?> object of the action won't be enough any more.

@lptr
Copy link
Member

lptr commented Jun 1, 2021

Can we use ImplementationSnapshot to carry this information for serializable lambdas?

@wolfs
Copy link
Member Author

wolfs commented Jun 1, 2021

Can we use ImplementationSnapshot to carry this information for serializable lambdas?

We can't use the current class, though I suppose we can create a subclass which has the necessary information and adds it accordingly to the hasher. We can create this new class in TaskActionWrapper.getActionImplementation to create the right ImplementationSnapshot subclass. This would allow using serializable lambdas for task actions, but not for nested inputs.

For fixing the problem for nested inputs we need to create the right snapshot when we attach the class object as an input property, since the class object won't be enough anymore for the snapshotter to determine the implementation snapshot.

@stale
Copy link

stale bot commented Jun 4, 2022

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you're interested in how we try to keep the backlog in a healthy state, please read our blog post on how we refine our backlog. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

@stale stale bot added the stale label Jun 4, 2022
@melix
Copy link
Contributor

melix commented Jun 7, 2022

unstale

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:feature A new functionality in:execution-engine incremental, up-to-date, overlapping outputs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants