Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mng 6957: Support Build Consumer while defining modules in reverse order #391

Closed
wants to merge 114 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
acaaf88
MNG-6957: extract inherit
rfscholte Jul 10, 2020
0a1cb05
MNG-6957: remove model from inherit method
rfscholte Jul 10, 2020
121bd9e
Simplify code
rfscholte Jul 11, 2020
a5e7ee6
Get modules from fileModel, respect profiles (which can also contain …
rfscholte Jul 11, 2020
208de10
Add activeExternalProfiles
rfscholte Jul 12, 2020
f03490b
No need to check getActiveExternalProfiles, can't contain modules
rfscholte Jul 12, 2020
2ed1627
Moving up setters of result
rfscholte Jul 12, 2020
cbf6c29
No need to keep activeProfiles in ModelData
rfscholte Jul 13, 2020
a77383a
result.setRawModel( modelId, rawModel ) can also be moved
rfscholte Jul 14, 2020
98afc31
lineage can just a Models
rfscholte Jul 14, 2020
598060d
Replace ModelSource with Source
rfscholte Jul 17, 2020
e4cd759
Make clear where rawModel and model should be used
rfscholte Jul 17, 2020
30f4f3a
Use rawModel to read get parent
rfscholte Jul 17, 2020
5a2b9b5
pick up tmpModel from result
rfscholte Jul 17, 2020
24109e0
extract profiles from result
rfscholte Jul 17, 2020
3c5f811
verify superpom by id
rfscholte Jul 17, 2020
0a04f51
check currentData by id
rfscholte Jul 17, 2020
a324124
ModelData.rawModel is never used
rfscholte Jul 17, 2020
d486b4e
pick up inherited model from lineage
rfscholte Jul 17, 2020
cb39b97
Don't update ModelData
rfscholte Jul 17, 2020
3265c13
No need to change GAV for ModelData
rfscholte Jul 17, 2020
e7d2801
checkPluginVersions must use lineage
rfscholte Jul 17, 2020
fa8de9c
Split collecting raw models from inheritence
rfscholte Jul 17, 2020
72c59d6
Set effectiveModel instance early
rfscholte Jul 18, 2020
a655fe9
start reading poms witout parent
rfscholte Jul 18, 2020
7b244ae
move effectivemodel to phase 2
rfscholte Jul 18, 2020
a9e603c
get activeExternalProfiles from result
rfscholte Jul 18, 2020
5ac87c4
in case of root, pick up effectivepom from result
rfscholte Jul 18, 2020
a044f10
some code is only required for non root
rfscholte Jul 18, 2020
ac44aa9
poolbuilder can use effectivemodel
rfscholte Jul 18, 2020
6b6df5d
move fileModel to result
rfscholte Jul 19, 2020
0c61fdf
properly assign effective model
rfscholte Jul 19, 2020
87e735f
Read modules from fileModel
rfscholte Jul 20, 2020
36ccdb7
During 2 phase builds use the fileModel
rfscholte Jul 20, 2020
87abbb9
Move setEffectiveModel
rfscholte Jul 20, 2020
b00bd46
initProject after phase two
rfscholte Jul 21, 2020
a32a6f7
prepare rawModels for phase two builds
rfscholte Jul 21, 2020
d914299
pick up effective pom later
rfscholte Jul 22, 2020
81e93bd
use fileModel for originalModel
rfscholte Jul 22, 2020
5520ac3
use pomFile as key of projectIndex
rfscholte Jul 22, 2020
628f50a
pick up pom from fileModel
rfscholte Jul 22, 2020
8b87e55
Move rawModels to phase 2
rfscholte Jul 23, 2020
c4e216d
modelSource is never null
rfscholte Jul 23, 2020
95b60a0
Always look in cache for model
rfscholte Jul 23, 2020
3802003
Use modelSource to set pomFile
rfscholte Jul 23, 2020
ec90603
use modelSource's inputStream to read model
rfscholte Jul 23, 2020
eeb49bc
select severity based on modelSource instance
rfscholte Jul 23, 2020
136d396
early check for fatal errors
rfscholte Jul 23, 2020
48db31d
extract readRawModel
rfscholte Jul 23, 2020
aaa7067
Always use request.getModelSource() as cacheKey
rfscholte Jul 24, 2020
20170b0
always cache fileModel
rfscholte Jul 24, 2020
7191f91
Split readRawModel from readModel
rfscholte Jul 24, 2020
e173320
put fileModel in cache when available
rfscholte Jul 24, 2020
2a51bf1
Use the same model
rfscholte Jul 25, 2020
213bc84
improve moment of cloning
rfscholte Jul 25, 2020
3d1270a
Don't set pom on model
rfscholte Jul 25, 2020
93b5a01
ProjectBuilder should use rawModel because of its profile activations
rfscholte Jul 25, 2020
b503507
directly set preferred source
rfscholte Jul 25, 2020
49ee2ae
No need to set the project file again
rfscholte Jul 25, 2020
2f53b5b
improve naming of variables
rfscholte Jul 25, 2020
7f79fdc
rawModels should be based on rawModel
rfscholte Jul 26, 2020
2a46f8c
rawModel most be activated, not the fileModel
rfscholte Jul 26, 2020
3e08457
switch methods
rfscholte Jul 26, 2020
4c69a1f
Move readRawModel to phase 2
rfscholte Jul 26, 2020
a96a27a
Use pomFile so context is used
rfscholte Aug 5, 2020
a669e08
verify version after groupId+artifactId
rfscholte Aug 5, 2020
3e6d30e
effective model should not be cached as raw model
rfscholte Sep 19, 2020
d699e71
rename method
rfscholte Sep 19, 2020
1ccd145
cache raw pom for inheritance
rfscholte Sep 19, 2020
cc239b8
Move buildconsumer to phase 2
rfscholte Sep 20, 2020
68ad1a5
Use stream for collecting profile ids
rfscholte Sep 20, 2020
2b41e8d
Combine methods to reduce cache
rfscholte Sep 25, 2020
2c7a891
move readRawModel, is only used once
rfscholte Sep 26, 2020
dce61af
always return new result
rfscholte Sep 26, 2020
daaf2f0
Don't auto cast
rfscholte Sep 26, 2020
587df50
start moving call readFileModel
rfscholte Sep 26, 2020
33c2dc0
use rawModel for artifactId
rfscholte Sep 27, 2020
e1f9853
move readFileModel for parents
rfscholte Sep 27, 2020
6bc43be
clean up code
rfscholte Sep 27, 2020
2376951
use result of readRawModel
rfscholte Sep 27, 2020
847de5e
TransformerContext must use ModelCache
rfscholte Sep 29, 2020
2e55731
Happy flow build consumer restored
rfscholte Sep 30, 2020
53bb4ed
No need to cache temp raw model during phase 1
rfscholte Oct 2, 2020
6e36900
No need to reset getModelIds anymore
rfscholte Oct 2, 2020
f254871
Provide option key for transformerContext
rfscholte Oct 3, 2020
7b4908d
inline code
rfscholte Oct 3, 2020
94a74e1
rename to readEffectiveModel, so we have all readXModel available
rfscholte Oct 3, 2020
ebb85d2
Remove pomFile parameter
rfscholte Oct 3, 2020
22038bc
prepare reading unknown dependency from build/consumer pom
rfscholte Oct 4, 2020
3224175
Fix early dependency reactor read for build/consumer
rfscholte Oct 5, 2020
8bdd48a
Fix buildconsumer for unknown parent
rfscholte Oct 6, 2020
3153d92
This can stay ModelBuildingRequest
rfscholte Oct 6, 2020
091eaf1
Remove redundant parameter of readFileModel
rfscholte Oct 11, 2020
d2e9e29
remove redundant parameter from readRawModel
rfscholte Oct 11, 2020
9e2a32c
Introduce DefaultTransformerContext, initialized during build, immuta…
rfscholte Oct 13, 2020
8d071ed
Pass ModelBuildingRequest
rfscholte Oct 13, 2020
20f7067
Refactor code
rfscholte Oct 14, 2020
eb406ff
Update documentation
rfscholte Oct 15, 2020
7f39d05
Add javadoc
rfscholte Oct 15, 2020
b44c333
Improve parentFilter, change requirement getRawModel by Path
rfscholte Oct 16, 2020
9638628
Extract interface, only DefaultModelBuilder should be aware of implem…
rfscholte Oct 24, 2020
faffc45
improve documentation, make tags name similar
rfscholte Oct 24, 2020
f3e8d97
improve documentation
rfscholte Oct 26, 2020
b9607f0
fix merge
rfscholte Oct 27, 2020
b6f0b12
Merge branch 'master' into MNG-6957
rfscholte Oct 30, 2020
2737305
Merge branch 'master' into MNG-6957
rfscholte Nov 28, 2020
337d180
Merge branch 'master' into MNG-6957
rfscholte Nov 29, 2020
459ba0a
Merge branch 'master' into MNG-6957
rfscholte Dec 4, 2020
50e8beb
Do not use the ReactorModelCache to store the mapping between GA<->So…
gnodet Nov 28, 2020
aab4950
Adjustments based on review comments
rfscholte Dec 13, 2020
db89566
Adjustments based on review comments
rfscholte Dec 13, 2020
a1de191
Adjustments based on review comments
rfscholte Dec 14, 2020
6f7f5be
Merge branch 'master' into MNG-6957
rfscholte Dec 14, 2020
e686be7
Remove unused innerclass
rfscholte Dec 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,13 @@
class ConsumerModelSourceTransformer extends AbstractModelSourceTransformer
{
@Override
protected AbstractSAXFilter getSAXFilter( Path pomFile,
protected AbstractSAXFilter getSAXFilter( Path pomFile,
TransformerContext context,
Consumer<LexicalHandler> lexicalHandlerConsumer )
throws TransformerConfigurationException, SAXException, ParserConfigurationException
{
return new DefaultConsumerPomXMLFilterFactory( new DefaultBuildPomXMLFilterFactory( context,
lexicalHandlerConsumer ) ).get( pomFile );
lexicalHandlerConsumer, true ) ).get( pomFile );
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@

import java.io.File;
import java.io.IOException;
import java.nio.file.Path;
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Arrays;
Expand All @@ -33,6 +32,7 @@
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;

import javax.inject.Inject;
import javax.inject.Named;
Expand All @@ -56,6 +56,7 @@
import org.apache.maven.model.Profile;
import org.apache.maven.model.ReportPlugin;
import org.apache.maven.model.building.ArtifactModelSource;
import org.apache.maven.model.building.TransformerContextBuilder;
import org.apache.maven.model.building.DefaultModelBuildingRequest;
import org.apache.maven.model.building.DefaultModelProblem;
import org.apache.maven.model.building.FileModelSource;
Expand Down Expand Up @@ -129,7 +130,7 @@ public ProjectBuildingResult build( File pomFile, ProjectBuildingRequest request
throws ProjectBuildingException
{
return build( pomFile, new FileModelSource( pomFile ),
new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null ) );
new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null, null ) );
}

private boolean useGlobalModelCache()
Expand All @@ -142,7 +143,7 @@ public ProjectBuildingResult build( ModelSource modelSource, ProjectBuildingRequ
throws ProjectBuildingException
{
return build( null, modelSource,
new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null ) );
new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null, null ) );
}

private ProjectBuildingResult build( File pomFile, ModelSource modelSource, InternalConfig config )
Expand Down Expand Up @@ -263,14 +264,7 @@ private DependencyResolutionResult resolveDependencies( MavenProject project, Re

private List<String> getProfileIds( List<Profile> profiles )
{
List<String> ids = new ArrayList<>( profiles.size() );

for ( Profile profile : profiles )
{
ids.add( profile.getId() );
}

return ids;
return profiles.stream().map( Profile::getId ).collect( Collectors.toList() );
}

private ModelBuildingRequest getModelBuildingRequest( InternalConfig config )
Expand All @@ -295,7 +289,7 @@ private ModelBuildingRequest getModelBuildingRequest( InternalConfig config )
request.setBuildStartTime( configuration.getBuildStartTime() );
request.setModelResolver( resolver );
request.setModelCache( config.modelCache );
request.setTransformerContext( (TransformerContext) config.session.getData().get( TransformerContext.KEY ) );
request.setTransformerContextBuilder( config.transformerContextBuilder );

return request;
}
Expand All @@ -314,7 +308,8 @@ public ProjectBuildingResult build( Artifact artifact, boolean allowStubModel, P
org.eclipse.aether.artifact.Artifact pomArtifact = RepositoryUtils.toArtifact( artifact );
pomArtifact = ArtifactDescriptorUtils.toPomArtifact( pomArtifact );

InternalConfig config = new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null );
InternalConfig config =
new InternalConfig( request, null, useGlobalModelCache() ? getModelCache() : null, null );

boolean localProject;

Expand Down Expand Up @@ -386,36 +381,13 @@ public List<ProjectBuildingResult> build( List<File> pomFiles, boolean recursive
ReactorModelPool.Builder poolBuilder = new ReactorModelPool.Builder();
final ReactorModelPool modelPool = poolBuilder.build();

if ( Features.buildConsumer().isActive() )
{
final TransformerContext context = new TransformerContext()
{
@Override
public String getUserProperty( String key )
{
return request.getUserProperties().getProperty( key );
}

@Override
public Model getRawModel( Path p )
{
return modelPool.get( p );
}

@Override
public Model getRawModel( String groupId, String artifactId )
{
return modelPool.get( groupId, artifactId, null );
}
};
request.getRepositorySession().getData().set( TransformerContext.KEY, context );
}

InternalConfig config = new InternalConfig( request, modelPool,
useGlobalModelCache() ? getModelCache() : new ReactorModelCache() );
InternalConfig config =
rfscholte marked this conversation as resolved.
Show resolved Hide resolved
new InternalConfig( request, modelPool, useGlobalModelCache() ? getModelCache() : new ReactorModelCache(),
modelBuilder.newTransformerContextBuilder() );

Map<String, MavenProject> projectIndex = new HashMap<>( 256 );
Map<File, MavenProject> projectIndex = new HashMap<>( 256 );

// phase 1: get file Models from the reactor.
boolean noErrors =
build( results, interimResults, projectIndex, pomFiles, new LinkedHashSet<>(), true, recursive,
config, poolBuilder );
Expand All @@ -424,6 +396,7 @@ public Model getRawModel( String groupId, String artifactId )

try
{
// Phase 2: get effective models from the reactor
noErrors =
build( results, new ArrayList<>(), projectIndex, interimResults, request,
new HashMap<>(), config.session ) && noErrors;
Expand All @@ -433,6 +406,12 @@ public Model getRawModel( String groupId, String artifactId )
Thread.currentThread().setContextClassLoader( oldContextClassLoader );
}

if ( Features.buildConsumer().isActive() )
{
request.getRepositorySession().getData().set( TransformerContext.KEY,
config.transformerContextBuilder.build() );
}

if ( !noErrors )
{
throw new ProjectBuildingException( results );
Expand All @@ -443,7 +422,7 @@ public Model getRawModel( String groupId, String artifactId )

@SuppressWarnings( "checkstyle:parameternumber" )
private boolean build( List<ProjectBuildingResult> results, List<InterimResult> interimResults,
Map<String, MavenProject> projectIndex, List<File> pomFiles, Set<File> aggregatorFiles,
Map<File, MavenProject> projectIndex, List<File> pomFiles, Set<File> aggregatorFiles,
boolean root, boolean recursive, InternalConfig config,
ReactorModelPool.Builder poolBuilder )
{
Expand All @@ -467,20 +446,19 @@ private boolean build( List<ProjectBuildingResult> results, List<InterimResult>

@SuppressWarnings( "checkstyle:parameternumber" )
private boolean build( List<ProjectBuildingResult> results, List<InterimResult> interimResults,
Map<String, MavenProject> projectIndex, File pomFile, Set<File> aggregatorFiles,
Map<File, MavenProject> projectIndex, File pomFile, Set<File> aggregatorFiles,
boolean isRoot, boolean recursive, InternalConfig config,
ReactorModelPool.Builder poolBuilder )
{
boolean noErrors = true;

ModelBuildingRequest request = getModelBuildingRequest( config );

MavenProject project = new MavenProject();
project.setFile( pomFile );

request.setPomFile( pomFile );
request.setTwoPhaseBuilding( true );
request.setLocationTracking( true );
ModelBuildingRequest request = getModelBuildingRequest( config )
.setPomFile( pomFile )
.setTwoPhaseBuilding( true )
.setLocationTracking( true );

DefaultModelBuildingListener listener =
new DefaultModelBuildingListener( project, projectBuildingHelper, config.request );
Expand All @@ -494,7 +472,7 @@ private boolean build( List<ProjectBuildingResult> results, List<InterimResult>
catch ( ModelBuildingException e )
{
result = e.getResult();
if ( result == null || result.getEffectiveModel() == null )
if ( result == null || result.getFileModel() == null )
{
results.add( new DefaultProjectBuildingResult( e.getModelId(), pomFile, e.getProblems() ) );

Expand All @@ -505,32 +483,17 @@ private boolean build( List<ProjectBuildingResult> results, List<InterimResult>
noErrors = false;
}

Model model = result.getEffectiveModel();
Model model = result.getFileModel().clone();

poolBuilder.put( model.getPomFile().toPath(), result.getRawModel() );
poolBuilder.put( model.getPomFile().toPath(), model );

try
{
// first pass: build without building parent.
initProject( project, projectIndex, false, result, new HashMap<>( 0 ), config.request );
}
catch ( InvalidArtifactRTException iarte )
{
result.getProblems().add( new DefaultModelProblem( null, ModelProblem.Severity.ERROR, null, model, -1, -1,
iarte ) );
}

projectIndex.put( result.getModelIds().get( 0 ), project );

InterimResult interimResult = new InterimResult( pomFile, request, result, listener, isRoot );
interimResults.add( interimResult );

if ( recursive && !model.getModules().isEmpty() )
if ( recursive )
{
File basedir = pomFile.getParentFile();

List<File> moduleFiles = new ArrayList<>();

for ( String module : model.getModules() )
{
if ( StringUtils.isEmpty( module ) )
Expand Down Expand Up @@ -608,6 +571,8 @@ private boolean build( List<ProjectBuildingResult> results, List<InterimResult>
noErrors = false;
}
}

projectIndex.put( pomFile, project );

return noErrors;
}
Expand Down Expand Up @@ -640,7 +605,7 @@ static class InterimResult
}

private boolean build( List<ProjectBuildingResult> results, List<MavenProject> projects,
Map<String, MavenProject> projectIndex, List<InterimResult> interimResults,
Map<File, MavenProject> projectIndex, List<InterimResult> interimResults,
ProjectBuildingRequest request, Map<File, Boolean> profilesXmls,
RepositorySystemSession session )
{
Expand Down Expand Up @@ -685,12 +650,14 @@ private boolean build( List<ProjectBuildingResult> results, List<MavenProject> p
catch ( ModelBuildingException e )
{
DefaultProjectBuildingResult result = null;
if ( project == null )
if ( project == null || interimResult.result.getEffectiveModel() == null )
{
result = new DefaultProjectBuildingResult( e.getModelId(), interimResult.pomFile, e.getProblems() );
}
else
{
project.setModel( interimResult.result.getEffectiveModel() );

result = new DefaultProjectBuildingResult( project, e.getProblems(), null );
}
results.add( result );
Expand All @@ -703,15 +670,14 @@ private boolean build( List<ProjectBuildingResult> results, List<MavenProject> p
}

@SuppressWarnings( "checkstyle:methodlength" )
private void initProject( MavenProject project, Map<String, MavenProject> projects,
private void initProject( MavenProject project, Map<File, MavenProject> projects,
boolean buildParentIfNotExisting, ModelBuildingResult result,
Map<File, Boolean> profilesXmls, ProjectBuildingRequest projectBuildingRequest )
{
Model model = result.getEffectiveModel();

project.setModel( model );
project.setOriginalModel( result.getRawModel() );
project.setFile( model.getPomFile() );
rfscholte marked this conversation as resolved.
Show resolved Hide resolved
project.setOriginalModel( result.getFileModel() );

initParent( project, projects, buildParentIfNotExisting, result, projectBuildingRequest );

Expand Down Expand Up @@ -938,7 +904,7 @@ HashMap<String, Artifact> compute()
}
}

private void initParent( MavenProject project, Map<String, MavenProject> projects, boolean buildParentIfNotExisting,
private void initParent( MavenProject project, Map<File, MavenProject> projects, boolean buildParentIfNotExisting,
ModelBuildingResult result, ProjectBuildingRequest projectBuildingRequest )
{
Model parentModel = result.getModelIds().size() > 1 && !result.getModelIds().get( 1 ).isEmpty()
Expand All @@ -957,7 +923,7 @@ private void initParent( MavenProject project, Map<String, MavenProject> project
// org.apache.maven.its.mng4834:parent:0.1
String parentModelId = result.getModelIds().get( 1 );
File parentPomFile = result.getRawModel( parentModelId ).getPomFile();
MavenProject parent = projects.get( parentModelId );
MavenProject parent = projects.get( parentPomFile );
if ( parent == null && buildParentIfNotExisting )
{
//
Expand Down Expand Up @@ -1095,16 +1061,22 @@ class InternalConfig
private final ReactorModelPool modelPool;

private final ReactorModelCache modelCache;

private final TransformerContextBuilder transformerContextBuilder;

InternalConfig( ProjectBuildingRequest request, ReactorModelPool modelPool, ReactorModelCache modelCache )
InternalConfig( ProjectBuildingRequest request, ReactorModelPool modelPool, ReactorModelCache modelCache,
TransformerContextBuilder transformerContextBuilder )
{
this.request = request;
this.modelPool = modelPool;
this.modelCache = modelCache;
this.transformerContextBuilder = transformerContextBuilder;

session =
LegacyLocalRepositoryManager.overlay( request.getLocalRepository(), request.getRepositorySession(),
repoSystem );
repositories = RepositoryUtils.toRepos( request.getRemoteRepositories() );

}

}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,7 @@ private static final class GavCacheKey
this.artifactId = ( artifactId != null ) ? artifactId : "";
this.version = ( version != null ) ? version : "";
this.tag = ( tag != null ) ? tag : "";

int hash = 17;
hash = hash * 31 + this.groupId.hashCode();
hash = hash * 31 + this.artifactId.hashCode();
hash = hash * 31 + this.version.hashCode();
hash = hash * 31 + this.tag.hashCode();
hashCode = hash;
this.hashCode = Objects.hash( groupId, artifactId, version, tag );
}

@Override
Expand All @@ -113,7 +107,6 @@ public int hashCode()
{
return hashCode;
}

}

private static final class SourceCacheKey
Expand Down
Loading