[WIP]webpack5 源码详解 - 封装模块 #18

Hazlank · 2022-05-01T12:58:04Z

封装模块

上一篇讲了关于webpack构建模块做的工作，接着讲讲封装模块做了些什么。

seal

webpack在make阶段完成过后，不再接收模块，会调用compilation.seal进行模块封装。

//compiler.js

this.hooks.make.callAsync(compilation, err => {
    //...
    logger.time("seal compilation");
    compilation.seal(err => {
	//...
    });
});


//Compilations.js

seal(callback) {
    //根据moduleGrape和hashFunction实例化ChunkGraph
    const chunkGraph = new ChunkGraph(
        this.moduleGraph,
        this.outputOptions.hashFunction
    );
    this.chunkGraph = chunkGraph;

    //将每个模块添加到chunkGraphForModuleMap
    if (this._backCompat) {
        for (const module of this.modules) {
            ChunkGraph.setChunkGraphForModule(module, chunkGraph);
        }
    }

    this.hooks.seal.call();

    //优化依赖
    this.logger.time("optimize dependencies");
    while (this.hooks.optimizeDependencies.call(this.modules)) {
            /* empty */
    }
    this.hooks.afterOptimizeDependencies.call(this.modules);
    this.logger.timeEnd("optimize dependencies");
    
}

首先会根据moduleGraph和options的配置实例化chunkGraph对象，再对应每个module，将其添加到chunkGraphForModuleMap里。

接着会调用optimizeDependencies hooks。webpack会根据mode来判断是否进行优化，比如当mode为production的时候，Optimization的配置就会生效

//defaults.js

//设置optimization相关配置
const applyOptimizationDefaults = (
    optimization,
    { production, development, css, records }
) => {
    //...
    D(optimization, "flagIncludedChunks", production);
    F(optimization, "moduleIds", () => {
        if (production) return "deterministic";
        if (development) return "named";
        return "natural";
    });
    F(optimization, "chunkIds", () => {
        if (production) return "deterministic";
        if (development) return "named";
        return "natural";
    });
    F(optimization, "sideEffects", () => (production ? true : "flag"));
    D(optimization, "providedExports", true);
    D(optimization, "usedExports", production);
    D(optimization, "innerGraph", production);
    D(optimization, "mangleExports", production);
    D(optimization, "concatenateModules", production);
    D(optimization, "runtimeChunk", false);
    D(optimization, "emitOnErrors", !production);
    D(optimization, "checkWasmTypes", production);
    D(optimization, "mangleWasmImports", false);
    D(optimization, "portableRecords", records);
    D(optimization, "realContentHash", production);
    D(optimization, "minimize", production);
    A(optimization, "minimizer", () => [
        {
            apply: compiler => {
                // Lazy load the Terser plugin
                const TerserPlugin = require("terser-webpack-plugin");
                new TerserPlugin({
                    terserOptions: {
                        compress: {
                            passes: 2
                        }
                    }
                }).apply(compiler);
            }
        }
    ]);
    //...
};

Terser插件也是在上面注册的，用于压缩代码，进行tree-shaking

回到compliation.seal函数, 调用optimizeDependencies hooks的位置。

optimizeDependencies hooks注册了两个插件，SideEffectsFlagPlugin和FlagDependencyUsagePlugin。第一个插件用于标记模块的副作用，第二个插件用于分析导出依赖是否有被使用，在之前遍历AST的时候已经收集了导出信息，所以之后只要分析当如果有导出的依赖被使用，会创建_usedInRuntime信息加入到moduleGraph对应的exportInfo

Create Entry Chunks

回到 seal 函数，接下来需要创建入口 chunk

seal () {
    //...
    this.logger.time("create chunks");
    this.hooks.beforeChunks.call();
    this.moduleGraph.freeze("seal");
    /** @type {Map<Entrypoint, Module[]>} */
    const chunkGraphInit = new Map();
    for (const [name, { dependencies, includeDependencies, options }] of this
        .entries) {
        
        //获取chunk对象
        const chunk = this.addChunk(name);
        
        //如果存在就赋值options.filename给chunk.filenameTemplate
        if (options.filename) {
            chunk.filenameTemplate = options.filename;
        }
        
        //根据options创建Entrypoint，entrypoint为chunkGroup对象
        const entrypoint = new Entrypoint(options);
        
        if (!options.dependOn && !options.runtime) {
        
            //在entrypoint._runtimeChunk设置chunk
            entrypoint.setRuntimeChunk(chunk);
        }
        //在entrypoint._entrypointChunk设置chunk
        entrypoint.setEntrypointChunk(chunk);
        
        //将entrypoint chunkGroup设置到namedChunkGroups Map
        this.namedChunkGroups.set(name, entrypoint);
        
        //将entrypoint chunkGroup设置到entrypoints Map
        this.entrypoints.set(name, entrypoint);
        
        //将entrypoint chunkGroup添加到chunkGroups集合
        this.chunkGroups.push(entrypoint);
        
        //创建chunk和ChunkGroup连接信息
        connectChunkGroupAndChunk(entrypoint, chunk);
    }
    //...

}


addChunk(name) {
    //name存在namedChunks则返回当前chunk
    if (name) {
        const chunk = this.namedChunks.get(name);
        if (chunk !== undefined) {
            return chunk;
        }
    }
    //新建chunk实例
    const chunk = new Chunk(name, this._backCompat);
    this.chunks.add(chunk);
    if (this._backCompat)
        //添加至ChunkGraphForChunk Map
        ChunkGraph.setChunkGraphForChunk(chunk, this.chunkGraph);
    if (name) {
        //添加至namedChunks Map
        this.namedChunks.set(name, chunk);
    }
    return chunk;
}

new Entrypoint(options)会新建 entrypoint 实例，它扩展于 chunkGroup 类，里面包含 chunks，origin 等属性。每一个入口就是单独的一个 chunkGroup 。之后会对 entrypoint 的属性进行设置并添加至 compliation.chunkGroups 集合里。

connectChunkGroupAndChunk 会将 ChunkGroup 和 chunk 进行连接，将 Entry chunk 加入 entrypoint 的 chunks 里。

compliation 实例里有许多集合和 Map 用来保存chunk的相关信息。

比如 namedChunkGroups ，它用于根据 chunk name 查询对应的 ChunkGroups, 然后将对应的 chunk 加入到 ChunkGroups 里。例如，入口文件如果没有设置 chunk name，默认的 name 就为 "main"，之后所有的同步依赖都会根据 "main" 找到入口块并将其打包在一起。异步的块也可以根据设置的 chunk name 选择分块或打包在一起。

比如 chunkGroups,它保存着所有 chunk 的集合，之后生成文件chunk的数量就会和 chunkGroups 的长度相同

seal() {
    for (const [name, { dependencies, includeDependencies, options }] of this
            .entries) {
        //...
        
        const entryModules = new Set();
        for (const dep of [...this.globalEntry.dependencies, ...dependencies]) {

            //添加Origin信息到entrypoint
            entrypoint.addOrigin(null, { name }, /** @type {any} */ (dep).request);

            //根据entry dep获取到entry module
            const module = this.moduleGraph.getModule(dep);
            if (module) {
                //创建chunk和entry module连接信息
                chunkGraph.connectChunkAndEntryModule(chunk, module, entrypoint);
                entryModules.add(module);

                //添加entrypoint至chunkGraphInit Map
                const modulesList = chunkGraphInit.get(entrypoint);
                if (modulesList === undefined) {
                    chunkGraphInit.set(entrypoint, [module]);
                } else {
                    modulesList.push(module);
                }
            }
        }

        //为每个模块设置depth
        this.assignDepths(entryModules);
    }
}

接下来会根据entries的dependencies进行循环，取出dep根据其获取module建立chunk和module的连接信息，并添加entrypoint和module至chunkGraphInit Map里

assignDepths会遍历入口模块所有的依赖，并对其设置依赖深度。比如A模块引用了B模块，B模块引用了C模块，那么A,B,C模块对应的depth就为0,1,2。

buildChunkGraph

buildChunkGraph是构建ChunkGraph的算法，它会建立module，chunk，chunkGroup之间的关系。

seal() {
   //...
   buildChunkGraph(this, chunkGraphInit);
   //...
}

const buildChunkGraph = (compilation, inputEntrypointsAndModules) => {
    const logger = compilation.getLogger("webpack.buildChunkGraph");
    
    // SHARED STATE
    /** @type {Map<AsyncDependenciesBlock, BlockChunkGroupConnection[]>} */
    const blockConnections = new Map();

    /** @type {Set<ChunkGroup>} */
    const allCreatedChunkGroups = new Set();

    /** @type {Map<ChunkGroup, ChunkGroupInfo>} */
    const chunkGroupInfoMap = new Map();

    /** @type {Set<DependenciesBlock>} */
    const blocksWithNestedBlocks = new Set();

    // PART ONE
    logger.time("visitModules");
    visitModules(
        logger,
        compilation,
        inputEntrypointsAndModules,
        chunkGroupInfoMap,
        blockConnections,
        blocksWithNestedBlocks,
        allCreatedChunkGroups
    );
    logger.timeEnd("visitModules");

    // PART TWO
    logger.time("connectChunkGroups");
    connectChunkGroups(
        compilation,
        blocksWithNestedBlocks,
        blockConnections,
        chunkGroupInfoMap
    );
    logger.timeEnd("connectChunkGroups");

    for (const [chunkGroup, chunkGroupInfo] of chunkGroupInfoMap) {
        for (const chunk of chunkGroup.chunks)
            chunk.runtime = mergeRuntime(chunk.runtime, chunkGroupInfo.runtime);
    }

    // Cleanup work
    logger.time("cleanup");
    cleanupUnconnectedGroups(compilation, allCreatedChunkGroups);
    logger.timeEnd("cleanup");
};

buildChunkGraph分为三个步骤

visitModules
connectChunkGroups
cleanupUnconnectedGroups

visitModules

visitModules 是 chunkGraph 算法的第一步，它会遍历每个Module、Dependency 和 AsyncDepBlock，并将 Modules 和 AsyncDepBlocks 与 Chunks 连接起来

当访问 module 时: 将 module 添加到当前 Chunk ，然后访问所有 Dependencies 和 AsyncDepBlocks

访问 Dependency 时: 将会访问引用的 module

访问 AsyncDepBlock 时：创建一个新的 Chunk，或者根据该 Chunk 设置的名字去寻找是否已经创建过此块，没有就创建新的 Chunk，有则将引用该 Chunk 的模块添加至 Chunk 的 origin 里。然后访问所有 Dependencies 和 AsyncDepBlocks

visitModules 通过使用队列迭代地工作。以前递归实现的版本会导致堆栈溢出，因此对其进行了重构

webpack 作者的对于 Chunk Graph 算法的写了一篇文章，大致解释关于该算法是怎么工作

const visitModules = (
    logger,
    compilation,
    inputEntrypointsAndModules,
    chunkGroupInfoMap,
    blockConnections,
    blocksWithNestedBlocks,
    allCreatedChunkGroups
) => {

    /** @type {QueueItem[]} */
    let queue = [];

    /** @type {Map<ChunkGroupInfo, Set<ChunkGroupInfo>>} */
    const queueConnect = new Map();
    /** @type {Set<ChunkGroupInfo>} */
    const chunkGroupsForCombining = new Set();

    // Fill queue with entrypoint modules
    // Create ChunkGroupInfo for entrypoints
    
    //拿出之前创建的entryPoint chunkGroup
    for (const [chunkGroup, modules] of inputEntrypointsAndModules) {
        
        //根据传参获取runtime 或 name
        const runtime = getEntryRuntime(
            compilation,
            chunkGroup.name,
            chunkGroup.options
        );
        /** @type {ChunkGroupInfo} */
        //创建chunkGroupInfo信息
        
        const chunkGroupInfo = {
            //块组
            chunkGroup,
            runtime,
            //chunkGroup 可追踪的最小 module 数据集
            minAvailableModules: undefined,
            minAvailableModulesOwned: false,
            availableModulesToBeMerged: [],
            //可跳过模块
            skippedItems: undefined,
            children: undefined,
            availableSources: undefined,
            availableChildren: undefined,
            preOrderIndex: 0,
            postOrderIndex: 0,
            chunkLoading:
                chunkGroup.options.chunkLoading !== undefined
                    ? chunkGroup.options.chunkLoading !== false
                    : compilation.outputOptions.chunkLoading !== false,
            asyncChunks:
                chunkGroup.options.asyncChunks !== undefined
                    ? chunkGroup.options.asyncChunks
                    : compilation.outputOptions.asyncChunks !== false
        };
        chunkGroup.index = nextChunkGroupIndex++;
        
        //如果当前chunk依赖父级Chunk时
        if (chunkGroup.getNumberOfParents() > 0) {
            // minAvailableModules for child entrypoints are unknown yet, set to undefined.
            // This means no module is added until other sets are merged into
            // this minAvailableModules (by the parent entrypoints)
            const skippedItems = new Set();
            for (const module of modules) {
                skippedItems.add(module);
            }
            //将 skippedItems 设置到当前 chunkGroupInfo.skippedItems
            chunkGroupInfo.skippedItems = skippedItems;
            //添加至 chunkGroupsForCombining
            chunkGroupsForCombining.add(chunkGroupInfo);
        } else {
            // The application may start here: We start with an empty list of available modules
            chunkGroupInfo.minAvailableModules = EMPTY_SET;
            
            //获取entry chunk
            const chunk = chunkGroup.getEntrypointChunk();
            
            //将模块推入队列
            for (const module of modules) {
                queue.push({
                    // ADD_AND_ENTER_MODULE = 1
                    action: ADD_AND_ENTER_MODULE,
                    block: module,
                    module,
                    chunk,
                    chunkGroup,
                    chunkGroupInfo
                });
            }
        }
        //设置chunkGroup对应的chunkGroupInfo信息
        chunkGroupInfoMap.set(chunkGroup, chunkGroupInfo);
        if (chunkGroup.name) {
            //设置chunkGroup name对应的chunkGroupInfo
            namedChunkGroups.set(chunkGroup.name, chunkGroupInfo);
        }
    }
    // Fill availableSources with parent-child dependencies between entrypoints
    //取出 chunkGroupsForCombining 里的每个 chunkGroupInfo
    for (const chunkGroupInfo of chunkGroupsForCombining) {
        const { chunkGroup } = chunkGroupInfo;
        chunkGroupInfo.availableSources = new Set();
        for (const parent of chunkGroup.parentsIterable) {
            //获取父 chunk 的 ChunkGroupInfo 
            const parentChunkGroupInfo = chunkGroupInfoMap.get(parent);
            //将父 chunkGroupInfo 添加到子 chunkGroupInfo.availableSources 集合
            chunkGroupInfo.availableSources.add(parentChunkGroupInfo);
            if (parentChunkGroupInfo.availableChildren === undefined) {
                parentChunkGroupInfo.availableChildren = new Set();
            }
            //将子 chunkGroupInfo 添加到父 chunkGroupInfo.availableChildren 集合
            parentChunkGroupInfo.availableChildren.add(chunkGroupInfo);
        }
    }
    // pop() is used to read from the queue
    // so it need to be reversed to be iterated in
    // correct order
    
    //将队列反转保证输出顺序正确
    queue.reverse();
}

首先 visitModules 会对之前创建的 entrypoint chunkGraph，迭代取出 chunkGroup 和 modules ，创建对应的 chunkGroupInfo 保存至 chunkGroupInfoMap 和 namedChunkGroups 里。

当 webpack 配置如下

module.exports = {
    entry: {
        home: "./index.js",
        family: {
            import: "./family.js",
            dependOn: "home"
        }
    }
};

family entry 依赖于 home entry 时，family chunkGroup 的 parent 就为 home entry

此时chunkGroup.getNumberOfParents() > 0，将会走相对应的逻辑，并添加至 chunkGroupsForCombining 以后续处理

最后再将 chunkGroupInfo，chunk，module 等信息添加到队列里处理，因为pop取的是最后的任务，所以为了保证顺序会 reverse，难道 webpack 作者不知道 unshift 吗? = =。

const visitModules = (
    logger,
    compilation,
    inputEntrypointsAndModules,
    chunkGroupInfoMap,
    blockConnections,
    blocksWithNestedBlocks,
    allCreatedChunkGroups
) => {
    //...
    
    while (queue.length || queueConnect.size) {
        processQueue();
        //...
    }

}

之后就是迭代队列，对之前添加至队列的任务进行逐步处理。

processQueue
processChunkGroupsForCombining
processChunkGroupsForMerging
processOutdatedChunkGroupInfo

const processQueue = () => {
    while (queue.length) {
        statProcessedQueueItems++;
        const queueItem = queue.pop();
        module = queueItem.module;
        block = queueItem.block;
        chunk = queueItem.chunk;
        chunkGroup = queueItem.chunkGroup;
        chunkGroupInfo = queueItem.chunkGroupInfo;

        switch (queueItem.action) {
            case ADD_AND_ENTER_ENTRY_MODULE:
                  ...//
            // fallthrough
            case ADD_AND_ENTER_MODULE: {
                //如果chunk已经存在该module就跳过连接
                if (chunkGraph.isModuleInChunk(module, chunk)) {
                    // already connected, skip it
                    break;
                }
                // We connect Module and Chunk
                //建立Module and Chunk的连接
                chunkGraph.connectChunkAndModule(chunk, module);
            }
            // fallthrough
            case ENTER_MODULE: {
                ...//
            case PROCESS_BLOCK: {
                ...//
            case PROCESS_ENTRY_BLOCK: {
                ...//
            case LEAVE_MODULE: {
                ...//
            }
        }
    }
};

processQueue会根据action进行不同的处理，在之前创建chunkGroupInfo的时候，初始action的值为ADD_AND_ENTER_MODULE

ADD_AND_ENTER_MODULE Case 会判断该模块是否存在过 Chunk 里，是的话说明 connection 过了，可以退出 switch。否则就调用 connectChunkAndModule(chunk, module)

connectChunkAndModule 会新建 ChunkGraphModule 和 ChunkGraphChunk ，并在里面添加传入的 Entry chunk 和 Entry module。

接着会走到 case ENTER_MODULE

case ENTER_MODULE : {
    //获取preOrderIndex
    const index = chunkGroup.getModulePreOrderIndex(module);
    //如果chunkGroup PreOrderIndex 为 undefined,从0开始设置
    if (index === undefined) {
        chunkGroup.setModulePreOrderIndex(
            module,
            chunkGroupInfo.preOrderIndex++
        );
    }
    
    //如果moduleGraph PreOrderIndex 为 undefined,从0开始设置
    if (
        moduleGraph.setPreOrderIndexIfUnset(
            module,
            nextFreeModulePreOrderIndex
        )
    ) {
        nextFreeModulePreOrderIndex++;
    }

    // reuse queueItem
    //将该queueItem的action设置为LEAVE_MODULE并推到队列里重用
    queueItem.action = LEAVE_MODULE;
    queue.push(queueItem);
}

首先获取 chunkGroup 的 PreOrderIndex 索引值，没有则从0开始，该索引是自上而下的，从最上层的module开始

moduleGraph 的 PreOrderIndex 也和上面的处理一样

之后将该任务的action设置为LEAVE_MODULE并推到队列里重用

最后会走到 case PROCESS_BLOCK 并调用 processBlock 访问依赖

case PROCESS_BLOCK: {
    processBlock(block);
    break;
}

const processBlock = block => {
    statProcessedBlocks++;
    // get prepared block info
    
    //获取同步 依赖
    const blockModules = getBlockModules(block, chunkGroupInfo.runtime);
    
    if (blockModules !== undefined) {
        const { minAvailableModules } = chunkGroupInfo;
        // Buffer items because order need to be reversed to get indices correct
        // Traverse all referenced modules
        for (let i = 0; i < blockModules.length; i += 2) {
            const refModule = /** @type {Module} */ (blockModules[i]);
            
            //如果引用的依赖已经存在于chunk里,跳过处理
            if (chunkGraph.isModuleInChunk(refModule, chunk)) {
                // skip early if already connected
                continue;
            }
            const activeState = /** @type {ConnectionState} */ (
                blockModules[i + 1]
            );
            if (activeState !== true) {
                skipConnectionBuffer.push([refModule, activeState]);
                if (activeState === false) continue;
            }
            //如果引用的依赖已经存在于父 chunk 里, 添加到 skipBuffer
            if (
                activeState === true &&
                (minAvailableModules.has(refModule) ||
                        minAvailableModules.plus.has(refModule))
            ) {
                // already in parent chunks, skip it for now
                skipBuffer.push(refModule);
                continue;
            }
            // enqueue, then add and enter to be in the correct order
            // this is relevant with circular dependencies
            
            //将依赖推入queueBuffer
            queueBuffer.push({
                action: activeState === true ? ADD_AND_ENTER_MODULE : PROCESS_BLOCK,
                block: refModule,
                module: refModule,
                chunk,
                chunkGroup,
                chunkGroupInfo
            });
        }
        // Add buffered items in reverse order
        if (skipConnectionBuffer.length > 0) {
            let { skippedModuleConnections } = chunkGroupInfo;
            if (skippedModuleConnections === undefined) {
                chunkGroupInfo.skippedModuleConnections = skippedModuleConnections =
                        new Set();
            }
            for (let i = skipConnectionBuffer.length - 1; i >= 0; i--) {
                skippedModuleConnections.add(skipConnectionBuffer[i]);
            }
            skipConnectionBuffer.length = 0;
        }
        
        //将所有的 skippedItems 添加到 ChunkGroupInfo.skippedItems
      
        if (skipBuffer.length > 0) {
            let { skippedItems } = chunkGroupInfo;
            if (skippedItems === undefined) {
                chunkGroupInfo.skippedItems = skippedItems = new Set();
            }
            for (let i = skipBuffer.length - 1; i >= 0; i--) {
                skippedItems.add(skipBuffer[i]);
            }
            skipBuffer.length = 0;
        }
        
        //将queueBuffer里的同步块推入队列
        if (queueBuffer.length > 0) {
            for (let i = queueBuffer.length - 1; i >= 0; i--) {
                queue.push(queueBuffer[i]);
            }
            queueBuffer.length = 0;
        }
    }

    // Traverse all Blocks
    //迭代异步依赖
    for (const b of block.blocks) {
        iteratorBlock(b);
    }
    
    //如果异步依赖里还引用了其他异步依赖, 添加到 blocksWithNestedBlocks 集合
    if (block.blocks.length > 0 && module !== block) {
        blocksWithNestedBlocks.add(block);
    }
};

processBlock会获取同步的依赖，推入到 queue 下次迭代处理。

(minAvailableModules.has(refModule) || minAvailableModules.plus.has(refModule))判断表示当前依赖已经存在父 chunk 里，所以可以将其添加到 skippedItems。

例如在入口文件, index 引用了模块 d 和异步模块 a ，且 a 模块也引用了 d ，所以 a 会把 d 添加到 ChunkGroupInfo.skippedItems 来跳过打包，直接使用父模块 index 里打包的 d，从而避免重复打包

之后就是调用 iteratorBlock 迭代处理异步依赖

const iteratorBlock = b => {
    // 1. We create a chunk group with single chunk in it for this Block
    // but only once (blockChunkGroups map)
    let cgi = blockChunkGroups.get(b);
    /** @type {ChunkGroup} */
    let c;
    /** @type {Entrypoint} */
    let entrypoint;
    const entryOptions = b.groupOptions && b.groupOptions.entryOptions;
    if (cgi === undefined) {
        const chunkName = (b.groupOptions && b.groupOptions.name) || b.chunkName;
        if (entryOptions) {
           //...
        } else if (!chunkGroupInfo.asyncChunks || !chunkGroupInfo.chunkLoading) {
          // Just queue the block into the current chunk group
          //...
        } else {
            //通过chunkName获取chunkGroupInfo
            cgi = chunkName && namedChunkGroups.get(chunkName);
            
            //如果没有chunkGroupInfo就创建新的chunk
            if (!cgi) {
                //根据异步块添加新的chunk到chunkGroup
                c = compilation.addChunkInGroup(
                    b.groupOptions || b.chunkName,
                    module,
                    b.loc,
                    b.request
                );
                c.index = nextChunkGroupIndex++;
                //创建chunkGroup信息
                cgi = {
                    chunkGroup: c,
                    runtime: chunkGroupInfo.runtime,
                    minAvailableModules: undefined,
                    minAvailableModulesOwned: undefined,
                    availableModulesToBeMerged: [],
                    skippedItems: undefined,
                    resultingAvailableModules: undefined,
                    children: undefined,
                    availableSources: undefined,
                    availableChildren: undefined,
                    preOrderIndex: 0,
                    postOrderIndex: 0,
                    chunkLoading: chunkGroupInfo.chunkLoading,
                    asyncChunks: chunkGroupInfo.asyncChunks
                };
                //所有新建ChunkGroups的集合
                allCreatedChunkGroups.add(c);
                
                //进行相关Map的设置
                chunkGroupInfoMap.set(c, cgi);
                if (chunkName) {
                    namedChunkGroups.set(chunkName, cgi);
                }
            } else {
                c = cgi.chunkGroup;
               
                //isInitial用于判断当前chunk group是否在初始化页面被加载
                if (c.isInitial()) {
                    compilation.errors.push(
                        new AsyncDependencyToInitialChunkError(chunkName, module, b.loc)
                    );
                    c = chunkGroup;
                }
                //添加b的groupOptions
                c.addOptions(b.groupOptions);
                //在chunkGroup Origin添加异步模块
                c.addOrigin(module, b.loc, b.request);
            }
            //添加异步块链接
            blockConnections.set(b, []);
        }
        //将异步块和chunkGroupInfo添加至Map
        blockChunkGroups.set(b, cgi);
    } else if (entryOptions) {
        entrypoint = /** @type {Entrypoint} */ (cgi.chunkGroup);
    } else {
        c = cgi.chunkGroup;
    }

    if (c !== undefined) {
        // 2. We store the connection for the block
        // to connect it later if needed
        
        //保存异步块的来源chunkGroupInfo和chunkGroup
        blockConnections.get(b).push({
            originChunkGroupInfo: chunkGroupInfo,
            chunkGroup: c
        });

        // 3. We enqueue the chunk group info creation/updatin
        
        //将异步块对应的chunkGroupInfo加入到connectList
        let connectList = queueConnect.get(chunkGroupInfo);
        if (connectList === undefined) {
            connectList = new Set();
            queueConnect.set(chunkGroupInfo, connectList);
        }
        connectList.add(cgi);

        // TODO check if this really need to be done for each traversal
        // or if it is enough when it's queued when created
        
        // 4. We enqueue the DependenciesBlock for traversa
        //将异步依赖推到queueDelayed队列推迟处理
        queueDelayed.push({
            action: PROCESS_BLOCK,
            block: b,
            module: module,
            chunk: c.chunks[0],
            chunkGroup: c,
            chunkGroupInfo: cgi
        });
    } else if (entrypoint !== undefined) {
        chunkGroupInfo.chunkGroup.addAsyncEntrypoint(entrypoint);
    }
};

iteratorBlock 会迭代所有的异步依赖，它会根据 chunkName 从 namedChunkGroups Map 获取对应的 chunkGroupInfo 信息。如果获取不到，就会调用 addChunkInGroup 新建 chunkGroup。

Dynamic Import 可以通过 Magic comments 的方式设置对应的 chunk name，所以我们可以自行设置，选择输出便于阅读的 chunk name 或者将不同的异步依赖打包在一起。

addChunkInGroup(groupOptions, module, loc, request) {
    if (typeof groupOptions === "string") {
        groupOptions = { name: groupOptions };
    }
    const name = groupOptions.name;
   
    //如果设置了chunk name，就将该chunk加入到当前的chunkGroup
    if (name) {
        const chunkGroup = this.namedChunkGroups.get(name);
        if (chunkGroup !== undefined) {
            chunkGroup.addOptions(groupOptions);
            if (module) {
                chunkGroup.addOrigin(module, loc, request);
            }
            return chunkGroup;
        }
    }
    
    //新建ChunkGroup
    const chunkGroup = new ChunkGroup(groupOptions);
    
    //对chunkGroup进行相关设置
    if (module) chunkGroup.addOrigin(module, loc, request);
    const chunk = this.addChunk(name);

    //连接 chunkGroup 和 chunk
    connectChunkGroupAndChunk(chunkGroup, chunk);

    //将chunkGroup推入到数组
    this.chunkGroups.push(chunkGroup);
    
    //如果设置了相关chunk name就添加到 namedChunkGroups 以便后续的其他依赖添加到相同的块
    if (name) {
        this.namedChunkGroups.set(name, chunkGroup);
    }
    return chunkGroup;
}

addChunkInGroup 会根据异步依赖的 chunkName 从 namedChunkGroups 获取对应 chunkGroup，如果能找到就添加并返回该 chunkGroup，否则就创建新的 chunkGroup，并对其进行相关设置

到这里 processBlock 的事情就做完了，将会 break 开始下一次循环，因为同步依赖也被添加到 queue ，所以也会取出来从 ADD_AND_ENTER_MODULE 或 PROCESS_BLOCK 步骤开始做跟前面相同的事情，直到它们的 action 为 LEAVE_MODULE

case LEAVE_MODULE: {
    const index = chunkGroup.getModulePostOrderIndex(module);
    if (index === undefined) {
        chunkGroup.setModulePostOrderIndex(
            module,
            chunkGroupInfo.postOrderIndex++
        );
    }

    if (
        moduleGraph.setPostOrderIndexIfUnset(
            module,
            nextFreeModulePostOrderIndex
        )
    ) {
        nextFreeModulePostOrderIndex++;
    }
    break;
}

LEAVE_MODULE 用于设置 chunkGroup 和 moduleGraph 对应 module 的 PostOrderIndex

当迭代完所有的同步模块后将会回收 processQueue 栈，进行下一步的处理

const visitModules = (
    //...
) => {
    //...
    while (queue.length || queueConnect.size) {
        //...
        if (chunkGroupsForCombining.size > 0) {
            logger.time("visitModules: combine available modules");
            processChunkGroupsForCombining();
            logger.timeEnd("visitModules: combine available modules");
        }
       //...
    }
    
    const processChunkGroupsForCombining = () => {
        for (const info of chunkGroupsForCombining) {
            for (const source of info.availableSources) {
                //如果依赖的父块没有 minAvailableModules, 则删除此 ChunkGroupInfo
                if (!source.minAvailableModules) {
                    chunkGroupsForCombining.delete(info);
                    break;
                }
            }
        }
        for (const info of chunkGroupsForCombining) {
            const availableModules = /** @type {ModuleSetPlus} */ (new Set());
            availableModules.plus = EMPTY_SET;
            const mergeSet = set => {
                if (set.size > availableModules.plus.size) {
                    for (const item of availableModules.plus) availableModules.add(item);
                        availableModules.plus = set;
                } else {
                    for (const item of set) availableModules.add(item);
                }
            };
            // combine minAvailableModules from all resultingAvailableModules
            for (const source of info.availableSources) {
                const resultingAvailableModules =
                    calculateResultingAvailableModules(source);
                //将父 chunk 的可用模块 merge 到子 chunk availableModules
                mergeSet(resultingAvailableModules);
                mergeSet(resultingAvailableModules.plus);
            }
            info.minAvailableModules = availableModules;
            info.minAvailableModulesOwned = false;
            info.resultingAvailableModules = undefined;
            outdatedChunkGroupInfo.add(info);
        }
        chunkGroupsForCombining.clear();
    };

}

processChunkGroupsForCombining 做的事情就是处理 chunkGroupsForCombining，chunkGroupsForCombining 只有在前面处理 Entry ChunkGroup 时有 dependOn 才会有相对应的集合。

processChunkGroupsForCombining 会计算父 chunk 的可用模块并 merge 到子 chunk availableModules。后面会介绍可用模块是什么。

const visitModules = (
    //...
) => {
    //...
    while (queue.length || queueConnect.size) {
        //...
        if (queueConnect.size > 0) {
            logger.time("visitModules: calculating available modules");
            processConnectQueue();
            logger.timeEnd("visitModules: calculating available modules");
        }
       //...
    }

}

处理完 chunkGroupsForCombining ，接着会调用 processConnectQueue 处理异步模块

const processConnectQueue = () => {
    // Figure out new parents for chunk groups
    // to get new available modules for these children
    
    //将异步模块设置到父chunkGroupInfo的子集
    for (const [chunkGroupInfo, targets] of queueConnect) {
        // 1. Add new targets to the list of children
        if (chunkGroupInfo.children === undefined) {
            chunkGroupInfo.children = targets;
        } else {
            for (const target of targets) {
                chunkGroupInfo.children.add(target);
            }
        }

        // 2. Calculate resulting available modules
        
        //计算当前chunkGroupInfo可用模块
        const resultingAvailableModules =
            calculateResultingAvailableModules(chunkGroupInfo);
            
        //...
    }
};

首先会将之前添加的 queueConnect 取出来, chunkGroupInfo.children 表示当前块的子块 ,首次执行的时候这个 chunkGroupInfo 是 Entry GroupInfo , 在入口文件，我们引用了一个异步依赖 a ，所以 Entry GroupInfo 的子块就为 a ，如果有多个异步依赖，将会变成一个集合并将所有异步依赖添加进去

接下来会调用 resultingAvailableModules 计算当前 chunkGroup 的可用模块数量

const calculateResultingAvailableModules = chunkGroupInfo => {
    if (chunkGroupInfo.resultingAvailableModules)
        return chunkGroupInfo.resultingAvailableModules;
    
    const minAvailableModules = chunkGroupInfo.minAvailableModules;

    // Create a new Set of available modules at this point
    // We want to be as lazy as possible. There are multiple ways doing this:
    // Note that resultingAvailableModules is stored as "(a) + (b)" as it's a ModuleSetPlus
    // - resultingAvailableModules = (modules of chunk) + (minAvailableModules + minAvailableModules.plus)
    // - resultingAvailableModules = (minAvailableModules + modules of chunk) + (minAvailableModules.plus)
    // We choose one depending on the size of minAvailableModules vs minAvailableModules.plus
    
    //根据 minAvailableModules 大小决定 resultingAvailableModules 的计算方式
    let resultingAvailableModules;
    if (minAvailableModules.size > minAvailableModules.plus.size) {
        // resultingAvailableModules = (modules of chunk) + (minAvailableModules + minAvailableModules.plus)
        
        //resultingAvailableModules = 空集合
        resultingAvailableModules = (new Set());
        
        //将所有minAvailableModules.plus的集合添加到minAvailableModules
        for (const module of minAvailableModules.plus)
            minAvailableModules.add(module);
            
        //置空minAvailableModules.plus
        minAvailableModules.plus = EMPTY_SET;
        
        resultingAvailableModules.plus = minAvailableModules;
        chunkGroupInfo.minAvailableModulesOwned = false;
    } else {
        // resultingAvailableModules = (minAvailableModules + modules of chunk) + (minAvailableModules.plus)
        
        //resultingAvailableModules = new Set(minAvailableModules)
        resultingAvailableModules = (
            new Set(minAvailableModules)
        );
        resultingAvailableModules.plus = minAvailableModules.plus;
    }

    // add the modules from the chunk group to the set
    
    //将当前chunk group的所有模块添加到集合里
    for (const chunk of chunkGroupInfo.chunkGroup.chunks) {
        for (const m of chunkGraph.getChunkModulesIterable(chunk)) {
            resultingAvailableModules.add(m);
        }
    }
    return (chunkGroupInfo.resultingAvailableModules =
        resultingAvailableModules);
};

当前 chunkGroup 可用模块个根据 minAvailableModules 和 minAvailableModules.plus 的大小有两种算法

minAvailableModules 为最小可用模块，是不包含模块本身

resultingAvailableModules 为计算可用模块，它等于 (minAvailableModules + modules of chunk) + (minAvailableModules.plus) 或 (modules of chunk) + (minAvailableModules + minAvailableModules.plus)

当 minAvailableModules 大的时候，将 minAvailableModules.plus 集合添加至 minAvailableModules 并清空，此时 resultingAvailableModules = new set()

当 minAvailableModules 小的时候，此时 resultingAvailableModules = newSet(minAvailableModules) ，将 resultingAvailableModules.plus = minAvailableModules.plus

最后再通过查找 chunkGraph 的将所有同步模块添加至 resultingAvailableModules

比如 index.js 里，一共四个依赖，三个同步依赖，一个异步的依赖。

在 Entry chunk 里, 它为最上层的依赖, 所以 minAvailableModules 为 0 。此时resultingAvailableModules = (minAvailableModules + modules of chunk) + (minAvailableModules.plus)，resultingAvailableModules 会把 minAvailableModules 加入到集合，并把所有同步块添加进来，所以计算出来的可用模块一共为四个[index, b, c, d] , 因为异步块 a 会分离出来所以并不会包含在里面

在 a chunk 里, minAvailableModules 为之前的父块里的 [index, b, c, d] ，此时 resultingAvailableModules = (modules of chunk) + (minAvailableModules + minAvailableModules.plus)。它会把 minAvailableModules 添加到 resultingAvailableModules.plus 再把其他同步块添加到 resultingAvailableModules。所以一共有五个可用模块 resultingAvailableModules.plus: [index, b, c ,d] + resultingAvailableModules: [a]

因为 entry chunk 和 a chunk 是父子的关系，所以 a 能够用父 chunk 的依赖，这就是为什么 a chunk 里的可用模块会包含 entry chunk 里的其他依赖

[
  0: {
    key: "main",
    value: {
      availableModulesToBeMerged: Array(0) 
      children: Set(1) {} // a
      chunkGroup: Entrypoint
      minAvailableModules: Set(0) {plus: Set(0), size: 0} 
      minAvailableModulesOwned: true
      //resultingAvailableModules = (minAvailableModules + modules of chunk) + (minAvailableModules.plus)
      resultingAvailableModules: Set(4) {plus: Set(0), size: 4} //[plus: [], index, b, c, d]
      skippedItems: Array(0)
    }
  },
  1: {
    key: "a"
    value: {
      availableModulesToBeMerged: Array(0)
      children: Set(1) {} // d
      chunkGroup: a
      minAvailableModules: {plus: Set(0), size: 4} // [plus: [], index, b ,c ,d]
      minAvailableModulesOwned: true
      //resultingAvailableModules = (modules of chunk) + (minAvailableModules + minAvailableModules.plus)
      resultingAvailableModules: Set(1) {plus: Set(4), size: 1}// [plus: [index, b, c, d], a]
      skippedItems: Array(0)
    },
  2: {
        key: "d"
    value: {
      availableModulesToBeMerged: Array(0)
      children: Set(0)
      chunkGroup: d
      minAvailableModules: {plus: Set(4), size: 1} // [plus: [index, b, c ,d], a]
      minAvailableModulesOwned: true
      //只有当前模块有异步依赖才会计算 resultingAvailableModules，因为 chunk d 没有其他依赖，所以为 undefined
      resultingAvailableModules: undefined
      skippedItems: Array(1) // d
    }
  }

计算完当前 chunkGroup 的可用模块后回到 processConnectQueue

const processConnectQueue = () => {
    const runtime = chunkGroupInfo.runtime;

        // 3. Update chunk group info
        //更新每个chunk group info
        for (const target of targets) {
            //将 resultingAvailableModules push 到 ChunkGroupInfo.availableModulesToBeMerged
            target.availableModulesToBeMerged.push(resultingAvailableModules);
            //将 ChunkGroupInfo 添加到 chunkGroupsForMerging
            chunkGroupsForMerging.add(target);
            const oldRuntime = target.runtime;
            const newRuntime = mergeRuntime(oldRuntime, runtime);
            if (oldRuntime !== newRuntime) {
                target.runtime = newRuntime;
                outdatedChunkGroupInfo.add(target);
            }
        }

        statConnectedChunkGroups += targets.size;
    }
    queueConnect.clear();
}

processConnectQueue 的后半段会将 resultingAvailableModules 添加到 ConnectQueue 里每个异步 ChunkGroupInfo 的 availableModulesToBeMerged 下，并添加到 processChunkGroupsForMerging 进行处理。

processChunkGroupsForMerging 做的事情，就是合并 availableModulesToBeMerged 里的集合,得到当前 chunkGroupInfo 的 minavailableModules

假设入口 index 有 c, b 两个同步依赖, c, b 都引用了异步依赖 a，a 引用了异步依赖 d，c 模块还单独引用了同步模块 e

此时 chunkGroupInfo d 的 availableModulesToBeMerged 拥有两个集合,[plus: [index, c, b], d] 和 [plus: [index, c, b, e], d]

最后 minavailableModules 会得到合并后的结果为 [plus: [index, c, b, e], d]

connectChunkGroups

visitModules 之后的第二个步骤就是 connectChunkGroups, connectChunkGroups 会建立异步依赖与它们的父模块的关系

const connectChunkGroups = (
    compilation,
    blocksWithNestedBlocks,
    blockConnections,
    chunkGroupInfoMap
) => {
    const { chunkGraph } = compilation;

    /**
     * Helper function to check if all modules of a chunk are available
     *
     * @param {ChunkGroup} chunkGroup the chunkGroup to scan
     * @param {ModuleSetPlus} availableModules the comparator set
     * @returns {boolean} return true if all modules of a chunk are available
     */
    const areModulesAvailable = (chunkGroup, availableModules) => {
        for (const chunk of chunkGroup.chunks) {
            for (const module of chunkGraph.getChunkModulesIterable(chunk)) {
                if (!availableModules.has(module) && !availableModules.plus.has(module))
                    return false;
            }
        }
        return true;
    };

    // For each edge in the basic chunk graph
    for (const [block, connections] of blockConnections) {
        // 1. Check if connection is needed
        // When none of the dependencies need to be connected
        // we can skip all of them
        // It's not possible to filter each item so it doesn't create inconsistent
        // connections and modules can only create one version
        // TODO maybe decide this per runtime
        if (
            // TODO is this needed?
            !blocksWithNestedBlocks.has(block) &&
            connections.every(({ chunkGroup, originChunkGroupInfo }) =>
                areModulesAvailable(
                    chunkGroup,
                    originChunkGroupInfo.resultingAvailableModules
                )
            )
   
   ) {
            continue;
        }

        // 2. Foreach edge
        for (let i = 0; i < connections.length; i++) {
            const { chunkGroup, originChunkGroupInfo } = connections[i];

            // 3. Connect block with chunk
            chunkGraph.connectBlockAndChunkGroup(block, chunkGroup);

            // 4. Connect chunk with parent
            connectChunkGroupParentAndChild(
                originChunkGroupInfo.chunkGroup,
                chunkGroup
            );
        }
    }
};

connectChunkGroups 会获取所有的异步依赖，然后建立与 originChunkGroupInfo 的父子关系，originChunkGroupInfo 就是异步依赖的父 chunk

如果父 chunk 里已经包含了该模块，就跳过关系建立

例如, 入口模块 index 引用了同步依赖 a 和异步依赖 b, b 引用了异步依赖 a。因为入口已经存在依赖 a 了，所以 b 里的异步依赖 a 不需要再创建了, 所以跳过关系建立

cleanupUnconnectedGroups

最后一步是调用 cleanupUnconnectedGroups，它会清理没有建立关系的 chunk，比如上面的例子，因为跳过了 connectChunkGroupParentAndChild，所以异步依赖 a 的 getNumberOfParents() 为 0，会移除当前的 chunk，不额外生成单独的 chunk

const cleanupUnconnectedGroups = (compilation, allCreatedChunkGroups) => {
    const { chunkGraph } = compilation;

    for (const chunkGroup of allCreatedChunkGroups) {
        if (chunkGroup.getNumberOfParents() === 0) {
            for (const chunk of chunkGroup.chunks) {
                //从
                compilation.chunks.delete(chunk);
                chunkGraph.disconnectChunk(chunk);
            }
            chunkGraph.disconnectChunkGroup(chunkGroup);
            chunkGroup.remove();
        }
    }
};

seal

构建完 chunkGroup 后, 之后会调用 optimizeChunks hook 处理 chunk 的优化。比如里面会有 MergeDuplicateChunksPlugin 合并重复的块，SplitChunksPlugin 进行分块处理等插件

seal 在最后会调用 optimizeTree hook 的异步钩子

seal() {
    //...
    
    //优化 chunks
    while (this.hooks.optimizeChunks.call(this.chunks, this.chunkGroups)) {
            /* empty */
    }
    
    this.hooks.optimizeTree.callAsync(this.chunks, this.modules, err => {
        //...
   

        this.hooks.optimizeChunkModules.callAsync(this.chunks, this.modules, err => {
            //...

            this.hooks.afterOptimizeChunkModules.call(this.chunks, this.modules);
           
            const shouldRecord = this.hooks.shouldRecord.call() !== false;

            this.hooks.reviveModules.call(this.modules, this.records);
            this.hooks.beforeModuleIds.call(this.modules);
            //为模块分配ID
            this.hooks.moduleIds.call(this.modules);
            this.hooks.optimizeModuleIds.call(this.modules);
            this.hooks.afterOptimizeModuleIds.call(this.modules);

+            this.hooks.reviveChunks.call(this.chunks, this.records);
            this.hooks.beforeChunkIds.call(this.chunks);
            //为chunk分配ID
            this.hooks.chunkIds.call(this.chunks);
            this.hooks.optimizeChunkIds.call(this.chunks);
            this.hooks.afterOptimizeChunkIds.call(this.chunks);

            this.assignRuntimeIds();

            this.logger.time("compute affected modules with chunk graph");
            this._computeAffectedModulesWithChunkGraph();
            this.logger.timeEnd("compute affected modules with chunk graph");

            this.sortItemsWithChunkIds();
            
            //记录模块和chunk
            if (shouldRecord) {
                this.hooks.recordModules.call(this.modules, this.records);
                this.hooks.recordChunks.call(this.chunks, this.records);
            }

            this.hooks.optimizeCodeGeneration.call(this.modules);
            this.logger.timeEnd("optimize");

            this.logger.time("module hashing");
            this.hooks.beforeModuleHash.call();
            this.createModuleHashes();
            this.hooks.afterModuleHash.call();
            this.logger.timeEnd("module hashing");

            this.logger.time("code generation");
            this.hooks.beforeCodeGeneration.call();
            this.codeGeneration(err => {
+               //...
            });
        });
    });
}

optimizeTree hook 回调里的第一个 hook 是 optimizeChunkModules hook，当 mode 为 production 时会往里注册一个插件叫 ModuleConcatenationPlugin ,它的作用是将所有模块提升至同一个闭包环境以增加运行速度。

之后会调用很多的钩子，例如给模块和 chunk 分配 ID，当构建期间没有错误时，shouldRecord !== false,会将模块和 chunk 信息记录到 records 里, records 会包含模块ID和模块对应的路径等信息，某些插件可以利用这些记录的信息在重构建时做持久化缓存。

createModuleHashes

优化的事情都做完后,接下来就是调用 createModuleHashes 开始为模块创建 hash

The text was updated successfully, but these errors were encountered:

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP]webpack5 源码详解 - 封装模块 #18

[WIP]webpack5 源码详解 - 封装模块 #18

Hazlank commented May 1, 2022 •

edited

Loading

[WIP]webpack5 源码详解 - 封装模块 #18

[WIP]webpack5 源码详解 - 封装模块 #18

Comments

Hazlank commented May 1, 2022 • edited Loading

封装模块

seal

Create Entry Chunks

buildChunkGraph

visitModules

connectChunkGroups

cleanupUnconnectedGroups

seal

createModuleHashes

Hazlank commented May 1, 2022 •

edited

Loading