背景

在大型 UE(Unreal Engine)项目中,“模块/插件"是维持工程可维护性的基本手段。但随着工程规模增长,很容易出现循环依赖问题:

  • 模块 A 为了复用工具函数依赖模块 B
  • 模块 B 为了注册某些类型或访问某些资源又依赖模块 A
  • 在某些平台或配置下(尤其是 Editor 构建或 Link 阶段需要聚合大量 .so/.a 时),最终表现为 循环依赖

UBT(UnrealBuildTool)能检测到循环依赖,但输出的是 ActionGraph 层面的信息:大量 Action #xxxx,并提示"这些 action 互相成环”。对于实际排障来说,主要痛点在于:

  1. Action ID 数量庞大、信息分散
  2. 一次循环往往牵涉多个 .so / .modules 文件
  3. 真正需要的答案是:“改动哪一条依赖能最快打破环?”

为此我编写了一个小工具:从 UBT 日志中提取循环依赖相关的 action,构建依赖图并找出更短、更值得优先处理的循环链路,辅助定位 最小改动点

本文按"日志分析 → 工具实现 → 输出解读 → 实际拆环"的顺序展开。

日志分析

首先分析 UBT 报告循环依赖时的典型日志片段(以下示例来自 Linux 构建 UnrealEditor,Windows/MSVC 下结构类似)。

这段日志包含两个关键信息:

  • Action #700 产出 libUnrealEditor-ActorSequence.so
  • 它 “Depends on cyclic actions”,列出了 action 2282/1270/1503/1263

从工程视角看,这意味着参与链接的几个库之间存在互相依赖,导致链接顺序无法拓扑排序。

flowchart LR
    A[Action #700\nLink ActorSequence.so]
    E[Action #2282\nproduce Engine.so]
    M[Action #1270\nproduce MovieScene.so]
    MT[Action #1503\nproduce MovieSceneTracks.so]
    T[Action #1263\nproduce TimeManagement.so]

    A --> E
    A --> M
    A --> MT
    A --> T

    %% cyclic actions 本质上是在说:这几个节点之间成环
    E --> M
    M --> MT
    MT --> T
    T --> E
Action #700: /bin/sh
    with arguments:  "/data/Stream_Depot/Client/Engine/Intermediate/Build/Linux/x64/UnrealEditor/Development/Link-libUnrealEditor-ActorSequence.so.link.sh"
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-Core.so
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-CoreUObject.so
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-Engine.so
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-MovieScene.so
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-MovieSceneTracks.so
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-TimeManagement.so
    depends on: /data/Stream_Depot/Client/Engine/Plugins/MovieScene/ActorSequence/Intermediate/Build/Linux/x64/UnrealEditor/Development/ActorSequence/Module.ActorSequence.cpp.o
    depends on: /data/Stream_Depot/Client/Engine/Plugins/MovieScene/ActorSequence/Intermediate/Build/Linux/x64/UnrealEditor/Development/ActorSequence/libUnrealEditor-ActorSequence.so.rsp
    produces:   /data/Stream_Depot/Client/Engine/Plugins/MovieScene/ActorSequence/Binaries/Linux/libUnrealEditor-ActorSequence.so
    Depends on cyclic actions:
        2282 (produces: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-Engine.so)
        1270 (produces: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-MovieScene.so)
        1503 (produces: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-MovieSceneTracks.so)
        1263 (produces: /data/Stream_Depot/Client/Engine/Binaries/Linux/libUnrealEditor-TimeManagement.so)

2)元数据生成阶段 .target/.modules 的 cyclic actions

循环依赖也可能发生在元数据生成阶段,而非 Link 阶段。这类日志的特点是:一个 action 产出 .target 文件,另一个 action 产出多个平台的 .modules 文件,两者互相依赖形成环。

Action #5333: /data/Stream_Depot/Client/Engine/Binaries/ThirdParty/DotNet/6.0.302/linux/dotnet
    with arguments: "/data/Stream_Depot/Client/Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll" -Mode=WriteMetadata -Input="/data/Stream_Depot/Client/Engine/Intermediate/Build/Linux/x64/UnrealEditor/Development/TargetMetadata.dat" -Version=2
    depends on: /data/Stream_Depot/Client/Engine/Binaries/Linux/UnrealEditor.version
    depends on: /data/Stream_Depot/Client/Engine/Intermediate/Build/Linux/x64/UnrealEditor/Development/TargetMetadata.dat
    produces:   /data/Stream_Depot/Client/Engine/Binaries/Linux/UnrealEditor.target
    Depends on cyclic actions:
        5332
            produces:   /data/Stream_Depot/Client/Engine/Binaries/Linux/Android/UnrealEditor.modules
            produces:   /data/Stream_Depot/Client/Engine/Binaries/Linux/Linux/UnrealEditor.modules
            produces:   /data/Stream_Depot/Client/Engine/Binaries/Linux/LinuxArm64/UnrealEditor.modules
            ...

为什么 UBT 的信息不便直接使用?

UBT 日志存在以下阅读障碍:

  1. Action ID 缺少上下文:需要从 produces: 反推对应的模块/目标。
  2. 只告知存在环,未指明拆解点:循环中每条依赖可能是必要的,也可能只是冗余的间接引用。
  3. 环的数量可能很多:大型工程中 UBT 可能报告多个循环点,需要确定优先级。

因此本工具的目标不是替代 UBT 检测,而是:

  • 将 UBT 输出的循环 action 信息结构化为图
  • 在图上进行轻量分析:找出更短的环,估算更值得优先拆解的链路

设计思路

将每个 Action #id 视为图中的一个节点:

  • 节点属性produces: 的文件列表(用于识别模块)
  • 有向边Action A 的 “Depends on cyclic actions” 中列出的 Action B,视为边 A -> B

这样"循环依赖"即为图中的 有向环

工具的处理流程如下:

flowchart LR
    L["UBT Build Log"] --> P["正则解析 Action/produces/cyclic-actions"]
    P --> G["构建 ActionGraph<br/>节点=Action, 边=A depends B"]
    G --> C["DFS 扫描环<br/>记录环上的 action 序列"]
    C --> S["按链路长度和权重排序"]
    S --> O["输出候选环<br/>并列出每个 action 的产物文件"]

工具实现

以下是核心实现的 Python 脚本。

实现要点

  • 使用正则匹配 Action #xxxproduces: 以及 Depends on cyclic actions: 后的 action id
  • 将依赖关系存储到图结构中
  • 通过 DFS 扫描依赖链,遇到栈内已存在的节点即认为找到一个环
  • 为环上的边计算启发式权重:被依赖次数(depended_count)越高的节点,越可能是公共库/关键节点,在环中越值得优先关注

注意:权重仅用于启发式排序,并非数学意义上的"最小反馈边集"。实践经验表明:先找出短环,再通过产物文件定位模块,排障效率更高。

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import codecs
import re
import argparse
import os
import sys

__VERSION__ = "1.0.0"

UE_LOG_ACTION_START = re.compile("^\\s*Action #\\s*(?P<ACTION_ID>\\d+)\\s*:")
UE_LOG_ACTION_PRODUCE = re.compile("^\\s\\s*produces\\s*:\\s*(?P<PRODUCE_FILE>.+)$")
UE_LOG_ACTION_DEP_START = re.compile(
    "^\\s\\s*Depends\\s+on\\s+cyclic\\s+actions", re.IGNORECASE
)
UE_LOG_ACTION_DEP_ID = re.compile(
    "^\\s\\s\\s*(?P<ACTION_ID>\\d+)\\s*(\\(\\s*produces\\s*:|$)"
)


class UEActionGraphNode:
    def __init__(self, action_id):
        self.action_id = action_id
        self.produce_files = []
        self.depend_actions = set()
        self.depended_count = 0

    def add_produce_file(self, file_path):
        if file_path not in self.produce_files:
            self.produce_files.append(file_path)

    def add_depend_action(self, action):
        if action not in self.depend_actions:
            self.depend_actions.add(action)
            action.depended_count += 1


class UEActionGraphSet:
    def __init__(self):
        self.actions = dict()

    def mutable_action(self, action_id):
        if action_id not in self.actions:
            self.actions[action_id] = UEActionGraphNode(action_id)
        return self.actions[action_id]

    def get_action(self, action_id):
        return self.actions.get(action_id, None)

    def __getitem__(self, action_id):
        return self.get_action(action_id)

    def __contains__(self, action_id):
        return action_id in self.actions


class UEActionGraphCircle:
    def __init__(self, from_action, to_action):
        self.from_action = from_action
        self.to_action = to_action
        self.weight = 0


def try_read_file(file_path):
    _err = None
    if not os.path.isabs(file_path):
        file_path = os.path.join(os.getcwd(), file_path)
    for try_encoding in ["utf-8", "utf-8-sig", "GB18030"]:
        try:
            ret = codecs.open(file_path, "r", encoding=try_encoding)
            return ret
        except Exception as e:
            if _err is None:
                _err = e
            if not os.path.exists(file_path):
                break
    raise _err


def build_deps_from_fd(file):
    last_action_node = None
    actions = UEActionGraphSet()
    deps_started = False
    for line in file:
        mat = UE_LOG_ACTION_START.match(line)
        if mat:
            last_action_node = actions.mutable_action(int(mat.group("ACTION_ID")))
            deps_started = False
            continue

        if last_action_node is not None:
            mat = UE_LOG_ACTION_DEP_START.match(line)
            if mat:
                deps_started = True
                continue

        if last_action_node is not None:
            if deps_started:
                mat = UE_LOG_ACTION_DEP_ID.match(line)
                if mat:
                    dep_action_id = int(mat.group("ACTION_ID"))
                    last_action_node.add_depend_action(
                        actions.mutable_action(dep_action_id)
                    )
                    continue
            else:
                mat = UE_LOG_ACTION_PRODUCE.match(line)
                if mat:
                    produce_file = mat.group("PRODUCE_FILE")
                    last_action_node.add_produce_file(produce_file)
                    continue

        if last_action_node is not None and (
            line.strip() == "" or (line and line[0] != " " and line[0] != "\t")
        ):
            last_action_node = None
            deps_started = False

    return actions


def show_actions(actions):
    for action_id, action_node in actions.actions.items():
        print(f"Action {action_id}:({action_node.depended_count} dependents)")
        print(f"  - produces files:")
        for f in action_node.produce_files:
            print(f"    - {f}")
        print(f"  - depends on actions:")
        for d in action_node.depend_actions:
            print(f"    - {d.action_id}")
    print(f"Total actions: {len(actions.actions)}")


def mutable_circle_node(
    circle_nodes, from_action: UEActionGraphNode, to_action: UEActionGraphNode
):
    if from_action not in circle_nodes:
        circle_nodes[from_action] = dict()
    from_set = circle_nodes[from_action]
    if to_action not in from_set:
        from_set[to_action] = UEActionGraphCircle(from_action, to_action)
    return from_set[to_action]


def get_circle_node(
    circle_nodes: dict, from_action: UEActionGraphNode, to_action: UEActionGraphNode
):
    if from_action not in circle_nodes:
        return None
    return circle_nodes[from_action].get(to_action, None)


def __concat_circle_actions(
    circle_nodes: dict,
    dep_link_stack: list,
    start: UEActionGraphCircle,
):
    sz = len(dep_link_stack)
    for i in range(sz - 1):
        from_action = dep_link_stack[i]
        to_action = dep_link_stack[i + 1]
        circle_node = mutable_circle_node(circle_nodes, from_action, to_action)
        circle_node.weight += start.weight

    last_circle_node = mutable_circle_node(
        circle_nodes, dep_link_stack[-1], start.from_action
    )
    last_circle_node.weight += start.weight


def __collect_circle_actions(
    circle_nodes: dict, dep_link_stack: list, action_node: UEActionGraphNode
):
    start_idx = -1
    for i in range(len(dep_link_stack)):
        if dep_link_stack[i] == action_node:
            start_idx = i
            break
    if start_idx < 0:
        raise RuntimeError(
            f"Internal error: start action {action_node.action_id} not found in stack {' -> '.join([str(n.action_id) for n in dep_link_stack])}"
        )

    sz = len(dep_link_stack)
    first_circle_node = None
    for i in range(sz - start_idx - 1):
        from_action = dep_link_stack[i + start_idx]
        to_action = dep_link_stack[i + start_idx + 1]
        circle_node = mutable_circle_node(circle_nodes, from_action, to_action)
        circle_node.weight += from_action.depended_count / sz
        if first_circle_node is None:
            first_circle_node = circle_node

    last_circle_node = mutable_circle_node(
        circle_nodes, dep_link_stack[-1], dep_link_stack[start_idx]
    )
    last_circle_node.weight += dep_link_stack[-1].depended_count

    if start_idx > 0:
        __concat_circle_actions(
            circle_nodes,
            dep_link_stack[0:start_idx],
            first_circle_node,
        )
    return dep_link_stack[start_idx:]


def __cal_circle_actions_weight(circle_nodes: dict, dep_link_stack: list):
    if len(dep_link_stack) <= 1:
        return 0

    sz = len(dep_link_stack)
    ret = 0
    for i in range(sz - 1):
        from_action = dep_link_stack[i]
        to_action = dep_link_stack[i + 1]
        circle_node = mutable_circle_node(circle_nodes, from_action, to_action)
        ret += circle_node.weight
    return ret


def __dfs_scan_actions(
    actions: UEActionGraphSet,
    circle_nodes: dict,
    dep_link_dict: dict,
    dep_link_stack: list,
    action_node: UEActionGraphNode,
    show_verbose: bool = False,
):
    ret = []
    dep_link_dict[action_node.action_id] = action_node
    dep_link_stack.append(action_node)

    # 优先找短的链:这里用 DFS + 遇到环就提前返回的策略,实践里足够快
    dfs_depend_actions = []
    for dep_action in action_node.depend_actions:
        existed_chain = get_circle_node(circle_nodes, action_node, dep_action)
        if existed_chain:
            __concat_circle_actions(circle_nodes, dep_link_stack, existed_chain)
            continue

        if dep_action.action_id in dep_link_dict:
            if show_verbose:
                print(
                    f"Found circle: "
                    + " -> ".join([str(n.action_id) for n in dep_link_stack])
                )
            ret.append(
                __collect_circle_actions(circle_nodes, dep_link_stack, dep_action)
            )
        else:
            dfs_depend_actions.append(dep_action)

    if not ret:
        for dep_action in dfs_depend_actions:
            res = __dfs_scan_actions(
                actions,
                circle_nodes,
                dep_link_dict,
                dep_link_stack,
                dep_action,
                show_verbose,
            )
            if res:
                ret += res
                break

    del dep_link_dict[action_node.action_id]
    del dep_link_stack[-1]

    return ret


def dfs_scan_actions(actions: UEActionGraphSet, show_verbose=False):
    circle_nodes = dict()  # (from, (to, UEActionGraphCircle))
    dep_link_dict = dict()
    dep_link_stack = []
    circle_chains = []
    for _, action_node in actions.actions.items():
        if action_node.depended_count == 0:
            continue

        if show_verbose:
            print(f"Scanning action {action_node.action_id} ...")
        for circle_stack in __dfs_scan_actions(
            actions,
            circle_nodes,
            dep_link_dict,
            dep_link_stack,
            action_node,
            show_verbose,
        ):
            circle_chains.append(
                {
                    "weight": __cal_circle_actions_weight(circle_nodes, circle_stack),
                    "chains": circle_stack,
                }
            )

    circle_chains.sort(key=lambda x: (len(x["chains"]), -x["weight"]))
    for circle_chain in circle_chains:
        print(
            f"Found circle chain: {' -> '.join([str(x.action_id) for x in circle_chain['chains']])}"
        )
        for circle_node in circle_chain["chains"]:
            print(
                f"  - ActionGraphNode #{circle_node.action_id} , depended: {circle_node.depended_count}, produces:"
            )
            for produce in circle_node.produce_files:
                print(f"    * {produce}")


def main():
    global __VERSION__

    parser = argparse.ArgumentParser(usage="%(prog)s [options...]")
    parser.add_argument("REMAINDER", nargs=argparse.REMAINDER, help="task names")
    parser.add_argument(
        "-v",
        "--version",
        action="store_true",
        help="show version and exit",
        dest="version",
        default=False,
    )

    parser.add_argument(
        "-V",
        "--verbose",
        action="store_true",
        help="show verbose",
        dest="verbose",
        default=False,
    )

    parser.add_argument(
        "-i",
        "--input",
        action="store",
        help="set input build log file, (use - to read from stdin)",
        dest="input",
        default="-",
    )

    options = parser.parse_args()
    if options.version:
        print(__VERSION__)
        return 0

    if options.input.strip() == "-":
        log_file = sys.stdin
    else:
        log_file = try_read_file(options.input)

    actions = build_deps_from_fd(log_file)
    if options.verbose:
        show_actions(actions)

    dfs_scan_actions(actions, show_verbose=options.verbose)

    return 0


if __name__ == "__main__":
    exit(main())

输出解读

脚本输出类似如下信息:

  • Found circle chain: 2282 -> 1270 -> 1503 -> 1263
  • 链路上每个 action 的 produces: 文件列表

这表明这些 action 之间存在有向环。下一步是将 action 级别的环映射回模块级别的环。

常用的映射规则:

产物路径模式对应内容
.../libUnrealEditor-XXX.so模块 XXX
Engine/Plugins/.../Binaries/.../libUnrealEditor-PluginName.so插件模块
.modules / .targetTarget/元数据生成流程

确认环上的产物对应哪些模块后,下一步是检查 *.Build.cs / Target.cs 中的依赖引用。

使用方法

1)收集构建日志

将 UBT 的输出重定向到文本文件,并启用 -Verbose 选项以获取完整的 action 信息。

Linux

./Engine/Build/BatchFiles/Linux/Build.sh UnrealEditor Linux Development -Verbose > build.log 2>&1

Windows PowerShell

./Engine/Build/BatchFiles/Build.bat UnrealEditor Win64 Development -Verbose *> build.log

2)运行脚本

python3 ue_cycle_finder.py -i build.log

支持从 stdin 读取:

cat build.log | python3 ue_cycle_finder.py -i -

效果示例

以上述日志为例,UBT 原始信息是:

Action #700 depends on cyclic actions: 2282/1270/1503/1263

真正需要的是"哪个模块与哪个模块形成环"。脚本输出环链路和 produces: 后,可以得到如下模块关系:

graph LR
    Engine[Engine.so] --> MovieScene[MovieScene.so]
    MovieScene --> MovieSceneTracks[MovieSceneTracks.so]
    MovieSceneTracks --> TimeManagement[TimeManagement.so]
    TimeManagement --> Engine

排障方式从:

  • 面对大量 Action #xxxx 无从下手

转变为:

  • Engine / MovieScene / MovieSceneTracks / TimeManagement 几个明确的模块间查找依赖引用

依赖环的常见形态

实际工程中,循环依赖不只存在"单一简单环"这一种形态。明确形态有助于确定拆解优先级。

1)单一简单环

graph LR
    A[ModuleA] --> B[ModuleB]
    B --> C[ModuleC]
    C --> A

这种情况较为直观:打断任意一条依赖边即可使图满足拓扑排序。

2)多个环共享链路

这种情况更常见,也更值得优先处理。以下示例包含两个环:

  • 环 1:A -> B -> C -> A
  • 环 2:D -> B -> C -> D

它们共享链路 B -> C

graph LR
    A[ModuleA] --> B[ModuleB]
    B --> C[ModuleC]
    C --> A

    D[ModuleD] --> B
    C --> D

这类结构提供了高性价比的排障策略:

  • 如果在共享链路上找到"可替换/可降级/可抽象"的依赖(如仅为 include,或可下沉到 Common/Interface 模块)
  • 拆掉共享链路可同时消除多个环

当脚本输出多个环时,应优先观察是否存在共享的 action/产物——共享链路往往意味着更少的改动和更大的收益。

3)共享更长的链路

以下示例中两个环都经过 B -> C -> E

graph LR
    A[ModuleA] --> B[ModuleB]
    B --> C[ModuleC]
    C --> E[ModuleE]
    E --> A

    D[ModuleD] --> B
    E --> D

如果 B -> CC -> EE -> ... 中某一段是"非必要链接依赖"(如仅为头文件可见),拆掉它即可同时断开多个环。

实践建议:优先从共享链路中"最弱"的依赖入手(如 include-only、工具模块、纯声明引用),避免拆解真正必需的核心链接依赖。

打破循环依赖的方法

工具的作用是更清晰地定位环,实际拆解仍需回到 UE 的模块依赖设计。

以下按优先级总结常用策略。

1)抽取公共模块(推荐)

如果模块 A 和 B 互相依赖只是为了共享少量代码或类型:

  • 将共享部分抽取到底层模块 Common
  • A、B 均依赖 Common

这是最干净、长期收益最高的方案:降低耦合、明确编译依赖。

2)引入接口模块

适用场景:A 需要调用 B 的能力,但不应强依赖 B 的实现细节。

做法:

  • 定义 BInterface 模块,仅包含纯接口和轻量声明(尽量使用前向声明)
  • A 只依赖 BInterface
  • B 依赖 BInterface 并提供实现(通过模块加载、Subsystem 注册或工厂注入等方式在运行时注入)

3)降级为 IncludePathModuleNames

如果依赖仅用于 include 头文件而不需要链接,可将其从 PrivateDependencyModuleNames / PublicDependencyModuleNames 降级为 PrivateIncludePathModuleNames / PublicIncludePathModuleNames,避免将"头文件可见"升级为"链接依赖"。

4)显式声明循环依赖(兜底方案)

如果短期无法拆解,UBT 曾提供显式声明循环依赖的机制:CircularlyReferencedDependentModules。Epic 官方知识库建议:优先移除循环依赖,无法移除时再显式声明以规避构建错误 1

注意:不同 UE 版本对此字段的支持可能有所调整。在较新版本的 UE 中,该接口已被移除,强制要求解除循环依赖——这对链接优化也有显著好处。

局限性与改进方向

本脚本刻意保持轻量,存在以下局限:

  • 仅分析日志中的 cyclic actions 段落,不复原完整的 UBT ActionGraph
  • 权重为启发式排序,不保证给出全局最优的"最小拆边集合"
  • DFS 实现偏向快速找出短链,而非枚举所有环

后续可考虑的增强方向:

  1. 输出 .dot / Graphviz 格式以便可视化
  2. 在候选拆边上进一步提示:打印 from produces -> to produces 作为建议检查的依赖对

总结

本文的核心经验:

  • UBT 报错的 ActionGraph 层信息对机器友好,对人不友好
  • 将日志结构化为图并进行轻量分析,能显著降低排障成本

工具本身并不复杂,但能将问题从"海量 action id"收敛到"几个明确模块之间的一条环"。之后即可用常规工程手段(抽取模块、接口隔离、降级依赖)完成拆解。

也欢迎有兴趣的小伙伴们互相交流探讨。