Unreal Engine 的 ReplicationGraph

在前述的UE服务端向客户端同步Actor的流程中,会以Tick的形式去每帧计算当前Connection感兴趣的Actor列表,并以特定的优先级排序,最后生成PriorityActors数组。当客户端数量为N而服务端参与同步的Actor数量为M时,单次NetDriver::TickFlush内生成所有连接的PriorityActors的最坏时间复杂度为N*M*Log(M), 当前World里参与同步的Actor数量变的很大时,此次TickFlush的执行时间将会急剧膨胀。EpicFortnite大世界中有多达50000的同步对象,同时有最多100个客户端连接,此时DS端的压力非常巨大。为了解决大量同步Actor引发的服务端性能问题,UE4.21中引入了ReplicationGraphReplicationGraph提供了一个更为高效的ServerReplicateActors实现,NetDriver::ServerReplicateActors这个函数的开头会去判断是否创建了一个ReplicationDriver对象,如果创建了则直接调用ReplicationDriver::ServerReplicateActors,不再走我们前述的Actor同步处理逻辑。

// NetDriver::ServerReplicateActors()
if (ReplicationDriver)
{
	return ReplicationDriver->ServerReplicateActors(DeltaSeconds);
}

下面我们来详细了解一下ReplicationGraph是如何实现一个更优的ServerReplicateActors

ReplicationGraph的Node功能介绍

ReplicationGraph将使用UReplicationGraphNode来管理所有参与同步的Actor,这是一个虚基类,只提供接口声明,核心接口是这个:

class REPLICATIONGRAPH_API UReplicationGraphNode : public UObject
{
public:
	/** Called once per frame prior to replication ONLY on root nodes (nodes created via UReplicationGraph::CreateNode) has RequiresPrepareForReplicationCall=true */
	void PrepareForReplication()
	/** Called when a network actor is spawned or an actor changes replication status */
	virtual void NotifyAddNetworkActor(const FNewReplicatedActorInfo& Actor ) PURE_VIRTUAL(UReplicationGraphNode::NotifyAddNetworkActor, );
	
	/** Called when a networked actor is being destroyed or no longer wants to replicate */
	virtual bool NotifyRemoveNetworkActor(const FNewReplicatedActorInfo& Actor, bool bWarnIfNotFound=true) PURE_VIRTUAL(UReplicationGraphNode::NotifyRemoveNetworkActor, return false; );

	// 根据传入的客户端连接参数来计算此连接可以同步的Actor集合
	virtual void GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params) = 0;

		/** Remove a child node from our list and flag it for destruction. Returns if the node was found or not */
	bool RemoveChildNode(UReplicationGraphNode* OutChildNode, UReplicationGraphNode::NodeOrdering NodeOrder=UReplicationGraphNode::NodeOrdering::IgnoreOrdering);

	/** Remove all null and about to be destroyed nodes from our list */
	void CleanChildNodes(UReplicationGraphNode::NodeOrdering NodeOrder);
protected:

	UPROPERTY()
	TArray< UReplicationGraphNode* > AllChildNodes;

	TSharedPtr<FReplicationGraphGlobalData>	GraphGlobals;
}

这里有一个AllChildNodes的成员来维持UReplicationGraphNode的树形结构。

这个类型的最简单子类是UReplicationGraphNode_ActorList,这个Node中包含了一个Actor列表,调用查询接口时会首先把这个Actor列表加入到结果中,然后再处理后续的查询:

class REPLICATIONGRAPH_API UReplicationGraphNode_ActorList : public UReplicationGraphNode
{
	/** The base list that most actors will go in */
	FActorRepListRefView ReplicationActorList;

	void GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params) override
	{
		Params.OutGatheredReplicationLists.AddReplicationActorList(ReplicationActorList);
		StreamingLevelCollection.Gather(Params);
		for (UReplicationGraphNode* ChildNode : AllChildNodes)
		{
			ChildNode->GatherActorListsForConnection(Params);
		}
	}
}

然后针对那些设置了bAlwaysRelevantActor(例如AGameState)有个对应的UReplicationGraphNode_AlwaysRelevant子类型,启动时注册这些AlwaysRelevantUClass,然后运行时把这个UClassActor都收集过来, 查询时直接返回之前收集好的Actor列表即可:

class REPLICATIONGRAPH_API UReplicationGraphNode_AlwaysRelevant : public UReplicationGraphNode
{
protected:

	UPROPERTY()
	UReplicationGraphNode* ChildNode = nullptr;

	// 启动时注册好的AlwaysRelevant的Uclass 
	TArray<UClass*>	AlwaysRelevantClasses;
public:
	virtual void PrepareForReplication() override
	{
		if (ChildNode == nullptr)
		{
			ChildNode = CreateChildNode<UReplicationGraphNode_ActorList>();
		}

		ChildNode->NotifyResetAllNetworkActors();
		for (UClass* ActorClass : AlwaysRelevantClasses)
		{
			for (TActorIterator<AActor> It(GetWorld(), ActorClass); It; ++It)
			{
				AActor* Actor = *It;
				if (IsActorValidForReplicationGather(Actor))
				{			
					ChildNode->NotifyAddNetworkActor( FNewReplicatedActorInfo(*It) );
				}
			}
		}
	}

	virtual void GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params) override
	{
		ChildNode->GatherActorListsForConnection(Params);
	}
}

另外对于对于某个客户端永远同步的Actor列表(例如当前客户端连接控制的ApawnAPlayerController),也封装了一个UReplicationGraphNode_AlwaysRelevant_ForConnection子类进行管理:

class REPLICATIONGRAPH_API UReplicationGraphNode_AlwaysRelevant_ForConnection : public UReplicationGraphNode_ActorList
{
	/** Rebuilt-every-frame list based on UNetConnection state */
	FActorRepListRefView ReplicationActorList;

	/** List of previously (or currently if nothing changed last tick) focused actor data per connection */
	UPROPERTY()
	TArray<FAlwaysRelevantActorInfo> PastRelevantActors;

	virtual void GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params) override
	{
		// Call super to add any actors that were explicitly given to use via NotifyAddNetworkActor
		Super::GatherActorListsForConnection(Params);
		// Reset and rebuild another list that will contains our current viewer/viewtarget
		ReplicationActorList.Reset();
		auto UpdateActor = [&](AActor* NewActor, AActor*& LastActor)
		{
			if (NewActor && !ReplicationActorList.Contains(NewActor))
			{
				ReplicationActorList.Add(NewActor);
			}
		};
		for (const FNetViewer& CurViewer : Params.Viewers)
		{
			if (CurViewer.Connection == nullptr)
			{
				continue;
			}

			FAlwaysRelevantActorInfo* LastData = PastRelevantActors.FindByKey<UNetConnection*>(CurViewer.Connection);

			// We've not seen this actor before, go ahead and add them.
			if (LastData == nullptr)
			{
				FAlwaysRelevantActorInfo NewActorInfo;
				NewActorInfo.Connection = CurViewer.Connection;
				LastData = &(PastRelevantActors[PastRelevantActors.Add(NewActorInfo)]);
			}

			check(LastData != nullptr);

			UpdateActor(CurViewer.InViewer, LastData->LastViewer);
			UpdateActor(CurViewer.ViewTarget, LastData->LastViewTarget);
		}
	}
}

可以看出这里的逻辑就是把当前Connection对应的NetViewer里的InViewerViewTarget加入到结果列表中,也就是对应的PlayerControllerPawn

除了上述几种基本的Node之外,引擎层还定义了多个其他种类的Node,业务层还可以自己扩展,例如Fortinite中就增加了TeamRelevantNode节点,用来对这个队伍内的所有客户端连接进行同步。这里重点讲一下对于查询优化最大的子类UReplicationGraphNode_GridSpatialization2D

基于网格划分的UReplicationGraphNode

这个类使用了前述的网格划分结构来加速查询,然后网格内的每个Cell都有一个对应的继承自UReplicationGraphNode_ActorListUReplicationGraphNode_GridCell节点,UReplicationGraphNode_GridSpatialization2D使用了一个二维数组来存储所有的UReplicationGraphNode_GridCell

TArray< TArray<UReplicationGraphNode_GridCell*> > Grid;

TArray<UReplicationGraphNode_GridCell*>& GetGridX(int32 X)
{
	if (Grid.Num() <= X)
	{
		Grid.SetNum(X+1);
	}
	return Grid[X];
}
UReplicationGraphNode_GridCell*& GetCell(TArray<UReplicationGraphNode_GridCell*>& GridX, int32 Y)
{
	if (GridX.Num() <= Y)
	{
		GridX.SetNum(Y+1);
	}
	return GridX[Y];
}		

UReplicationGraphNode_GridCell*& GetCell(int32 X, int32 Y)
{
	TArray<UReplicationGraphNode_GridCell*>& GridX = GetGridX(X);
	return GetCell(GridX, Y);
}

当添加一个Actor时,会根据这个Actor是场景里的静态Actor(位置不会移动)还是动态Actor(位置会移动)进入不同的接口:

  1. 当添加的是静态Actor时,根据其NetCullingDistance获取覆盖的GridNode,然后对于每个GridNode执行添加静态Actor的接口
void UReplicationGraphNode_GridSpatialization2D::AddActorInternal_Static_Implementation(const FNewReplicatedActorInfo& ActorInfo, FGlobalActorReplicationInfo& ActorRepInfo, bool bDormancyDriven)
{
	AActor* Actor = ActorInfo.Actor;
	const FVector Location3D = Actor->GetActorLocation();
	ActorRepInfo.WorldLocation = Location3D;

	StaticSpatializedActors.Emplace(Actor, FCachedStaticActorInfo(ActorInfo, bDormancyDriven));

	GetGridNodesForActor(ActorInfo.Actor, ActorRepInfo, GatheredNodes);
	for (UReplicationGraphNode_GridCell* Node : GatheredNodes)
	{
		Node->AddStaticActor(ActorInfo, ActorRepInfo, bDormancyDriven);
	}
}
  1. 如果添加的是动态Actor,则先放到一个临时数组DynamicSpatializedActors中,
void UReplicationGraphNode_GridSpatialization2D::AddActorInternal_Dynamic(const FNewReplicatedActorInfo& ActorInfo)
{
	DynamicSpatializedActors.Emplace(ActorInfo.Actor, ActorInfo);
}

这个数组会在每一帧的UReplicationGraphNode_GridSpatialization2D::PrepareForReplication中进行处理:

for (auto& MapIt : DynamicSpatializedActors)
{
	FActorRepListType& DynamicActor = MapIt.Key;
	FCachedDynamicActorInfo& DynamicActorInfo = MapIt.Value;
	FActorCellInfo& PreviousCellInfo = DynamicActorInfo.CellInfo;
	FNewReplicatedActorInfo& ActorInfo = DynamicActorInfo.ActorInfo;
	// Update location
	FGlobalActorReplicationInfo& ActorRepInfo = GlobalRepMap->Get(DynamicActor);

	// Check if this resets spatial bias
	const FVector Location3D = DynamicActor->GetActorLocation();
	ActorRepInfo.WorldLocation = Location3D;
	// Get the new CellInfo
	const FActorCellInfo NewCellInfo = GetCellInfoForActor(DynamicActor, Location3D, ActorRepInfo.Settings.GetCullDistance());

	if (PreviousCellInfo.IsValid())
	{
		bool bDirty = false;

		if (UNLIKELY(NewCellInfo.StartX > PreviousCellInfo.EndX || NewCellInfo.EndX < PreviousCellInfo.StartX ||
				NewCellInfo.StartY > PreviousCellInfo.EndY || NewCellInfo.EndY < PreviousCellInfo.StartY))
		{
			// No longer intersecting, we just have to remove from all previous nodes and add to all new nodes
			
			bDirty = true;

			GetGridNodesForActor(DynamicActor, PreviousCellInfo, GatheredNodes);
			for (UReplicationGraphNode_GridCell* Node : GatheredNodes)
			{
				Node->RemoveDynamicActor(ActorInfo);
			}

			GetGridNodesForActor(DynamicActor, NewCellInfo, GatheredNodes);
			for (UReplicationGraphNode_GridCell* Node : GatheredNodes)
			{
				Node->AddDynamicActor(ActorInfo);
			}
		}
		else
		{
			// Some overlap so lets find out what cells need to be added or removed
		}
	}
	else
	{
		// First time - Just add
		GetGridNodesForActor(DynamicActor, NewCellInfo, GatheredNodes);
		for (UReplicationGraphNode_GridCell* Node : GatheredNodes)
		{
			Node->AddDynamicActor(ActorInfo);
		}

		PreviousCellInfo = NewCellInfo;
	}
}

这段代码的核心逻辑是判断这个DynamicActor的当前位置覆盖的同步区域NewCellInfo是否与之前计算的同步区域PreviousCellInfo有交集,然后再做处理:

  1. 如果没有交集,则先从网格中删除这个Actor,再重新添加

  2. 如果有交集,则计算两个集合的差集:

    1. 计算PreviousCellInfo - NewCellInfoNode集合RemoveSet,遍历RemoveSet中的所有Node执行RemoveDynamicActor(Actor)
    2. 计算NewCellInfo - PreviousCellInfo的集合AddSet,遍历AddSet中所有的Node执行AddDynamicActor(Actor)

这里区分静态Actor和动态Actor,也是出于效率考量,因为静态Actor位置不会改变,所以所覆盖的GridNode是不会变的(假如其同步半径不修改的情况),而动态Actor位置会不断的变化,必须每次在PrepareForReplication时重算其影响范围。

这里的UReplicationGraphNode_GridCell实现上也分别提供了静态Actor和动态Actor的相关接口:

class REPLICATIONGRAPH_API UReplicationGraphNode_GridCell : public UReplicationGraphNode_ActorList
{
	UPROPERTY()
	UReplicationGraphNode* DynamicNode = nullptr;

	UPROPERTY()
	UReplicationGraphNode_DormancyNode* DormancyNode = nullptr;
}
void UReplicationGraphNode_GridCell::AddStaticActor(const FNewReplicatedActorInfo& ActorInfo, FGlobalActorReplicationInfo& ActorRepInfo, bool bParentNodeHandlesDormancyChange)
{
	if (ActorRepInfo.bWantsToBeDormant)
	{
		// Pass to dormancy node
		GetDormancyNode()->AddDormantActor(ActorInfo, ActorRepInfo);
	}
	else
	{	
		// Put him in our non dormancy list
		Super::NotifyAddNetworkActor(ActorInfo);
	}
	// We need to be told if this actor changes dormancy so we can move him between nodes. Unless our parent is going to do it.
	if (!bParentNodeHandlesDormancyChange)
	{
		ActorRepInfo.Events.DormancyChange.AddUObject(this, &UReplicationGraphNode_GridCell::OnStaticActorNetDormancyChange);
	}
}
void UReplicationGraphNode_GridCell::AddDynamicActor(const FNewReplicatedActorInfo& ActorInfo)
{
	GetDynamicNode()->NotifyAddNetworkActor(ActorInfo);
}

然后执行查询时,遍历传入的Connection的所有FNetViewer,获取这个Viewer的坐标点对应的网格(CellX, CellY)

// void UReplicationGraphNode_GridSpatialization2D::GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params)
TArray<FPlayerGridCellInformation, FReplicationGraphConnectionsAllocator> ActiveGridCells;

// 下面的是循环体
FPlayerGridCellInformation NewPlayerCell(FIntPoint(CellX, CellY));
FLastLocationGatherInfo* GatherInfoForConnection = nullptr;

// Save this information out for later.
if (CurViewer.Connection != nullptr)
{
	GatherInfoForConnection = LastLocationArray.FindByKey<UNetConnection*>(CurViewer.Connection);

	// Add any missing last location information that we don't have
	if (GatherInfoForConnection == nullptr)
	{
		GatherInfoForConnection = &LastLocationArray[LastLocationArray.Emplace(CurViewer.Connection, FVector(ForceInitToZero))];
	}
}

FVector LastLocationForConnection = GatherInfoForConnection ? GatherInfoForConnection->LastLocation : ClampedViewLoc;

//@todo: if this is clamp view loc this is now redundant...
if (GridBounds.IsValid)
{
	// Clean up the location data for this connection to be grid bound
	LastLocationForConnection = GridBounds.GetClosestPointTo(LastLocationForConnection);
}
else
{
	// Prevent extreme locations from causing the Grid to grow too large
	LastLocationForConnection = LastLocationForConnection.BoundToCube(HALF_WORLD_MAX);
}

// Try to determine the previous location of the user.
NewPlayerCell.PrevLocation.X = FMath::Max(0, (int32)((LastLocationForConnection.X - SpatialBias.X) / CellSize));
NewPlayerCell.PrevLocation.Y = FMath::Max(0, (int32)((LastLocationForConnection.Y - SpatialBias.Y) / CellSize));

// If we have not operated on this cell yet (meaning it's not shared by anyone else), gather for it.
if (!UniqueCurrentLocations.Contains(NewPlayerCell.CurLocation))
{
	TArray<UReplicationGraphNode_GridCell*>& GridX = GetGridX(CellX);
	if (GridX.Num() <= CellY)
	{
		GridX.SetNum(CellY + 1);
	}

	UReplicationGraphNode_GridCell* CellNode = GridX[CellY];
	if (CellNode)
	{
		CellNode->GatherActorListsForConnection(Params);
	}

	UniqueCurrentLocations.Add(NewPlayerCell.CurLocation);
}

// Add this to things we consider later.
ActiveGridCells.Add(NewPlayerCell);

在上面的代码逻辑中,此次查询先获取查询的中心点NewPlayerCell

  1. 如果当前帧内没有执行过以此为中心的查询请求,则找到对应的CellNode执行查询
  2. 记录当前查询点信息到ActiveGridCellsUniqueCurrentLocations

这样做的好处是对于同一个Grid内发起的多个请求只会执行一次,然后对于不参与查询的Grid则不需要做子节点的查询。

UReplicationGraphNode_GridCell中,静态的Actor如果是Dormant的则使用DomancyNode来管理,如果不是Dormant的则使用父类UReplicationGraphNode_ActorList来管理,而动态的Actor则使用一个DynamicNode来管理:

class REPLICATIONGRAPH_API UReplicationGraphNode_GridCell : public UReplicationGraphNode_ActorList
{
	UPROPERTY()
	UReplicationGraphNode* DynamicNode = nullptr;

	UPROPERTY()
	UReplicationGraphNode_DormancyNode* DormancyNode = nullptr;
	void AddStaticActor(const FNewReplicatedActorInfo& ActorInfo, FGlobalActorReplicationInfo& ActorRepInfo, bool bParentNodeHandlesDormancyChange)
	{
		if (ActorRepInfo.bWantsToBeDormant)
		{
			// Pass to dormancy node
			GetDormancyNode()->AddDormantActor(ActorInfo, ActorRepInfo);
		}
		else
		{	
			// Put him in our non dormancy list
			Super::NotifyAddNetworkActor(ActorInfo);
		}

		// We need to be told if this actor changes dormancy so we can move him between nodes. Unless our parent is going to do it.
		if (!bParentNodeHandlesDormancyChange)
		{
			ActorRepInfo.Events.DormancyChange.AddUObject(this, &UReplicationGraphNode_GridCell::OnStaticActorNetDormancyChange);
		}
	}
	void AddDynamicActor(const FNewReplicatedActorInfo& ActorInfo)
	{
		GetDynamicNode()->NotifyAddNetworkActor(ActorInfo);
	}
}

这个DynamicNode的具体类型其实是UReplicationGraphNode_ActorListFrequencyBuckets:

UReplicationGraphNode* UReplicationGraphNode_GridCell::GetDynamicNode()
{
	if (DynamicNode == nullptr)
	{
		if (CreateDynamicNodeOverride)
		{
			DynamicNode = CreateDynamicNodeOverride(this);
		}
		else
		{
			DynamicNode = CreateChildNode<UReplicationGraphNode_ActorListFrequencyBuckets>();
		}
	}

	return DynamicNode;
}

UReplicationGraphNode_ActorListFrequencyBuckets,这个类主要处理在 Non StreamingLevel 时大规模 Actors 的负载均衡(实现思路也很简单,根据所有 Actors 的数量,分了几组 Buckets,每组设置 Actor 的上限。添加Actor时选择Buckets里包含Actor数量最少的Bucket进行添加。动态添加和删除Actor 时会检查Bucket数量与Actor数量之间的关系是否满足限定条件,不满足时重新分配合适数量的Buckets并将Actor进行随机分配。

// Readd/Rebalance
for (int32 idx=0; idx < FullList.Num(); ++idx)
{
	NonStreamingCollection[idx % NewSize].Add( FullList[idx] );
}

然后在执行关联Actor查询的时候根据当前帧数依次选择一组Buckets进行同步:

// void UReplicationGraphNode_ActorListFrequencyBuckets::GatherActorListsForConnection(const FConnectionGatherActorListParameters& Params)

// Default path only: don't return lists in "off" frames.
const int32 idx = Params.ReplicationFrameNum % NonStreamingCollection.Num();
Params.OutGatheredReplicationLists.AddReplicationActorList(NonStreamingCollection[idx]);

到这里,基于空间索引的Connection关联Actor查询优化结构UReplicationGraphNode_GridSpatialization2D的实现基本明晰了。

ReplicationGraph的执行流程

在于关键性的函数 UReplicationGraph::ServerReplicateActors(float DeltaSeconds)这个是以NetDriver::TickFlush()来驱动的 。函数的开头会遍历所有Node执行PrepareForReplication来做本次Tick内的数据准备:

{
	QUICK_SCOPE_CYCLE_COUNTER(NET_PrepareReplication);

	for (UReplicationGraphNode* Node : PrepareForReplicationNodes)
	{
		Node->PrepareForReplication();
	}
}

在前面介绍的几个UReplicationGraphNode的子类中,主要有这几个子类有数据准备逻辑:

  1. UReplicationGraphNode_GridSpatialization2D 这个类在准备阶段负责把所有的动态Actor填充到其同步覆盖范围内的所有GridCell
  2. UReplicationGraphNode_AlwaysRelevant 这个类在准备阶段根据注册过来的永远同步的ActorUClass来收集相关的Actor实例列表

做好了数据准备工作之后,开始遍历所有的UNetReplicationGraphConnection,针对每个UNetReplicationGraphConnection收集这个UNetReplicationGraphConnection对应的TArray<FNetViewer>数组,以这个数组构造查询参数,然后遍历所有的顶层的UReplicationGraphNode来进行查询:

FGatheredReplicationActorLists GatheredReplicationListsForConnection;

TSet<FName> AllVisibleLevelNames;
ConnectionManager->GetClientVisibleLevelNames(AllVisibleLevelNames);
const FConnectionGatherActorListParameters Parameters(ConnectionViewers, *ConnectionManager, AllVisibleLevelNames, FrameNum, GatheredReplicationListsForConnection);

{
	QUICK_SCOPE_CYCLE_COUNTER(NET_ReplicateActors_GatherForConnection);

	for (UReplicationGraphNode* Node : GlobalGraphNodes)
	{
		Node->GatherActorListsForConnection(Parameters);
	}

	for (UReplicationGraphNode* Node : ConnectionManager->ConnectionGraphNodes)
	{
		Node->GatherActorListsForConnection(Parameters);
	}
	// 执行当前`ConnectionViewers`数组内每个`Viewer`在`ReplicationGraph`里的位置更新。
	ConnectionManager->UpdateGatherLocationsForConnection(ConnectionViewers, DestructionSettings);

	if (GatheredReplicationListsForConnection.NumLists() == 0)
	{
		// No lists were returned, kind of weird but not fatal. Early out because code below assumes at least 1 list
		UE_LOG(LogReplicationGraph, Warning, TEXT("No Replication Lists were returned for connection"));
		return 0;
	}
}

这里GlobalGraphNodes就是一个可以供所有连接进行查询的UReplicationGraphNode数组,在官方样例ShooterGame中,这个数组是这样初始化的:

void UShooterReplicationGraph::InitGlobalGraphNodes()
{
	// -----------------------------------------------
	//	Spatial Actors
	// -----------------------------------------------

	GridNode = CreateNewNode<UReplicationGraphNode_GridSpatialization2D>();
	GridNode->CellSize = CVar_ShooterRepGraph_CellSize;
	GridNode->SpatialBias = FVector2D(CVar_ShooterRepGraph_SpatialBiasX, CVar_ShooterRepGraph_SpatialBiasY);

	if (CVar_ShooterRepGraph_DisableSpatialRebuilds)
	{
		GridNode->AddSpatialRebuildBlacklistClass(AActor::StaticClass()); // Disable All spatial rebuilding
	}
	
	AddGlobalGraphNode(GridNode);

	// -----------------------------------------------
	//	Always Relevant (to everyone) Actors
	// -----------------------------------------------
	AlwaysRelevantNode = CreateNewNode<UReplicationGraphNode_ActorList>();
	AddGlobalGraphNode(AlwaysRelevantNode);

	// -----------------------------------------------
	//	Player State specialization. This will return a rolling subset of the player states to replicate
	// -----------------------------------------------
	UShooterReplicationGraphNode_PlayerStateFrequencyLimiter* PlayerStateNode = CreateNewNode<UShooterReplicationGraphNode_PlayerStateFrequencyLimiter>();
	AddGlobalGraphNode(PlayerStateNode);
}

可以看出这个数组里存储了三个Node,分别是基于空间索引的GridNode,所有节点都同步的AlwaysRelevantNode和用来处理PlayerState同步的PlayerStateNode

ConnectionManager->ConnectionGraphNodes则是只对当前Connection提供服务的UReplicationGraphNode数组,在ShooterGame中使用此函数进行初始化:

void UShooterReplicationGraph::InitConnectionGraphNodes(UNetReplicationGraphConnection* RepGraphConnection)
{
	Super::InitConnectionGraphNodes(RepGraphConnection);

	UShooterReplicationGraphNode_AlwaysRelevant_ForConnection* AlwaysRelevantConnectionNode = CreateNewNode<UShooterReplicationGraphNode_AlwaysRelevant_ForConnection>();

	// This node needs to know when client levels go in and out of visibility
	RepGraphConnection->OnClientVisibleLevelNameAdd.AddUObject(AlwaysRelevantConnectionNode, &UShooterReplicationGraphNode_AlwaysRelevant_ForConnection::OnClientLevelVisibilityAdd);
	RepGraphConnection->OnClientVisibleLevelNameRemove.AddUObject(AlwaysRelevantConnectionNode, &UShooterReplicationGraphNode_AlwaysRelevant_ForConnection::OnClientLevelVisibilityRemove);

	AddConnectionGraphNode(AlwaysRelevantConnectionNode, RepGraphConnection);
}
void UReplicationGraph::InitConnectionGraphNodes(UNetReplicationGraphConnection* ConnectionManager)
{
	// This handles tear off actors. Child classes should call Super::InitConnectionGraphNodes.
	ConnectionManager->TearOffNode = CreateNewNode<UReplicationGraphNode_TearOff_ForConnection>();
	ConnectionManager->AddConnectionGraphNode(ConnectionManager->TearOffNode);
}

Super::InitConnectionGraphNodes添加了用来同步Actor销毁通知的TearOffNode,然后在UShooterReplicationGraph::InitConnectionGraphNodes添加了用来收集当前连接所有NetViewerAlwaysRelevantConnectionNode。 在收集需要同步的Actor集合之后, 接下来开始真正的Actor同步流程:

// --------------------------------------------------------------------------------------------------------------
// PROCESS gathered replication lists
// --------------------------------------------------------------------------------------------------------------
{
	QUICK_SCOPE_CYCLE_COUNTER(NET_ReplicateActors_ProcessGatheredLists);

	ReplicateActorListsForConnections_Default(ConnectionManager, GatheredReplicationListsForConnection, ConnectionViewers);
	ReplicateActorListsForConnections_FastShared(ConnectionManager, GatheredReplicationListsForConnection, ConnectionViewers);
}

ReplicateActorListsForConnections_Default中会先对之前收集的Actor列表进行优先级计算,然后进行排序:

PrioritizedReplicationList.Reset();
TArray<FPrioritizedRepList::FItem>* SortingArray = &PrioritizedReplicationList.Items;

NumGatheredListsOnConnection += GatheredReplicationListsForConnection.NumLists();

const float MaxDistanceScaling = PrioritizationConstants.MaxDistanceScaling;
const uint32 MaxFramesSinceLastRep = PrioritizationConstants.MaxFramesSinceLastRep;

const TArray<FActorRepListConstView>& GatheredLists = GatheredReplicationListsForConnection.GetLists(EActorRepListTypeFlags::Default);
for (const FActorRepListConstView& List : GatheredLists)
{
	// Add actors from gathered list
	NumGatheredActorsOnConnection += List.Num();
	for (AActor* Actor : List)
	{
		float AccumulatedPriority = GlobalData.Settings.AccumulatedNetPriorityBias;
		// 一大堆计算优先级的函数
		SortingArray->Emplace(FPrioritizedRepList::FItem(AccumulatedPriority, Actor, &GlobalData, &ConnectionData));
	}
}
{
	// Sort the merged priority list. We could potentially move this into the replicate loop below, this could potentially save use from sorting arrays that don't fit into the budget
	RG_QUICK_SCOPE_CYCLE_COUNTER(NET_ReplicateActors_PrioritizeForConnection_Sort);
	NumPrioritizedActorsOnConnection += SortingArray->Num();
	SortingArray->Sort();
}

值得注意的是这里的AccumulatedPriority数值越小代表优先级越高,所以排序的时候不需要传入相反的比较器了。 这里的优先级计算考虑了很多因素,进行了多个分数的累加:

  1. 根据Actor与当前Connection里所有FNetViwer的距离最小值SmallestDistanceSq计算出一个分数,距离越小分数越小
  2. 根据Actor上次同步时间与当前时间之间的差值FramesSinceLastRep计算出一个分数,上次同步时间越久则分数越小
  3. 如果此Actor最近刚切换到dormant状态,则加上一个-1.5的分数,临时的提高这些Actor的同步优先级
  4. 如果此Actor是当前Connection里的Viewer,则根据全局设置选项CVar_ForceConnectionViewerPriority来决定是直接将AccumulatedPriority重置为负无穷大还是直接扣10分,这样的操作保证了当前连接主控的那些Actor的优先级最高

排序完成之后,按照同步优先级从高到低遍历排序之后的结果,调用ReplicateSingleActor进行向下同步,在遍历过程中如果发现当前Connection等待下发的数据太多,则终止遍历,用以节省流量,降低网络负荷:

for (int32 ActorIdx = 0; ActorIdx < PrioritizedReplicationList.Items.Num(); ++ActorIdx)
{
	const FPrioritizedRepList::FItem& RepItem = PrioritizedReplicationList.Items[ActorIdx];

	AActor* Actor = RepItem.Actor;
	FConnectionReplicationActorInfo& ActorInfo = *RepItem.ConnectionData;

	// Always skip if we've already replicated this frame. This happens if an actor is in more than one replication list
	if (ActorInfo.LastRepFrameNum == FrameNum)
	{
		INC_DWORD_STAT_BY(STAT_NetRepActorListDupes, 1);
		continue;
	}

	FGlobalActorReplicationInfo& GlobalActorInfo = *RepItem.GlobalData;

	int64 BitsWritten = ReplicateSingleActor(Actor, ActorInfo, GlobalActorInfo, ConnectionActorInfoMap, *ConnectionManager, FrameNum);
	if (IsConnectionReady(NetConnection) == false)
	{
		// We've exceeded the budget for this category of replication list.
		RG_QUICK_SCOPE_CYCLE_COUNTER(NET_ReplicateActors_PartialStarvedActorList);
		HandleStarvedActorList(PrioritizedReplicationList, ActorIdx + 1, ConnectionActorInfoMap, FrameNum);
		NotifyConnectionSaturated(*ConnectionManager);
		break;
	}
}

这里的ReplicateSingleActor其实调用的就是我们在前面介绍的ActorChannel::ReplicateActor:

// int64 UReplicationGraph::ReplicateSingleActor(AActor* Actor, FConnectionReplicationActorInfo& ActorInfo, FGlobalActorReplicationInfo& GlobalActorInfo, FPerConnectionActorInfoMap& ConnectionActorInfoMap, UNetReplicationGraphConnection& ConnectionManager, const uint32 FrameNum)
if (UNLIKELY(ActorInfo.bTearOff))
{
	// Replicate and immediately close in tear off case
	BitsWritten = ActorInfo.Channel->ReplicateActor();
	BitsWritten += ActorInfo.Channel->Close(EChannelCloseReason::TearOff);
}
else
{
	// Just replicate normally
	BitsWritten = ActorInfo.Channel->ReplicateActor();
}