Skip to main content
Version: Next

MetaVersion Workflow

DHE hot updates previously utilized two workflows:

  • DHAO Workflow
  • MetaVersion Workflow

In versions v7.6.0 and earlier, the build workflow was based on the DHAO Workflow. The DHAO Workflow used dhao files to record which types and functions had changed. When HybridCLR loaded a DHE assembly, it would determine whether to call the original AOT function or execute the new code based on the changes recorded in the dhao file.

The dhao file was generated by comparing the latest hot update DHE DLL with the original DHE DLL in the main package. While its principle was straightforward, the downside was that if multiple main packages were released, a separate dhao file would be required for each main package. This process became complex and difficult to manage when there were many main packages.

Starting from version v7.7.0, the MetaVersion Workflow was introduced to address these shortcomings of the DHAO Workflow.

Principle​

The MetaVersion Workflow assigns a unique version number to each type and function in a DLL. When loading a DHE assembly using RuntimeApi.LoadDifferentialHybridAssemblyWithMetaVersion, it compares the meta version of the latest hot update assembly with the meta version of the AOT snapshot in the current main package to determine whether a type or function has changed. This approach only requires generating a single meta version file for each hot update DHE DLL, which can be used for hot updates across all main packages.

Basic Concepts​

Before diving into the MetaVersion Workflow, it is essential to understand several key terms:

  • Snapshot
  • Inject Rule files
  • Meta Version files
  • Meta Spec files
  • manifest.json
  • signature-mapper.json

Snapshot​

A Snapshot is a collection of files required to generate meta version files, including:

  • DLL files
  • Inject Rule files
  • manifest.json
  • Meta Version files
  • Meta Spec files
  • signature-mapper.json

There are two types of Snapshots: AOT Snapshot and HotUpdate Snapshot.

Inject Rule Files​

By default, code is injected at the beginning of almost all DHE functions. This injection can significantly mitigate the problem of dirty function propagation, but it increases code size and introduces a small amount of additional overhead. Inject Rule files are used to customize injection rules, allowing certain functions to be excluded from injection. For detailed documentation, see Function Injection Policies.

manifest.json​

This file records information such as the list of DHE assemblies. When comparing and calculating the meta version files of two AOT Snapshots, it checks the consistency of the DHE assembly list. Only AOT Snapshots with identical DHE assembly lists can correctly calculate the meta version.

Meta Version Files​

Meta Version files store version information for all metadata. When loading a DHE assembly, the system compares the meta version file of the hot update assembly with the original DHE assembly's meta version file to determine whether changes have occurred, and subsequently decides whether to call the original AOT function or execute the interpreted function.

Meta Version files use the .mv.bytes suffix.

Meta Spec Files​

Meta Spec files are human-readable versions of Meta Version files. These files are not used during runtime. It is recommended to add them to version control, but they should not be included in the hot update resource system, as they serve no purpose there!

There are two types of Meta Spec files:

  • *.mv.spec: Records complete meta version information.
  • *.mv.diff.spec: Records changes in the meta version information. This file can be used to quickly identify which types and functions have changed.

signature-mapper.json​

When C# code is modified (especially when types or functions are added or removed), the tokens for the same types and functions in the generated DLL may change. If the meta version file only records these metadata tokens, it would take a significant amount of time at runtime to establish the mapping between old and new tokens. A better solution is to record the signature information of each token and quickly establish the mapping by comparing signatures.

Since signature strings are very long, to save space and facilitate runtime comparison, these strings are mapped to unique integers in advance. The signature-mapper.json file records this mapping relationship.

The signature-mapper.json file is only needed when generating meta version files and is not used during runtime. It is recommended to add this file to version control, but it should not be included in the hot update resource system!

Build and Hot Update Workflow​

  • Building the Main Package

    • Export the main package project or build the main package directly.
    • Create the corresponding AOT Snapshot.
    • Add the meta version files from the AOT Snapshot to the main package's StreamingAssets directory (optional but highly recommended).

  • Publishing Hot Updates

    • Create a HotUpdate Snapshot.
    • Add the hot update DLLs and meta version files from the HotUpdate Snapshot to the hot update resource system.

AOT Snapshot​

The AOT Snapshot records the AOT information of the main package and has several core functions:

  • It records its own meta version information.
  • It is used to calculate the meta version information for subsequent main package versions.
  • It is used to calculate the meta version information for hot update code.

The directory structure of an AOT Snapshot is as follows:

  AotSnapshotDir
    ├── *.dll
    ├── InjectRules
    ├── MetaVersions
      ├── *.mv.bytes
      ├── *.mv.spec
      ├── *.mv.diff.spec
    ├── signature-mapper.json
    └── manifest.json

The AOT Snapshot is fully determined during the main package build process. Please add it to your project's version control system.

Since the *.mv.bytes file from the AOT Snapshot is required when loading a DHE assembly using RuntimeApi::LoadDifferentialHybridAssemblyWithMetaVersion, it is recommended to include this file with the main package.

Creating an AOT Snapshot​

danger

The DLLs in the AOT Snapshot must match the binary code of the main package exactly. Please create the AOT Snapshot after exporting the project or building the main package! Do not use the AOT DLLs generated by HybridCLR/Generate/All!

The process is as follows:

  • Create the base AOT snapshot file.
  • Generate the meta version for the DHE DLLs in the base AOT snapshot.

Creating the Base AOT Snapshot File​

Call MetaVersionWorkflow.CreateAotSnapshot(BuildTarget target, string outputSnapshotDir) to create the base snapshot file.

CreateAotSnapshot performs the following tasks:

  • Copies all AOT and DHE DLLs.
  • Copies the inject rule files to the InjectRules directory.
  • Creates a manifest.json file, which records the list of all DHE assemblies.

The base snapshot directory structure is as follows:

  AotSnapshot
  ├── *.dll
  ├── InjectRules
        ├── rule1.xml
        ├── rule2.xml
  └── manifest.json

Generating Meta Version Files for DHE DLLs in the Base AOT Snapshot​

Call MetaVersionWorkflow.GenerateAotSnapshotMetaVersionFiles(string prevSnapshotDir, string curSnapshotDir) to generate the meta version files for the DHE DLLs in the snapshot.

The rules for the prevSnapshotDir parameter in MetaVersionWorkflow.GenerateAotSnapshotMetaVersionFiles are as follows:

  • If this is the first AOT Snapshot (i.e., there is no older AOT Snapshot), prevSnapshotDir should be null.
  • If the project uses a single main package mode (i.e., only one main package is valid at a time and a new main package invalidates the old one), prevSnapshotDir should be null.
  • If the project uses a multi-main package mode (i.e., old main packages remain valid after a new one is released), the prevSnapshotDir parameter should point to the most recently released AOT Snapshot directory with the same build target.

GenerateAotSnapshotMetaVersionFiles generates the following files:

  • Meta Version files, placed in the MetaVersions directory.
  • signature-mapper.json file.

The directory structure before generation is as follows:

  Snapshots
  ├── PrevAotSnapshotDir
      ├── *.dll
      ├── InjectRules
      ├── MetaVersions
        ├── *.mv.bytes
        ├── *.mv.spec
        ├── *.mv.diff.spec
      ├── signature-mapper.json
      └── manifest.json
  ├── CurrentSnapshotDir
      ├── *.dll
      ├── InjectRules
      └── manifest.json

After generation, the MetaVersions directory and signature-mapper.json file are added. The final directory structure is as follows:

  Snapshots
  ├── PrevAotSnapshotDir
      ├── *.dll
      ├── InjectRules
      ├── MetaVersions
        ├── *.mv.bytes
        ├── *.mv.spec
        ├── *.mv.diff.spec
      ├── signature-mapper.json
      └── manifest.json
  ├── CurrentAotSnapshotDir
      ├── *.dll
      ├── InjectRules
      ├── MetaVersions (New)
        ├── *.mv.bytes
        ├── *.mv.spec
        ├── *.mv.diff.spec
      ├── signature-mapper.json (New)
      └── manifest.json

Multi-Platform Considerations​

Most games are released on multiple platforms. Since there are significant differences in the main package AOT DLLs between platforms, it is strongly recommended to maintain separate AOT Snapshot trees for each platform. The directory structure should be similar to the following:

  Snapshots
  ├── PreAotSnapshotDir-StandaloneWindows64
  ├── CurrentAotSnapshotDir-StandaloneWindows64
  ├── PreAotSnapshotDir-Android64
  ├── CurrentAotSnapshotDir-Android64
  ├── PreAotSnapshotDir-iOS
  ├── CurrentAotSnapshotDir-iOS

For example, when calculating the meta version for a newly released Win64 main package, use the snapshot directory of the previously released Win64 main package and the new main package's snapshot to compute the meta version for the new main package.

Version Control​

The directory structure mentioned in the previous section is not convenient for version control. A more suitable directory structure for version control is as follows:

  Snapshots
  ├── AotSnapshotDir-StandaloneWindows64
  ├── AotSnapshotDir-Android64
  ├── AotSnapshotDir-iOS

The Snapshot update process is as follows:

  • Create a temporary CurrentAotSnapshotDir-{buildTarget} directory for the latest main package.
  • Compare CurrentAotSnapshotDir-{buildTarget} with AotSnapshotDir-{buildTarget} to generate the meta version files for CurrentAotSnapshotDir-{buildTarget}.
  • Replace AotSnapshotDir-{buildTarget} with CurrentAotSnapshotDir-{buildTarget}.
  • Commit AotSnapshotDir-{buildTarget} to the repository.

HotUpdate Snapshot​

A HotUpdate Snapshot is created each time a code hot update is published. Compared to an AOT Snapshot, a HotUpdate Snapshot only contains the following files:

  • The latest hot update assembly files
  • The meta version information for the latest DHE assemblies
  • signature-mapper.json

The directory structure is as follows:

  HotUpdateSnapshotDir
    ├── *.dll
    ├── MetaVersions
      ├── *.mv.bytes
      ├── *.mv.spec
      ├── *.mv.diff.spec
    ├── signature-mapper.json (New)

When publishing a hot update, only the hot update assembly DLL files and the meta version files for the DHE assemblies are needed. Add these files to the hot update resource system.

Creating a HotUpdate Snapshot​

The creation process is as follows:

  • Use HybridCLR/CompileDll/ActivedBuildTarget to generate the hot update DLLs.
  • Generate the meta version files for the hot update assemblies.

Call MetaVersionWorkflow.GenerateHotUpdateMetaVersionFiles(string aotSnapshotDir, string hotUpdateSnapshotDir) to generate the meta version files for the hot update assemblies. Here, aotSnapshotDir is the latest AotSnapshot-{buildTarget} directory, and hotUpdateSnapshotDir is the directory containing the latest hot update assemblies.

After calling this function, the meta version files for the DHE assemblies will be generated in the {hotUpdateSnapshotDir}/MetaVersions directory. The directory structure is similar to the following:

  HotUpdateSnapshot
  ├── *.dll