UnityRef is currently in early development. Some features may be incomplete and/or not functioning.

UNITYREF

Your Pit Stop For Solving ANYTHING in Unity

assets

[Sentis] Fix ONNX Model Import: Kokoro-82M TTS Failures

Solution

audio systemadministrationneural networksAI

Unity 2023.2.x - Unity 6.3.x

Published Sun, Mar 8

Author Profile Photo

Reviewed by _architect

Chief Editor

80 posts
Issue

 Unity Sentis encountered import errors when processing the Kokoro-82M Text-to-Speech (TTS) ONNX model, specifically reporting non-support for critical ONNX operations including ConcatFromSequence, Loop, SequenceEmpty, and SplitToSequence.

Explanation

Sentis now supports the Kokoro-82M Text-to-Speech (TTS) model through updated support for sequence-based ONNX operators.

  1. Open the Package Manager and update Sentis to version 2.1.0 or later to ensure compatibility with Loop and Sequence operations.
  2. Import your ONNX file into the Project window to allow the Sentis importer to generate a ModelAsset automatically.
  3. Use your text processing library to convert raw text into tokenized phoneme IDs before passing them to the Sentis runtime.
  4. Initialize a Worker via the WorkerFactory using the ComputePrecompiled or GPUCompute backend for optimal performance.
  5. Execute the Model and extract the resulting TensorFloat data into an AudioSource using SetData.

Additional Tips

  • Monitor the Profiler to manage the memory footprint of the Kokoro model during runtime as sequence operations can be memory intensive.
  • Verify that your ONNX file uses Opset 13 or higher for optimal compatibility with the Sentis importer.

TL;DR

The Kokoro-82M Text-to-Speech (TTS) ONNX model is now fully supported by Sentis through enhanced operator compatibility, enabling high-quality neural speech synthesis within the engine.


Related Posts Haven't quite found a solution to your problem? We think these posts might help you.
[Audio] Resolve Import Failures for Converted OGG Assets[2D Sprites] Master Dynamic Visual Depth Control[Canvas System] Duplicated UI Assets: Awake Initialization Skip

Content inspired by a Unity discussion post.