AI Development/TensorRT

[TensorRT] InstanceNormalization_TRT Plugin

꾸준희 2021. 9. 1. 17:42
728x90
반응형

 

 

아래와 같이 Engine Serialization 과정에서 InstanceNormalization_TRT 플러그인을 못찾겠다는 에러가 발생했다. 

 

Instance Normalization Paper : https://arxiv.org/pdf/1607.08022.pdf

[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin InstanceNormalization_TRT version 1
[TensorRT] ERROR: safeDeserializationUtils.cpp (322) - Serialization Error in load: 0 (Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[TensorRT] ERROR: INVALID_STATE: std::exception
[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed.

 

 

나의 모델에는 ONNX parsing 과정에서 InstanceNormalization operator 가 존재한다.

 

 

 

 

 

 

 

 

 

ONNX operators

https://github.com/onnx/onnx/blob/master/docs/Operators.md#InstanceNormalization

 

TensorRT Plugins

https://github.com/NVIDIA/TensorRT/blob/eb5de99b523c76c2f3ae997855ad86d3a1e86a31/plugin/README.md

 

InstanceNormalizationPlugin

https://github.com/NVIDIA/TensorRT/tree/eb5de99b523c76c2f3ae997855ad86d3a1e86a31/plugin/instanceNormalizationPlugin

 

TensorRT Support Matrix 

https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#fntarg_12

 

Support Matrix :: NVIDIA Deep Learning TensorRT Documentation

The section lists the supported TensorRT layers and each of the features. About this task Note: Supports broadcast indicates support for broadcast in this layer. This layer allows its two input tensors to be of dimensions [1, 5, 4, 3] and [1, 5, 1, 1], and

docs.nvidia.com

 

위 문서들에 의하면 이 operator 는 TensorRT 6 버전 부터 지원한다고 하는데 왜 플러그인을 찾지 못하는 것인지 구글링해보았다. 플러그인을 초기화 해주면 된다고 한다. 

 

 

 

 

 

1. Python API

 

아래 예시 코드와 같이 TensorRT 플러그인을 초기화 했더니 해결되었다. 

trt.init_libnvinfer_plugins(TRT_LOGGER, '')

 

예시 코드 1.

https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#add_custom_layer_python

import tensorrt as trt
import numpy as np

TRT_LOGGER = trt.Logger()

trt.init_libnvinfer_plugins(TRT_LOGGER, '')
PLUGIN_CREATORS = trt.get_plugin_registry().plugin_creator_list

def get_trt_plugin(plugin_name):
        plugin = None
        for plugin_creator in PLUGIN_CREATORS:
            if plugin_creator.name == plugin_name:
                lrelu_slope_field = trt.PluginField("neg_slope", np.array([0.1], dtype=np.float32), trt.PluginFieldType.FLOAT32)
                field_collection = trt.PluginFieldCollection([lrelu_slope_field])
                plugin = plugin_creator.create_plugin(name=plugin_name, field_collection=field_collection)
        return plugin

def main():
    builder = trt.Builder(TRT_LOGGER) 
    network = builder.create_network()
    config = builder.create_builder_config()
    config.max_workspace_size = 2**20
    input_layer = network.add_input(name="input_layer", dtype=trt.float32, shape=(1, 1))
    lrelu = network.add_plugin_v2(inputs=[input_layer], plugin=get_trt_plugin("LReLU_TRT"))
    lrelu.get_output(0).name = "outputs"
    network.mark_output(lrelu.get_output(0))

 

 

 

 

 

 

 

 

 

2. C++ API

 

 

사용 예시 1. 

https://github.com/mlcommons/inference_results_v1.0/blob/718818922edeef0a0c801841a8c7ee64f8f39c1f/closed/Fujitsu/code/harness/harness_triton_mig/main_server.cc#L665

 initLibNvInferPlugins(&gLogger.getTRTLogger(), "");

 

사용 예시 2. 

https://github.com/rajeshroy402/jetson-object-detection/blob/8454910ef056a86c548e2c2d42974f863bc75acc/c/tensorNet.cpp#L893

	/*
	 * load NV inference plugins
	 */
#if NV_TENSORRT_MAJOR > 4
	static bool loadedPlugins = false;

	if( !loadedPlugins )
	{
		LogVerbose(LOG_TRT "loading NVIDIA plugins...\n");

		loadedPlugins = initLibNvInferPlugins(&gLogger, "");

		if( !loadedPlugins )
			LogError(LOG_TRT "failed to load NVIDIA plugins\n");
		else
			LogVerbose(LOG_TRT "completed loading NVIDIA plugins.\n");
	}
#endif

 

 

 

 

 

 

 

 

 

 

 

참고자료 1 : https://stackoverflow.com/questions/60967438/unable-load-tensor-rt-savedmodel-after-conversion-in-tensorflow-2-1

 

Unable load Tensor RT SavedModel after conversion in Tensorflow 2.1

I have been attempting to convert a YOLOv3 model implemented in Tensorflow 2 to Tensor RT by following the tutorial on the NVIDIA website (https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-

stackoverflow.com

 

참고자료 2 : https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/_nv_infer_plugin_8h.html#a5238fbc83e103bce4c85291c8c5b85a6

 

TensorRT: NvInferPlugin.h File Reference

This is the API for the Nvidia provided TensorRT plugins. The BatchedNMS Plugin performs non_max_suppression on the input boxes, per batch, across all classes. It greedily selects a subset of bounding boxes in descending order of score. Prunes away boxes t

docs.nvidia.com

 

참고자료 3 : https://github.com/onnx/onnx-tensorrt/issues/597

 

[TRT] INVALID_ARGUMENT: getPluginCreator could not find plugin InstanceNormalization_TRT version 1 [12/22/2020-12:46:22] [E] [TR

[TRT] INVALID_ARGUMENT: getPluginCreator could not find plugin InstanceNormalization_TRT version 1 [12/22/2020-12:46:22] [E] [TRT] safeDeserializationUtils.cpp (322) - Serialization Error in load: ...

github.com

 

참고자료 4 : https://forums.developer.nvidia.com/t/tensorrt-cc-98-tensorrt-error-getplugincreator-could-not-find-plugin-normalize-trt-version-1-namespace/68708

 

TensorRT.cc:98] [TensorRT.ERROR]: getPluginCreator could not find plugin Normalize_TRT version 1 namespace

The giexec generate ssd.engine. It’s OK! But get error when inferencing in ssd.engine. CUDA:10 TRT:5.0.2 Driver:410.48

forums.developer.nvidia.com

 

728x90
반응형