mirror of https://github.com/llvm/torch-mlir
[torch-mlir] remove trailing whitespace from md documentation (#2853)
parent
24b8c8672a
commit
d1cd117998
|
@ -1,17 +1,17 @@
|
||||||
# How to Add Ops to Torch-Mlir
|
# How to Add Ops to Torch-Mlir
|
||||||
|
|
||||||
Collected links and contacts for how to add ops to torch-mlir.
|
Collected links and contacts for how to add ops to torch-mlir.
|
||||||
|
|
||||||
|
|
||||||
<details>
|
<details>
|
||||||
<summary>Turbine Camp: Start Here</summary>
|
<summary>Turbine Camp: Start Here</summary>
|
||||||
This document was previously known as `turbine-camp.md` to Nod.ai. "Turbine Camp" is part of Nod.ai's onboarding process. Welcome to turbine camp. This document originated at Nod.ai as a part of onboardding process, where new nod-ai folks learn about the architecture of our work by adding support for 2 ops to torch-mlir. I decided to put this into torch mlir because a lot of this is about torch-mlir.
|
This document was previously known as `turbine-camp.md` to Nod.ai. "Turbine Camp" is part of Nod.ai's onboarding process. Welcome to turbine camp. This document originated at Nod.ai as a part of onboardding process, where new nod-ai folks learn about the architecture of our work by adding support for 2 ops to torch-mlir. I decided to put this into torch mlir because a lot of this is about torch-mlir.
|
||||||
|
|
||||||
Written & maintained by @renxida
|
Written & maintained by @renxida
|
||||||
|
|
||||||
Guides by other folks that were used during the creation of this document:
|
Guides by other folks that were used during the creation of this document:
|
||||||
- [Chi Liu](https://gist.github.com/AmosLewis/dd31ab37517977b1c499d06495b4adc2)
|
- [Chi Liu](https://gist.github.com/AmosLewis/dd31ab37517977b1c499d06495b4adc2)
|
||||||
- [Sunsoon](https://docs.google.com/document/d/1H79DwW_wnVzUU81EogwY5ueXgnl-QzKet1p2lnqPar4/edit?pli=1)
|
- [Sunsoon](https://docs.google.com/document/d/1H79DwW_wnVzUU81EogwY5ueXgnl-QzKet1p2lnqPar4/edit?pli=1)
|
||||||
|
|
||||||
## Before you begin...
|
## Before you begin...
|
||||||
|
|
||||||
|
|
|
@ -4,7 +4,7 @@
|
||||||
|
|
||||||
As part of adding support for a Torch operator in Torch-MLIR, it is usually
|
As part of adding support for a Torch operator in Torch-MLIR, it is usually
|
||||||
necessary to define a shape and dtype function so that the compiler can infer
|
necessary to define a shape and dtype function so that the compiler can infer
|
||||||
the shapes and dtypes of result tensors for the operator. We use the
|
the shapes and dtypes of result tensors for the operator. We use the
|
||||||
[abstract interpretation library](abstract_interp_lib.md) for this process.
|
[abstract interpretation library](abstract_interp_lib.md) for this process.
|
||||||
|
|
||||||
## Step-by-step guide
|
## Step-by-step guide
|
||||||
|
@ -19,7 +19,7 @@ We will use the example of adding support for the `torch.aten.tanh` op.
|
||||||
file is the "rosetta stone" that allows translating between
|
file is the "rosetta stone" that allows translating between
|
||||||
e.g. `torch.aten.tanh`, `AtenTanhOp`, and the shape and dtype
|
e.g. `torch.aten.tanh`, `AtenTanhOp`, and the shape and dtype
|
||||||
function signatures are:
|
function signatures are:
|
||||||
|
|
||||||
- `def aten〇tanh〡shape(self: List[int]) -> List[int]:`
|
- `def aten〇tanh〡shape(self: List[int]) -> List[int]:`
|
||||||
- `def aten〇tanh〡dtype(self_rank_dtype: Tuple[int, int]) -> int:`
|
- `def aten〇tanh〡dtype(self_rank_dtype: Tuple[int, int]) -> int:`
|
||||||
|
|
||||||
|
@ -39,10 +39,10 @@ We will use the example of adding support for the `torch.aten.tanh` op.
|
||||||
But in general, you will need to write the function and test it
|
But in general, you will need to write the function and test it
|
||||||
(see the comments about "Shape, dtype, and decomposition function
|
(see the comments about "Shape, dtype, and decomposition function
|
||||||
testing infrastructure" in `testing_framework.py`). New shape
|
testing infrastructure" in `testing_framework.py`). New shape
|
||||||
functions should be added upstream following the example of [this PR](https://github.com/pytorch/pytorch/pull/76889),
|
functions should be added upstream following the example of [this PR](https://github.com/pytorch/pytorch/pull/76889),
|
||||||
though it can be useful to iterate locally in `abstract_interp_lib_gen.py`
|
though it can be useful to iterate locally in `abstract_interp_lib_gen.py`
|
||||||
first.
|
first.
|
||||||
|
|
||||||
Similarly, dtype functions should ideally just be a call to the helper
|
Similarly, dtype functions should ideally just be a call to the helper
|
||||||
`promote_dtypes` defined in `library_generator.py`. However, some ops will
|
`promote_dtypes` defined in `library_generator.py`. However, some ops will
|
||||||
require some extra logic to calculate the right result types. While dtypes
|
require some extra logic to calculate the right result types. While dtypes
|
||||||
|
|
|
@ -442,5 +442,5 @@ characteristics.
|
||||||
|
|
||||||
### Presentations and Talks
|
### Presentations and Talks
|
||||||
|
|
||||||
* 2021-10-07: MLIR ODM: Introduction to Torch-MLIR. ([recording](https://www.youtube.com/watch?v=QbNkex-gizs) and [slides](https://docs.google.com/presentation/d/1ZhzfE4EK6XV7AdQTYicrsE_OYjkER_yiB0vBeszRfzY/edit#slide=id.gf56404f79c_1_55))
|
* 2021-10-07: MLIR ODM: Introduction to Torch-MLIR. ([recording](https://www.youtube.com/watch?v=QbNkex-gizs) and [slides](https://docs.google.com/presentation/d/1ZhzfE4EK6XV7AdQTYicrsE_OYjkER_yiB0vBeszRfzY/edit#slide=id.gf56404f79c_1_55))
|
||||||
* 2022-08-20: Overview of Torch-MLIR passes. ([recording](https://www.youtube.com/watch?v=ZpwlVxsD9_U) and [slides](https://drive.google.com/file/d/1ZSlk1HGttRuVhJSxtP6spWt_hxClit2T/view))
|
* 2022-08-20: Overview of Torch-MLIR passes. ([recording](https://www.youtube.com/watch?v=ZpwlVxsD9_U) and [slides](https://drive.google.com/file/d/1ZSlk1HGttRuVhJSxtP6spWt_hxClit2T/view))
|
||||||
|
|
|
@ -11,8 +11,8 @@ for the reference importer which complies with the rules below.
|
||||||
With the exception of certain special or complicated ONNX operators, most
|
With the exception of certain special or complicated ONNX operators, most
|
||||||
are relatively straight-forward to map, following this general procedure:
|
are relatively straight-forward to map, following this general procedure:
|
||||||
|
|
||||||
* Plan the ops you wish to support by consulting the
|
* Plan the ops you wish to support by consulting the
|
||||||
[ONNX operator database](https://onnx.ai/onnx/operators/).
|
[ONNX operator database](https://onnx.ai/onnx/operators/).
|
||||||
* This database has detailed diffs wrt different support versions but
|
* This database has detailed diffs wrt different support versions but
|
||||||
at the level of detail we operate, most version diffs are inconsequential
|
at the level of detail we operate, most version diffs are inconsequential
|
||||||
and just require a bit more pattern support.
|
and just require a bit more pattern support.
|
||||||
|
@ -24,7 +24,7 @@ are relatively straight-forward to map, following this general procedure:
|
||||||
corresponding with the alphabetic sort of the op and add a conversion.
|
corresponding with the alphabetic sort of the op and add a conversion.
|
||||||
* Generate successful test cases:
|
* Generate successful test cases:
|
||||||
* All `onnx_importer.py` tests are dumped to the test temp dir (success
|
* All `onnx_importer.py` tests are dumped to the test temp dir (success
|
||||||
or failure). This is typically located under
|
or failure). This is typically located under
|
||||||
`tools/torch-mlir/test/python/onnx_importer/Output`. The `.mlir` files
|
`tools/torch-mlir/test/python/onnx_importer/Output`. The `.mlir` files
|
||||||
under there should provide good variants to drive lit test coverage of
|
under there should provide good variants to drive lit test coverage of
|
||||||
conversion.
|
conversion.
|
||||||
|
@ -34,25 +34,25 @@ are relatively straight-forward to map, following this general procedure:
|
||||||
* There are often many variants of tests for checking conformance of
|
* There are often many variants of tests for checking conformance of
|
||||||
different historic ONNX encodings, but these are often not load bearing
|
different historic ONNX encodings, but these are often not load bearing
|
||||||
at the MLIR level.
|
at the MLIR level.
|
||||||
* Pick a handful of test cases and add them to
|
* Pick a handful of test cases and add them to
|
||||||
`test/Conversion/TorchOnnxToTorch/simple_ops_x_to_y.mlir` corresponding to
|
`test/Conversion/TorchOnnxToTorch/simple_ops_x_to_y.mlir` corresponding to
|
||||||
an alphabetic breakdown. At this time, ignore tests that are not exercising
|
an alphabetic breakdown. At this time, ignore tests that are not exercising
|
||||||
useful differences in the pattern implementations.
|
useful differences in the pattern implementations.
|
||||||
* (Optionally) Use `torch-mlir-opt` to validate the outputs of the new op.
|
* (Optionally) Use `torch-mlir-opt` to validate the outputs of the new op.
|
||||||
First, build the project using
|
First, build the project using
|
||||||
`cmake --build build --target tools/torch-mlir/all`. This will generate
|
`cmake --build build --target tools/torch-mlir/all`. This will generate
|
||||||
the conversion binary, `torch-mlir-opt`. Then call `torch-mlir-opt` with
|
the conversion binary, `torch-mlir-opt`. Then call `torch-mlir-opt` with
|
||||||
the MLIR pass `convert-torch-onnx-to-torch`:
|
the MLIR pass `convert-torch-onnx-to-torch`:
|
||||||
```
|
```
|
||||||
build/bin/torch-mlir-opt -convert-torch-onnx-to-torch \
|
build/bin/torch-mlir-opt -convert-torch-onnx-to-torch \
|
||||||
-split-input-file [DESIRED_ONNX_FILE].mlir
|
-split-input-file [DESIRED_ONNX_FILE].mlir
|
||||||
```
|
```
|
||||||
* Generate failure test cases:
|
* Generate failure test cases:
|
||||||
* Some ops have forms that do not (easily) map to torch-mlir. If you leave
|
* Some ops have forms that do not (easily) map to torch-mlir. If you leave
|
||||||
an op under-implemented, add a failing test case to
|
an op under-implemented, add a failing test case to
|
||||||
`test/Conversion/TorchOnnxToTorch/unsupported_simple_ops.mlir`.
|
`test/Conversion/TorchOnnxToTorch/unsupported_simple_ops.mlir`.
|
||||||
* Optional but recommended: Use your test case files to fuzz against the
|
* Optional but recommended: Use your test case files to fuzz against the
|
||||||
torch-mlir backend of your choice by running a backend conversion pipeline
|
torch-mlir backend of your choice by running a backend conversion pipeline
|
||||||
and fixing any crashes/issues.
|
and fixing any crashes/issues.
|
||||||
* Send a patch with your changes.
|
* Send a patch with your changes.
|
||||||
|
|
||||||
|
@ -115,7 +115,7 @@ not yet implemented.
|
||||||
The `IsolatedFromAbove` parent of the ops can contain the following
|
The `IsolatedFromAbove` parent of the ops can contain the following
|
||||||
metadata:
|
metadata:
|
||||||
|
|
||||||
* `torch.onnx_meta.ir_version`: 64bit `IntegerAttr` corresponding to
|
* `torch.onnx_meta.ir_version`: 64bit `IntegerAttr` corresponding to
|
||||||
`ModelProto.ir_version`.
|
`ModelProto.ir_version`.
|
||||||
* `torch.onnx_meta.producer_name`: `StringAttr` corresponding to
|
* `torch.onnx_meta.producer_name`: `StringAttr` corresponding to
|
||||||
`ModelProto.producer_name`.
|
`ModelProto.producer_name`.
|
||||||
|
@ -135,7 +135,7 @@ are only minor variations of an op. Major variations should use
|
||||||
|
|
||||||
### Special op forms
|
### Special op forms
|
||||||
|
|
||||||
Certain ONNX operators map to different structural components of
|
Certain ONNX operators map to different structural components of
|
||||||
torch-mlir's representation:
|
torch-mlir's representation:
|
||||||
|
|
||||||
* `ConstantOfShape`: Mapped to `torch.vtensor.literal` with
|
* `ConstantOfShape`: Mapped to `torch.vtensor.literal` with
|
||||||
|
|
|
@ -103,7 +103,7 @@ At some point, the tensors will be synced in order to execute the computation --
|
||||||
>>> torch._lazy.mark_step()
|
>>> torch._lazy.mark_step()
|
||||||
```
|
```
|
||||||
|
|
||||||
This triggers a call to `LazyGraphExecutor::SyncLiveTensorsGraph` somewhere in the guts of LTC, which collects all the `TorchMlirNode`s (technically `torch::lazy::Node`s at this point) from the current trace and
|
This triggers a call to `LazyGraphExecutor::SyncLiveTensorsGraph` somewhere in the guts of LTC, which collects all the `TorchMlirNode`s (technically `torch::lazy::Node`s at this point) from the current trace and
|
||||||
creates an instance of `TorchMlirLoweringContext`. Here, the `TorchMlirNode`s are lowered to JIT via `mlir_node_lowering.cpp` and inserted into a `jit::Graph`.
|
creates an instance of `TorchMlirLoweringContext`. Here, the `TorchMlirNode`s are lowered to JIT via `mlir_node_lowering.cpp` and inserted into a `jit::Graph`.
|
||||||
|
|
||||||
Next, `TorchMlirLoweringContext::Build` is executed and the final `jit::Graph` is sent to `torch_mlir::importJitFunctionAsFuncOp` to generate MLIR using the existing infrastructure from Torch-MLIR.
|
Next, `TorchMlirLoweringContext::Build` is executed and the final `jit::Graph` is sent to `torch_mlir::importJitFunctionAsFuncOp` to generate MLIR using the existing infrastructure from Torch-MLIR.
|
||||||
|
@ -121,7 +121,7 @@ Finally, the compiled computation is sent to `TorchMlirBackendImpl::ExecuteCompu
|
||||||
|
|
||||||
## Implementing a custom backend
|
## Implementing a custom backend
|
||||||
|
|
||||||
A reference implementation of a custom backend is available [here](../python/torch_mlir/csrc/reference_lazy_backend/).
|
A reference implementation of a custom backend is available [here](../python/torch_mlir/csrc/reference_lazy_backend/).
|
||||||
All the work involved with generating MLIR is handled in the base LTC backend, so vendors only need to worry about implementing `Compile`, `ExecuteComputation`, and some other minor methods to interface with the device.
|
All the work involved with generating MLIR is handled in the base LTC backend, so vendors only need to worry about implementing `Compile`, `ExecuteComputation`, and some other minor methods to interface with the device.
|
||||||
|
|
||||||
A pybind is needed to invoke C++ code to register the autogen PyTorch kernels and the custom backend itself.
|
A pybind is needed to invoke C++ code to register the autogen PyTorch kernels and the custom backend itself.
|
||||||
|
|
|
@ -33,18 +33,18 @@ Received 1 arguments, and returned 2 results during ExecuteCompile!
|
||||||
|
|
||||||
Results: tensor([[0.7616, 0.9640, 0.9951, 0.9993, 0.9999]], device='lazy:0')
|
Results: tensor([[0.7616, 0.9640, 0.9951, 0.9993, 0.9999]], device='lazy:0')
|
||||||
|
|
||||||
JIT Graph:
|
JIT Graph:
|
||||||
graph(%p0 : Float(1, 5)):
|
graph(%p0 : Float(1, 5)):
|
||||||
%1 : Float(1, 5) = aten::tanh(%p0)
|
%1 : Float(1, 5) = aten::tanh(%p0)
|
||||||
return (%p0, %1)
|
return (%p0, %1)
|
||||||
|
|
||||||
MLIR:
|
MLIR:
|
||||||
func.func @graph(%arg0: !torch.vtensor<[1,5],f32>) -> (!torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>) {
|
func.func @graph(%arg0: !torch.vtensor<[1,5],f32>) -> (!torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>) {
|
||||||
%0 = torch.aten.tanh %arg0 : !torch.vtensor<[1,5],f32> -> !torch.vtensor<[1,5],f32>
|
%0 = torch.aten.tanh %arg0 : !torch.vtensor<[1,5],f32> -> !torch.vtensor<[1,5],f32>
|
||||||
return %arg0, %0 : !torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>
|
return %arg0, %0 : !torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>
|
||||||
}
|
}
|
||||||
|
|
||||||
Input/Output Alias Mapping:
|
Input/Output Alias Mapping:
|
||||||
Output: 0 -> Input param: 0
|
Output: 0 -> Input param: 0
|
||||||
|
|
||||||
In Mark Step: true
|
In Mark Step: true
|
||||||
|
|
Loading…
Reference in New Issue