[torch-mlir] remove trailing whitespace from md documentation (#2853)

pull/2856/head
Aart Bik 2024-02-02 11:02:53 -08:00 committed by GitHub
parent 24b8c8672a
commit d1cd117998
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 26 additions and 26 deletions

View File

@ -1,17 +1,17 @@
# How to Add Ops to Torch-Mlir # How to Add Ops to Torch-Mlir
Collected links and contacts for how to add ops to torch-mlir. Collected links and contacts for how to add ops to torch-mlir.
<details> <details>
<summary>Turbine Camp: Start Here</summary> <summary>Turbine Camp: Start Here</summary>
This document was previously known as `turbine-camp.md` to Nod.ai. "Turbine Camp" is part of Nod.ai's onboarding process. Welcome to turbine camp. This document originated at Nod.ai as a part of onboardding process, where new nod-ai folks learn about the architecture of our work by adding support for 2 ops to torch-mlir. I decided to put this into torch mlir because a lot of this is about torch-mlir. This document was previously known as `turbine-camp.md` to Nod.ai. "Turbine Camp" is part of Nod.ai's onboarding process. Welcome to turbine camp. This document originated at Nod.ai as a part of onboardding process, where new nod-ai folks learn about the architecture of our work by adding support for 2 ops to torch-mlir. I decided to put this into torch mlir because a lot of this is about torch-mlir.
Written & maintained by @renxida Written & maintained by @renxida
Guides by other folks that were used during the creation of this document: Guides by other folks that were used during the creation of this document:
- [Chi Liu](https://gist.github.com/AmosLewis/dd31ab37517977b1c499d06495b4adc2) - [Chi Liu](https://gist.github.com/AmosLewis/dd31ab37517977b1c499d06495b4adc2)
- [Sunsoon](https://docs.google.com/document/d/1H79DwW_wnVzUU81EogwY5ueXgnl-QzKet1p2lnqPar4/edit?pli=1) - [Sunsoon](https://docs.google.com/document/d/1H79DwW_wnVzUU81EogwY5ueXgnl-QzKet1p2lnqPar4/edit?pli=1)
## Before you begin... ## Before you begin...

View File

@ -4,7 +4,7 @@
As part of adding support for a Torch operator in Torch-MLIR, it is usually As part of adding support for a Torch operator in Torch-MLIR, it is usually
necessary to define a shape and dtype function so that the compiler can infer necessary to define a shape and dtype function so that the compiler can infer
the shapes and dtypes of result tensors for the operator. We use the the shapes and dtypes of result tensors for the operator. We use the
[abstract interpretation library](abstract_interp_lib.md) for this process. [abstract interpretation library](abstract_interp_lib.md) for this process.
## Step-by-step guide ## Step-by-step guide
@ -19,7 +19,7 @@ We will use the example of adding support for the `torch.aten.tanh` op.
file is the "rosetta stone" that allows translating between file is the "rosetta stone" that allows translating between
e.g. `torch.aten.tanh`, `AtenTanhOp`, and the shape and dtype e.g. `torch.aten.tanh`, `AtenTanhOp`, and the shape and dtype
function signatures are: function signatures are:
- `def atentanh〡shape(self: List[int]) -> List[int]:` - `def atentanh〡shape(self: List[int]) -> List[int]:`
- `def atentanh〡dtype(self_rank_dtype: Tuple[int, int]) -> int:` - `def atentanh〡dtype(self_rank_dtype: Tuple[int, int]) -> int:`
@ -39,10 +39,10 @@ We will use the example of adding support for the `torch.aten.tanh` op.
But in general, you will need to write the function and test it But in general, you will need to write the function and test it
(see the comments about "Shape, dtype, and decomposition function (see the comments about "Shape, dtype, and decomposition function
testing infrastructure" in `testing_framework.py`). New shape testing infrastructure" in `testing_framework.py`). New shape
functions should be added upstream following the example of [this PR](https://github.com/pytorch/pytorch/pull/76889), functions should be added upstream following the example of [this PR](https://github.com/pytorch/pytorch/pull/76889),
though it can be useful to iterate locally in `abstract_interp_lib_gen.py` though it can be useful to iterate locally in `abstract_interp_lib_gen.py`
first. first.
Similarly, dtype functions should ideally just be a call to the helper Similarly, dtype functions should ideally just be a call to the helper
`promote_dtypes` defined in `library_generator.py`. However, some ops will `promote_dtypes` defined in `library_generator.py`. However, some ops will
require some extra logic to calculate the right result types. While dtypes require some extra logic to calculate the right result types. While dtypes

View File

@ -442,5 +442,5 @@ characteristics.
### Presentations and Talks ### Presentations and Talks
* 2021-10-07: MLIR ODM: Introduction to Torch-MLIR. ([recording](https://www.youtube.com/watch?v=QbNkex-gizs) and [slides](https://docs.google.com/presentation/d/1ZhzfE4EK6XV7AdQTYicrsE_OYjkER_yiB0vBeszRfzY/edit#slide=id.gf56404f79c_1_55)) * 2021-10-07: MLIR ODM: Introduction to Torch-MLIR. ([recording](https://www.youtube.com/watch?v=QbNkex-gizs) and [slides](https://docs.google.com/presentation/d/1ZhzfE4EK6XV7AdQTYicrsE_OYjkER_yiB0vBeszRfzY/edit#slide=id.gf56404f79c_1_55))
* 2022-08-20: Overview of Torch-MLIR passes. ([recording](https://www.youtube.com/watch?v=ZpwlVxsD9_U) and [slides](https://drive.google.com/file/d/1ZSlk1HGttRuVhJSxtP6spWt_hxClit2T/view)) * 2022-08-20: Overview of Torch-MLIR passes. ([recording](https://www.youtube.com/watch?v=ZpwlVxsD9_U) and [slides](https://drive.google.com/file/d/1ZSlk1HGttRuVhJSxtP6spWt_hxClit2T/view))

View File

@ -11,8 +11,8 @@ for the reference importer which complies with the rules below.
With the exception of certain special or complicated ONNX operators, most With the exception of certain special or complicated ONNX operators, most
are relatively straight-forward to map, following this general procedure: are relatively straight-forward to map, following this general procedure:
* Plan the ops you wish to support by consulting the * Plan the ops you wish to support by consulting the
[ONNX operator database](https://onnx.ai/onnx/operators/). [ONNX operator database](https://onnx.ai/onnx/operators/).
* This database has detailed diffs wrt different support versions but * This database has detailed diffs wrt different support versions but
at the level of detail we operate, most version diffs are inconsequential at the level of detail we operate, most version diffs are inconsequential
and just require a bit more pattern support. and just require a bit more pattern support.
@ -24,7 +24,7 @@ are relatively straight-forward to map, following this general procedure:
corresponding with the alphabetic sort of the op and add a conversion. corresponding with the alphabetic sort of the op and add a conversion.
* Generate successful test cases: * Generate successful test cases:
* All `onnx_importer.py` tests are dumped to the test temp dir (success * All `onnx_importer.py` tests are dumped to the test temp dir (success
or failure). This is typically located under or failure). This is typically located under
`tools/torch-mlir/test/python/onnx_importer/Output`. The `.mlir` files `tools/torch-mlir/test/python/onnx_importer/Output`. The `.mlir` files
under there should provide good variants to drive lit test coverage of under there should provide good variants to drive lit test coverage of
conversion. conversion.
@ -34,25 +34,25 @@ are relatively straight-forward to map, following this general procedure:
* There are often many variants of tests for checking conformance of * There are often many variants of tests for checking conformance of
different historic ONNX encodings, but these are often not load bearing different historic ONNX encodings, but these are often not load bearing
at the MLIR level. at the MLIR level.
* Pick a handful of test cases and add them to * Pick a handful of test cases and add them to
`test/Conversion/TorchOnnxToTorch/simple_ops_x_to_y.mlir` corresponding to `test/Conversion/TorchOnnxToTorch/simple_ops_x_to_y.mlir` corresponding to
an alphabetic breakdown. At this time, ignore tests that are not exercising an alphabetic breakdown. At this time, ignore tests that are not exercising
useful differences in the pattern implementations. useful differences in the pattern implementations.
* (Optionally) Use `torch-mlir-opt` to validate the outputs of the new op. * (Optionally) Use `torch-mlir-opt` to validate the outputs of the new op.
First, build the project using First, build the project using
`cmake --build build --target tools/torch-mlir/all`. This will generate `cmake --build build --target tools/torch-mlir/all`. This will generate
the conversion binary, `torch-mlir-opt`. Then call `torch-mlir-opt` with the conversion binary, `torch-mlir-opt`. Then call `torch-mlir-opt` with
the MLIR pass `convert-torch-onnx-to-torch`: the MLIR pass `convert-torch-onnx-to-torch`:
``` ```
build/bin/torch-mlir-opt -convert-torch-onnx-to-torch \ build/bin/torch-mlir-opt -convert-torch-onnx-to-torch \
-split-input-file [DESIRED_ONNX_FILE].mlir -split-input-file [DESIRED_ONNX_FILE].mlir
``` ```
* Generate failure test cases: * Generate failure test cases:
* Some ops have forms that do not (easily) map to torch-mlir. If you leave * Some ops have forms that do not (easily) map to torch-mlir. If you leave
an op under-implemented, add a failing test case to an op under-implemented, add a failing test case to
`test/Conversion/TorchOnnxToTorch/unsupported_simple_ops.mlir`. `test/Conversion/TorchOnnxToTorch/unsupported_simple_ops.mlir`.
* Optional but recommended: Use your test case files to fuzz against the * Optional but recommended: Use your test case files to fuzz against the
torch-mlir backend of your choice by running a backend conversion pipeline torch-mlir backend of your choice by running a backend conversion pipeline
and fixing any crashes/issues. and fixing any crashes/issues.
* Send a patch with your changes. * Send a patch with your changes.
@ -115,7 +115,7 @@ not yet implemented.
The `IsolatedFromAbove` parent of the ops can contain the following The `IsolatedFromAbove` parent of the ops can contain the following
metadata: metadata:
* `torch.onnx_meta.ir_version`: 64bit `IntegerAttr` corresponding to * `torch.onnx_meta.ir_version`: 64bit `IntegerAttr` corresponding to
`ModelProto.ir_version`. `ModelProto.ir_version`.
* `torch.onnx_meta.producer_name`: `StringAttr` corresponding to * `torch.onnx_meta.producer_name`: `StringAttr` corresponding to
`ModelProto.producer_name`. `ModelProto.producer_name`.
@ -135,7 +135,7 @@ are only minor variations of an op. Major variations should use
### Special op forms ### Special op forms
Certain ONNX operators map to different structural components of Certain ONNX operators map to different structural components of
torch-mlir's representation: torch-mlir's representation:
* `ConstantOfShape`: Mapped to `torch.vtensor.literal` with * `ConstantOfShape`: Mapped to `torch.vtensor.literal` with

View File

@ -103,7 +103,7 @@ At some point, the tensors will be synced in order to execute the computation --
>>> torch._lazy.mark_step() >>> torch._lazy.mark_step()
``` ```
This triggers a call to `LazyGraphExecutor::SyncLiveTensorsGraph` somewhere in the guts of LTC, which collects all the `TorchMlirNode`s (technically `torch::lazy::Node`s at this point) from the current trace and This triggers a call to `LazyGraphExecutor::SyncLiveTensorsGraph` somewhere in the guts of LTC, which collects all the `TorchMlirNode`s (technically `torch::lazy::Node`s at this point) from the current trace and
creates an instance of `TorchMlirLoweringContext`. Here, the `TorchMlirNode`s are lowered to JIT via `mlir_node_lowering.cpp` and inserted into a `jit::Graph`. creates an instance of `TorchMlirLoweringContext`. Here, the `TorchMlirNode`s are lowered to JIT via `mlir_node_lowering.cpp` and inserted into a `jit::Graph`.
Next, `TorchMlirLoweringContext::Build` is executed and the final `jit::Graph` is sent to `torch_mlir::importJitFunctionAsFuncOp` to generate MLIR using the existing infrastructure from Torch-MLIR. Next, `TorchMlirLoweringContext::Build` is executed and the final `jit::Graph` is sent to `torch_mlir::importJitFunctionAsFuncOp` to generate MLIR using the existing infrastructure from Torch-MLIR.
@ -121,7 +121,7 @@ Finally, the compiled computation is sent to `TorchMlirBackendImpl::ExecuteCompu
## Implementing a custom backend ## Implementing a custom backend
A reference implementation of a custom backend is available [here](../python/torch_mlir/csrc/reference_lazy_backend/). A reference implementation of a custom backend is available [here](../python/torch_mlir/csrc/reference_lazy_backend/).
All the work involved with generating MLIR is handled in the base LTC backend, so vendors only need to worry about implementing `Compile`, `ExecuteComputation`, and some other minor methods to interface with the device. All the work involved with generating MLIR is handled in the base LTC backend, so vendors only need to worry about implementing `Compile`, `ExecuteComputation`, and some other minor methods to interface with the device.
A pybind is needed to invoke C++ code to register the autogen PyTorch kernels and the custom backend itself. A pybind is needed to invoke C++ code to register the autogen PyTorch kernels and the custom backend itself.

View File

@ -33,18 +33,18 @@ Received 1 arguments, and returned 2 results during ExecuteCompile!
Results: tensor([[0.7616, 0.9640, 0.9951, 0.9993, 0.9999]], device='lazy:0') Results: tensor([[0.7616, 0.9640, 0.9951, 0.9993, 0.9999]], device='lazy:0')
JIT Graph: JIT Graph:
graph(%p0 : Float(1, 5)): graph(%p0 : Float(1, 5)):
%1 : Float(1, 5) = aten::tanh(%p0) %1 : Float(1, 5) = aten::tanh(%p0)
return (%p0, %1) return (%p0, %1)
MLIR: MLIR:
func.func @graph(%arg0: !torch.vtensor<[1,5],f32>) -> (!torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>) { func.func @graph(%arg0: !torch.vtensor<[1,5],f32>) -> (!torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>) {
%0 = torch.aten.tanh %arg0 : !torch.vtensor<[1,5],f32> -> !torch.vtensor<[1,5],f32> %0 = torch.aten.tanh %arg0 : !torch.vtensor<[1,5],f32> -> !torch.vtensor<[1,5],f32>
return %arg0, %0 : !torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32> return %arg0, %0 : !torch.vtensor<[1,5],f32>, !torch.vtensor<[1,5],f32>
} }
Input/Output Alias Mapping: Input/Output Alias Mapping:
Output: 0 -> Input param: 0 Output: 0 -> Input param: 0
In Mark Step: true In Mark Step: true