llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/test/Dialect/Torch/reduce-op-variants.mlir

206 lines
15 KiB
MLIR
Raw Normal View History

[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// RUN: torch-mlir-opt -torch-reduce-op-variants --split-input-file %s | FileCheck %s
Significantly restructure torch/aten import design. This is a really major and invasive restructuring of the way we get torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into MLIR. Please forgive the challenging review, but due to the sheer invasiveness, it wasn't really practical do do it in sane smaller pieces. This fully replaces everything that was already working on the TorchScript path (actually, more -- we added tanh support to TorchToLinalg in order to delete the older code paths). Additionally, I've kept the lights on for the acap path too, including what little e2e stuff was working before (for expediency I made a few tiny compromises along the way that will be easy to undo when we give that path proper attention). Overview of the new design: - The torch operator `somens::someunqualname.someoverloadname` is imported as `torch.somens.someunqualname.someoverloadname` (skip the last dotted part if the overload name is empty), OR, if we don't have such an op registered, it is imported as `torch.operator "somens.someunqualname.someoverloadname" (...) : ...`. - The addition of the "overload name" is a critical element here, as the `(ns,unqual,overload)` triple is unique, which solves a lot of problems we were having. - This involves having separate MLIR ops for the `trailing_` and `.out` variants and all the different overloads. This seemed necessary, because the set of overloads is so wild and varied and unstructured. The previous design was leaning into some underlying structure that just isn't there -- the default situation is the "random overload that we want to manage on the MLIR side", rather than that being an exception. E.g. `aten::ne` (not-equal) has 21 overloads, only 4 of which are c10 dispatcher ops see [gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1), and the "out" variant is really called `.Tensor_out` instead of `.out` as it frequently is for other ops. - Rationale for all being in `torch` namespace: the set of operators are so varied and unstructured that "dialect per namespace" doesn't result in anything resembling the typical MLIR dialect boundary expectations. We could maybe draw the boundary at dispatcher ops vs non-dispatcher ops, but that doesn't seem to really result in very much useful structure at this point in time. - Note: within the torch operator registry, we effectively have a mini-basicpy subdialect (already type-resolved), which is reasonably structured. - The existing Torch op interfaces are also removed -- now that we track the overload name, we can losslessly find the original operator. - Instead of `ATenRecognizeKernelsPass`, we now have a `ReduceOpVariantsPass` that keys off certain traits (and perhaps eventually interfaces) to reduce variants of ops to a smaller set, ideally operating on immutable tensors and using surrounding ops to model the mutability/aliasing aspects. - Note: `torch.ns.unqual.overload` ops allow both immutable and mutable tensors (unlike the previous hard distinction in the common case). This is a premonition for a future change that will introduce a bona fide `!torch.tensor` type that will clean up a bunch of stuff. - `TorchToLinalg` / `TorchToStd` supercede the existing "ATen->TCF->TCP->Linalg" path. - The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`. It should look somewhat familiar, but the benefit of hindsight has allowed a lot of simplifications. The overall trend seems to be to make the `torch` dialect a nice layer independent of anything else. It feels like as a natural result of various future changes we will be removing the reliance on basicpy+numpy dialects and have a nice self-contained type system too that properly models the TorchScript type system (including proper subtyping, mutable/immutable tensors, optional dtype, etc.). Recommended review order: - Start at some of the new import IR, e.g. in `frontends/pytorch/test/node_import/prim.py`, `frontends/pytorch/test/acap_export/test_export_add3.py`, and other tests. - `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py` and associated generated files: - `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td` - `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td` - Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h` - Various code changes in the import path in `frontends/pytorch/csrc/builder`. Probably most interesting is the new code in `torch_to_mlir_utils.cpp` that has the logic to create the `torch.operator` ops or `torch.ns.unqual.overload` ops. This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe), just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @convert_to_value_semantic_tensors(
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// CHECK-SAME: %[[ARG:.*]]: !torch.tensor<[],f32>) -> !torch.tensor<[],f32> {
Make MaximizeValueSemantics a bit smarter. This adds a pattern to MaximizeValueSemantics which does a simple abstract interpretation within a block, which handles simple cases of `torch.overwrite_tensor`, enough to remove all the unnecessary uses of non-value tensors in ResNet right now. Before/after IR: [gist](https://gist.github.com/silvasean/a3e1ef625b19dfc63579f73cd3b543b6) Also, - Split `torch.copy.tensor` into `torch.copy.to_tensor` and `torch.copy.to_vtensor` which convert between value and non-value semantic tensors. This is a much cleaner factorization as they have very separate use cases and properties (e.g. different side effects) - Remove the various canonicalization patterns they had, which were confusing because they resulted in limited forms of maximizing value semantics throughout the pipeline. We should structure our compilation pipeline such that only MaximizeValueSemantics should be maximizing value semantics. - Adjust pass pipeline to only run MaximizeValueSemantics once. - Make OverwriteTensorOp `$value` always be a value tensor and `$overwritten` be a non-value tensor.
2021-06-19 04:47:47 +08:00
// CHECK: %[[OPERAND_TENSOR:.*]] = torch.copy.to_vtensor %[[ARG]] : !torch.vtensor<[],f32>
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// CHECK: %[[RESULT_TENSOR:.*]] = torch.aten.tanh %[[OPERAND_TENSOR]] : !torch.vtensor<[],f32> -> !torch.vtensor<[],f32>
Make MaximizeValueSemantics a bit smarter. This adds a pattern to MaximizeValueSemantics which does a simple abstract interpretation within a block, which handles simple cases of `torch.overwrite_tensor`, enough to remove all the unnecessary uses of non-value tensors in ResNet right now. Before/after IR: [gist](https://gist.github.com/silvasean/a3e1ef625b19dfc63579f73cd3b543b6) Also, - Split `torch.copy.tensor` into `torch.copy.to_tensor` and `torch.copy.to_vtensor` which convert between value and non-value semantic tensors. This is a much cleaner factorization as they have very separate use cases and properties (e.g. different side effects) - Remove the various canonicalization patterns they had, which were confusing because they resulted in limited forms of maximizing value semantics throughout the pipeline. We should structure our compilation pipeline such that only MaximizeValueSemantics should be maximizing value semantics. - Adjust pass pipeline to only run MaximizeValueSemantics once. - Make OverwriteTensorOp `$value` always be a value tensor and `$overwritten` be a non-value tensor.
2021-06-19 04:47:47 +08:00
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[RESULT_TENSOR]] : !torch.tensor<[],f32>
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// CHECK: return %[[RET]] : !torch.tensor<[],f32>
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @convert_to_value_semantic_tensors(%arg0: !torch.tensor<[],f32>) -> !torch.tensor<[],f32> {
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
%0 = torch.aten.tanh %arg0 : !torch.tensor<[],f32> -> !torch.tensor<[],f32>
return %0 : !torch.tensor<[],f32>
Significantly restructure torch/aten import design. This is a really major and invasive restructuring of the way we get torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into MLIR. Please forgive the challenging review, but due to the sheer invasiveness, it wasn't really practical do do it in sane smaller pieces. This fully replaces everything that was already working on the TorchScript path (actually, more -- we added tanh support to TorchToLinalg in order to delete the older code paths). Additionally, I've kept the lights on for the acap path too, including what little e2e stuff was working before (for expediency I made a few tiny compromises along the way that will be easy to undo when we give that path proper attention). Overview of the new design: - The torch operator `somens::someunqualname.someoverloadname` is imported as `torch.somens.someunqualname.someoverloadname` (skip the last dotted part if the overload name is empty), OR, if we don't have such an op registered, it is imported as `torch.operator "somens.someunqualname.someoverloadname" (...) : ...`. - The addition of the "overload name" is a critical element here, as the `(ns,unqual,overload)` triple is unique, which solves a lot of problems we were having. - This involves having separate MLIR ops for the `trailing_` and `.out` variants and all the different overloads. This seemed necessary, because the set of overloads is so wild and varied and unstructured. The previous design was leaning into some underlying structure that just isn't there -- the default situation is the "random overload that we want to manage on the MLIR side", rather than that being an exception. E.g. `aten::ne` (not-equal) has 21 overloads, only 4 of which are c10 dispatcher ops see [gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1), and the "out" variant is really called `.Tensor_out` instead of `.out` as it frequently is for other ops. - Rationale for all being in `torch` namespace: the set of operators are so varied and unstructured that "dialect per namespace" doesn't result in anything resembling the typical MLIR dialect boundary expectations. We could maybe draw the boundary at dispatcher ops vs non-dispatcher ops, but that doesn't seem to really result in very much useful structure at this point in time. - Note: within the torch operator registry, we effectively have a mini-basicpy subdialect (already type-resolved), which is reasonably structured. - The existing Torch op interfaces are also removed -- now that we track the overload name, we can losslessly find the original operator. - Instead of `ATenRecognizeKernelsPass`, we now have a `ReduceOpVariantsPass` that keys off certain traits (and perhaps eventually interfaces) to reduce variants of ops to a smaller set, ideally operating on immutable tensors and using surrounding ops to model the mutability/aliasing aspects. - Note: `torch.ns.unqual.overload` ops allow both immutable and mutable tensors (unlike the previous hard distinction in the common case). This is a premonition for a future change that will introduce a bona fide `!torch.tensor` type that will clean up a bunch of stuff. - `TorchToLinalg` / `TorchToStd` supercede the existing "ATen->TCF->TCP->Linalg" path. - The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`. It should look somewhat familiar, but the benefit of hindsight has allowed a lot of simplifications. The overall trend seems to be to make the `torch` dialect a nice layer independent of anything else. It feels like as a natural result of various future changes we will be removing the reliance on basicpy+numpy dialects and have a nice self-contained type system too that properly models the TorchScript type system (including proper subtyping, mutable/immutable tensors, optional dtype, etc.). Recommended review order: - Start at some of the new import IR, e.g. in `frontends/pytorch/test/node_import/prim.py`, `frontends/pytorch/test/acap_export/test_export_add3.py`, and other tests. - `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py` and associated generated files: - `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td` - `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td` - Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h` - Various code changes in the import path in `frontends/pytorch/csrc/builder`. Probably most interesting is the new code in `torch_to_mlir_utils.cpp` that has the logic to create the `torch.operator` ops or `torch.ns.unqual.overload` ops. This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe), just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
}
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @convert_to_value_semantic_tensors_list(
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK-SAME: %[[VT0:.*]]: !torch.vtensor, %[[VT1:.*]]: !torch.vtensor,
// CHECK-SAME: %[[VT2:.*]]: !torch.vtensor) -> !torch.tensor {
// CHECK: %[[T0:.*]] = torch.copy.to_tensor %[[VT0]] : !torch.tensor
// CHECK: %[[T1:.*]] = torch.copy.to_tensor %[[VT1]] : !torch.tensor
// CHECK: %[[T2:.*]] = torch.copy.to_tensor %[[VT2]] : !torch.tensor
// CHECK: %[[DIM:.*]] = torch.constant.int 1
// CHECK: %[[LIST_ORIG:.*]] = torch.prim.ListConstruct %[[T0]], %[[T1]], %[[T2]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: (!torch.tensor, !torch.tensor, !torch.tensor) -> !torch.list<tensor>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[VT0_COPY:.*]] = torch.copy.to_vtensor %[[T0]] : !torch.vtensor
// CHECK: %[[VT1_COPY:.*]] = torch.copy.to_vtensor %[[T1]] : !torch.vtensor
// CHECK: %[[VT2_COPY:.*]] = torch.copy.to_vtensor %[[T2]] : !torch.vtensor
// CHECK: %[[LIST_NEW:.*]] = torch.prim.ListConstruct
// CHECK-SAME: %[[VT0_COPY]], %[[VT1_COPY]], %[[VT2_COPY]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: (!torch.vtensor, !torch.vtensor, !torch.vtensor) -> !torch.list<vtensor>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[VRET:.*]] = torch.aten.cat %[[LIST_NEW]], %[[DIM]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.list<vtensor>, !torch.int -> !torch.vtensor
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[VRET]] : !torch.tensor
// CHECK: return %[[RET]] : !torch.tensor
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @convert_to_value_semantic_tensors_list(%vt0: !torch.vtensor, %vt1: !torch.vtensor, %vt2: !torch.vtensor) -> !torch.tensor {
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
%t0 = torch.copy.to_tensor %vt0 : !torch.tensor
%t1 = torch.copy.to_tensor %vt1 : !torch.tensor
%t2 = torch.copy.to_tensor %vt2 : !torch.tensor
%int1 = torch.constant.int 1
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
%list = torch.prim.ListConstruct %t0, %t1, %t2 : (!torch.tensor, !torch.tensor, !torch.tensor) -> !torch.list<tensor>
%ret = torch.aten.cat %list, %int1 : !torch.list<tensor>, !torch.int -> !torch.tensor
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
return %ret : !torch.tensor
}
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @convert_to_value_semantic_tensors_optional(
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK-SAME: %[[INPUT:.*]]: !torch.tensor, %[[FLOAT_TENSOR:.*]]: !torch.tensor<[4],f32>,
// CHECK-SAME: %[[TRAINING:.*]]: !torch.bool, %[[CUDNN_ENABLE:.*]]: !torch.bool,
// CHECK-SAME: %[[FLOAT:.*]]: !torch.float) -> !torch.tensor {
// CHECK: %[[NONE:.*]] = torch.constant.none
// CHECK: %[[FLOAT_TENSOR_OPTIONAL:.*]] = torch.derefine %[[FLOAT_TENSOR]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.tensor<[4],f32> to !torch.optional<tensor>
// CHECK: %[[BIAS_NONE_OPTIONAL:.*]] = torch.derefine %[[NONE]] : !torch.none to !torch.optional<tensor>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[VINPUT:.*]] = torch.copy.to_vtensor %[[INPUT]] : !torch.vtensor
// CHECK: %[[FLOAT_VTENSOR:.*]] = torch.copy.to_vtensor %[[FLOAT_TENSOR]] : !torch.vtensor<[4],f32>
// CHECK: %[[WEIGHTS_TENSOR_OPTIONAL:.*]] = torch.derefine %[[FLOAT_VTENSOR]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.vtensor<[4],f32> to !torch.optional<vtensor<[4],f32>>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[FLOAT_VTENSOR:.*]] = torch.copy.to_vtensor %[[FLOAT_TENSOR]] : !torch.vtensor<[4],f32>
// CHECK: %[[MEAN_VTENSOR_OPTIONAL:.*]] = torch.derefine %[[FLOAT_VTENSOR]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.vtensor<[4],f32> to !torch.optional<vtensor<[4],f32>>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[FLOAT_VTENSOR:.*]] = torch.copy.to_vtensor %[[FLOAT_TENSOR]] : !torch.vtensor<[4],f32>
// CHECK: %[[VAR_VTENSOR_OPTIONAL:.*]] = torch.derefine %[[FLOAT_VTENSOR]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.vtensor<[4],f32> to !torch.optional<vtensor<[4],f32>>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK: %[[VRET:.*]] = torch.aten.batch_norm %[[VINPUT]], %[[WEIGHTS_TENSOR_OPTIONAL]],
// CHECK-SAME: %[[BIAS_NONE_OPTIONAL]], %[[MEAN_VTENSOR_OPTIONAL]], %[[VAR_VTENSOR_OPTIONAL]],
// CHECK-SAME: %[[TRAINING]], %[[FLOAT]], %[[FLOAT]], %[[CUDNN_ENABLE]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: !torch.vtensor, !torch.optional<vtensor<[4],f32>>, !torch.optional<tensor>,
// CHECK-SAME: !torch.optional<vtensor<[4],f32>>, !torch.optional<vtensor<[4],f32>>,
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
// CHECK-SAME: !torch.bool, !torch.float, !torch.float, !torch.bool -> !torch.vtensor
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[VRET]] : !torch.tensor
// CHECK: return %[[RET]] : !torch.tensor
// CHECK: }
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @convert_to_value_semantic_tensors_optional(%t: !torch.tensor,
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
%ft: !torch.tensor<[4],f32>,
%training: !torch.bool,
%cudnn_enable: !torch.bool,
%f : !torch.float) -> !torch.tensor {
%none = torch.constant.none
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
%tensor_optional = torch.derefine %ft: !torch.tensor<[4],f32> to !torch.optional<tensor>
%none_optional = torch.derefine %none : !torch.none to !torch.optional<tensor>
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
%ret = torch.aten.batch_norm %t, %tensor_optional, %none_optional, %tensor_optional,
%tensor_optional, %training, %f, %f, %cudnn_enable:
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
!torch.tensor, !torch.optional<tensor>, !torch.optional<tensor>,
!torch.optional<tensor>, !torch.optional<tensor>,
E2e implementation for `aten.cat`,`aten.gather`, `aten.bmm` Also contains the following changes: - Remove derefineOp canonicalizer because it's not safe. - Support for optional tensor and list tensors in reduceOpVariant. This only works for some special detected and easy to handle cases. For list, it covers the case list is got from a `ListConstruct`. For optional, it covers the case optional is constructed from a `DerefineOp`. - Remove the `inferReturnTypes` for `FromBuiltinTensorOp` because it's not safe to deduce types from the input. For example, a built-in tensor of i8 could be converted to si8 or ui8. It's better to let the user specify the return type explicitly.
2021-09-14 08:57:59 +08:00
!torch.bool, !torch.float, !torch.float, !torch.bool -> !torch.tensor
return %ret: !torch.tensor
}
Significantly restructure torch/aten import design. This is a really major and invasive restructuring of the way we get torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into MLIR. Please forgive the challenging review, but due to the sheer invasiveness, it wasn't really practical do do it in sane smaller pieces. This fully replaces everything that was already working on the TorchScript path (actually, more -- we added tanh support to TorchToLinalg in order to delete the older code paths). Additionally, I've kept the lights on for the acap path too, including what little e2e stuff was working before (for expediency I made a few tiny compromises along the way that will be easy to undo when we give that path proper attention). Overview of the new design: - The torch operator `somens::someunqualname.someoverloadname` is imported as `torch.somens.someunqualname.someoverloadname` (skip the last dotted part if the overload name is empty), OR, if we don't have such an op registered, it is imported as `torch.operator "somens.someunqualname.someoverloadname" (...) : ...`. - The addition of the "overload name" is a critical element here, as the `(ns,unqual,overload)` triple is unique, which solves a lot of problems we were having. - This involves having separate MLIR ops for the `trailing_` and `.out` variants and all the different overloads. This seemed necessary, because the set of overloads is so wild and varied and unstructured. The previous design was leaning into some underlying structure that just isn't there -- the default situation is the "random overload that we want to manage on the MLIR side", rather than that being an exception. E.g. `aten::ne` (not-equal) has 21 overloads, only 4 of which are c10 dispatcher ops see [gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1), and the "out" variant is really called `.Tensor_out` instead of `.out` as it frequently is for other ops. - Rationale for all being in `torch` namespace: the set of operators are so varied and unstructured that "dialect per namespace" doesn't result in anything resembling the typical MLIR dialect boundary expectations. We could maybe draw the boundary at dispatcher ops vs non-dispatcher ops, but that doesn't seem to really result in very much useful structure at this point in time. - Note: within the torch operator registry, we effectively have a mini-basicpy subdialect (already type-resolved), which is reasonably structured. - The existing Torch op interfaces are also removed -- now that we track the overload name, we can losslessly find the original operator. - Instead of `ATenRecognizeKernelsPass`, we now have a `ReduceOpVariantsPass` that keys off certain traits (and perhaps eventually interfaces) to reduce variants of ops to a smaller set, ideally operating on immutable tensors and using surrounding ops to model the mutability/aliasing aspects. - Note: `torch.ns.unqual.overload` ops allow both immutable and mutable tensors (unlike the previous hard distinction in the common case). This is a premonition for a future change that will introduce a bona fide `!torch.tensor` type that will clean up a bunch of stuff. - `TorchToLinalg` / `TorchToStd` supercede the existing "ATen->TCF->TCP->Linalg" path. - The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`. It should look somewhat familiar, but the benefit of hindsight has allowed a lot of simplifications. The overall trend seems to be to make the `torch` dialect a nice layer independent of anything else. It feels like as a natural result of various future changes we will be removing the reliance on basicpy+numpy dialects and have a nice self-contained type system too that properly models the TorchScript type system (including proper subtyping, mutable/immutable tensors, optional dtype, etc.). Recommended review order: - Start at some of the new import IR, e.g. in `frontends/pytorch/test/node_import/prim.py`, `frontends/pytorch/test/acap_export/test_export_add3.py`, and other tests. - `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py` and associated generated files: - `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td` - `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td` - Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h` - Various code changes in the import path in `frontends/pytorch/csrc/builder`. Probably most interesting is the new code in `torch_to_mlir_utils.cpp` that has the logic to create the `torch.operator` ops or `torch.ns.unqual.overload` ops. This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe), just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @reduce_trailing_underscore_inplace_variant(
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// CHECK-SAME: %[[ARG0:.*]]: !torch.tensor<[2,2],f32>,
// CHECK-SAME: %[[ARG1:.*]]: !torch.tensor<[2,2],f32>) -> (!torch.tensor<[2,2],f32>, !torch.tensor<[2,2],f32>) {
Add `!torch.int` type. This replaces the ad-hoc use of `i64` throughout the Torch layer, and helps to keep it crystal clear the distinction between `!torch.int` (which is modeling the Python `int` type) and the various types that serve as dtypes of tensors, which are a totally different type universe. Changes: - `!torch.int` type and C bindings. - Change `torch.constant.int` parser to not need the `: i64` at the end. - `m_TorchConstantInt` matcher to aid with matching constants. - BackendTypeConversion changes for `!torch.int` -> `i64` type conversion. - Refactor finalizing patterns in FinalizingBackendTypeConversionPass (they were getting very repetitive). - Mechanical rewriting of `!torch.int` to `i64` in all the tests, and `AnyTorchIntType` to `Torch_IntType` in the `.td` files.
2021-06-17 06:53:15 +08:00
// CHECK: %[[C1:.*]] = torch.constant.int 1
Make MaximizeValueSemantics a bit smarter. This adds a pattern to MaximizeValueSemantics which does a simple abstract interpretation within a block, which handles simple cases of `torch.overwrite_tensor`, enough to remove all the unnecessary uses of non-value tensors in ResNet right now. Before/after IR: [gist](https://gist.github.com/silvasean/a3e1ef625b19dfc63579f73cd3b543b6) Also, - Split `torch.copy.tensor` into `torch.copy.to_tensor` and `torch.copy.to_vtensor` which convert between value and non-value semantic tensors. This is a much cleaner factorization as they have very separate use cases and properties (e.g. different side effects) - Remove the various canonicalization patterns they had, which were confusing because they resulted in limited forms of maximizing value semantics throughout the pipeline. We should structure our compilation pipeline such that only MaximizeValueSemantics should be maximizing value semantics. - Adjust pass pipeline to only run MaximizeValueSemantics once. - Make OverwriteTensorOp `$value` always be a value tensor and `$overwritten` be a non-value tensor.
2021-06-19 04:47:47 +08:00
// CHECK: %[[TENSOR0:.*]] = torch.copy.to_vtensor %[[ARG0]] : !torch.vtensor<[2,2],f32>
// CHECK: %[[TENSOR1:.*]] = torch.copy.to_vtensor %[[ARG1]] : !torch.vtensor<[2,2],f32>
Add `!torch.int` type. This replaces the ad-hoc use of `i64` throughout the Torch layer, and helps to keep it crystal clear the distinction between `!torch.int` (which is modeling the Python `int` type) and the various types that serve as dtypes of tensors, which are a totally different type universe. Changes: - `!torch.int` type and C bindings. - Change `torch.constant.int` parser to not need the `: i64` at the end. - `m_TorchConstantInt` matcher to aid with matching constants. - BackendTypeConversion changes for `!torch.int` -> `i64` type conversion. - Refactor finalizing patterns in FinalizingBackendTypeConversionPass (they were getting very repetitive). - Mechanical rewriting of `!torch.int` to `i64` in all the tests, and `AnyTorchIntType` to `Torch_IntType` in the `.td` files.
2021-06-17 06:53:15 +08:00
// CHECK: %[[TENSOR_RESULT:.*]] = torch.aten.add.Tensor %[[TENSOR0]], %[[TENSOR1]], %[[C1]] : !torch.vtensor<[2,2],f32>, !torch.vtensor<[2,2],f32>, !torch.int -> !torch.vtensor<[2,2],f32>
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// Note: This somewhat redundant conversion back and forth
Significantly restructure torch/aten import design. This is a really major and invasive restructuring of the way we get torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into MLIR. Please forgive the challenging review, but due to the sheer invasiveness, it wasn't really practical do do it in sane smaller pieces. This fully replaces everything that was already working on the TorchScript path (actually, more -- we added tanh support to TorchToLinalg in order to delete the older code paths). Additionally, I've kept the lights on for the acap path too, including what little e2e stuff was working before (for expediency I made a few tiny compromises along the way that will be easy to undo when we give that path proper attention). Overview of the new design: - The torch operator `somens::someunqualname.someoverloadname` is imported as `torch.somens.someunqualname.someoverloadname` (skip the last dotted part if the overload name is empty), OR, if we don't have such an op registered, it is imported as `torch.operator "somens.someunqualname.someoverloadname" (...) : ...`. - The addition of the "overload name" is a critical element here, as the `(ns,unqual,overload)` triple is unique, which solves a lot of problems we were having. - This involves having separate MLIR ops for the `trailing_` and `.out` variants and all the different overloads. This seemed necessary, because the set of overloads is so wild and varied and unstructured. The previous design was leaning into some underlying structure that just isn't there -- the default situation is the "random overload that we want to manage on the MLIR side", rather than that being an exception. E.g. `aten::ne` (not-equal) has 21 overloads, only 4 of which are c10 dispatcher ops see [gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1), and the "out" variant is really called `.Tensor_out` instead of `.out` as it frequently is for other ops. - Rationale for all being in `torch` namespace: the set of operators are so varied and unstructured that "dialect per namespace" doesn't result in anything resembling the typical MLIR dialect boundary expectations. We could maybe draw the boundary at dispatcher ops vs non-dispatcher ops, but that doesn't seem to really result in very much useful structure at this point in time. - Note: within the torch operator registry, we effectively have a mini-basicpy subdialect (already type-resolved), which is reasonably structured. - The existing Torch op interfaces are also removed -- now that we track the overload name, we can losslessly find the original operator. - Instead of `ATenRecognizeKernelsPass`, we now have a `ReduceOpVariantsPass` that keys off certain traits (and perhaps eventually interfaces) to reduce variants of ops to a smaller set, ideally operating on immutable tensors and using surrounding ops to model the mutability/aliasing aspects. - Note: `torch.ns.unqual.overload` ops allow both immutable and mutable tensors (unlike the previous hard distinction in the common case). This is a premonition for a future change that will introduce a bona fide `!torch.tensor` type that will clean up a bunch of stuff. - `TorchToLinalg` / `TorchToStd` supercede the existing "ATen->TCF->TCP->Linalg" path. - The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`. It should look somewhat familiar, but the benefit of hindsight has allowed a lot of simplifications. The overall trend seems to be to make the `torch` dialect a nice layer independent of anything else. It feels like as a natural result of various future changes we will be removing the reliance on basicpy+numpy dialects and have a nice self-contained type system too that properly models the TorchScript type system (including proper subtyping, mutable/immutable tensors, optional dtype, etc.). Recommended review order: - Start at some of the new import IR, e.g. in `frontends/pytorch/test/node_import/prim.py`, `frontends/pytorch/test/acap_export/test_export_add3.py`, and other tests. - `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py` and associated generated files: - `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td` - `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td` - Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h` - Various code changes in the import path in `frontends/pytorch/csrc/builder`. Probably most interesting is the new code in `torch_to_mlir_utils.cpp` that has the logic to create the `torch.operator` ops or `torch.ns.unqual.overload` ops. This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe), just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
// (which is cleaned up by canonicalization) is an artifact of two patterns
// being applied in sequence.
Make MaximizeValueSemantics a bit smarter. This adds a pattern to MaximizeValueSemantics which does a simple abstract interpretation within a block, which handles simple cases of `torch.overwrite_tensor`, enough to remove all the unnecessary uses of non-value tensors in ResNet right now. Before/after IR: [gist](https://gist.github.com/silvasean/a3e1ef625b19dfc63579f73cd3b543b6) Also, - Split `torch.copy.tensor` into `torch.copy.to_tensor` and `torch.copy.to_vtensor` which convert between value and non-value semantic tensors. This is a much cleaner factorization as they have very separate use cases and properties (e.g. different side effects) - Remove the various canonicalization patterns they had, which were confusing because they resulted in limited forms of maximizing value semantics throughout the pipeline. We should structure our compilation pipeline such that only MaximizeValueSemantics should be maximizing value semantics. - Adjust pass pipeline to only run MaximizeValueSemantics once. - Make OverwriteTensorOp `$value` always be a value tensor and `$overwritten` be a non-value tensor.
2021-06-19 04:47:47 +08:00
// CHECK: %[[ARRAY_RESULT:.*]] = torch.copy.to_tensor %[[TENSOR_RESULT]] : !torch.tensor<[2,2],f32>
[Torch] Fix mixP case for non value semantic ops (#2540) NonValueSemantic Ops like Add_, div_, etc. expect result DType to be the same as the first input. However, current implementation would result in wrong result type for case like: ```python a = torch.randn(3, 3).half() # float16 b = torch.randn(3, 3) # float32 a += b # i.e. torch.ops.aten.add_(a, b) ``` torch expects `a` to be float16, but dtype refinement would infer float32 type, since it's replaced by `aten.add`.
2023-11-02 12:40:08 +08:00
// CHECK: %[[NONE:.*]] = torch.constant.none
// CHECK: %[[FALSE:.*]] = torch.constant.bool false
// CHECK: %[[DTYPE:.*]] = torch.constant.int 6
// CHECK: %[[DTYPE_RESULT:.*]] = torch.aten.to.dtype %[[ARRAY_RESULT]], %[[DTYPE]], %[[FALSE]], %[[FALSE]], %[[NONE]] : !torch.tensor<[2,2],f32>, !torch.int, !torch.bool, !torch.bool, !torch.none -> !torch.tensor<[2,2],f32>
// CHECK: %[[TENSOR_AGAIN:.*]] = torch.copy.to_vtensor %[[DTYPE_RESULT]] : !torch.vtensor<[2,2],f32>
Add operand type invariant to `torch.overwrite.tensor.contents` (#606) This commit adds the invariant to the op `torch.overwrite.tensor.contents` that both of its operands have the same shape and size. In order to maintain the invariant, special handling of this op is added to the `RefineTypes` pass.
2022-02-23 03:41:46 +08:00
// CHECK: torch.overwrite.tensor.contents %[[TENSOR_AGAIN]] overwrites %[[ARG0]] : !torch.vtensor<[2,2],f32>, !torch.tensor<[2,2],f32>
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
// CHECK: return %[[ARG0]], %[[ARG0]] : !torch.tensor<[2,2],f32>, !torch.tensor<[2,2],f32>
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @reduce_trailing_underscore_inplace_variant(%arg0: !torch.tensor<[2,2],f32>, %arg1: !torch.tensor<[2,2],f32>) -> (!torch.tensor<[2,2],f32>, !torch.tensor<[2,2],f32>) {
Add `!torch.int` type. This replaces the ad-hoc use of `i64` throughout the Torch layer, and helps to keep it crystal clear the distinction between `!torch.int` (which is modeling the Python `int` type) and the various types that serve as dtypes of tensors, which are a totally different type universe. Changes: - `!torch.int` type and C bindings. - Change `torch.constant.int` parser to not need the `: i64` at the end. - `m_TorchConstantInt` matcher to aid with matching constants. - BackendTypeConversion changes for `!torch.int` -> `i64` type conversion. - Refactor finalizing patterns in FinalizingBackendTypeConversionPass (they were getting very repetitive). - Mechanical rewriting of `!torch.int` to `i64` in all the tests, and `AnyTorchIntType` to `Torch_IntType` in the `.td` files.
2021-06-17 06:53:15 +08:00
%c1 = torch.constant.int 1
%0 = torch.aten.add_.Tensor %arg0, %arg1, %c1 : !torch.tensor<[2,2],f32>, !torch.tensor<[2,2],f32>, !torch.int -> !torch.tensor<[2,2],f32>
Introduce `!torch.tensor` / `!torch.vtensor` types. This removes our reliance on the numpy dialect and avoids our off-label use of the builtin tnesor type for modeling unknown dtypes. The `!torch.vtensor` (`ValueTensorType`) type is a value-semantic tensor. The `!torch.tensor` (`NonValueTensorType`) type is a non-value-semantic tensor. The new types look as follows syntactically: ``` // Least-static-information, non-value-semantic tensor. !torch.tensor // Explicit form of least-static-information variant. !torch.tensor<*,unk> // Least-static-information, value-semantic tensor. !torch.vtensor // Explicit form of least-static-information variant. !torch.vtensor<*,unk> // Fixed-set of allowable element types, with first-class support for // Torch's frontend signedness semantics. !torch.tensor<*,si32> // First-class support for unknown dtypes. !torch.tensor<[?,?,?],unk> // Standard MLIR representation of `?` for unknown dimensions. !torch.tensor<[?,2,?,4],unk> // Statically shaped / dtyped example. !torch.vtensor<[1,2,3,4],f32> ``` This required fairly significant changes throughout the compiler, but overall it is a big cleanup. We now have a much clearer layering of "the Torch frontend lowering" vs "lowering to std + linalg + etc.". At the C++ level, there is `ValueTensorType`, `NonValueTensorType`. We also have a helper `BaseTensorType` (kind of like ShapedType) which interoperates with those two. Included changes: - New `torch.tensor(dense<0.0> : tensor<5xf32>) : !torch.tensor` op for creating torch tensor literals in the frontend. - Consistently use signedness for the types (except i1 which I didn't touch -- we need to sort out the situation with !basicpy.BoolType there anyway so will be attending to that soon) - Frontend can annotate whether an argument to the function has value semantics. We currently require this, as our backend contract does not currently allow us to even model the non-value-semantic case. Before, the value-semantic assumption was randomly injected in the middle of the pass pipeline. - Move ArrayToTensor (now called MaximizeValueSemantics) and RefinePublicReturn passes to torch dialect. - The TorchToStd and TorchToLinalg passes are now type conversions from `!torch.vtensor` to `tensor` and use the dialect conversion infra. The overall conversion pipeline is set up following the best practices of the "Type Conversions the Not-So-Hard Way" talk. This required introducing `torch-func-builtin-tensorize` and `torch-finalizing-builtin-tensorize` passes analogous to the upstream bufferization passes with the corresponding names (mostly just copypasta from there). - Misc Torch-level canonicalizations -- we now cleanly layer the lowering to std later in the pipeline, so we are gradually lessening our reliance on random std constant folding before we get to that point. Recommended review order: - New types in TorchTypes.td/TorchTypes.h/TorchDialect.cpp - New ops in TorchOps.td / TorchOps.cpp - Less important / more mechanical stuff - Frontend changes. - Pass changes/additions in `Torch/Transforms` and `Conversion/`
2021-05-21 08:07:18 +08:00
return %0, %arg0 : !torch.tensor<[2,2],f32>, !torch.tensor<[2,2],f32>
Significantly restructure torch/aten import design. This is a really major and invasive restructuring of the way we get torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into MLIR. Please forgive the challenging review, but due to the sheer invasiveness, it wasn't really practical do do it in sane smaller pieces. This fully replaces everything that was already working on the TorchScript path (actually, more -- we added tanh support to TorchToLinalg in order to delete the older code paths). Additionally, I've kept the lights on for the acap path too, including what little e2e stuff was working before (for expediency I made a few tiny compromises along the way that will be easy to undo when we give that path proper attention). Overview of the new design: - The torch operator `somens::someunqualname.someoverloadname` is imported as `torch.somens.someunqualname.someoverloadname` (skip the last dotted part if the overload name is empty), OR, if we don't have such an op registered, it is imported as `torch.operator "somens.someunqualname.someoverloadname" (...) : ...`. - The addition of the "overload name" is a critical element here, as the `(ns,unqual,overload)` triple is unique, which solves a lot of problems we were having. - This involves having separate MLIR ops for the `trailing_` and `.out` variants and all the different overloads. This seemed necessary, because the set of overloads is so wild and varied and unstructured. The previous design was leaning into some underlying structure that just isn't there -- the default situation is the "random overload that we want to manage on the MLIR side", rather than that being an exception. E.g. `aten::ne` (not-equal) has 21 overloads, only 4 of which are c10 dispatcher ops see [gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1), and the "out" variant is really called `.Tensor_out` instead of `.out` as it frequently is for other ops. - Rationale for all being in `torch` namespace: the set of operators are so varied and unstructured that "dialect per namespace" doesn't result in anything resembling the typical MLIR dialect boundary expectations. We could maybe draw the boundary at dispatcher ops vs non-dispatcher ops, but that doesn't seem to really result in very much useful structure at this point in time. - Note: within the torch operator registry, we effectively have a mini-basicpy subdialect (already type-resolved), which is reasonably structured. - The existing Torch op interfaces are also removed -- now that we track the overload name, we can losslessly find the original operator. - Instead of `ATenRecognizeKernelsPass`, we now have a `ReduceOpVariantsPass` that keys off certain traits (and perhaps eventually interfaces) to reduce variants of ops to a smaller set, ideally operating on immutable tensors and using surrounding ops to model the mutability/aliasing aspects. - Note: `torch.ns.unqual.overload` ops allow both immutable and mutable tensors (unlike the previous hard distinction in the common case). This is a premonition for a future change that will introduce a bona fide `!torch.tensor` type that will clean up a bunch of stuff. - `TorchToLinalg` / `TorchToStd` supercede the existing "ATen->TCF->TCP->Linalg" path. - The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`. It should look somewhat familiar, but the benefit of hindsight has allowed a lot of simplifications. The overall trend seems to be to make the `torch` dialect a nice layer independent of anything else. It feels like as a natural result of various future changes we will be removing the reliance on basicpy+numpy dialects and have a nice self-contained type system too that properly models the TorchScript type system (including proper subtyping, mutable/immutable tensors, optional dtype, etc.). Recommended review order: - Start at some of the new import IR, e.g. in `frontends/pytorch/test/node_import/prim.py`, `frontends/pytorch/test/acap_export/test_export_add3.py`, and other tests. - `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py` and associated generated files: - `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td` - `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td` - Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h` - Various code changes in the import path in `frontends/pytorch/csrc/builder`. Probably most interesting is the new code in `torch_to_mlir_utils.cpp` that has the logic to create the `torch.operator` ops or `torch.ns.unqual.overload` ops. This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe), just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
}
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
Add `torch.vtensor.literal` op. This op is much better behaved than the `torch.tensor.literal` op (which is the new name of the `torch.tensor` op). In particular `torch.tensor.literal`: - always has a maximally refined type. - always has value semantics. - can be constant folded / CSE'd. ReduceOpVariants is changed to perform the transformation from `torch.tensor.literal` to `torch.vtensor.literal` (which in general involves static information casts and copies. This new op also allowed tightening up `torch.tensor.literal` to only accept NonValueTensorType (instead of any tensor type). This new ".literal" name is more descriptive. It was getting too confusing seeing an op called just `torch.tensor` (we originally called it that because that's the name of the similar function in the Torch Python API, but it just doesn't fit here).
2021-06-17 23:52:13 +08:00
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @torch.tensor.literal() -> !torch.tensor {
Add `torch.vtensor.literal` op. This op is much better behaved than the `torch.tensor.literal` op (which is the new name of the `torch.tensor` op). In particular `torch.tensor.literal`: - always has a maximally refined type. - always has value semantics. - can be constant folded / CSE'd. ReduceOpVariants is changed to perform the transformation from `torch.tensor.literal` to `torch.vtensor.literal` (which in general involves static information casts and copies. This new op also allowed tightening up `torch.tensor.literal` to only accept NonValueTensorType (instead of any tensor type). This new ".literal" name is more descriptive. It was getting too confusing seeing an op called just `torch.tensor` (we originally called it that because that's the name of the similar function in the Torch Python API, but it just doesn't fit here).
2021-06-17 23:52:13 +08:00
// CHECK: %[[VTENSOR:.*]] = torch.vtensor.literal(dense<0.000000e+00> : tensor<7xf32>) : !torch.vtensor<[7],f32>
// CHECK: %[[SIZES_ERASED:.*]] = torch.tensor_static_info_cast %[[VTENSOR]] : !torch.vtensor<[7],f32> to !torch.vtensor
Make MaximizeValueSemantics a bit smarter. This adds a pattern to MaximizeValueSemantics which does a simple abstract interpretation within a block, which handles simple cases of `torch.overwrite_tensor`, enough to remove all the unnecessary uses of non-value tensors in ResNet right now. Before/after IR: [gist](https://gist.github.com/silvasean/a3e1ef625b19dfc63579f73cd3b543b6) Also, - Split `torch.copy.tensor` into `torch.copy.to_tensor` and `torch.copy.to_vtensor` which convert between value and non-value semantic tensors. This is a much cleaner factorization as they have very separate use cases and properties (e.g. different side effects) - Remove the various canonicalization patterns they had, which were confusing because they resulted in limited forms of maximizing value semantics throughout the pipeline. We should structure our compilation pipeline such that only MaximizeValueSemantics should be maximizing value semantics. - Adjust pass pipeline to only run MaximizeValueSemantics once. - Make OverwriteTensorOp `$value` always be a value tensor and `$overwritten` be a non-value tensor.
2021-06-19 04:47:47 +08:00
// CHECK: %[[TENSOR:.*]] = torch.copy.to_tensor %[[SIZES_ERASED]] : !torch.tensor
Add `torch.vtensor.literal` op. This op is much better behaved than the `torch.tensor.literal` op (which is the new name of the `torch.tensor` op). In particular `torch.tensor.literal`: - always has a maximally refined type. - always has value semantics. - can be constant folded / CSE'd. ReduceOpVariants is changed to perform the transformation from `torch.tensor.literal` to `torch.vtensor.literal` (which in general involves static information casts and copies. This new op also allowed tightening up `torch.tensor.literal` to only accept NonValueTensorType (instead of any tensor type). This new ".literal" name is more descriptive. It was getting too confusing seeing an op called just `torch.tensor` (we originally called it that because that's the name of the similar function in the Torch Python API, but it just doesn't fit here).
2021-06-17 23:52:13 +08:00
// CHECK: return %[[TENSOR]] : !torch.tensor
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @torch.tensor.literal() -> !torch.tensor {
Add `torch.vtensor.literal` op. This op is much better behaved than the `torch.tensor.literal` op (which is the new name of the `torch.tensor` op). In particular `torch.tensor.literal`: - always has a maximally refined type. - always has value semantics. - can be constant folded / CSE'd. ReduceOpVariants is changed to perform the transformation from `torch.tensor.literal` to `torch.vtensor.literal` (which in general involves static information casts and copies. This new op also allowed tightening up `torch.tensor.literal` to only accept NonValueTensorType (instead of any tensor type). This new ".literal" name is more descriptive. It was getting too confusing seeing an op called just `torch.tensor` (we originally called it that because that's the name of the similar function in the Torch Python API, but it just doesn't fit here).
2021-06-17 23:52:13 +08:00
%0 = torch.tensor.literal(dense<0.0> : tensor<7xf32>) : !torch.tensor
return %0 : !torch.tensor
}
[MLIR][TORCH] Add E2E support for aten.index.Tensor op This commit adds lowering of `aten.index.Tensor` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2021-12-17 22:36:53 +08:00
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @convert_to_value_semantic_tensors_optional_list(
[MLIR][TORCH] Add E2E support for aten.index.Tensor op This commit adds lowering of `aten.index.Tensor` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2021-12-17 22:36:53 +08:00
// CHECK-SAME: %[[SELF:.*]]: !torch.tensor<[5],f32>,
// CHECK-SAME: %[[INDICES:.*]]: !torch.tensor<[2,3],si64>) -> !torch.tensor {
// CHECK: %[[INDICES_OPTIONAL_LIST:.*]] = torch.prim.ListConstruct %[[INDICES]] :
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
// CHECK-SAME: (!torch.tensor<[2,3],si64>) -> !torch.list<optional<tensor<[2,3],si64>>>
[MLIR][TORCH] Add E2E support for aten.index.Tensor op This commit adds lowering of `aten.index.Tensor` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2021-12-17 22:36:53 +08:00
// CHECK: %[[SELF_VTENSOR:.*]] = torch.copy.to_vtensor %[[SELF]] : !torch.vtensor<[5],f32>
// CHECK: %[[INDICES_VTENSOR:.*]] = torch.copy.to_vtensor %[[INDICES]] : !torch.vtensor<[2,3],si64>
Add basic support for list of optional tensors in reduce-op-variants (#971) This commit adds support for lists of type `list<optional<tensor>>` where each element in the list is either a `!torch.tensor` or a `!torch.none`.
2022-07-09 02:12:15 +08:00
// CHECK: %[[INDICES_LIST:.*]] = torch.prim.ListConstruct %[[INDICES_VTENSOR]] : (!torch.vtensor<[2,3],si64>) -> !torch.list<optional<vtensor<[2,3],si64>>>
// CHECK: %[[VRET:.*]] = torch.aten.index.Tensor %[[SELF_VTENSOR]], %[[INDICES_LIST]] : !torch.vtensor<[5],f32>, !torch.list<optional<vtensor<[2,3],si64>>> -> !torch.vtensor
[MLIR][TORCH] Add E2E support for aten.index.Tensor op This commit adds lowering of `aten.index.Tensor` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2021-12-17 22:36:53 +08:00
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[VRET]] : !torch.tensor
// CHECK: return %[[RET]] : !torch.tensor
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @convert_to_value_semantic_tensors_optional_list(%self: !torch.tensor<[5],f32>, %indices: !torch.tensor<[2,3],si64>) -> !torch.tensor {
Elide `!torch.` prefix in nested dialect types. This leads to much more succinct types in many cases: ``` !torch.list<!torch.int> !torch.list<int> !torch.tuple<!torch.list<!torch.int>, !torch.list<!torch.int>> !torch.tuple<list<int>, list<int>> !torch.optional<!torch.list<!torch.int>> !torch.optional<list<int>> !torch.list<list<list<tensor>>> !torch.list<!torch.list<!torch.list<!torch.tensor>>> ``` I would like to take this further and allow omitting the `!torch.` prefix in all cases, but that's harder -- for example, we currently use `FuncOp` for functions, and so I don't think we can customize the printing there. It seems like it will be a longer road to getting that level of customization.
2022-03-16 07:22:56 +08:00
%tensor_optional_list = torch.prim.ListConstruct %indices : (!torch.tensor<[2,3],si64>) -> !torch.list<optional<tensor<[2,3],si64>>>
%ret = torch.aten.index.Tensor %self, %tensor_optional_list : !torch.tensor<[5],f32>, !torch.list<optional<tensor<[2,3],si64>>> -> !torch.tensor
[MLIR][TORCH] Add E2E support for aten.index.Tensor op This commit adds lowering of `aten.index.Tensor` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2021-12-17 22:36:53 +08:00
return %ret : !torch.tensor
}
[Torch][Linalg] Add basic support for RNG This PR include the following pieces: - Add torch `Generator` type. `Generator` type is converted to i64 in refbackend type converter. - Add seed managment support for the default global generator. `torch_c.getNextSeed` op is used to get the seed. On refbackend, the `torch_c.getNextSeed` is lowered to load/store from [0] of global variable `default_generator` memref<i64> in `InsertRngGlobals` pass. - Add `aten.uniform_` and testing as an example op for RNG ops. Add `torch.pseudo.aten.uniform` op. It has the same operands and return as the `aten.uniform_` from the op registry except for value semantics.
2022-01-12 09:59:42 +08:00
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
Add basic support for list of optional tensors in reduce-op-variants (#971) This commit adds support for lists of type `list<optional<tensor>>` where each element in the list is either a `!torch.tensor` or a `!torch.none`.
2022-07-09 02:12:15 +08:00
// CHECK-LABEL: func.func @convert_to_value_semantic_tensors_optional_list_nones_and_tensors(
// CHECK-SAME: %[[SELF:.*]]: !torch.tensor<[5],f32>,
// CHECK-SAME: %[[INDICES_0:.*]]: !torch.tensor<[2,3],si64>,
// CHECK-SAME: %[[INDICES_1:.*]]: !torch.tensor<[3],si64>) -> !torch.tensor {
// CHECK: %[[NONE:.*]] = torch.constant.none
// CHECK: %[[INDICES_OPTIONAL_LIST:.*]] = torch.prim.ListConstruct %[[NONE]], %[[INDICES_0]], %[[INDICES_1]] :
// CHECK-SAME: (!torch.none, !torch.tensor<[2,3],si64>, !torch.tensor<[3],si64>) -> !torch.list<optional<tensor>>
// CHECK: %[[SELF_VTENSOR:.*]] = torch.copy.to_vtensor %[[SELF]] : !torch.vtensor<[5],f32>
// CHECK: %[[INDICES_VTENSOR_0:.*]] = torch.copy.to_vtensor %[[INDICES_0]] : !torch.vtensor<[2,3],si64>
// CHECK: %[[INDICES_VTENSOR_1:.*]] = torch.copy.to_vtensor %[[INDICES_1]] : !torch.vtensor<[3],si64>
// CHECK: %[[INDICES_LIST:.*]] = torch.prim.ListConstruct %[[NONE]], %[[INDICES_VTENSOR_0]], %[[INDICES_VTENSOR_1]] : (!torch.none, !torch.vtensor<[2,3],si64>, !torch.vtensor<[3],si64>) -> !torch.list<optional<vtensor>>
// CHECK: %[[VRET:.*]] = torch.aten.index.Tensor %[[SELF_VTENSOR]], %[[INDICES_LIST]] : !torch.vtensor<[5],f32>, !torch.list<optional<vtensor>> -> !torch.vtensor
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[VRET]] : !torch.tensor
// CHECK: return %[[RET]] : !torch.tensor
func.func @convert_to_value_semantic_tensors_optional_list_nones_and_tensors(%self: !torch.tensor<[5],f32>, %indices0: !torch.tensor<[2,3],si64>, %indices1: !torch.tensor<[3],si64>) -> !torch.tensor {
%none = torch.constant.none
%tensor_optional_list = torch.prim.ListConstruct %none, %indices0, %indices1 : (!torch.none, !torch.tensor<[2,3],si64>, !torch.tensor<[3],si64>) -> !torch.list<optional<tensor>>
%ret = torch.aten.index.Tensor %self, %tensor_optional_list : !torch.tensor<[5],f32>, !torch.list<optional<tensor>> -> !torch.tensor
return %ret : !torch.tensor
}
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
// CHECK-LABEL: func.func @torch.aten.bernoulli_.float(
[LINALG] Add value tensor variant to `bernoulli_.float` (#597) This commit adds the op `PseudoAtenBernoulliFloatOp` that represents `AtenBernoulli_FloatOp` without the underscore. This is needed to make sure that the `ReduceOpVariants` pass turns the in-place op into an op that takes value tensors as inputs, otherwise the `MaximizeValueSemantics` pass will not be able to add value semantics correctly.
2022-02-15 10:58:48 +08:00
// CHECK-SAME: %[[T:.*]]: !torch.tensor) -> !torch.tensor {
// CHECK: %[[GENERATOR:.*]] = torch.constant.none
// CHECK: %[[P:.*]] = torch.constant.float 5.000000e-01
// CHECK: %[[T_VTENSOR:.*]] = torch.copy.to_vtensor %[[T]] : !torch.vtensor
Improve "pseudo" op terminology. The term "pseudo" is very vague and was getting confusing (I felt I had to explain it in every comment referencing it). Instead, rework the "pseudo" ops to instead be named: - MLIR Syntax: `torch.valsem.*` - C++ / ODS: `ValsemVariant*Op` This makes it clear what the concept is, and avoids confusion with other things that might be called "pseudo", since these are very specific and should be 100% consistently named w.r.t. the non-valsem-variant ops that they correspond to.
2022-03-16 07:57:33 +08:00
// CHECK: %[[VRET:.*]] = torch.valsem.aten.bernoulli.float %[[T_VTENSOR]], %[[P]], %[[GENERATOR]] : !torch.vtensor, !torch.float, !torch.none -> !torch.vtensor
[LINALG] Add value tensor variant to `bernoulli_.float` (#597) This commit adds the op `PseudoAtenBernoulliFloatOp` that represents `AtenBernoulli_FloatOp` without the underscore. This is needed to make sure that the `ReduceOpVariants` pass turns the in-place op into an op that takes value tensors as inputs, otherwise the `MaximizeValueSemantics` pass will not be able to add value semantics correctly.
2022-02-15 10:58:48 +08:00
// CHECK: %[[RET:.*]] = torch.copy.to_tensor %[[VRET]] : !torch.tensor
// CHECK: %[[COPY_VTENSOR:.*]] = torch.copy.to_vtensor %[[RET]] : !torch.vtensor
Add operand type invariant to `torch.overwrite.tensor.contents` (#606) This commit adds the invariant to the op `torch.overwrite.tensor.contents` that both of its operands have the same shape and size. In order to maintain the invariant, special handling of this op is added to the `RefineTypes` pass.
2022-02-23 03:41:46 +08:00
// CHECK: torch.overwrite.tensor.contents %[[COPY_VTENSOR]] overwrites %[[T]] : !torch.vtensor, !torch.tensor
[LINALG] Add value tensor variant to `bernoulli_.float` (#597) This commit adds the op `PseudoAtenBernoulliFloatOp` that represents `AtenBernoulli_FloatOp` without the underscore. This is needed to make sure that the `ReduceOpVariants` pass turns the in-place op into an op that takes value tensors as inputs, otherwise the `MaximizeValueSemantics` pass will not be able to add value semantics correctly.
2022-02-15 10:58:48 +08:00
// CHECK: return %[[T]] : !torch.tensor
mlir: bump llvm tag to 5380e3 (#856) In addition to updating the llvm-project submodule, this patch also: 1. updates shape functions and tests so that `func` and `call` operations refer to the `func` dialect 2. avoid duplicate registration of dialects
2022-05-17 03:54:35 +08:00
func.func @torch.aten.bernoulli_.float(%t: !torch.tensor) -> !torch.tensor {
[LINALG] Add value tensor variant to `bernoulli_.float` (#597) This commit adds the op `PseudoAtenBernoulliFloatOp` that represents `AtenBernoulli_FloatOp` without the underscore. This is needed to make sure that the `ReduceOpVariants` pass turns the in-place op into an op that takes value tensors as inputs, otherwise the `MaximizeValueSemantics` pass will not be able to add value semantics correctly.
2022-02-15 10:58:48 +08:00
%generator = torch.constant.none
%p = torch.constant.float 5.000000e-01
%ret = torch.aten.bernoulli_.float %t, %p, %generator : !torch.tensor, !torch.float, !torch.none -> !torch.tensor
return %ret : !torch.tensor
}
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// -----
// CHECK-LABEL: func.func @scaled_dot_product_flash_attention_for_cpu
// CHECK-SAME: %[[ARG0:.+]]: !torch.vtensor<[1,1,5,5],f32>, %[[ARG1:.+]]: !torch.vtensor<[1,1,5,5],f32>, %[[ARG2:.+]]: !torch.vtensor<[1,1,5,5],f32>
// CHECK: %[[ZERO:.+]] = torch.constant.float 0.000000e+00
// CHECK: %[[FALSE:.+]] = torch.constant.bool false
// CHECK: %[[NONE0:.+]] = torch.constant.none
// CHECK: %[[NONE1:.+]] = torch.constant.none
build: manually update PyTorch version (#3627) Set PyTorch and TorchVision version to nightly release 2024-08-18. This commit also updates the `scaled_dot_product_attention` op. A new attribute `enable_gqa` has been added. As of now, only the default value for the same is supported. Signed-Off By: Vivek Khandelwal <vivekkhandelwal1424@gmail.com>
2024-08-19 14:33:56 +08:00
// CHECK: %[[ATTEN:.+]] = torch.aten.scaled_dot_product_attention %[[ARG0]], %[[ARG1]], %[[ARG2]], %[[NONE0]], %[[ZERO]], %[[FALSE]], %[[NONE1]], %[[FALSE]]
[torch] Fix tm_tensor.attention for end-to-end (#2907) Some operations include a backend matcher for specialized operations. We map these back to generics so they appropriately match to the high performance versions. This is done for the attention operation.
2024-02-14 13:18:01 +08:00
// CHECK: return %[[ATTEN]]
func.func @scaled_dot_product_flash_attention_for_cpu(%arg0: !torch.vtensor<[1,1,5,5],f32>, %arg1: !torch.vtensor<[1,1,5,5],f32>, %arg2: !torch.vtensor<[1,1,5,5],f32>) -> !torch.vtensor<[1,1,5,5],f32> {
%float0.000000e00 = torch.constant.float 0.000000e+00
%false = torch.constant.bool false
%none = torch.constant.none
%none_0 = torch.constant.none
%0:2 = torch.operator "torch.aten._scaled_dot_product_flash_attention_for_cpu"(%arg0, %arg1, %arg2, %float0.000000e00, %false, %none, %none_0) : (!torch.vtensor<[1,1,5,5],f32>, !torch.vtensor<[1,1,5,5],f32>, !torch.vtensor<[1,1,5,5],f32>, !torch.float, !torch.bool, !torch.none, !torch.none) -> (!torch.vtensor<[1,1,5,5],f32>, !torch.vtensor<[1,1,5],f32>)
return %0#0 : !torch.vtensor<[1,1,5,5],f32>
}