2020-11-25 05:10:27 +08:00
|
|
|
//===- mlir_utils.h ---------------------------------------------*- C++ -*-===//
|
|
|
|
//
|
|
|
|
// This file is licensed under a pytorch-style license
|
|
|
|
// See frontends/pytorch/LICENSE for license information.
|
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
|
|
|
|
#ifndef NPCOMP_FRONTENDS_PYTORCH_CSRC_MLIR_UTILS_H
|
|
|
|
#define NPCOMP_FRONTENDS_PYTORCH_CSRC_MLIR_UTILS_H
|
|
|
|
|
|
|
|
#include <cstring>
|
|
|
|
#include <string>
|
2021-02-02 09:59:42 +08:00
|
|
|
#include <vector>
|
2020-11-25 05:10:27 +08:00
|
|
|
|
2020-12-15 06:30:51 +08:00
|
|
|
#include "mlir-c/IR.h"
|
2020-11-25 05:10:27 +08:00
|
|
|
#include "mlir-c/Support.h"
|
|
|
|
|
Significantly restructure torch/aten import design.
This is a really major and invasive restructuring of the way we get
torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into
MLIR. Please forgive the challenging review, but due to the sheer
invasiveness, it wasn't really practical do do it in sane smaller
pieces.
This fully replaces everything that was already working on the
TorchScript path (actually, more -- we added tanh support to
TorchToLinalg in order to delete the older code paths). Additionally,
I've kept the lights on for the acap path too, including what little e2e
stuff was working before (for expediency I made a few tiny compromises
along the way that will be easy to undo when we give that path proper
attention).
Overview of the new design:
- The torch operator `somens::someunqualname.someoverloadname` is
imported as `torch.somens.someunqualname.someoverloadname` (skip the
last dotted part if the overload name is empty), OR, if we don't have
such an op registered, it is imported as
`torch.operator "somens.someunqualname.someoverloadname" (...) : ...`.
- The addition of the "overload name" is a critical element here, as
the `(ns,unqual,overload)` triple is unique, which solves a lot of
problems we were having.
- This involves having separate MLIR ops for the `trailing_` and
`.out` variants and all the different overloads. This seemed
necessary, because the set of overloads is so wild and varied and
unstructured. The previous design was leaning into some underlying
structure that just isn't there -- the default situation is
the "random overload that we want to manage on the MLIR side",
rather than that being an exception. E.g. `aten::ne` (not-equal)
has 21 overloads, only 4 of which are c10 dispatcher ops see
[gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1),
and the "out" variant is really called `.Tensor_out` instead of
`.out` as it frequently is for other ops.
- Rationale for all being in `torch` namespace: the set of operators
are so varied and unstructured that "dialect per namespace"
doesn't result in anything resembling the typical MLIR dialect
boundary expectations. We could maybe draw the boundary at
dispatcher ops vs non-dispatcher ops, but that doesn't seem to
really result in very much useful structure at this point in time.
- Note: within the torch operator registry, we effectively have a
mini-basicpy subdialect (already type-resolved), which is reasonably
structured.
- The existing Torch op interfaces are also removed -- now that we
track the overload name, we can losslessly find the original
operator.
- Instead of `ATenRecognizeKernelsPass`, we now have a
`ReduceOpVariantsPass` that keys off certain traits (and perhaps
eventually interfaces) to reduce variants of ops to a smaller set,
ideally operating on immutable tensors and using surrounding ops to
model the mutability/aliasing aspects.
- Note: `torch.ns.unqual.overload` ops allow both immutable and
mutable tensors (unlike the previous hard distinction in the common
case). This is a premonition for a future change that will introduce a
bona fide `!torch.tensor` type that will clean up a bunch of stuff.
- `TorchToLinalg` / `TorchToStd` supercede the existing
"ATen->TCF->TCP->Linalg" path.
- The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`.
It should look somewhat familiar, but the benefit of hindsight has
allowed a lot of simplifications.
The overall trend seems to be to make the `torch` dialect a nice layer
independent of anything else. It feels like as a natural result of
various future changes we will be removing the reliance on basicpy+numpy
dialects and have a nice self-contained type system too that properly
models the TorchScript type system (including proper subtyping,
mutable/immutable tensors, optional dtype, etc.).
Recommended review order:
- Start at some of the new import IR, e.g. in
`frontends/pytorch/test/node_import/prim.py`,
`frontends/pytorch/test/acap_export/test_export_add3.py`, and other
tests.
- `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py`
and associated generated files:
- `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td`
- `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td`
- Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new
traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h`
- Various code changes in the import path in
`frontends/pytorch/csrc/builder`. Probably most interesting is the new
code in `torch_to_mlir_utils.cpp` that has the logic to create the
`torch.operator` ops or `torch.ns.unqual.overload` ops.
This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe),
just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
|
|
|
#include "c10/util/ArrayRef.h"
|
2021-02-20 08:21:21 +08:00
|
|
|
#include "c10/util/Optional.h"
|
|
|
|
|
2020-11-25 05:10:27 +08:00
|
|
|
namespace torch_mlir {
|
|
|
|
|
|
|
|
inline MlirStringRef toMlirStringRef(const std::string &s) {
|
|
|
|
return mlirStringRefCreate(s.data(), s.size());
|
|
|
|
}
|
|
|
|
|
|
|
|
inline MlirStringRef toMlirStringRef(const char *s) {
|
|
|
|
return mlirStringRefCreate(s, std::strlen(s));
|
|
|
|
}
|
|
|
|
|
2020-12-15 06:30:51 +08:00
|
|
|
inline MlirNamedAttribute toMlirNamedAttribute(const char *s,
|
|
|
|
MlirAttribute attr) {
|
|
|
|
MlirContext context = mlirAttributeGetContext(attr);
|
|
|
|
MlirIdentifier ident = mlirIdentifierGet(context, toMlirStringRef(s));
|
|
|
|
return mlirNamedAttributeGet(ident, attr);
|
|
|
|
}
|
|
|
|
|
2021-01-28 08:35:44 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
MlirNamedAttribute namedAttr) {
|
|
|
|
mlirOperationStateAddAttributes(&state, 1, &namedAttr);
|
|
|
|
}
|
|
|
|
|
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
MlirRegion region) {
|
|
|
|
mlirOperationStateAddOwnedRegions(&state, 1, ®ion);
|
|
|
|
}
|
|
|
|
|
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
MlirValue value) {
|
|
|
|
mlirOperationStateAddOperands(&state, 1, &value);
|
|
|
|
}
|
|
|
|
|
2021-02-02 09:59:42 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
2021-02-06 09:57:38 +08:00
|
|
|
const std::vector<MlirValue> &values) {
|
2021-02-02 09:59:42 +08:00
|
|
|
mlirOperationStateAddOperands(&state, values.size(), values.data());
|
|
|
|
}
|
|
|
|
|
Significantly restructure torch/aten import design.
This is a really major and invasive restructuring of the way we get
torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into
MLIR. Please forgive the challenging review, but due to the sheer
invasiveness, it wasn't really practical do do it in sane smaller
pieces.
This fully replaces everything that was already working on the
TorchScript path (actually, more -- we added tanh support to
TorchToLinalg in order to delete the older code paths). Additionally,
I've kept the lights on for the acap path too, including what little e2e
stuff was working before (for expediency I made a few tiny compromises
along the way that will be easy to undo when we give that path proper
attention).
Overview of the new design:
- The torch operator `somens::someunqualname.someoverloadname` is
imported as `torch.somens.someunqualname.someoverloadname` (skip the
last dotted part if the overload name is empty), OR, if we don't have
such an op registered, it is imported as
`torch.operator "somens.someunqualname.someoverloadname" (...) : ...`.
- The addition of the "overload name" is a critical element here, as
the `(ns,unqual,overload)` triple is unique, which solves a lot of
problems we were having.
- This involves having separate MLIR ops for the `trailing_` and
`.out` variants and all the different overloads. This seemed
necessary, because the set of overloads is so wild and varied and
unstructured. The previous design was leaning into some underlying
structure that just isn't there -- the default situation is
the "random overload that we want to manage on the MLIR side",
rather than that being an exception. E.g. `aten::ne` (not-equal)
has 21 overloads, only 4 of which are c10 dispatcher ops see
[gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1),
and the "out" variant is really called `.Tensor_out` instead of
`.out` as it frequently is for other ops.
- Rationale for all being in `torch` namespace: the set of operators
are so varied and unstructured that "dialect per namespace"
doesn't result in anything resembling the typical MLIR dialect
boundary expectations. We could maybe draw the boundary at
dispatcher ops vs non-dispatcher ops, but that doesn't seem to
really result in very much useful structure at this point in time.
- Note: within the torch operator registry, we effectively have a
mini-basicpy subdialect (already type-resolved), which is reasonably
structured.
- The existing Torch op interfaces are also removed -- now that we
track the overload name, we can losslessly find the original
operator.
- Instead of `ATenRecognizeKernelsPass`, we now have a
`ReduceOpVariantsPass` that keys off certain traits (and perhaps
eventually interfaces) to reduce variants of ops to a smaller set,
ideally operating on immutable tensors and using surrounding ops to
model the mutability/aliasing aspects.
- Note: `torch.ns.unqual.overload` ops allow both immutable and
mutable tensors (unlike the previous hard distinction in the common
case). This is a premonition for a future change that will introduce a
bona fide `!torch.tensor` type that will clean up a bunch of stuff.
- `TorchToLinalg` / `TorchToStd` supercede the existing
"ATen->TCF->TCP->Linalg" path.
- The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`.
It should look somewhat familiar, but the benefit of hindsight has
allowed a lot of simplifications.
The overall trend seems to be to make the `torch` dialect a nice layer
independent of anything else. It feels like as a natural result of
various future changes we will be removing the reliance on basicpy+numpy
dialects and have a nice self-contained type system too that properly
models the TorchScript type system (including proper subtyping,
mutable/immutable tensors, optional dtype, etc.).
Recommended review order:
- Start at some of the new import IR, e.g. in
`frontends/pytorch/test/node_import/prim.py`,
`frontends/pytorch/test/acap_export/test_export_add3.py`, and other
tests.
- `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py`
and associated generated files:
- `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td`
- `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td`
- Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new
traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h`
- Various code changes in the import path in
`frontends/pytorch/csrc/builder`. Probably most interesting is the new
code in `torch_to_mlir_utils.cpp` that has the logic to create the
`torch.operator` ops or `torch.ns.unqual.overload` ops.
This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe),
just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
c10::ArrayRef<MlirValue> values) {
|
|
|
|
mlirOperationStateAddOperands(&state, values.size(), values.data());
|
|
|
|
}
|
|
|
|
|
2021-01-28 08:35:44 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
MlirType resultType) {
|
|
|
|
mlirOperationStateAddResults(&state, 1, &resultType);
|
|
|
|
}
|
|
|
|
|
2021-02-02 09:59:42 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
2021-02-06 09:57:38 +08:00
|
|
|
const std::vector<MlirType> &resultTypes) {
|
2021-02-02 09:59:42 +08:00
|
|
|
mlirOperationStateAddResults(&state, resultTypes.size(), resultTypes.data());
|
|
|
|
}
|
|
|
|
|
Significantly restructure torch/aten import design.
This is a really major and invasive restructuring of the way we get
torch operators (`torch::jit::Operator` / `c10::OperatorHandle`) into
MLIR. Please forgive the challenging review, but due to the sheer
invasiveness, it wasn't really practical do do it in sane smaller
pieces.
This fully replaces everything that was already working on the
TorchScript path (actually, more -- we added tanh support to
TorchToLinalg in order to delete the older code paths). Additionally,
I've kept the lights on for the acap path too, including what little e2e
stuff was working before (for expediency I made a few tiny compromises
along the way that will be easy to undo when we give that path proper
attention).
Overview of the new design:
- The torch operator `somens::someunqualname.someoverloadname` is
imported as `torch.somens.someunqualname.someoverloadname` (skip the
last dotted part if the overload name is empty), OR, if we don't have
such an op registered, it is imported as
`torch.operator "somens.someunqualname.someoverloadname" (...) : ...`.
- The addition of the "overload name" is a critical element here, as
the `(ns,unqual,overload)` triple is unique, which solves a lot of
problems we were having.
- This involves having separate MLIR ops for the `trailing_` and
`.out` variants and all the different overloads. This seemed
necessary, because the set of overloads is so wild and varied and
unstructured. The previous design was leaning into some underlying
structure that just isn't there -- the default situation is
the "random overload that we want to manage on the MLIR side",
rather than that being an exception. E.g. `aten::ne` (not-equal)
has 21 overloads, only 4 of which are c10 dispatcher ops see
[gist](https://gist.github.com/silvasean/190ba918c550c956260e21254e1b8aa1),
and the "out" variant is really called `.Tensor_out` instead of
`.out` as it frequently is for other ops.
- Rationale for all being in `torch` namespace: the set of operators
are so varied and unstructured that "dialect per namespace"
doesn't result in anything resembling the typical MLIR dialect
boundary expectations. We could maybe draw the boundary at
dispatcher ops vs non-dispatcher ops, but that doesn't seem to
really result in very much useful structure at this point in time.
- Note: within the torch operator registry, we effectively have a
mini-basicpy subdialect (already type-resolved), which is reasonably
structured.
- The existing Torch op interfaces are also removed -- now that we
track the overload name, we can losslessly find the original
operator.
- Instead of `ATenRecognizeKernelsPass`, we now have a
`ReduceOpVariantsPass` that keys off certain traits (and perhaps
eventually interfaces) to reduce variants of ops to a smaller set,
ideally operating on immutable tensors and using surrounding ops to
model the mutability/aliasing aspects.
- Note: `torch.ns.unqual.overload` ops allow both immutable and
mutable tensors (unlike the previous hard distinction in the common
case). This is a premonition for a future change that will introduce a
bona fide `!torch.tensor` type that will clean up a bunch of stuff.
- `TorchToLinalg` / `TorchToStd` supercede the existing
"ATen->TCF->TCP->Linalg" path.
- The new `torch_ods_gen.py` supercedes `torch_signature_ods_gen.py`.
It should look somewhat familiar, but the benefit of hindsight has
allowed a lot of simplifications.
The overall trend seems to be to make the `torch` dialect a nice layer
independent of anything else. It feels like as a natural result of
various future changes we will be removing the reliance on basicpy+numpy
dialects and have a nice self-contained type system too that properly
models the TorchScript type system (including proper subtyping,
mutable/immutable tensors, optional dtype, etc.).
Recommended review order:
- Start at some of the new import IR, e.g. in
`frontends/pytorch/test/node_import/prim.py`,
`frontends/pytorch/test/acap_export/test_export_add3.py`, and other
tests.
- `frontends/pytorch/python/torch_mlir_utils/codegen/torch_ods_gen.py`
and associated generated files:
- `include/npcomp/Dialect/Torch/IR/GeneratedAtenOps.td`
- `include/npcomp/Dialect/Torch/IR/GeneratedPrimOps.td`
- Inspect `ReduceOpVariants.cpp` / `reduce-op-variants.mlir` and the new
traits in `include/npcomp/Dialect/Torch/IR/TorchTraits.h`
- Various code changes in the import path in
`frontends/pytorch/csrc/builder`. Probably most interesting is the new
code in `torch_to_mlir_utils.cpp` that has the logic to create the
`torch.operator` ops or `torch.ns.unqual.overload` ops.
This is the [new ResNet IR](https://gist.github.com/silvasean/5407aafb710d07612b7b5b92eabecebe),
just to be able to look at a substantial sample of IR in the new style.
2021-05-05 05:42:50 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state,
|
|
|
|
c10::ArrayRef<MlirType> resultTypes) {
|
|
|
|
mlirOperationStateAddResults(&state, resultTypes.size(), resultTypes.data());
|
|
|
|
}
|
|
|
|
|
2021-02-20 08:21:21 +08:00
|
|
|
template <typename T>
|
|
|
|
void addToMlirOperationState(MlirOperationState &state, c10::optional<T> o) {
|
|
|
|
if (o.has_value()) {
|
|
|
|
addToMlirOperationState(state, o.value());
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-02-06 09:57:38 +08:00
|
|
|
inline void addToMlirOperationState(MlirOperationState &state) {}
|
|
|
|
|
|
|
|
template <typename T, typename U, typename... Ts>
|
|
|
|
void addToMlirOperationState(MlirOperationState &state, T &&t, U &&u,
|
|
|
|
Ts &&...ts) {
|
2021-02-02 09:59:42 +08:00
|
|
|
addToMlirOperationState(state, std::forward<T>(t));
|
2021-02-06 09:57:38 +08:00
|
|
|
addToMlirOperationState(state, std::forward<U>(u), std::forward<Ts>(ts)...);
|
2021-01-28 08:35:44 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
template <typename... Ts>
|
|
|
|
MlirOperation createMlirOperation(std::string name, MlirLocation loc,
|
|
|
|
Ts &&...ts) {
|
|
|
|
MlirOperationState state = mlirOperationStateGet(toMlirStringRef(name), loc);
|
|
|
|
addToMlirOperationState(state, std::forward<Ts>(ts)...);
|
|
|
|
return mlirOperationCreate(&state);
|
|
|
|
}
|
|
|
|
|
|
|
|
template <typename... Ts>
|
|
|
|
MlirOperation createMlirOperationAtEnd(MlirBlock block, std::string name,
|
|
|
|
MlirLocation loc, Ts &&...ts) {
|
|
|
|
MlirOperation operation =
|
|
|
|
createMlirOperation(name, loc, std::forward<Ts>(ts)...);
|
|
|
|
mlirBlockInsertOwnedOperationBefore(block, mlirBlockGetTerminator(block),
|
|
|
|
operation);
|
|
|
|
return operation;
|
|
|
|
}
|
|
|
|
|
2020-11-25 05:10:27 +08:00
|
|
|
} // namespace torch_mlir
|
|
|
|
|
|
|
|
#endif // NPCOMP_FRONTENDS_PYTORCH_CSRC_MLIR_UTILS_H
|