llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/e2e_testing/torchscript/xfail_sets.py

340 lines
12 KiB
Python
Raw Normal View History

Dual license the torch-mlir project. This commit (with approval from all contributors) dual licenses the torch-mlir project under both the standard LLVM license and the standard PyTorch license. This will facilitate moving code between torch-mlir and the two upstream projects. The standard file comment is now: ``` // This file is licensed under the Apache License v2.0 with LLVM Exceptions. // See https://llvm.org/LICENSE.txt for license information. // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // Also available under a BSD-style license. See LICENSE. ``` See `LICENSE` in the project root for the terms of both licenses.
2021-09-30 00:03:40 +08:00
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
# Also available under a BSD-style license. See LICENSE.
Add IREE support in TorchScript e2e tests. - Add support for "expected failures" in test reporting. The new error reports look like [this](https://gist.github.com/silvasean/6ffd95e1d55302b699673da201da210d). - We will now be able to put these tests into CI, since the harness understand which tests are expected to pass and fail. - Refactor RefBackendTestConfig to NpcompBackendTestConfig which supports both RefBackend and IREE. - Add instructions for installing IREE dependencies (both from packages and for local builds of IREE) - Add `tools/torchscript_e2e_test.sh` for invoking the e2e test harness (this makes invoking a bit easier, as it doesn't rely on a loose Python invocation).
2021-07-01 05:13:21 +08:00
# This file describes the sets of tests expected to fail for each config.
# This information is deliberately kept in a side table, rather than
# in-situ on the test, as a deliberate layering decision: tests should
# have unique keys to identify them and enable side tables of various kinds
# (this includes down into lower parts of the stack, where a side table
# might be used to keep more elaborate sets of testing configurations).
Move COMMON_TORCH_MLIR_LOWERING_XFAILS into test_suite That way, downstreams don't have to duplicate this list. Also, remove "external config" feature, since it is subsumed by just importing the test suite.
2022-04-20 03:35:56 +08:00
from torch_mlir_e2e_test.test_suite import COMMON_TORCH_MLIR_LOWERING_XFAILS
Small fixes for slice edge cases (#476)
2021-12-16 05:54:41 +08:00
REFBACKEND_XFAIL_SET = COMMON_TORCH_MLIR_LOWERING_XFAILS
E2E framework: Add support for list/dict/scalar values Most of the change is in the reporting code to give error messages that are useful, and adjusting TraceItem to be semantically correct w.r.t. Python's modeling of return values. This allows writing a test like `ListLiteralModule_basic` for list functionality, which we will soon be hooking up to IREE. The IR for that test currently gets this far: ``` builtin.func @forward(%arg0: f64) -> !torch.list<!torch.float> { %0 = torch.from_f64 %arg0 %1 = torch.prim.ListConstruct %0, %0 : (!torch.float, !torch.float) -> !torch.list<!torch.float> return %1 : !torch.list<!torch.float> } ``` It should be sufficient to just add a conversion of `torch.prim.ListConstruct` (+ relevant type conversion) to necessary IREE primitives. For lists of *tensors* (rather than scalar floats), it gets more complicated, as we need to deal with changing their element type to ValueTensorType first (by default, they will all be NonValueTensorType). It seems that IREE might have a type we can lower into for non-value tensors as well, TBD.
2021-08-11 07:10:31 +08:00
add ci tests (#754)
2022-05-26 03:59:59 +08:00
EAGER_MODE_XFAIL_SET = {
# RefBackend fails
"TableBatchEmbeddingModule_basic",
[MLIR][TORCH] Fix CI failure This commit fixes the CI failure by temporarily adding the failing test to xfail set. Signed-Off By: Vivek Khandelwal<vivek@nod-labs.com>
2022-08-03 19:50:33 +08:00
"QuantizedMLP_basic",
"Matmul_vecmat"
add ci tests (#754)
2022-05-26 03:59:59 +08:00
}
This PR implements an eager mode backend for PyTorch through the torch-mlir framework. This is accomplished by overriding the `__torch_dispatch__` class method on wrapper subclass `TorchMLIRTensor(torch.Tensor)`. Effectively, this mode works by compiling op by op as the NN is eagerly executed by PyTorch. Entailed in that compilation is building a representation of the op that can be `torch.jit.script`ed, importing using `ModuleBuilder`, and then executing (e.g., with `RefBackendLinalgOnTensorsBackend`). This mode includes a fallback to conventional PyTorch if anything in the torch-mlir compilation process fails (e.g., unsupported op). Currently, all e2e tests pass execpt for two that involve an upstream PyTorch bug (https://github.com/pytorch/pytorch/issues/74400). High priority next steps: 1. A compile cache in order to speed up reruns of the same NN. 2. Integration with IREE (though not in this repo). 3. Integration with `torch.distributed`.
2022-03-03 07:02:18 +08:00
Add a basic TOSA E2E backend. We lower through linalg-on-tensors and use RefBackend to run it. This adds enough support for a "tanh" op. Adding more ops should be fairly mechanical now that things are wired up. Run with: ``` ./tools/torchscript_e2e_test.sh -c tosa ``` The backend structure is very similar to linalg-on-tensors based E2E backends and is a nice parallel (see `tosa_backend.py`). Actually, this forced a nice refactoring to the layering here. We removed `torchscript-module-to-linalg-on-tensors-backend-pipeline` and instead require separately running ``` torchscript-function-to-torch-backend-pipeline,torch-backend-to-linalg-on-tensors-backend-pipeline ``` This highlights the step that lowers to the "torch backend contract" of cleaned up `torch` dialect ops is a critical step in the lowering. Going forward, that is the key load-bearing contract of the torch-mlir project, not the linalg-on-tensors backend contract. Recommended review order: - `TorchToTosa.cpp` / `TorchToTosa/basic.mlir` - `python/torch_mlir_e2e_test/torchscript/configs/tosa_backend.py` and the new `utils.py` file there. - `python/torch_mlir_e2e_test/tosa_backends/linalg_on_tensors.py` and `abc.py` in that directory for the TOSA backend e2e interface. - other misc mechanical changes
2021-10-08 10:07:03 +08:00
# Write the TOSA set as a "passing" set as it is very early in development
# and very few tests work yet.
Bump llvm-project to f1b922188ead5ca492c8d8edd47921b013a22ae0. Includes a fix to use `add_mlir_public_c_api_library` for Torch-MLIR's CAPI library, which is now required (note: upstream sample has it the right way). Disabled a TOSA test per discussion: https://github.com/llvm/torch-mlir/issues/379
2021-10-26 02:43:21 +08:00
TOSA_PASS_SET = {
Rename `tosa-to-linalg-on-tensors` to `tosa-to-linalg` The pass name changed upstream.
2021-10-26 04:43:54 +08:00
"ElementwiseUnaryModule_basic",
Update external llvm to 966b72098363d44adf2882b9c34 The external llvm is updated to point to https://reviews.llvm.org/rG966b72098363d44adf2882b9c34fcdbe344ff913. Some of the changes wrt. NamedAttr has been addressed. Signed-off-by: Prashant Kumar <prashant@nod-labs.com>
2021-12-04 15:31:38 +08:00
"ElementwiseBinaryModule_basic",
[tosa] Add Torch.sigmoid fp32 to TOSA (#386) * [tosa] Add Torch.sigmoid fp32 to TOSA Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2021-10-29 01:09:12 +08:00
"ElementwiseSigmoidModule_basic",
Add non-default dtype support for a few elementwise math ops. (#687) * fix type inference * fix Torch2Linalg conversion * add test cases
2022-03-24 04:35:43 +08:00
"ElementwiseExpModule_basic",
[tosa] Add structure for eltwise ops Add a bunch of op legalizations. Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2021-11-11 11:03:36 +08:00
"ElementwiseReluModule_basic",
"ElementwiseFloorModule_basic",
"ElementwiseLogModule_basic",
Make broadcasting result shape more static This involes the following 2 parts: - Change refine type to propagate more static shape info. - Get as much static shape info as possible when creating the result tensor when converting to linalg.
2021-12-29 10:11:07 +08:00
"ElementwiseBinaryStaticShapeModule_basic",
Add aten.hardtanh e2e support.
2022-02-09 04:57:23 +08:00
"ElementwiseMinimumModule_basic",
"ElementwiseMinimumIntModule_basic",
"ElementwiseMaximumModule_basic",
"ElementwiseMaximumIntModule_basic",
[mlir][tosa] Refactor conversions to use templates (#416) - Remove use of conversion construction macros - Add mul and div op conversions - Add corresponding tests Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2021-11-12 08:15:58 +08:00
"TanhBackward_basic",
[tosa] Add TosaMakeBroadcastable pass to torch-to-tosa pipeline. Fixes broken e2e test ElementwiseAddModule_basic Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2021-11-24 14:25:59 +08:00
"ElementwiseAddModule_basic",
Add support for multiple return values This change is to unblock the work of some backprop ops returning more than one tensors. We will need to think of a more scalable approach in the future if more flexible return types combinations are needed.
2021-11-08 23:56:40 +08:00
"ReturnThreeTensorFloat32_basic",
lowered addcmul and addcdiv to linalg
2021-11-25 06:01:48 +08:00
"AddCMulModule_basic",
"AddCDivModule_basic",
Fix forward XPASS
2021-12-01 05:48:10 +08:00
"SqueezeModule_broadcast",
Add support for passing & returning memref of bool types Support for passing memref of bool types as a function argument and return is added in ref-backend. Signed-off-by: Prashant Kumar <prashant@nod-labs.com>
2021-12-08 22:05:02 +08:00
"BoolTensorReturnFalseModule_basic",
"BoolTensorReturnTrueModule_basic",
"BoolTensorReturnMixedModule_basic",
Fixes https://github.com/llvm/torch-mlir/issues/751 where `torch.bool` is parsed as signless `i1`. (#752)
2022-04-14 01:28:27 +08:00
"BoolTensorHandleSignless_basic",
* [tosa] Support for Rsqrt legalization (#480) Signed-off-by: Anup Gangwar <anup.gangwar@arm.com> Co-authored-by: Anup Gangwar <anup.gangwar@arm.com>
2021-12-15 02:03:58 +08:00
"ElementwiseRsqrtModule_basic",
[tosa] Support for Torch.squeeze (#487)
2021-12-16 13:40:29 +08:00
"SqueezeModule_static",
"SqueezeModule_noUnitDim",
"SqueezeModule_allUnitDim",
Added aten::t() Op
2021-12-17 12:08:07 +08:00
"TModuleRank1_basic",
"TModuleRank0_basic",
[TORCH][MLIR] Fold trivial cases of `aten.to.dtype` and `aten.view` op - It folds `aten.to.dtype` when the input tensor type and result type are exactly same. - It folds `aten.view` when the rank of both the input tensor type and result type is unity. Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
2021-12-23 20:04:29 +08:00
"ElementwiseToDtypeIdentityModule_basic",
"View1DFoldModule_basic",
[TORCH][MLIR] Add E2E support for `aten._unsafe_view` op. This commit adds decomposition of `aten._unsafe_view` op into `aten.view` op. Signed-Off-By: Prateek Gupta<prateek@nod-labs.com>
2022-02-10 16:11:05 +08:00
"UnsafeView1DFoldModule_basic",
[tosa] Implement squeeze.dim support (#511) Templated variants for squeeze and squeeze.dim
2022-01-07 00:31:29 +08:00
"SqueezeDimModule_static",
"SqueezeDimModule_identity",
"SqueezeDimModule_unitDim",
Add support for two return values This commit adds support for two return values of type memref f32 and i64. Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-01-10 21:24:08 +08:00
"ReturnTwoTensorF32I64_basic",
[tosa] Add tosa-to-standard before tosa-to-linalg pass (#524) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-01-15 03:05:11 +08:00
"ElementwisePowModule_basic",
[tosa] Enable tosa-to-linalg-named so Matmul works again (#530) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-01-20 04:10:04 +08:00
"BmmModule_basic",
Add static type information support to `aten.mm` (#602) This commit adds static type information support to `aten.mm`. This is needed for the forward pass of Bert training.
2022-02-19 01:56:48 +08:00
"MmDagModule_basic",
[LINALG] Lower `aten.Matmul` to `linalg.BatchMatmul` This commit lowers `aten.matmul` to `linalg.BatchMatmul` under the following conditions: 1. The result of matrix multiplication must have batch dimensions, i.e., rank greater than 2. 2. The resultant matrix must have at most 1 dynamic batch dimension. It also handles broadcasting of batch dimensions when batch dimensions of the matrices are broadcastable. Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
2022-06-16 23:45:10 +08:00
"Matmul4dStatic_basic",
[tosa] Enable tosa-to-linalg-named so Matmul works again (#530) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-01-20 04:10:04 +08:00
"Matmul_dot",
"Matmul_3d",
[MLIR][TORCH] Add integer dtype support for aten.rsub.Scalar op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-06-14 20:31:30 +08:00
"RsubFloatModule_basic",
"RsubFloatModule_noalpha_basic",
* [tosa] Support for AtenRsubScalarOp for scalar constants (#531) * [tosa] Support for AtenCeilOp and AtenReciprocalOp * [tosa] Support for comparator ops, Aten[Gt|Lt|Eq][Tensor|Scalar]Op with scalar constant * [tosa] Support for Scalar variants of Aten[Mul|Div|Add|Sub] Ops with scalar constants Signed-off-by: Anup Gangwar <anup.gangwar@arm.com> Co-authored-by: Anup Gangwar <anup.gangwar@arm.com>
2022-01-21 02:58:30 +08:00
"ElementwiseGtFloatScalarModule_basic",
"ElementwiseGtIntScalarModule_basic",
"ElementwiseGtMixed2ScalarModule_basic",
"ElementwiseGtFloatTensorModule_basic",
"ElementwiseGtIntTensorModule_basic",
"ElementwiseLtFloatScalarModule_basic",
"ElementwiseLtIntScalarModule_basic",
"ElementwiseLtDiffWidthScalarModule_basic",
"ElementwiseLtFloatTensorModule_basic",
"ElementwiseLtIntTensorModule_basic",
"ElementwiseEqFloatScalarModule_basic",
"ElementwiseEqIntScalarModule_basic",
"ElementwiseEqDiffWidthScalarModule_basic",
"ElementwiseEqFloatTensorModule_basic",
"ElementwiseEqIntTensorModule_basic",
"ElementwiseMulScalarModule_int",
"ElementwiseMulScalarModule_float",
"ElementwiseMulTensorIntModule_basic",
"ElementwiseDivScalarModule_basic",
"ElementwiseSubScalarFloatModule_basic",
"ElementwiseAddScalarFloatModule_basic",
"ElementwiseMulScalarModule_float",
"ElementwiseCeilModule_basic",
"ElementwiseReciprocalModule_basic",
"TypePromotionAlphaWiderModule_basic",
[tosa] Implement conv2d support (#541) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-01-27 11:16:13 +08:00
"Conv2dWithPaddingDilationStrideStaticModule_basic",
Revert "[LINALG] Decompose `aten.batch_norm` into `aten.native_batch_norm`" This reverts commit 442ff4605c54f836afa17d2780bd9bddcc2a5ed8.
2022-02-25 03:41:55 +08:00
"BatchNorm1DModule_basic",
Fix typo in `inputRank` check of `AtenBatchNormOp` (#1046) The original conversion pattern for `AtenBatchNormOp` required that the input rank be greater than 2; however, the only expectation in the conversion pattern and in Pytorch is that the input rank is greater than 1, since the second dimension of the input must match the size of the `weight`, `bias`, `runningMean`, and `runningVar` inputs. This commit fixes the `inputRank` check.
2022-07-16 00:35:59 +08:00
"BatchNorm1DWith2DInputModule_basic",
Revert "[LINALG] Decompose `aten.batch_norm` into `aten.native_batch_norm`" This reverts commit 442ff4605c54f836afa17d2780bd9bddcc2a5ed8.
2022-02-25 03:41:55 +08:00
"BatchNorm2DModule_basic",
"BatchNorm3DModule_basic",
* [tosa] Support for AtenFlattenUsingIntsOp (#548)
2022-01-29 13:38:56 +08:00
"FlattenStaticModule_basic",
"FlattenRank0Module_basic",
"ElementwiseFlattenBroadcastModule_basic",
[TORCH] Add aten.std e2e support
2022-01-30 01:10:50 +08:00
"SquareModule_basic",
[tosa] Add maxpool2d and adaptive_avgpool2d support (#550) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-02-01 05:34:09 +08:00
"MaxPool2dStaticModule_basic",
Revert "[LINALG] Decompose `aten.batch_norm` into `aten.native_batch_norm`" This reverts commit 442ff4605c54f836afa17d2780bd9bddcc2a5ed8.
2022-02-25 03:41:55 +08:00
"ResNet18StaticModule_basic",
* [tosa] Support for AtenNativeLayerNormOp * [tosa] Support for AtenPermuteOp Signed-off-by: Anup Gangwar <anup.gangwar@arm.com>
2022-02-04 06:08:19 +08:00
"NativeLayerNormModule4D_basic",
"LayerNormNormalizeOverAllDimsModule_basic",
"PermuteModule_basic",
"PermuteNegativeIndexModule_basic",
[tosa] Support for some ops and fix for Issue #532 (#575) * [tosa] Support for AtenNe[Tensor|Scalar]Op, AtenLog2Op, AtenBitwiseAndTensorOp, AtenSquareOp and AtenThresholdOp * Fix for Issue #532 - Mixed input types for few ops and updated few tests to use i32 instead of i64 Signed-off-by: Anup Gangwar <anup.gangwar@arm.com> Co-authored-by: Anup Gangwar <anup.gangwar@arm.com>
2022-02-12 04:30:02 +08:00
"ElementwiseLog2Module_basic",
"Threshold1dIntI32Module_basic",
"Threshold1dFloatModule_basic",
"Threshold2dFloatModule_basic",
"Threshold3dFloatModule_basic",
"ElementwiseSubScalarIntModule_basic",
"ElementwiseAddScalarIntModule_basic",
"ElementwiseMulScalarModule_basic",
[tosa] Support for Aten[Zeros|Ones|Fill_Scalar] ops (#604) Signed-off-by: Anup Gangwar <anup.gangwar@arm.com> Co-authored-by: Anup Gangwar <anup.gangwar@arm.com>
2022-02-17 01:53:51 +08:00
"ZerosModuleDefaultDtype_basic",
"ZerosModuleInt2D_basic",
"ZerosModuleInt3D_basic",
"ZerosModuleFloat2D_basic",
"ZerosModuleFloat3D_basic",
"ZerosModuleFalsePinMemory_basic",
"OnesModuleDefaultDtype_basic",
"OnesModuleInt_basic",
"OnesModuleFloat_basic",
"OnesModuleFalsePinMemory_basic",
[MLIR][TORCH] Add E2E support for aten.new_zeros, aten.new_ones op This commit adds lowering of `aten.new_zeros` and `aten.new_ones` op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-02-28 14:14:40 +08:00
"NewZerosModuleDefaultDtype_basic",
"NewZerosModuleInt2D_basic",
"NewZerosModuleInt3D_basic",
"NewZerosModuleFloat2D_basic",
"NewZerosModuleFloat3D_basic",
"NewZerosModuleFalsePinMemory_basic",
"NewOnesModuleDefaultDtype_basic",
"NewOnesModuleInt2D_basic",
"NewOnesModuleInt3D_basic",
"NewOnesModuleFloat2D_basic",
"NewOnesModuleFloat3D_basic",
"NewOnesModuleFalsePinMemory_basic",
Decompose aten.silu op Decomposition of aten.silu.op is added as silu(x) = x * sigmoid(x).
2022-03-02 01:30:58 +08:00
"SiluModule_basic",
[LINALG] Add decomposition of `aten.dropout` op - This commit adds decomposition of `aten.dropout` op. It also covers the training mode of the same op. - It also adds lowering of `aten.sub.float` op. Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
2022-02-17 00:34:03 +08:00
"DropoutEvalIntModule_basic",
"DropoutEvalFloatModule_basic",
[tosa] Support for Aten[Unsqueeze|Contiguous|Dropout|Reshape|View] ops (#700)
2022-03-26 05:15:07 +08:00
"ContiguousModule_basic",
"DropoutModule_basic",
"ViewExpandModule_basic",
[LINALG] Fix handling of size-1 dims in `aten.view` (#962) This commit adds support for several size-1 dims in a row in an expansion using `aten.view`.
2022-06-23 06:58:40 +08:00
"ViewExpandOnesModule_basic",
[LINALG] Fix handling of size-1 dims in `aten.view` again. (#992) A previous fix to the handling of size-1 dims in `aten.view` (https://github.com/llvm/torch-mlir/pull/962) resulted in the wrong grouping of dimensions when size-1 dims where between two dims of size greater than 1. This commit fixes that.
2022-07-01 07:39:25 +08:00
"ViewExpandOnesBeforeAndAfterModule_basic",
"ViewExpandOnesMiddleModule_basic",
Allow expanding and collapsing in aten::view (#1082) - Supports cases where the view op expands and collapses dims simulataneously. This does not handle the case where it is neither expanding nor collapsing (e.g. [2, 3] -> [3, 2]) - Additionally fixes a previous bug with adding 1-sized dims on both sides of a tensor with aten.view
2022-07-21 05:35:51 +08:00
"ViewExpandCollapseModule_basic",
"ViewExpandCollapseWithOnesModule_basic",
[tosa] Support for Aten[Unsqueeze|Contiguous|Dropout|Reshape|View] ops (#700)
2022-03-26 05:15:07 +08:00
"ViewCollapseInferredDimModule_basic",
"ViewExpandInferredDimModule_basic",
"ViewNoChangeStaticModule_basic",
"UnsafeViewExpandModule_basic",
"ReshapeCollapseModule_basic",
tosa] Support for Aten[Gelu|GeluBackward] ops (#720) Signed-off-by: Anup Gangwar <anup.gangwar@arm.com> Co-authored-by: Anup Gangwar <anup.gangwar@arm.com>
2022-03-31 08:00:55 +08:00
"ElementwiseGeluModule_basic",
"GeluBackwardModule_basic",
[LINALG] Add the lowering of `aten.ne.Scalar` op The lowering of `aten.ne.Scalar` op has been added to the linalg backend.
2022-04-04 00:19:01 +08:00
"ElementwiseNeIntScalarModule_basic",
"ElementwiseNeFloatTensorModule_basic",
Add lowering for _convolution
2022-07-08 14:44:03 +08:00
"Convolution2DStaticModule_basic",
test: add end-to-end test for aten.neg (#760)
2022-04-16 03:37:57 +08:00
"ElementwiseNegModule_basic",
[REFBACKEND] Add support for returning multiple different return types. Added the dynamic registration of return function to the execution engine. This makes sure that different/multiple return types are supported. Also, updated the .style.yapf indentation to 4.
2022-04-19 21:47:47 +08:00
"TestMultipleTensorReturn_basic",
[tosa] Support for AtenAvgPool2d op Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-05-19 20:35:59 +08:00
"AdaptiveAvgPool2dUnitOutputSizeStaticModule_basic",
[MLIR][TORCH] Add E2E support for aten.baddbmm op This commit decomposes `aten.baddbmm` op into `aten.bmm`, `aten.mul.Scalar`, and `aten.add.Tensor` op. Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-05-30 16:08:54 +08:00
"BaddbmmDynamicModule_basic",
"BaddbmmStaticModule_basic",
"BaddbmmWithAlphaBetaModule_basic",
"BaddbmmWithAlphaModule_basic",
"BaddbmmWithBetaModule_basic",
"BaddbmmBroadcast1DInputModule_basic",
"BaddbmmBroadcast2DInputModule_basic",
[MLIR][TORCH] Add decomposition of aten.numpy_T op This commit adds the decomposition of `aten.numpy_T` op into `aten.t` or `aten.permute` op. Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
2022-06-03 20:38:59 +08:00
"NumpyTRank1Module_basic",
"NumpyTRank2Module_basic",
"NumpyTRankNStaticModule_basic",
"NumpyTRankNDynamicModule_basic",
[tosa] aten.embedding op support (#991) Enables BERT legalization. Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-07-01 04:13:52 +08:00
"EmbeddingModuleI32Static_basic",
[tosa] aten.transpose.int support (#1017) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-07-08 04:05:33 +08:00
"TModuleRank2_basic",
"TransposeIntModule_basic",
"TransposeIntNegDimsModule_basic",
[tosa] aten.max.dim , aten.slice.tensor ops (#1027) Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com>
2022-07-14 01:10:18 +08:00
"ArgmaxModule_keepDim",
"ArgmaxModule_with_dim",
"_LogSoftmaxModuleStable_basic",
Bump llvm-project to f1b922188ead5ca492c8d8edd47921b013a22ae0. Includes a fix to use `add_mlir_public_c_api_library` for Torch-MLIR's CAPI library, which is now required (note: upstream sample has it the right way). Disabled a TOSA test per discussion: https://github.com/llvm/torch-mlir/issues/379
2021-10-26 02:43:21 +08:00
}
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
LTC_XFAIL_SET = {
Blacklist _convolution op (#1048) * Blacklist _convolution op in LTC * Removed duplicate Torch_AtenSelectScatterOp instance from autogen .td * Removed duplicate Torch_AtenSliceScatterOp instance from autogen .td
2022-07-14 01:28:05 +08:00
"_Convolution2DAllFalseModule_basic",
"_Convolution2DBenchmarkModule_basic",
"_Convolution2DCudnnModule_basic",
"_Convolution2DDeterministicModule_basic",
"_Convolution2DTF32Module_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"AdaptiveAvgPool2dNonUnitOutputSizeDynamicModule_basic",
"AdaptiveAvgPool2dNonUnitOutputSizeStaticModule_basic",
"AddIntModule_basic",
"BernoulliFloatModule_basic",
"BernoulliModule_basic",
"BernoulliTensorModule_basic",
"BincountMinlengthModule_basic",
"BincountModule_basic",
"BincountStaticSizeModule_basic",
"BoolFloatFalseModule_basic",
"BoolFloatTrueModule_basic",
"BoolIntFalseModule_basic",
"BoolIntTrueModule_basic",
"CeilFloatModule_basic",
"DivFloatModule_basic",
"DropoutTrainModule_basic",
Prune xfail e2e LTC tests & fix bugs from functionalization pass (#1044) - Pruned number of xfailed e2e LTC tests from 305 to 134 - Reviewed every failure to ensure the error genuinely warrants an xfail - Fixed bug where non-tensor outputs of LTC computation had `.to('cpu')` called, which caused a failure and inflated the xfail count - Fixed bug with `HBC_basic` test where a constant tensor was created in its constructor without being declared as a buffer, which prevented the device from being updated when the parent `torch.nn.Module` got moved to the `lazy` device - Note that this test is still xfail'd due to some unsupported ops. Left a comment about some potential issues that may arise if it gets reenabled in the future - Updated autogen `GeneratedTorchOps.td` to reflect the latest set of supported ops - Renamed `aten.zero.functionalization` to `aten.zero` to reflect upstream PyTorch changes
2022-07-13 03:21:50 +08:00
"ElementwiseAtenFloorDivideBroadcastModule_basic",
"ElementwiseAtenFloorDivideModule_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"ElementwiseWhereScalarModule_basic",
"ElementwiseWhereScalarOtherModule_basic",
"ElementwiseWhereScalarSelfModule_basic",
"ElementwiseWhereSelfModule_basic",
"EmptyLikeMemoryFormatModule_basic",
"EmptyLikeModule_defaultDtype",
"EmptyLikeModule_falsePinMemory",
"EmptyLikeModule_float",
"EmptyLikeModule_int",
"EqIntModule_basic",
"Fill_TensorFloat64WithFloat32_basic",
"Fill_TensorFloat64WithFloat64_basic",
"Fill_TensorFloat64WithInt64_basic",
"FullLikeModuleDefaultDtype_basic",
"FullLikeModuleFalsePinMemory_basic",
"FullLikeModuleFloat2D_basic",
"FullLikeModuleFloat3DStatic_basic",
"FullLikeModuleFloat3D_basic",
"FullLikeModuleInt2DStatic_basic",
"FullLikeModuleInt2D_basic",
"FullLikeModuleInt3D_basic",
"GeFloatIntModule_basic",
"GeFloatModule_basic",
Resolve remaining LTC CI failures (#1110) * Replace CHECK_EQ with TORCH_CHECK_EQ * Check value of TORCH_MLIR_USE_INSTALLED_PYTORCH during LTC build * Update LTC XFAIL with NewZerosModule ops * Explicitly blacklist _like ops * Automatically blacklist new_/_like ops * Prune away unused Python dependencies from LTC * Add flag to disable LTC * Autogen dummy _REFERENCE_LAZY_BACKEND library when LTC is disabled * Implement compute_shape_var * Removed Var tests from XFAIL Set * XFAIL tests using _local_scalar_dense or index.Tensor * Add StdDim tests to XFAIL set * Autogen aten::cat
2022-07-30 17:54:40 +08:00
"GeIntModule_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"GtFloatIntModule_basic",
"GtIntModule_basic",
"HBC_basic",
"HardTanhIntModule_basic",
"HardTanhModule_basic",
"IndexPut1DFloatAccumulateModule_basic",
"IndexPut1DFloatNonAccumulateModule_basic",
"IndexPut1DIntAccumulateModule_basic",
"IndexPut1DIntNonAccumulateModule_basic",
"IndexPut2DFloatAccumulateModule_basic",
"IndexPut2DFloatNonAccumulateModule_basic",
"IndexPut2DIntAccumulateModule_basic",
"IndexPut2DIntNonAccumulateModule_basic",
"IndexPut3DFloatAccumulateModule_basic",
"IndexPut3DFloatNonAccumulateModule_basic",
"IndexPut3DIntAccumulateModule_basic",
"IndexPut3DIntNonAccumulateModule_basic",
"IndexPutHackedTwin1DFloatAccumulateModule_basic",
"IndexPutHackedTwin1DFloatNonAccumulateModule_basic",
"IndexPutHackedTwin1DIntAccumulateModule_basic",
"IndexPutHackedTwin1DIntNonAccumulateModule_basic",
"IndexPutHackedTwin2DFloatAccumulateModule_basic",
"IndexPutHackedTwin2DFloatNonAccumulateModule_basic",
"IndexPutHackedTwin2DIntAccumulateModule_basic",
"IndexPutHackedTwin2DIntNonAccumulateModule_basic",
"IndexPutHackedTwin3DFloatAccumulateModule_basic",
"IndexPutHackedTwin3DFloatNonAccumulateModule_basic",
"IndexPutHackedTwin3DIntAccumulateModule_basic",
"IndexPutHackedTwin3DIntNonAccumulateModule_basic",
"IndexPutImpl1DFloatAccumulateModule_basic",
"IndexPutImpl1DFloatNonAccumulateModule_basic",
"IndexPutImpl1DIntAccumulateModule_basic",
"IndexPutImpl1DIntNonAccumulateModule_basic",
"IndexPutImpl2DFloatAccumulateModule_basic",
"IndexPutImpl2DFloatNonAccumulateModule_basic",
"IndexPutImpl3DFloatAccumulateModule_basic",
"IndexPutImpl3DFloatNonAccumulateModule_basic",
Prune xfail e2e LTC tests & fix bugs from functionalization pass (#1044) - Pruned number of xfailed e2e LTC tests from 305 to 134 - Reviewed every failure to ensure the error genuinely warrants an xfail - Fixed bug where non-tensor outputs of LTC computation had `.to('cpu')` called, which caused a failure and inflated the xfail count - Fixed bug with `HBC_basic` test where a constant tensor was created in its constructor without being declared as a buffer, which prevented the device from being updated when the parent `torch.nn.Module` got moved to the `lazy` device - Note that this test is still xfail'd due to some unsupported ops. Left a comment about some potential issues that may arise if it gets reenabled in the future - Updated autogen `GeneratedTorchOps.td` to reflect the latest set of supported ops - Renamed `aten.zero.functionalization` to `aten.zero` to reflect upstream PyTorch changes
2022-07-13 03:21:50 +08:00
"IndexTensorModule3dInput_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"IndexTensorModule_basic",
Resolve remaining LTC CI failures (#1110) * Replace CHECK_EQ with TORCH_CHECK_EQ * Check value of TORCH_MLIR_USE_INSTALLED_PYTORCH during LTC build * Update LTC XFAIL with NewZerosModule ops * Explicitly blacklist _like ops * Automatically blacklist new_/_like ops * Prune away unused Python dependencies from LTC * Add flag to disable LTC * Autogen dummy _REFERENCE_LAZY_BACKEND library when LTC is disabled * Implement compute_shape_var * Removed Var tests from XFAIL Set * XFAIL tests using _local_scalar_dense or index.Tensor * Add StdDim tests to XFAIL set * Autogen aten::cat
2022-07-30 17:54:40 +08:00
"IndexTensorMultiInputContiguousCenter_basic",
"IndexTensorMultiInputNonContiguous_basic",
"IndexTensorMultiInputOneDim_basic",
"IndexTensorMultiInputThreeIndexers_basic",
"IndexTensorMultiInput_basic",
Resolve CI testing failure for Lazy Tensor Core (#1088) * Xfail unsupported ops * Register FuncDialect * Include dynamic_ir in build * Code reformat * Enable LTC tests for macOS and Source Build
2022-07-26 07:40:25 +08:00
"IndexTensorSelectDimModule_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"Matmul_dot",
"Matmul_matvec",
"MulIntModule_basic",
"NeFloatIntModule_basic",
"NeIntModule_basic",
"NewEmptyModuleDefaultDtype_basic",
"NewEmptyModuleFalsePinMemory_basic",
"NewEmptyModuleFloat2D_basic",
"NewEmptyModuleFloat3D_basic",
"NewEmptyModuleInt2D_basic",
"NewEmptyModuleInt3D_basic",
"NewEmptyModuleLayoutIntDtype_basic",
"NewEmptyModuleNonDefaultFloatDtype_basic",
"NewEmptyModuleNonDefaultIntDtype_basic",
"NewOnesModuleDefaultDtype_basic",
"NewOnesModuleFalsePinMemory_basic",
"NewOnesModuleFloat2D_basic",
"NewOnesModuleFloat3D_basic",
"NewOnesModuleInt2D_basic",
"NewOnesModuleInt3D_basic",
Resolve remaining LTC CI failures (#1110) * Replace CHECK_EQ with TORCH_CHECK_EQ * Check value of TORCH_MLIR_USE_INSTALLED_PYTORCH during LTC build * Update LTC XFAIL with NewZerosModule ops * Explicitly blacklist _like ops * Automatically blacklist new_/_like ops * Prune away unused Python dependencies from LTC * Add flag to disable LTC * Autogen dummy _REFERENCE_LAZY_BACKEND library when LTC is disabled * Implement compute_shape_var * Removed Var tests from XFAIL Set * XFAIL tests using _local_scalar_dense or index.Tensor * Add StdDim tests to XFAIL set * Autogen aten::cat
2022-07-30 17:54:40 +08:00
"NewZerosModuleDefaultDtype_basic",
"NewZerosModuleFalsePinMemory_basic",
"NewZerosModuleFloat2D_basic",
"NewZerosModuleFloat3D_basic",
"NewZerosModuleInt2D_basic",
"NewZerosModuleInt3D_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"OnesLikeModule_defaultDtype",
"OnesLikeModule_falsePinMemory",
"OnesLikeModule_float",
"OnesLikeModule_int",
"QuantizedMLP_basic",
"RandLikeDtypeModule_basic",
"RandLikeModule_basic",
"ScalarImplicitFloatModule_basic",
"ScalarImplicitIntModule_basic",
"SliceEndSleStartModule_basic",
"SliceOutOfUpperBoundIndexModule_basic",
"SliceStartEqEndModule_basic",
"SqrtIntModule_basic",
"StdBiasedModule_basic",
Resolve remaining LTC CI failures (#1110) * Replace CHECK_EQ with TORCH_CHECK_EQ * Check value of TORCH_MLIR_USE_INSTALLED_PYTORCH during LTC build * Update LTC XFAIL with NewZerosModule ops * Explicitly blacklist _like ops * Automatically blacklist new_/_like ops * Prune away unused Python dependencies from LTC * Add flag to disable LTC * Autogen dummy _REFERENCE_LAZY_BACKEND library when LTC is disabled * Implement compute_shape_var * Removed Var tests from XFAIL Set * XFAIL tests using _local_scalar_dense or index.Tensor * Add StdDim tests to XFAIL set * Autogen aten::cat
2022-07-30 17:54:40 +08:00
"StdDimBiasedModule_basic",
"StdDimKeepDimFalseModule_basic",
"StdDimKeepDimTrueModule_basic",
[MLIR][TORCH] Add support for dim=None to Aten[Var|Std]DimOp PyTorch recently added support for `dim=None` in the `torch.var` (https://github.com/pytorch/pytorch/commit/5ca9b2b6fa4d5871ad900a9d4e5ba34c02934bb3) and `torch.std`op (https://github.com/pytorch/pytorch/commit/eb0e30e0bc86477f5ae17e9d5651a9fbc01a91e3). This commit adds the corresponding support in torch-mlir. Signed-Off By: Vivek Khandelwal<vivek@nod-labs.com>
2022-08-05 14:24:00 +08:00
"StdDimEmptyDimModule_basic",
"StdDimNoneDimModule_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"StdUnbiasedModule_basic",
"SubFloatModule_basic",
"SubIntModule_basic",
"TableBatchEmbeddingModule_basic",
Resolve CI testing failure for Lazy Tensor Core (#1088) * Xfail unsupported ops * Register FuncDialect * Include dynamic_ir in build * Code reformat * Enable LTC tests for macOS and Source Build
2022-07-26 07:40:25 +08:00
"TensorsConcatNegativeDimModule_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
"TensorToBoolZeroRank_basic",
"TensorToBool_basic",
"TensorToFloatZeroRank_basic",
"TensorToFloat_basic",
"TensorToIntZeroRank_basic",
"TensorToInt_basic",
"TensorsConcatModule_basic",
"UniformModule_basic",
"UniformStaticModule_basic",
"UnsafeViewCollapseDynamicWithAtenSizeIntModule_basic",
"ViewCollapseDynamicWithAtenSizeIntModule_basic",
Add embedding Bag e2e case in xfail set (#1130)
2022-08-02 06:23:45 +08:00
"AtenEmbeddingBagSumExample_basic",
Added e2e LTC tests (#916) * Added e2e LTC Torch MLIR tests * Fix seed for reproducability * Check if computation is None before getting debug string * Updated unit tests, and added numeric tests * Print name of the model layer that fails numeric validation * Run LTC e2e test with CI/CD * Set seed in main function, instead of beginning of execution * Add comment to specify number of digits of precision * Fixed typo * Remove tests for LTC example models * Added LTC option to torchscript e2e * Implement compile and run for LTC e2e test * xfail all tests that use ops that aren't currently supported
2022-06-10 03:56:01 +08:00
}