2018-02-15 22:53:19 +01:00
|
|
|
(def
|
|
|
|
|
(ident fn)
|
2018-08-14 17:56:52 -07:00
|
|
|
(decl
|
|
|
|
|
(list
|
|
|
|
|
(param
|
|
|
|
|
(ident x)
|
|
|
|
|
(variable (ident Tensor)))
|
|
|
|
|
(param
|
|
|
|
|
(ident y)
|
|
|
|
|
(variable (ident Tensor)))
|
|
|
|
|
(param
|
|
|
|
|
(ident z)
|
|
|
|
|
(variable (ident Tensor))))
|
|
|
|
|
(option))
|
2018-02-15 22:53:19 +01:00
|
|
|
(list
|
Fix the clamp special case and gradient problem on None, add None to JIT (#9596)
Summary:
Supersedes #8925
This PR fixes #8502, it fixes the gradients problem for clamp when passing None to the function, and add support for the NoneLiteral and NoneType in script to enable clamp tests. Now we could have corner cases like:
```python
torch.jit.script
def func():
x = torch.randn(3, 3, requires_grad=True)
y = torch.clamp(x, None, 0) # max = 0
y = torch.clamp(x, min=None, max=0)
```
In both JIT and Aten, we use Scalar(NAN) as a sentinel value when passing None type to function clamp, this is the current way we used to support None type in JIT and to solve the gradient problem when user explicitly passing None into clamp.
In JIT side, we create a tensor(NAN) and undefinedTensor if we encounter None when matching the function schema, and later in the interpreter, it will translate to Scalar(NAN) if needed.
Ideally we don't need clamp_min and clamp_max in ATenNative/Autograd and could only support clamp after this change, but since bunch of other operators (e.g. Activation.cpp, Loss.cpp) is using clamp_min in several places, we will still have the functions available, but all python invocations will only call clamp instead of clamp_min/max (with calling underlying th_max/th_min in clamp).
zdevito jamesr66a
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9596
Reviewed By: zdevito
Differential Revision: D8940839
Pulled By: wanchaol
fbshipit-source-id: c543a867b82e0ab8c99384773b173fdde2605d28
2018-07-27 22:47:29 -07:00
|
|
|
(assign
|
|
|
|
|
(list (variable (ident q)))
|
|
|
|
|
(None))
|
2018-02-15 22:53:19 +01:00
|
|
|
(assign
|
2018-04-09 19:34:51 -07:00
|
|
|
(list (variable (ident q)))
|
2018-02-15 22:53:19 +01:00
|
|
|
(-
|
|
|
|
|
(+
|
|
|
|
|
(variable (ident x))
|
|
|
|
|
(variable (ident y)))
|
2018-02-24 11:15:55 +01:00
|
|
|
(apply
|
2018-03-12 06:52:40 -07:00
|
|
|
(.
|
|
|
|
|
(variable (ident z))
|
|
|
|
|
(ident sigmoid))
|
|
|
|
|
(list)
|
2018-02-24 11:15:55 +01:00
|
|
|
(list))))
|
|
|
|
|
(expression statement
|
2018-04-09 19:34:51 -07:00
|
|
|
(list
|
|
|
|
|
(apply
|
|
|
|
|
(variable (ident print))
|
|
|
|
|
(list (variable (ident q)))
|
|
|
|
|
(list))))
|
2018-02-15 22:53:19 +01:00
|
|
|
(assign
|
2018-04-09 19:34:51 -07:00
|
|
|
(list (variable (ident w)))
|
|
|
|
|
(unary minus
|
|
|
|
|
(variable (ident z))))
|
2018-02-15 22:53:19 +01:00
|
|
|
(if
|
|
|
|
|
(and
|
|
|
|
|
(and
|
|
|
|
|
(not (variable (ident x)))
|
|
|
|
|
(not (variable (ident y))))
|
|
|
|
|
(variable (ident z)))
|
|
|
|
|
(list
|
|
|
|
|
(assign
|
2018-04-09 19:34:51 -07:00
|
|
|
(list (variable (ident m)))
|
2018-02-15 22:53:19 +01:00
|
|
|
(if
|
|
|
|
|
(not (variable (ident z)))
|
|
|
|
|
(variable (ident x))
|
|
|
|
|
(variable (ident y)))))
|
|
|
|
|
(list))
|
|
|
|
|
(while
|
|
|
|
|
(and
|
|
|
|
|
(<
|
|
|
|
|
(variable (ident x))
|
|
|
|
|
(variable (ident y)))
|
|
|
|
|
(>
|
|
|
|
|
(variable (ident y))
|
|
|
|
|
(variable (ident z))))
|
|
|
|
|
(list
|
|
|
|
|
(assign
|
2018-04-09 19:34:51 -07:00
|
|
|
(list (variable (ident q)))
|
2018-02-15 22:53:19 +01:00
|
|
|
(variable (ident x)))))
|
2018-11-01 09:58:37 -07:00
|
|
|
(assert
|
|
|
|
|
(eq (const 1) (const 1))
|
|
|
|
|
(option (string_literal hello)))
|
2018-03-06 13:14:53 +01:00
|
|
|
(return
|
|
|
|
|
(list (variable (ident x))))))
|