pytorch

mirror of https://github.com/zebrajr/pytorch.git synced 2026-01-15 12:15:51 +00:00

Files

Lin Jiang 1f158adeee Add support for attention weight in SparseLookup (#26748 )

Summary:
Support attention weights input to SparseLookup. In attention sum pooling, if attention weights can be pre-calculated before embedding lookup,  they can be passed to SparseLookup and processed by SparseLengthsWeightedSum op. One example is id_score attention sum pooling.

Essentially the net is converted from:
  LengthsSum(Mul(Gather(keys, w), att_weight))
to:
  SpaseLenghtsWeightedSum(keys, w, att_weight)

It unblocks potential efficiency gain with distributed training.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/26748

Test Plan: unit test

Reviewed By: chocjy

Differential Revision: D17553345

Pulled By: wheatkit

fbshipit-source-id: 60cc3c4b0bc1eade5459ac598e85286f3849a412

2019-10-08 20:22:25 -07:00

docs

Fix several DeprecationWarning: invalid escape sequence (#15733 )

2019-01-05 08:53:35 -08:00

examples

Adding models to jenkins benchmark script (#21010 )

2019-05-30 15:17:40 -07:00

helpers

Add elementwise_affine for LayerNormGradientOp (#19982 )

2019-05-03 15:33:46 -07:00

ideep

Removed work around for convolution transpose op since the bug has be… (#22184 )

2019-06-25 14:34:34 -07:00

layers

Add support for attention weight in SparseLookup (#26748 )

2019-10-08 20:22:25 -07:00

mint

re-enable copy of python files, but be careful that the copy is only … (#14982 )

2018-12-11 16:54:08 -08:00

mkl

implement operators for DNNLOWP (#18656 )

2019-04-10 12:04:39 -07:00

modeling

gradient clipping by norm

2019-08-22 11:20:40 -07:00

models

Adding ShufflenetV2 to caffe2's benchmark suite. (#20180 )

2019-05-23 20:40:17 -07:00

onnx

Automatic update of fbcode/onnx to 23bb6ea1a71f08e200114a153f48bd7adb66d486 (#26441 )

2019-09-24 05:38:52 -07:00

operator_test

Expose PiecewiseLinearTransform to PyTorch

2019-09-27 12:49:04 -07:00

predictor

Supporting Manifold DB in Predictor Exporter (#22334 )

2019-07-01 16:17:02 -07:00

rnn

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

serialized_test

Output sequence probability with CTC beam search, optional multiple output sequences (#21927 )

2019-07-02 17:29:13 -07:00

test

Enforce import order to make protobuf cpp implementation in python work (#18560 )

2019-04-03 13:17:08 -07:00

trt

Skip tests if C2/ONNX models cannot be read (#18494 )

2019-03-27 11:21:44 -07:00

__init__.py

Revert #17191 and #17215 that no longer apply on Windows (#17567 )

2019-03-01 10:37:27 -08:00

_import_c_extension.py

Enforce import order to make protobuf cpp implementation in python work (#18560 )

2019-04-03 13:17:08 -07:00

allcompare_test.py

…

attention.py

…

benchmark_generator.py

…

binarysize.py

…

brew_test.py

…

brew.py

Testing for folded conv_bn_relu (#19298 )

2019-04-16 19:04:06 -07:00

build.py

…

cached_reader.py

Pass loop_over optional parameter for cached reader properly. (#21929 )

2019-06-19 18:15:32 -07:00

caffe_translator_test.py

Fix several ResourceWarning: unclosed file (#15746 )

2019-01-09 15:36:53 -08:00

caffe_translator.py

Fix several ResourceWarning: unclosed file (#15746 )

2019-01-09 15:36:53 -08:00

checkpoint_test.py

…

checkpoint.py

Remove setting logger level in caffe2.python.checkpoint (#19803 )

2019-05-10 07:00:58 -07:00

CMakeLists.txt

Fix CMakeLists.txt for Int8 python bindings (#15047 )

2018-12-11 10:48:47 -08:00

cnn.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

compatibility.py

…

context_test.py

…

context.py

…

control_ops_grad_test.py

Fix the weird bug in control_flow_op_test.py (#26931 )

2019-09-26 20:44:03 -07:00

control_ops_grad.py

DeviceScope support for CUDA and testing (#15357 )

2019-01-30 18:42:12 -08:00

control_ops_util.py

…

control_test.py

…

control.py

…

convert_test.py

New serialization format (#12384 )

2018-10-16 16:36:58 -07:00

convert.py

New serialization format (#12384 )

2018-10-16 16:36:58 -07:00

convnet_benchmarks_test.py

Skip convnets benchmark in rocm CI (#17331 )

2019-02-20 21:12:24 -08:00

convnet_benchmarks.py

…

core_gradients_test.py

Back out "Back out "[Caffe2] Fix device_option propagation"" (#25908 )

2019-09-17 04:01:36 -07:00

core_test.py

Extend Net.RunAllOnGPU() to support RecurrentNetwork op (#15713 )

2019-02-08 15:48:42 -08:00

core.py

BlobReference __getattr__ can only throw AttributeError (#26654 )

2019-09-23 13:01:00 -07:00

crf_predict.py

…

crf_viterbi_test.py

…

crf.py

…

data_parallel_model_test.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

data_parallel_model.py

skip import nccl and gloo_gpu in cpu machine (#22522 )

2019-07-10 11:56:56 -07:00

data_workers_test.py

…

data_workers.py

…

dataio_test.py

Pass loop_over optional parameter for cached reader properly. (#21929 )

2019-06-19 18:15:32 -07:00

dataio.py

Rearrange stopping condition in CompositeReader (#20062 )

2019-05-06 15:06:32 -07:00

dataset.py

…

db_file_reader.py

…

db_test.py

…

device_checker.py

…

dlpack.h

Upgrade DLPack

2018-11-12 15:59:46 -08:00

dyndep.py

guard dyndep with a lock (#26153 )

2019-09-13 11:38:14 -07:00

embedding_generation_benchmark.py

…

experiment_util.py

…

extension_loader.py

always restore dlopen flag in dyndep (#22958 )

2019-07-17 10:26:25 -07:00

filler_test.py

caffe2 - Expose tensor filler util to Python (#18886 )

2019-04-08 11:54:10 -07:00

functional_test.py

…

functional.py

…

fused_8bit_rowwise_conversion_ops_test.py

…

gradient_check_test.py

Unify gpu_support variable in python tests (#16748 )

2019-02-07 00:29:51 -08:00

gradient_checker.py

Adding gradient to Boolean Mask operator (#21423 )

2019-06-06 20:48:47 -07:00

gru_cell.py

…

hip_test_util.py

Make CUDNN an alias of MIOPEN for HIP ops (#12278 )

2018-10-24 17:07:31 -07:00

hsm_util.py

…

hypothesis_test_util.py

Hypothesis tests: add ability to enforce shape inference (#23935 )

2019-08-13 05:32:41 -07:00

hypothesis_test.py

Enable Unique operator tests on ROCm

2019-09-11 16:36:14 -07:00

ideep_test_util.py

…

layer_model_helper.py

Integrate FC fp16 exporter into Dper2 (#26582 )

2019-09-29 10:19:28 -07:00

layer_model_instantiator.py

…

layer_parameter_sharing_test.py

Add validator for optimizers when parameters are shared

2019-04-17 21:10:38 -07:00

layer_test_util.py

…

layers_test.py

Add support for attention weight in SparseLookup (#26748 )

2019-10-08 20:22:25 -07:00

lengths_reducer_fused_8bit_rowwise_ops_test.py

make the threshold for acurracy more precise (#17194 )

2019-02-20 13:14:11 -08:00

lengths_reducer_rowwise_8bit_ops_test.py

…

lstm_benchmark.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

memonger_test.py

Unify gpu_support variable in python tests (#16748 )

2019-02-07 00:29:51 -08:00

memonger.py

Fix typo "properlyh"

2019-08-09 11:43:04 -07:00

mkl_test_util.py

…

model_device_test.py

Unify gpu_support variable in python tests (#16748 )

2019-02-07 00:29:51 -08:00

model_helper_test.py

…

model_helper.py

Remove the identical if branch (#18019 )

2019-03-15 13:14:26 -07:00

modifier_context.py

…

mpi_python.cc

…

muji_test.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

muji.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

net_builder_test.py

…

net_builder.py

…

net_drawer.py

Allow customization of blob node in net_drawer (#16915 )

2019-02-12 15:02:50 -08:00

net_printer_test.py

…

net_printer.py

Fix spelling errors (#21665 )

2019-06-13 15:21:55 -07:00

nomnigraph_test.py

nomnigraph - support subgraph visualization (#13795 )

2018-11-16 08:19:20 -08:00

nomnigraph_transformations_test.py

Add transpose network pass (#13437 )

2018-11-01 14:27:07 -07:00

nomnigraph_transformations.py

Add transpose network pass (#13437 )

2018-11-01 14:27:07 -07:00

nomnigraph.py

createUniqueDataNode

2018-10-31 11:16:38 -07:00

normalizer_context.py

…

normalizer_test.py

…

normalizer.py

Enable alternative LayerNorm impl in FisherGan (#12178 )

2018-10-11 17:36:11 -07:00

numa_benchmark.py

Revert D13205604: Move numa.{h, cc} to c10/util

2018-12-07 10:01:25 -08:00

numa_test.py

Move numa.{h, cc} to c10/util (#15024 )

2018-12-12 12:21:10 -08:00

observer_test.py

…

operator_fp_exceptions_test.py

Caffe2 - Add flag to fails if float point exceptions is detected in operator runs (#18040 )

2019-03-16 12:28:05 -07:00

optimizer_context.py

…

optimizer_test_util.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

optimizer_test.py

Unify gpu_support variable in python tests (#16748 )

2019-02-07 00:29:51 -08:00

optimizer.py

Add validator for optimizers when parameters are shared

2019-04-17 21:10:38 -07:00

parallel_workers_test.py

Add event and event_counter columns to caffe2_usage_tracer table (#21739 )

2019-06-13 12:06:02 -07:00

parallel_workers.py

…

parallelize_bmuf_distributed_test.py

Unify gpu_support variable in python tests (#16748 )

2019-02-07 00:29:51 -08:00

pipeline_test.py

…

pipeline.py

…

predictor_constants.py

…

pybind_state_dlpack.cc

Upgrade DLPack

2018-11-12 15:59:46 -08:00

pybind_state_dlpack.h

Remove PythonOp non-CPU path and PytorchOp (#15417 )

2019-01-02 16:36:37 -08:00

pybind_state_gpu.cc

add simple memory analyzer and log warning if GPU underutilized (#21024 )

2019-05-28 19:58:54 -07:00

pybind_state_hip.cc

Fix get_gpu_memory_info in non-cuda builds (#21054 )

2019-05-28 23:05:15 -07:00

pybind_state_ideep.cc

Upgrade mkldnn-bridge for dnnlowp support (#16308 )

2019-04-03 12:47:17 -07:00

pybind_state_int8.cc

Renaming meta() to dtype() - 2/2 (#13334 )

2018-10-30 18:24:30 -07:00

pybind_state_nomni.cc

nomnigraph - support subgraph visualization (#13795 )

2018-11-16 08:19:20 -08:00

pybind_state_registry.cc

…

pybind_state_registry.h

…

pybind_state.cc

weight_names bug fix (#23848 )

2019-08-06 10:58:43 -07:00

pybind_state.h

Support unpickle py2 NetDef object in py3 (#26147 )

2019-09-18 02:02:34 -07:00

python_op_test.py

Clean up a couple of items in the C2 test scaffolding (WIP) (#7847 )

2018-11-07 09:16:13 -08:00

queue_util.py

…

record_queue.py

…

recurrent.py

…

regularizer_context.py

…

regularizer_test.py

Implement "trimmed lasso" regularization and support all available regularization in a single interface (#22966 )

2019-07-17 16:12:31 -07:00

regularizer.py

Implement LpNorm regularizer to be used on the inputs for feature importance (#26376 )

2019-10-03 09:39:42 -07:00

rnn_cell.py

Unify cuda and hip device types in Caffe2 python front end (#14221 )

2018-11-29 14:00:16 -08:00

schema_test.py

Pass LRU hash output evicted_values to SparseLookup (#21389 )

2019-07-02 11:27:37 -07:00

schema.py

Pass LRU hash output evicted_values to SparseLookup (#21389 )

2019-07-02 11:27:37 -07:00

scope_test.py

Add EmptyNameScope to allow you jump out from current scope. (#14631 )

2018-12-12 01:39:50 -08:00

scope.py

Add EmptyNameScope to allow you jump out from current scope. (#14631 )

2018-12-12 01:39:50 -08:00

session_test.py

…

session.py

…

sparse_to_dense_mask_test.py

Increase static tolerance for negative feature ids

2019-05-20 19:09:22 -07:00

sparse_to_dense_test.py

…

task_test.py

caffe2/python/task: added __repr__ methods to all task definitions (#15250 )

2018-12-17 16:02:16 -08:00

task.py

A trivial typo fix in caffe2.python (#15907 )

2019-01-17 04:57:34 -08:00

test_util.py

caffe2 - support flaky operator tests for caffe2 build (#18155 )

2019-03-25 16:58:34 -07:00

text_file_reader.py

Create Node2Vec ModuleKeeper

2019-04-01 10:36:23 -07:00

timeout_guard.py

…

toy_regression_test.py

Enable junk fill for the default CPU allocator (#13377 )

2018-11-08 00:02:37 -08:00

transformations_test.py

Remove sinkMaxPool transformation (#17694 )

2019-03-12 20:10:46 -07:00

transformations.py

support pre-convert filter format for mkldnn training mode and change 'OptimizeForIdeep' to 'OptimizeForMkldnn' (#15171 )

2019-03-29 19:00:48 -07:00

tt_core_test.py

…

tt_core.py

…

utils_test.py

Convert Arguments to dictionary (#13436 )

2018-11-01 14:27:05 -07:00

utils.py

Query caffe2 operator stats for detailed execution info (#20924 )

2019-06-13 23:41:04 -07:00

visualize.py

…

workspace_test.py

Pass Variable into Caffe2 ops, by requiring that the Variable doesn't require grad (#22473 )

2019-07-08 11:31:10 -07:00

workspace.py

Fix get_gpu_memory_info in non-cuda builds (#21054 )

2019-05-28 23:05:15 -07:00