Dhruv Choudhary f0ea6862ba Support for pruning delays in Adagrad Optimizer (#34527) ...

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34527

Adding support for prune_delays and prune ratios in Adagrad optimizer.

Test Plan:
Tested via unit tests in masked_adagrad_optimizer_test. Added unit test  for prune_delay versions of MaskedAdagrad

buck build caffe2/caffe2/fb/optimizers:masked_adagrad_optimizer_test; buck-out/gen/caffe2/caffe2/fb/optimizers/masked_adagrad_optimizer_test#binary.par

buck test caffe2/caffe2/fb/dper/layer_models/tests/split_1:sparse_nn_test -- 'test_pruning'

All Dper tests passed https://our.intern.facebook.com/intern/testinfra/testrun/7599824380741217

Reviewed By: chocjy

Differential Revision: D20313419

fbshipit-source-id: 5c2c8d4e0fc2ec538bcd6f145c6b87a2381f90f3

2020-04-09 12:59:23 -07:00

..

benchmarks

[caffe2] open source 2/4-bit SLS operators (#34903)

2020-03-17 22:55:10 -07:00

docs

…

examples

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

helpers

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

ideep

caffe2 python ideep conv_op test_int8_convolution skip for python 3

2019-10-08 21:31:11 -07:00

layers

Move gating part of SparseFeatureGating to local

2020-02-16 14:18:27 -08:00

mint

…

mkl

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

modeling

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

models

Make caffe2/caffe2/python/models/seq2seq python3 compatible

2020-02-04 10:51:47 -08:00

onnx

Revert D20121169: [pytorch][PR] ONNX Export Support for CrossEntropyLoss

2020-03-12 20:30:54 -07:00

operator_test

idtt + zch distributed inference (#35763)

2020-04-03 12:09:34 -07:00

predictor

[DPER3][Shape inference] Update Shape Information in dper3 backend (#34475)

2020-03-19 13:49:34 -07:00

rnn

…

serialized_test

[Caffe2] Fix shape inference for element-wise operators (#33431)

2020-02-25 09:03:06 -08:00

test

…

trt

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

__init__.py

Fix dll load logic for Python 3.8 on Windows (#32215)

2020-01-22 08:33:34 -08:00

_import_c_extension.py

[AMD] Remove num_gpu check for remote execution (#34318)

2020-03-06 09:53:57 -08:00

allcompare_test.py

…

attention.py

…

benchmark_generator.py

…

binarysize.py

…

brew_test.py

…

brew.py

…

build.py

…

cached_reader.py

…

caffe_translator_test.py

…

caffe_translator.py

…

checkpoint_test.py

…

checkpoint.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

CMakeLists.txt

…

cnn.py

…

compatibility.py

…

context_test.py

…

context.py

…

control_ops_grad_test.py

Fix the weird bug in control_flow_op_test.py (#26931)

2019-09-26 20:44:03 -07:00

control_ops_grad.py

…

control_ops_util.py

…

control_test.py

…

control.py

…

convert_test.py

…

convert.py

…

convnet_benchmarks_test.py

…

convnet_benchmarks.py

…

core_gradients_test.py

Back out "Back out "[Caffe2] Fix device_option propagation"" (#25908)

2019-09-17 04:01:36 -07:00

core_test.py

…

core.py

[net_transform] only skip ConstantFill for autogen_grad (#34628)

2020-03-11 19:09:52 -07:00

crf_predict.py

…

crf_viterbi_test.py

…

crf.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

data_parallel_model_test.py

Skips test_equiv_recurrent (#29255)

2019-11-06 13:29:23 -08:00

data_parallel_model.py

…

data_workers_test.py

Disables test_atomic_ops and testInputOrder (#29145)

2019-11-05 16:53:53 -08:00

data_workers.py

…

dataio_test.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

dataio.py

…

dataset.py

…

db_file_reader.py

…

db_test.py

…

device_checker.py

…

dlpack.h

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

dyndep.py

guard dyndep with a lock (#26153)

2019-09-13 11:38:14 -07:00

embedding_generation_benchmark.py

…

experiment_util.py

…

extension_loader.py

…

filler_test.py

…

functional_test.py

…

functional.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

fused_8bit_rowwise_conversion_ops_test.py

[caffe2] make fused rowwise quant/dequant op work for N-dim tensors (#33426)

2020-02-19 23:29:42 -08:00

gradient_check_test.py

…

gradient_checker.py

[caffe2] fix type and shape inference for common gradient ops (#35857)

2020-04-02 11:17:04 -07:00

gru_cell.py

…

hip_test_util.py

…

hsm_util.py

…

hypothesis_test_util.py

[caffe2] fix type and shape inference for common gradient ops (#35857)

2020-04-02 11:17:04 -07:00

hypothesis_test.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

ideep_test_util.py

…

layer_model_helper.py

Add transfer_learning_blob_name_mappings into layer_model_helper to support layer model transfer learning

2020-03-18 15:28:00 -07:00

layer_model_instantiator.py

…

layer_parameter_sharing_test.py

…

layer_test_util.py

…

layers_test.py

FCTransposed to FbFCPacked (#29766)

2019-12-10 10:18:21 -08:00

lengths_reducer_fused_8bit_rowwise_ops_test.py

[caffe2] fix how np.clip is used in lengths_reducer_fused_{4,8}_rowwise_ops_test (#32086)

2020-01-14 22:53:28 -08:00

lengths_reducer_rowwise_8bit_ops_test.py

…

lstm_benchmark.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

memonger_test.py

…

memonger.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

mkl_test_util.py

…

model_device_test.py

…

model_helper_test.py

…

model_helper.py

Fix TensorProtosDBInput AttributeError (#32274)

2020-01-29 12:05:43 -08:00

modifier_context.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

mpi_python.cc

Replace c10::guts::stuff with std::stuff (#30915)

2019-12-16 13:57:19 -08:00

muji_test.py

…

muji.py

…

net_builder_test.py

…

net_builder.py

…

net_drawer.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

net_printer_test.py

…

net_printer.py

…

nomnigraph_test.py

…

nomnigraph_transformations_test.py

…

nomnigraph_transformations.py

…

nomnigraph.py

…

normalizer_context.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

normalizer_test.py

…

normalizer.py

Scale init for batch-norm and layer-norm (#31983)

2020-01-10 11:55:56 -08:00

numa_benchmark.py

…

numa_test.py

…

observer_test.py

…

operator_fp_exceptions_test.py

…

optimizer_context.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

optimizer_test_util.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

optimizer_test.py

Build Unit Test of SparseRAdam

2019-11-18 15:22:37 -08:00

optimizer.py

Support for pruning delays in Adagrad Optimizer (#34527)

2020-04-09 12:59:23 -07:00

parallel_workers_test.py

ParallelWorkersTest.testParallelWorkersInitFun is flaky (#29045)

2019-11-01 13:59:02 -07:00

parallel_workers.py

get rid of deprecated thread.isAlive() to use py2.6 modern form is_alive()

2019-10-22 15:37:31 -07:00

parallelize_bmuf_distributed_test.py

…

pipeline_test.py

…

pipeline.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

predictor_constants.py

…

pybind_state_dlpack.cc

…

pybind_state_dlpack.h

…

pybind_state_gpu.cc

…

pybind_state_hip.cc

Make caffe2/fb folder compatible with AMD (#29131)

2019-11-04 16:40:29 -08:00

pybind_state_ideep.cc

Upgrade MKL-DNN to DNNL v1.2 (#32422)

2020-03-26 22:07:59 -07:00

pybind_state_int8.cc

…

pybind_state_nomni.cc

…

pybind_state_registry.cc

…

pybind_state_registry.h

…

pybind_state.cc

Enable splitSparseLengthsSumSparse in onnxifi (#35555)

2020-03-30 13:34:59 -07:00

pybind_state.h

caffe2: preserve python exception type from PythonOp (#36267)

2020-04-09 12:43:24 -07:00

python_op_test.py

caffe2: preserve python exception type from PythonOp (#36267)

2020-04-09 12:43:24 -07:00

queue_util.py

…

record_queue.py

…

recurrent.py

…

regularizer_context.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

regularizer_test.py

…

regularizer.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

rnn_cell.py

[caffe2] Remove python2 from operator_test (#33977)

2020-03-02 08:55:53 -08:00

schema_test.py

…

schema.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

scope_test.py

…

scope.py

…

session_test.py

…

session.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

sparse_to_dense_mask_test.py

…

sparse_to_dense_test.py

…

task_test.py

…

task.py

Fix typos (#30606)

2019-12-02 20:17:42 -08:00

test_util.py

…

text_file_reader.py

…

timeout_guard.py

…

toy_regression_test.py

…

transformations_test.py

…

transformations.py

…

tt_core_test.py

…

tt_core.py

…

utils_test.py

[C2] Introduce extra_info force CPU tags for auto-generated iteration counter blobs (#32607)

2020-02-05 23:49:27 -08:00

utils.py

[C2] Introduce extra_info force CPU tags for auto-generated iteration counter blobs (#32607)

2020-02-05 23:49:27 -08:00

visualize.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00

workspace_test.py

Revert "Revert D18171156: Merge Tensor and Variable." (#29299)

2019-11-08 09:11:20 -08:00

workspace.py

Fix typos, via a Levenshtein-type corrector (#31523)

2020-01-17 16:03:19 -08:00