Aapo Kyrola cdb50fbf2b add optimizer support to data_parallel_model; Use MomentumSGDUpdate ...

Summary:
This diff does two things:
- add supports for optimizer to data_parallel_model. User can supply optimizer_builder_fun instead of param_update_builder_fun. The latter is called for each GPU separately with proper namescope and devicescope, while optimizer builder only is called once and adds optimizes to the whole model.

- use MomentumSGDUpdate instead of MomentumSGD + WeightedSum. This bring major perf benefits.

Changes resnet50 trainer to use optimizer.

This relies on D5133652

Reviewed By: dzhulgakov

Differential Revision: D5142973

fbshipit-source-id: 98e1114f5fae6c657314b3296841ae2dad0dc0e2

2017-05-30 12:49:57 -07:00

..

char_rnn.py

rnn with brew

2017-05-16 13:33:44 -07:00

lmdb_create_example.py

doxygen python block added

2017-03-29 06:46:16 -07:00

resnet50_trainer.py

add optimizer support to data_parallel_model; Use MomentumSGDUpdate

2017-05-30 12:49:57 -07:00