Commit Graph

261 Commits

Author SHA1 Message Date
torzdf
a8f22cc019 Extract updates:
- Default CPU detector to MTCNN
  - add basic Aligner false positive filters
  - Typing: align + plugins
  - Use specific AlignerBatch class for alignment
  -
2022-09-18 19:44:41 +01:00
torzdf
952d79922b Bugfixes:
- Extract - batch mode. Exclude folders with no images
  - Train. Trigger the correct preview/mask update from gui trigger
2022-09-13 18:54:01 +01:00
torzdf
7da2cc3dd2 Training - Use custom preview pop-out 2022-09-07 11:49:52 +01:00
torzdf
ae7793e876 bugfix: convert - Process final items on truncated batch 2022-09-02 12:42:52 +01:00
torzdf
13cfb3f39e extract: Add batch processing mode 2022-08-31 19:48:47 +01:00
torzdf
04b197ec5f convert: reduce RAM consumption 2022-08-27 08:13:45 +01:00
torzdf
1022651eb8 Bugfix: convert - Gif Writer
- Fix non-launch error on Gif Writer
  - convert plugins - linting
  - convert/fs_media/preview/queue_manager - typing
  - Change convert items from dict to Dataclass
2022-08-26 23:56:03 +01:00
torzdf
9e503bdaa2 bugfix: debug landmarks 2022-08-23 19:19:16 +01:00
torzdf
2beceffad9 Data Augmentation update (#1263)
- lib.detected_face
    - Subclass Masks for Landmark based masks
    - Add training mask propery + methods to DetectedFace
  - lib.training_training
    - subclass TrainingDataGenerator for training and preview data
    - Split cache into own module
    - Reduce thread count to 1 to prevent image corruption + data re-use
    - Process on largest model input/output size rather than stored image size
    - Size and crop masks during caching stage
    - Implement ring buffer for data flow
    - Fix preview reload bug
  - augmentation
    - typing
    - switch color aug order
    - better initialization
    - Fix warp + landmark warp to correctly apply at different image scales
    - Slightly improved warp caching
    - Don't store whether image is_preview. Handle all data as training images implicitly
  - plugins.trainer: Typing and fixes to work with trainingdata refactor
2022-08-21 18:59:45 +01:00
torzdf
26e26c6288 Update TF to 2.9
- Update TF to 2.7 to 2.9
    - Bump dependencies
    - Remove decode from pynvml calls
    - force keras predict functions to non-verbose
    - update tests
    - update Tensorboard logging
    - Update docs
2022-08-02 17:12:26 +01:00
torzdf
2ea05623bd Update Distibution Strategies:
- Add Central Storage Stategy
  - Deprecate 'distributed' cli argument
2022-07-19 18:23:16 +01:00
torzdf
c8122bc499 bugfix: Stop preview window from stealing focus 2022-07-06 11:15:51 +01:00
torzdf
3c73ae4ec9 bugfix: Update preview screen in GUI 2022-07-01 19:06:42 +01:00
torzdf
7b9fc0454d Live Preview - Replace cv2 with matplotlib viewer 2022-07-01 13:07:01 +01:00
torzdf
b1a8183ab4 scripts.train - type checking 2022-06-30 18:41:58 +01:00
torzdf
c279ac3518 Disable saving optimizer state option 2022-06-24 13:43:26 +01:00
torzdf
06468c97d4 Training: Add setting option to save optimizer weights 2022-06-23 14:54:09 +01:00
torzdf
0d23714875 bugfix: extract - stop progress bar from going over max value 2022-05-16 23:26:42 +01:00
torzdf
b11ed6ead0 typofix 2022-05-13 12:10:51 +01:00
torzdf
0f7ee1603f training - Enable resize in popup preview image 2022-05-13 12:08:25 +01:00
Dhyey Patel
feac8a0108 trainer: add support for non-interactive jobs (#1193)
Add support for training in non-interactive shell environment such
as Sun Grid Engine, Univa Grid Engine and others.

Tested on Univa Grid Engine.

Reference: https://stackoverflow.com/questions/967369/python-find-out-if-running-in-shell-or-not-e-g-sun-grid-engine-queue.
2022-05-04 23:07:31 +01:00
torzdf
cf4b567cc6 Tensorflow 2.6 Support (#1182)
* lib.cli.launcher - Bump max tf version to 2.6

* Remove pathlib requirement

* Update requirements files

* Update setup.py

* bugfix - GUI: Supress errors when attempting to load previews in extract

* GUI: Suppress ptxas error messages for Windows
2021-09-05 02:25:12 +01:00
torzdf
b5b0da42c8 convert - bugfix - Race condition when checking for predicted mask existence 2021-08-07 12:15:56 +01:00
torzdf
ac22d40a91 extract: mask - Delete any mask from outside of frame boundaries 2021-07-22 13:04:29 +01:00
torzdf
60a4231832 Bugfixes
- extract - debug landmarks - fix non integer errror
  - training (gui) - fix mask toggling
2021-05-30 11:45:03 +01:00
torzdf
18e3139275 Updates
- Change shortcut key to M
  - Slight optimizations
  - GUI support for mask toggling
2021-05-28 19:41:11 +01:00
AnDenixa
486375ec99 - added push t to disable mask display at preview window
trsprmvd

- added extra checks for mask presence in the set
2021-05-28 18:00:08 +01:00
torzdf
3d914ee382 bugfix: Convert - don't error if no mask is selected 2021-05-21 10:52:39 +01:00
torzdf
8ec4b4fb82 bugfix: GUI - Correctly handle shutdown when closing during a running task 2021-05-04 10:07:41 +00:00
torzdf
29cfdaad46 Training startup updates
- Remove alignments path option from cli
  - Restructure training_data.py
  - Load alignment data from PNG header during first epoch
  - lib.image.read_image_batch - Add option to return metadata
  - lib.utils.get_image_paths - Add option for explicit extension
  - plugins.train.trainer._base - remove pre-cache alignments code
  - scripts.train - Check first image in training folders for metadata
  - Documentation
2021-03-21 23:29:31 +00:00
torzdf
cff0f5698b GUI updates
- Control panel fixes
  - Basic themeing support
  - Fix tools to display more natively
  - Give all config items a group
2021-03-16 19:46:14 +00:00
torzdf
8758a29a5c GUI - Fix slider backgrounds for Linux 2021-03-15 00:56:33 +00:00
torzdf
46edd5e5ae GUI fix - Make grey slider backgrounds go away 2021-03-15 00:38:18 +00:00
torzdf
1c0d112588 Minor GUI Updates
- Fix spacing between buttons on Control Panels
  - Configurable background on Control Panels
  - Fix background color of TreeView menu in settings pop up
  - Change TreeView selected item highlight color
  - Change console background color to match control panel
  - Fix unfilled color in control panel background
2021-03-14 23:16:08 +00:00
torzdf
7115ad3550 GUI - Minor Updates
- Add style support to control panel
  - Theme settings to red
  - Use standard tk buttons
  - Fix some graphical glitches
2021-03-14 20:27:04 +00:00
torzdf
5eea32833d GUI - Make groups collapsible 2021-03-14 02:50:06 +00:00
torzdf
1d07dbaa5d train - Add option to output model summary and exit 2021-03-11 01:35:00 +00:00
torzdf
d8d88cb654 scripts.convert -RGB model support 2021-03-10 12:10:05 +00:00
torzdf
ef22c576f6 Bugfix - Extract - Fix skip existing faces 2021-02-20 14:55:45 +00:00
torzdf
770daa3182 Bugfix - scripts.extract
- Store the source filename, not the full frame in png meta header
2021-02-18 10:34:32 +00:00
torzdf
b1cfbe458c Update extracted faces to use PNG EXIF data (#1123)
Documentation
  - Update Usage.md, align.rst and image.rst
lib.image.py
  - read_image - Remove hash return, add metadata return
  - Remove read_image_hash functions
  - Add read_image_meta functios
  - Replace encode_image_with_hash with encode_image (to store metadata)
  - Add png meta reading and writing functions
  - Update Image Loaders/Savers to handle metadata rather than hashes
lib.training_data
  - Naming updates to remove references to hashes
lib.align.Alignments
  - Add versioning notes
  - Increment alignments version to 2.1
  - Deprecate hashing lookup functions
  - Replace filter_hashes with filter_faces
lib.align.detected_face
  - DetectedFace
    - Remove hash property
    - Add png header data serializing/deserializing functions
  - Mask
    - Add png header data serializing/deserializing functions
  - add update_legacy_png_header function to update png meta data
lib.cli.args - Deprecate alignments files for training
- plugins.train.trainer
  - Update alignments/mask code to read png header data
- scripts.convert
  - Aligned images folder - read data from png headers
- scripts.extract
  - Write png header information and no longer store hash of face
- tools.alignments
  - remove leftover-faces, merge and update-hashes jobs
  - Update jobs to use png meta data rather than hashes
- tools.manual
  - Update extract code to output png meta data and don't store hashes
  - Perform check on launch that tool is not pointing at a faces folder
tools.mask
  - Update to use png meta data
tools.sort
  - Update to use png meta data
2021-02-14 16:49:55 +00:00
torzdf
7d44aab33c Minor GUI Updates
- Switch paned window class to ttk
  - Properly pad StatusBar
2021-02-07 02:06:31 +00:00
torzdf
cd24f25f9a Extract - Always output faces as .png 2021-02-05 12:03:06 +00:00
torzdf
d392dfbdf7 Bugfixes
- Manual Tool:
    - Hide annotations for faces not meeting criteria
    - Update landmarks on face add/del
    - Clearer landmark annotations
    - Handle non-numerics in frame number box
  - Training
    - Fix mis-aligned preview images
    - Allows mixing legacy + new alignments for A and B
    - Catch non-training images in training folder
    - Catch inconsistently sized training images
    - Standardize coverage ratio calculation
    - lib.image - Add option to get image shape along with hash
Dfaker model:
  - Add 256px mode
2020-12-15 12:11:26 +00:00
torzdf
05018f6119 Extract - Increase area and move centering (#1095)
* Extract
  - Implement aligner re-feeding
  - Add extract type to pipeline.ExtractMedia
  - Add pose annotation to debug
* Convert
  - implement centering
  - remove usage of feed and reference face properties
  - Remove distributed option from convert
  - Force update of alignments file on legacy receive
* Train
  - Resize preview image to model output size
  - Force legacy centering if centering does not exist in model's state file
  - Enable training on legacy face sets

* Alignments Tool
  - Update draw to include head/pose
  - Remove DFL drop + linting
  - Remove remove-frames job
  - remove align-eyes option
  - Update legacy masks to new extract type
  - Exit if attempting to merge version 1.0 alignments files with version 2.0 alignments files
  - Re-generate thumbnails on legacy upgrade
* Mask Tool
  - Update for new extract + bugfix full frame
* Manual Tool
  - Update to new extraction method
   - Disable legacy alignments,
   - extract box bugfix
   - extract faces - size to 512 and center on head
* Preview Tool
  - Display based on model centering
* Sort Tool
  - Use alignments for sort by face

* lib.aligner
  - Add Pose Class
  - Add AlignedFace Class
  - center _MEAN_FACE on x
  - Add meta information with versioning to alignments file
  - lib.aligner.get_align_matrix to use landmarks not face
  - Refactor aligned faces in lib.faces_detect
* lib.logger
  - larger file log padding
* lib.config
  - Fix global changeable_items
* lib.face_filter
  - Use new extracted face images
* lib.image
  - bump thumbnail default size to 96px
2020-12-08 01:31:56 +00:00
torzdf
b7ecad058d GUI: Consolidate settings to single menu 2020-10-24 00:06:52 +01:00
torzdf
0532c22dbf Bugfix: Extract - Fix Skip Saving Faces 2020-10-19 18:30:56 +01:00
torzdf
fe664e2748 Bugfix - Fix Timelapse when alignments files are used 2020-10-09 23:15:40 +01:00
torzdf
dfc118fd6d bugfix: Training - Make timelapse image extensions case insensitive 2020-09-24 10:21:38 +01:00
torzdf
1daa7dc6f6 GUI - Stats optimization (#1067)
* Faster stat loading + caching

* Compress data in cache

* Optimize some calculations

* Vectorize smoothing

* stats.Calculations optimized

* Load latest training data from live iterator

* Add options to training graph
2020-09-24 01:23:25 +01:00