* RecoveryState is updated before confirmation(s)
* New confirmation prompt that supports both cloning and imaging modes
* Refactored show_selection_details() to use new objects
* Allows resumed state to be detected and prompt switched to "Resume?"
* Renamed function show_safety_check() to double_confirm_clone() for clarity
* Dest/map paths are now set in two steps:
* The filename prefix is set when creating the DevObj()
* The full paths are set when creating the BlockPair()
* Merged dest safety checks into RecoveryState.add_block_pair()
* Mostly check_dest_paths() and dest_safety_check()
* Moved dir RWX checks to is_writable_dir()
* Moved mount RW check to is_writable_filesystem()
* Started merging menu_clone() and menu_image() into menu_ddrescue()
* Colored report data is generated during obj instantiation
* Code has been moved into its own function
* Entire colored string is now stored for each Obj
* (Should make show_selection, etc more mode/Obj agnostic)
* loopback_dev vs image_path is now better separated
* losetup is called in ImageObj.set_details()
* loopback -D is still called during program cleanup/wrapup
* get_device_size_in_bytes() has been renamed get_size_in_bytes()
* load_map() is called on BlockPair() instantiation
* This partially replaces resume_from_map()
* Also fixed improper method declarations lacking the self argument
Added two classes:
* BlockPair()
* Track source/dest pair specific data
* update_progress() method for its own data
* RecoveryState()
* Track BlockPair objects and overall state
* update_progress() method for overall data
Reasons:
* Code readability
* Better status updates, code currently split between:
* get_recovery_scope_size()
* resume_from_map()
* update_progress()
* Functions that should probably be merged into other functions:
* get_recovery_scope_size()
* set_dest_image_paths()
* check_dest_paths()
* Logic that needs to be cleaned up:
* Calculating overall recovery size
* Pass "Done"ness and status strings need separated
* Pass "Done"ness at the device and overall levels
* Updating output for side pane status display
* --test-mode disabled by default
* Fixed bug that prevented escaping auto_run via Ctrl-c
* Fixed no-trim / no-scrape flag handling
* Only proceed device(s) have been selected in menu_select_children
* Added read_map_file() which uses ddrescuelog to create dict of current state
* Added --test-mode= option to expert menu
* Add size (in bytes) to all devs
* Allows to calculate real total percent recovered
* Detect 100% completion via ddrescuelog -D
* Moved mark_complete / mark_incomplete code to update_progress()
* Update progress every 30s during ddrescue passes
* Fixed auto_run logic
* Enabled by default
* Based on static thresholds per pass.
* Pass 1: 85%
* Pass 2: 98%
* If using child devices, all must be above the threshold to continue
* Updating device / child device status/progress done in mark_*() functions
* Add current pass description to main menu
* Current pass (overall) only updated if all children have passed
* Fix Pass 4 crash
* Refresh rate is now handled by 'watch --interval'
* Allows for much simpler ddrescue execution / tracking
* Removed all 'SMART Report' sections from functions/ddrescue.py
* functions/hw_diags.py has been further extended
* Supports full device paths (only for displaying attributes ATM)
* Adds a timestamp when only displaying attributes
* Common code moved to select_device()
* Existing select_device() renamed menu_select_device()
* Fixed skip_device code
* Refactored source/dest vars into dicts
* Added confirmation after source/dest are selected
* Renamed get_mounted_data to get_mounted_volumes
* Report data is now a dict for better clarity
* Widened report hoping that LVM names will fit (they probably wont)
* This fixes#38
* Show UNKNOWN for missing DMI variables
* Suppress errors for systems w/out sound cards
* Suppress errors for glxinfo
* Keep revision numbers for network devices
* Better names (e.g. 'Letter' instead of 'Source')
* Added 'Local' bool to simplify network-vs-local installs
* Only reassign the source letter if necessary
* Bugfix: Update 'Path' when reassigning a volume letter
* Fixes#33 (again)
* Known build data moved to settings/windows_builds.py
* OS version statically set in windows_builds.py
* Added Insider Previews to recognized builds
* Shown in red due to their very short support windows
* Windows 8 and Windows 8.1 are now separate versions
* Added codenames to Vista and 7 (for fun!)
* Unrecognized builds have the CurrentBuild number in the DisplayName
* All tests: runs if SMART=CS/OVERRIDE & BADBLOCKS=CS
* All drive tests: runs if SMART=CS/OVERRIDE & BADBLOCKS=CS
* I/O tests are readonly, blocksize=4M, and limited to 16 Gb
* Fix issue #23
* Using the BASH3 Boilerplate as a base for the new script
* The new version requires explicitly setting all sources
* Will avoid ambiguous matches / collision issues seen in issue #15
* Better ensures that the intended sources are used
* Still verify that '.bin' exists to ensure a valid "Main Kit" source
* Fixes recursion bug when Windows.old folders are present
* Combined logic for file/folder sources and WIM sources
* Code uses proper folder separators for the running OS
* (e.g. '\' for Windows and '/' for the rest)
* Renamed run_smart to run_nvme_smart to match rest of the code
* Mark NVMe/SMART status as NS earlier in a couple more cases
* Prevent crash when all tests are run and then aborted during Prime95
* The script waits for the self-test for polling-time+5min
* Unless it detects completion after 60 seconds
* Fixed missing linebreaks in progress panes
* BREAKING: TESTS data structure reworked to support per-dev status
* Renamed 'Health OK' to 'Quick Health OK'
* Better implies that we're not done testing
* Renamed SMART sections to NVMe/SMART
* Dropping SMART short-tests for NVMe drives
* Replaced smartmontools with smartmontool-svn
* Adds JSON exporting which makes parsing the data easier
* Using nvme-cli for NVMe drives instead of smartctl
* NVMe support in smartctl is still experimental and inconsistent
* New "Health OK" detection
* This is used to determine if a disk is okay to be tested
* e.g. an IDE disk w/out any SMART data now requires explicit overriding
* NVMe disks are set using the "critical_warning" attribute
* SMART disks are set using the "overall-health" attribute
* Code has been merged into Build Linux
* (releng copy is updated in place)
* SSH key for wktech now generated at build time
* Hostname is now set using main.py
* Passwords are now set using main.py
* Timezone is now set using main.py
## Main Kit ##
* Updated build script to work with new folder layout
* Browser sections have better profile & installation detection/reporting
* Browser extension section updated to support Firefox Quantum
* User Data Transfers will display local sources before remote sources
* Various bugfixes
## Windows PE ##
* WizardKit PE has been integrated into this project
* The build script has been replaced with a new PowerShell script
* This downloads the tools, builds a new PE image, and exports x32 & x64 ISOs
* main.py is opened for editing before the PE images are built
* ISO names use KIT_NAME_SHORT (e.g. WK-WinPE-20XX-XX-XX-amd64.iso)
* BitLocker CLI tool "manage-bde" has been added
* All PE Python scripts have been updated
* These scripts also use the main.py file for customization
* Better handling of damaged drives during backup & setup sections
* Partition type descriptions have been trimmed to fit on a single line
* Backup and Setup sections are now logged
* These logs can be reviewed and saved before returning to the main menu
* "No disk" errors seen when scanning empty card-readers should be silenced
* Various bugfixes
* Safer registry queries
* Only query CurrentBuild, CurrentVersion, and ProductName
* Service Pack / update status determined by the build number
* Set build & version to -1 instead of 'Unknown' if str to int/float fails
* Might prevent crashes by not mixing str/num types
* Show download progress again
* Done by leaving $progressPreference set to system default
* If there are errors during download, then bail before extracting data
* Moving to a new style guide for PowerShell scripts
* Using this: https://poshcode.gitbooks.io/powershell-practice-and-style/
* Encoding set to UTF-8 BOM instead of UTF-8
## Major Update ##
* Build Kit.cmd
* This script will build a fully functional kit requiring only PowerShell 3.0+ be installed prior.
* All launchers are now generated during builds and updates (by update_kit.py)
* This allows for a much leaner code-base to manage
* Lots of bugfixes, cleanup, and other adjustments.
* VCRedist 2017 installs as an update for 2015 (http://disq.us/p/1hjcta0)
* This prevents installing or uninstalling the 2015 versions after installing 2017
* (Because both 2015 and 2017 use these regkeys: HKEY_LOCAL_MACHINE(Wow6432Node|)\SOFTWARE\Microsoft\DevDiv\VC\Servicing\14.0)
If Launch.cmd aborted it would cause the launcher to abort as well.
If the error was handled by Launch.cmd then we can assume the launcher ran correctly.
* Generated launchers were using inconsistent endings causing very strange crashes.
* New plan: strip current endings and let Python handle them instead.
# Plan #
* Repo should be in a non-functional state
* This is to avoid confusion when most, if not all, scripts fail to run when first downloaded.
* The first step should be to run Build Kit.cmd which does the following
* Downloads 7-Zip, ConEmu, Notepad++, & Python (including wheel packages)
* (i.e. just enough to run update_kit.py)
* Open .bin/Scripts/settings/main.py in Notepad++ for configuration
* Move the kit folders into place
* (i.e. /.root_items/* ->> /, /Build Kit.cmd --> /.bin/Scripts/Build Kit.cmd, etc)
* Added Build Kit.cmd
* Launches build_kit.ps1
* Added build_kit.ps1
* Downloads 7-Zip, ConEmu, and Python
* Launches update_kit.py unless errors are encountered
* Removed all launchers
* update_kit.py will generate them in a future update
* Removed PywScript
* Setting L_NCMD with PSScript or PyScript will now use their respective native windows (instead of ConEmu)
* Relaunch in ConEmu code reworked to avoid launching ConEmu twice in this case
* i.e. CMD (Launch.cmd) --> ConEmu (Launch.cmd) --> ConEmu (some_script.py)
* Updated PSScript section to match PyScript
## MAJOR refactoring done ##
* All .cmd Command scripts
* Brandind / Settings variables now set via .bin/Scripts/settings/main.py
* Window titles now set using KIT_FULL_NAME
* All .py Python scripts
* All ClientDir paths should now use KIT_SHORT_NAME
* Long lines wrapped to better follow PEP8
* String formatting now more consistant
* Updated run_program() and popen_program() calls to use lists
* (e.g. cmd = ['', '', '']; run_program(cmd))
** Should improve clarity IMO
* Update window titles AFTER init_global_vars() so KIT_FULL_NAME can be used
* Branding / Settings
* Support tech now configurable
* (e.g. "Please let {tech} know and they'll look into it")
* Timezone now configurable
* Upload info can now be disabled/enabled in .bin/Scripts/settings/main.py
* CHKDSK
* Combined read-only and fix scripts and added menu
* DISM
* Combined ScanHealth and RestoreHealth scripts and added menu
* functions/common.py
* BREAKING: run_program() and popen_program() no longer accept 'args' variable
* Misc
* Removed Win7 NVMe launcher
* Never used and Win7 is deprecated
* Removed "DeviceRemover" and "Display Driver Uninstaller" launchers
* Both cut too deep and were not useful
* Removed Nirsoft utilities and Sysinternals Suite launchers
* Too many tools unused.
* Added .url links to the websites in case the tools are needed
* Replaced WinDirStat with TreeSizeFree
* Replaced Q-Dir launcher with XYplorer launcher
* Q-Dir was running into issues on Windows 10
* Removed C.IntRep, ESET, and MBAM launchers from "OSR & VR"
* Removed JRT
* Deprecated and discontinued by MBAM
* Removed unsupported QuickBooks launchers (2014 and older)
* Removed unsupported Office launchers (2010 and 2013\365)
* Removed "Revo Uninstaller" launcher
* Removed infrequently used tools from "Diagnostics"
* Auslogics DiskDefrag
* BatteryInfoView
* BIOSCodes
* GpuTest
* HeavyLoad
* Bugfixes
* major_exception() try-blocks should catch CTL+c again
* Allows for manual script bailing
## MAJOR refactoring in progress ##
# BROKEN SCRIPTS #
(These scripts need updated to use the new global_vars & try_and_print)
* user_data_transfer.py
* install_sw_bundle.py
* All Python scripts
* PLAN: Replace vars_wk with global_vars
* functions.py
* PLAN: Will hold (nearly?) all functions going forward
* Should simplify the remaining scripts
* (e.g. reset_browsers and sw_diagnostics both had a backup_browsers())
* PLAN: Move all program definitions to functions.py
* (e.g. vars_wk['SevenZip'] --> global_vars['Programs']['SevenZip'])
* Added major_exception()
* Should be called if anything goes wrong to alert user and provide info
* Added non_clobber_rename()
* Appends a number (if necessary) to avoid overwriting data
* Added popen_program()
* Removes subprocess req for sw_checklist
* Added try_and_print()
* Run a function and show CS/NS/etc
* Can add additional "result" strings via Exception classes
* Passes unrecognized args/kwargs to the function
* Used by scripts to standardize message formatting and error handling
* exit_script() now handles opening the log (if set) and killing caffeine.exe
* Refactored init_vars_wk/os code to work with global_vars
* Uses the try_and_print function for better formatting and error handling
* BREAKING: 'OS' vars now located in global_vars['OS'] (including 'Arch')
* OS labeled as 'outdated', 'very outdated', and 'unrecognized' as needed
* BREAKING: renamed PROGRAMS to TOOLS
* Expanded TOOLS
* Removed log_file as a required argument from all functions
* Removed vars_wk/global_vars as a required argument from all functions
* Removed unused WinPE functions
* Sorted functions
* user_checklist() (formerly reset_browsers())
* Most functions moved to functions.py (see above)
* Browsers
* Work has been split into backup, reset, and config sections
* Script now asks all questions before starting backup/reset/config
* Script now forces the homepage to DEFAULT_HOMEPAGE (set in functions.py)
* (i.e. it no longer asks first, it just does it)
* Current homepages are listed where possible before backup/reset/config
* Launch/Launchers
* Added QuickBooks support
* Removed launchers
* CPU-Z
* HWMonitor
* PerfMonitor2
* SanDisk Express Cache
* Shortcut Cleaner
* SuperAntiSpyware (SAS)
* Replaced Notepad2-Mod with Notepad++
* NEW: CompressedBin folder .cbin
* This folder holds compressed versions of what was in .bin
* These files are 7-zip compressed using file + header encryption
* (This is to avoid false-positive AV alerts)
* NEW: Python conversion
* All scripts rewritten / ported to Python
* All PoSH scripts removed
* All scripts adjusted to no longer use vars as a variable
* This is because vars is a built-in function
* Also, slightly more clear as vars_wk == variables for Wizard Kit
* vars_os merged with vars_wk for since both are based on the current system
* Launch.cmd no longer uses %path% and instead directly modifies L_PATH
* fixes bug where an empty %PATH% would halt various scripts
* Copy WizardKit
* Now only copies the required folders in .bin
* Avoids potentially massive slowdowns on often-used UFDs
* Updated to support the new .cbin folder
* User Data Transfer expanded
* File-based main data selection is now done first
* Default inclusions and exclusions adjusted
* Cleanup should now unhide TransferDir
* Launch.cmd and Launchers updated
* Launch and Launcher_Template reworked to support the new .cbin method
* Launch will extract the archive (if exists) and then launch the item
* Launch.cmd now automatically reloads inside ConEmu
* Launch.cmd now runs from L_ITEM's dir in most cases
* Added L_NCMD to use the native command window instead of ConEmu
* Added PywScript mode that uses Python native console window
* Launchers are customized at the top of the files now
* FindBin and Flags functions have been moved to the end of the file
* Launchers and Launch.cmd now use more descriptive variable names
* Helps readability and troubleshooting
* Ported code from copy_office.cmd into Launch.cmd
* Office setup local folders now have better naming
* Scripts should now print details about the ENV when aborting
* Should help diagnose future breaks, typos, etc..
* Added a gen_office.bash script for creating the Office Setup Launchers
* Fixed Office variable issue (needed delayedexpansion)
* SW Diagnostics / SW Checklist
* (Re)added Kill All Processes section using ProcessKiller 2.0 from Tron
* Added Everything - dumps a file listing of the %systemdrive%
* Added Opera to the browser backup section
* AdwCleaner is no longer removed during the checklist
* HWiNFO has replaced HWMonitor due to false AMD/ATI readings
* HWiNFO is not opened until after Enter is pressed to exit the checklist
* Installed OS warnings expanded to mark many more versions as outdated
* SAS is no longer force-removed at the end of the script
* The new user data size function is reading data for all users
# Migration to Python started #
* PoSH has an extreme slowdown for some systems while it runs an optimization
** pass for .NET on the first PoSH script execution.
** This is reason enough to move to an alternative.
* New additions:
* User Data Transfer script
* Will extract from a WIM or copy from a folder
* Uses wimlib-imagex for images and FastCopy for folders
* Removes undesired items after transfer/extraction
* HWiNFO
* Missing ODD repair registry patch
* Q-Dir
* SW Bundle install script
* ConEmu
* Moving back to ConEmu for better performance.
* Copy-WizardKit
* Now uses FastCopy
* functions.py
* Ported init.ps1 to Python using functions.py (from WinPE) as a base
* Launch.cmd
* Elevating programs/scripts now done using a temp VBScript file
* Can run Python scripts (using either the 32 or 64 bit runtime)
* transferred_keys.cmd
* Expanded searched paths
* Misc
* Lots of variables and files renamed
* Lots of hard-coded paths are now in variables
* Should only be set in scripts in %bin%\Scripts
* Moved a subset of the Diagnostics launchers to a new 'Extras' folder
* The launchers moved are those that are less-often used
* Refactored FindBin code to be more concise
* Renamed "KitDir" "ClientDir" to indicate that it is on the client's system
* Removed GeForce Experience launcher as it now requires an account
* Added link to NVIDIA's driver webpage to download the correct driver
* Removed AMD's Gaming Evolved launcher
* This is usually bundled with the GPU driver anyway
* Switched back to ConEmu
* Variable and script names are now more descriptive
* i.e. checklist -> final_checklist, and HH -> %kit_dir%
* (not happy with %kit_dir%, will probably change again)
Major .bin and extraction code overhaul
* All .cmd files now have code for searching the for the .bin folder
* This starts at the script's parent folder and moves upward
* If no .bin folder is found then print an error to the screen
* 7z has been replaced with 7za
* This is for better handling of x32/x64 usage
* This also fixed a few bugs with extracting ProduKey/MailPassView/etc
* We no longer require the full power of 7z.exe/7z.dll
* (Since wimlib-imagex will probably be used for WIM files soon-ish)
* Bugfixes
* A few calls of pushd were unsafe (without quotes)
* SFC scan no longer closes immediately
* Added OutlookAttachView
* Added PerfMonitor2
* CHKDSK
* Will use the newer repair features on Windows 8 or newer
* MailPassView & ProduKey
* Now saved in a 7-Zip encrypted archive to avoid deletion by AV
* SW Checklist / Diagnostics
* Windows 10 Blocked automatically
* (Too late but whatevs it was in testing before the cutoff)
* Bugfixes
* Activate Windows (with BIOS key)
* Wasn't extracting ProduKey before trying to run it
* SW Diagnostics
* Should now report the preview size from BleachBit correctly
* DISM
* Much safer WinVer check
* Enter / Exit SafeMode
* Now enables MSI (un)installers in SafeMode w/ Networking
* Final Checklist
* NEW: can now be run directly from UFD
* Hide Windows 10 Upgrade.reg
* Added more registry blocks
* Removed Chocolately
* Too much volatility
* Ninite will regain default installer status
* Replaced ConEmu with Cmder
* Cmder has a heavily modified ConEmu
* Changed background
* Added Cmder launchers in Misc
* SW Diagnostics
* NEW: can now be run directly from UFD
* BleachBit / Winapp2.ini updated
* Expanded selections
* Now only checks temp data size; DOES NOT delete temp data
* Internet connection test will now loop until the manually exited
* Now warns you that it can't check Windows activation in SafeMode
* Bugfixes
* reset_browsers.ps1
* Should skip "resetting" the backup folders (i.e. Default.wkbak)
