![]() |
Quote:
HDDs are very slow, I recently upgraded to nvme based storage and I can feel the speed. |
Dedup
Quote:
srep does a very good job finding matches that are located far away! that in order to happen makes a dictionary! IF you sort the files you feed srep you can actually reduce ram needed and its speed Sorting preprocession can speedup the process and cost less ram !! and remove IO overhead because NO temps |
Quote:
Quote:
Quote:
Believe me I have several ideas of how to reduce memory usage before even relying on virtual memory. Optimisation is my middle name. |
Quote:
if you add stdio, it would be great.. |
xcompress if pure windows 10 compression, and i think it supports it by default.
|
1 Attachment(s)
Can you guys test this. I added preflate as an alternative of reflate in case if you are having issues with crc errors when you use reflate or prolonged precompression times such as this:
Quote:
I ran a benchmark and here are the results benchmark results on "game1.resources" from "Dishonored Death of the Outsiders" xtool_2019: Code:
8.30 GB >> ?? >> ?? >> ?? Code:
8.30 GB >> 11.3 GB >> 8.74 GB >> 6.24 GB Code:
8.30 GB >> 11.1 GB >> 8.79 GB >> 6.28 GB |
Dishonored Death of the Outsiders
game1.resources Quote:
|
1 Attachment(s)
Quote:
|
what is ztool and xtool and lolz?
|
Quote:
|
|
Quote:
|
can't wait to test the new xtool :D
|
1 Attachment(s)
Here's an early working version of xtool. I'm still busy adding preflate, reflate and other things so currently only the zlib function works which means this will not work on Dishonored 2 or DOOM but it will work on most titles.
Drag and drop game files that are zlib compressed on the bat file Things to test: * Stability * Can you cancel installation when xtool was used in setup * If you have 16 threads or more, make xtool use 100% of your cpu to see if it can handle it * if you have 32GB+ ram, set high chunk size, something like -c1gb and see if xtool was able to allocate the necessary memory * If you have potato pc, don't worry I got a job for you :) check if there are no problems if you are using 1 thread both in encoding and decoding Whatever you do, don't place preflate_dll.dll near the exe, not yet. else it will fail, the preflate code hasn't been added yet :D Notes: The file check must report FC: no differences encountered if the test went successfully, if it failed kindly upload the file it failed on. This xtool uses slightly more memory than the 2019 version, I opted for stability than less memory usage xtool now accepts maths expressions in command line, if you wanted it to use all cores minus 1, you can just write -t100p-1 or -t100-2, totally up to you Next release/test focuses on: + preflate + reflate (reflate actually isn't replaced by preflate, it sometimes gives better output than preflate so I'll keep it just in case you are after best results) + depth setting |
Hello Razor, here is my test (I did make a post before but I had to delete it since I used a reflate files by mistake):
My spec: Ryzen 7 2700 (8c/16t) + 16GB RAM. Bat file editied for 128mb chunk size. Here's is XTool using all 16 of my threads: https://i.imgur.com/jGFDfgB.png Testing on pak file from Astroneer: Start size: 2.13GB End Size: 3.75GB No differences encountered ;) Not entirely sure how I'd go about testing it in a setup... |
All times are GMT -7. The time now is 13:00. |
Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com