FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   XTool 2020 (Main Project) (https://fileforums.com/showthread.php?t=102832)

shazzla 14-07-2019 02:08

Did it five times ,as you requested.

Layers of fear 2 - pakchunk2-WindowsNoEditor.pak - OK !
Legacy of the void - data.006 - OK !
X4 Foundations - 05.dat - OK !

Interesting : in all testfiles the last two numbers were the same/file :

05.dat ends : 11372 ,11372.
data.006 : 110507,110507.
pakchunk2-WindowsNoEditor.pak : 34990,34990.
Maybe is it a bug ?!

Thanks for your work !

KaktoR 14-07-2019 04:12

Did it aswell with Kingdom Come Deliverance and a few pdf files.

All seems to be the way it should be!

Razor12911 14-07-2019 15:36

Quote:

Originally Posted by shazzla (Post 481790)
Did it five times ,as you requested.

Layers of fear 2 - pakchunk2-WindowsNoEditor.pak - OK !
Legacy of the void - data.006 - OK !
X4 Foundations - 05.dat - OK !

Interesting : in all testfiles the last two numbers were the same/file :

05.dat ends : 11372 ,11372.
data.006 : 110507,110507.
pakchunk2-WindowsNoEditor.pak : 34990,34990.
Maybe is it a bug ?!

Thanks for your work !

I actually saw those last repeating numbers, I already fixed the problem after uploading the attachment, it's not really a big deal but thanks for the test.

@everyone
there are more tests to come.

doofoo24 14-07-2019 15:41

Quote:

Originally Posted by Razor12911 (Post 481800)
@everyone
there are more tests to come.

lz4,lzo,zstd...
is there any speed improvement in raw2hif_dll/hif2raw_dll, for exmaple xtool is slow on doom/cod ghost ?

Razor12911 15-07-2019 16:22

well that is the thing, I didn't make those libraries, if they are slow, they are slow nothing I can do unless if you contact the creator here:
https://encode.ru/threads/1399-refla...e-recompressor

as for doom, well I had plans for that game, there will be a plugin that will come with xtool called idtech6.dll or something similar. The plugin is specially designed to work best for games that use that engine (also dishonored 2, doto), handling zlib specifically for them unlike the universal zlib that tries all possible methods to give you the best output, the plugin will know beforehand what to do and how to handle the streams from those games and speed up the process.

xtool.exe e:precomp:t100p:idtech6 - -
is how you will use it, there are several games that if they require a special touch, they will just come separate from xtool in plugin form like dunia, unity and etc, which is why I created a separate thread here because I reckon there is going to be many.

Razor12911 17-07-2019 15:19

1 Attachment(s)
Here's a speed test

prepare two types of inputs, one with deflate streams and one without deflate streams, can be a video file.

drag and drop the test files onto test.bat, there will be a console output, upload those results as test report.

The test is between xtool v012 and the upcoming release.

Note: Your input shouldn't be from the game that use idTech6 engine (DOOM, Dishonored 2 and Death of the Outsider)

doofoo24 17-07-2019 16:54

1 Attachment(s)
test on HellbladeSenuasSacrifice (HellbladeGame-WindowsNoEditor.pak)...

Process ID : 5172
Thread ID : 3824
Process Exit Code: 0
Thread Exit Code : 0

User Time : 153.018s
Kernel Time : 28.031s
Process Time : 181.049s
Clock Time : 129.848s

Working Set : 881712 KB
Paged Pool : 109 KB
Nonpaged Pool : 18 KB
Pagefile : 1133300 KB
Page Fault Count : 541119

IO Read : 16928698 KB (in 264520 reads )
IO Write : 26142474 KB (in 3999156 writes)
IO Other : 2 KB (in 130 others)

WOW Super fast 360mb/s with half cpu usage...

Andu21 17-07-2019 20:34

Echo
 
CI_Sc090.mp4

Code:

xtool_new.exe:

Process ID      : 3920
Thread ID        : 2620
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          20.779s
Kernel Time      :          0.265s
Process Time    :          21.044s
Clock Time      :          6.797s

Working Set      :          104788 KB
Paged Pool      :              91 KB
Nonpaged Pool    :              7 KB
Pagefile        :          163964 KB
Page Fault Count : 26277

IO Read          :          354842 KB (in            5548 reads )
IO Write        :          354826 KB (in              46 writes)
IO Other        :              5 KB (in            240 others)

xtool_old.exe:

Process ID      : 4104
Thread ID        : 4888
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          40.856s
Kernel Time      :          0.187s
Process Time    :          41.043s
Clock Time      :          11.781s

Working Set      :          106836 KB
Paged Pool      :              93 KB
Nonpaged Pool    :              7 KB
Pagefile        :          164968 KB
Page Fault Count : 26918

IO Read          :          354843 KB (in            5547 reads )
IO Write        :          354826 KB (in            5545 writes)
IO Other        :              1 KB (in            220 others)

Echo-WindowsNoEditor.pak

Code:

xtool_new.exe:

Process ID      : 4832
Thread ID        : 2644
Process Exit Code: 0
Thread Exit Code : 0

User Time        :        196.046s
Kernel Time      :          3.322s
Process Time    :        199.368s
Clock Time      :        489.965s

Working Set      :          213032 KB
Paged Pool      :              91 KB
Nonpaged Pool    :              9 KB
Pagefile        :          282924 KB
Page Fault Count : 90883

IO Read          :        4233301 KB (in          66149 reads )
IO Write        :        4476621 KB (in          171112 writes)
IO Other        :              6 KB (in            240 others)

xtool_old.exe:

Process ID      : 2736
Thread ID        : 4384
Process Exit Code: 0
Thread Exit Code : 259

User Time        :        376.648s
Kernel Time      :          2.792s
Process Time    :        379.440s
Clock Time      :        464.932s

Working Set      :          214460 KB
Paged Pool      :              93 KB
Nonpaged Pool    :              9 KB
Pagefile        :          287852 KB
Page Fault Count : 100289

IO Read          :        4233303 KB (in          66148 reads )
IO Write        :        4446197 KB (in          69472 writes)
IO Other        :              1 KB (in            232 others)


dixen 17-07-2019 21:32

bundle_pc.ipk (Rayman Origins)

Quote:

xtool_new.exe:

Process ID : 9272
Thread ID : 1356
Process Exit Code: 0
Thread Exit Code : 0

User Time : 60.453s
Kernel Time : 1.546s
Process Time : 61.999s
Clock Time : 21.767s

Working Set : 448404 KB
Paged Pool : 110 KB
Nonpaged Pool : 11 KB
Pagefile : 540260 KB
Page Fault Count : 211743

IO Read : 2046443 KB (in 31979 reads )
IO Write : 3038292 KB (in 5711 writes)
IO Other : 6 KB (in 202 others)

xtool_old.exe:


Process ID : 1220
Thread ID : 4636
Process Exit Code: 0
Thread Exit Code : 0

User Time : 130.625s
Kernel Time : 2.593s
Process Time : 133.218s
Clock Time : 44.855s

Working Set : 2084988 KB
Paged Pool : 111 KB
Nonpaged Pool : 10 KB
Pagefile : 2291268 KB
Page Fault Count : 1055801

IO Read : 2046448 KB (in 31978 reads )
IO Write : 3038286 KB (in 47474 writes)
IO Other : 3 KB (in 195 others)

Razor12911 18-07-2019 15:10

Thanks for the tests, 95% complete. the new release will first ship with 1 codec (zlib) just to see if everything works as expected

IgorKolesnik 19-07-2019 06:49

Quote:

Originally Posted by Razor12911 (Post 481842)
Thanks for the tests, 95% complete. the new release will first ship with 1 codec (zlib) just to see if everything works as expected

can update game file scaner? thank

Razor12911 20-07-2019 17:37

Quote:

Originally Posted by IgorKolesnik (Post 481844)
can update game file scaner? thank

and add what feature? :rolleyes:

@everyone

here's a benchmark

some file from Pro Evolution Soccer

old xtool = 275MB >> 381MB took 11.63 seconds using 4 threads
new xtool = 275MB >> 381MB took 06.09 seconds using 4 threads

same memory usage and I/O

note: I only reworked the encoder to use the cpu effectively so you should expect roughly x1.25-2 more speed from old version while using same resources and this applies to all codecs reflate, oodle and etc

Simorq 21-07-2019 01:35

Hi Razor12911
Code:

v: skip verification
This option works for oodle:D, crilayla:D, lzo:D.
Please do not delete this option in the new version.

KaktoR 21-07-2019 02:10

Quote:

Originally Posted by Razor12911 (Post 481868)
and add what feature? :rolleyes:

Just fix the hang bug :D
Quote:

Originally Posted by Razor12911 (Post 481868)
@everyone

here's a benchmark

some file from Pro Evolution Soccer

old xtool = 275MB >> 381MB took 11.63 seconds using 4 threads
new xtool = 275MB >> 381MB took 06.09 seconds using 4 threads

same memory usage and I/O

note: I only reworked the encoder to use the cpu effectively so you should expect roughly x1.25-2 more speed from old version while using same resources and this applies to all codecs reflate, oodle and etc

This is great. :cool:

dixen 21-07-2019 04:43

Quote:

Originally Posted by Razor12911 (Post 481868)
and add what feature? :rolleyes:

oodle detect, for example:D


All times are GMT -7. The time now is 13:14.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com