FileForums - XTool 2020 (Main Project)

Page 3 of 37

Last »

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - XTool 2020 (Main Project) (https://fileforums.com/showthread.php?t=102832)

shazzla

21-07-2019 13:20

If game uses oodle,then you can find its DLLs in game folder somewhere.
Imho : oodle stream scanner is obsolete ?!

Razor12911

21-07-2019 15:29

Quote:

Originally Posted by Simorq (Post 481873)

Hi Razor12911

Code:

v: skip verification

This option works for oodle:D, crilayla:D, lzo:D.
Please do not delete this option in the new version.

The option will be there

Quote:

Originally Posted by dixen (Post 481875)

oodle detect, for example:D

I remember including an oodle scanner, but fine I'll think of something for the scanner.

Quote:

Originally Posted by KaktoR (Post 481874)

This is great. :cool:

how about this?

Quote:

There will be a new feature in xtool called history data, what it does is simply store processed stream information and be able to recall it at the later stage.

If you are wondering how this helps, well there are certain games made by lazy developers who repeat game resources instead of just reusing them, resulting in bigger game size for no absolute reason, apart from that, if they are compressed streams it means that xtool will have to process the same stream over and over again, storing history data allows it to prevent the same stream from being reprocessed instead, it will just look at the history and find out how the previous copy was processed then automatically assume it's the same stream and then apply the same settings, resulting in more speed.

I ran a test on a file I repeated on purpose and these were the results.

old xtool took 196.671 seconds and used 335MB ram
new xtool took 154.174 seconds and used 319MB ram
new xtool with history feature took 72.223 seconds and used 439MB ram

the cost of the feature depends on how much history you want to be stored but with default settings, it should use an additonal 192mb ram usage than without, you can always disable it or order it to store more history, depends on how much memory you have to spare.

Note:
In a practical situation, you can shave off a few seconds but then again, depends on the input

KaktoR

23-07-2019 07:18

This is also good :D

Did you have to make changes to the libraries too? Or just the source of xtool executable? Just corious :)

Razor12911

23-07-2019 15:04

Only the source of xtool, all I did was optimize optimize and optimize, the CPU is forever doing something, I tried to minimize the gaps between reads and writes by making xtool process stuff while it is reading and writing so yeah, the threads hardly get rests now.

Razor12911

26-07-2019 16:55

1 Attachment(s)

Project status = Welp...

I hope this isn't strike 2 in terms losing my sources, my PC died about 2 days ago, motherboard and GPU got fried this time XD. As for my sources well they were stored on my main PC (the dead one) and I don't know if they are safe since I don't have a spare PC to boot up the drive to grab important files and store them to my backup drive, so I can't really tell if the only components that died were only the board and GPU and not my HDD, but I guess that's a story for some other time. If you are wondering what this means in terms of xtool 2019...

Well since I normally code from desktop because I don't need to go far to do things and etc, it was different for xtool somehow... because I was coding from a backup drive only for that project like I knew some shit like this would happen so yes the source is safe (lucky af), if you are wondering about the development now that my main PC is busted.

Well I just had to dust an old laptop and continued development there, issue is it's incredibly slow but it will have to do. If you are wondering why I only provided x86 binaries, well. OS is 32-bit :D, somehow can't compile for x64.

Anyways enough about that, this was suppose to be the last test before the first release, but before I uploaded, I noticed some shitty error in made in the reflate code and I have to rewrite some parts of it so after fixing that, I'll post the final test and if all is good, I'll be making a release.

there are 3 codecs in xtool right now (zlib, reflate, rzlib). The bug was in reflate in multi threaded mode when decoding, but if you want to try it out anyways, open the test bat file. replace zlib with reflate or rzlib but use 1 thread to decode.

zlib = xtool uses zlib1.dll/zlibwapi.dll to process data, fails on some streams
reflate = xtool uses reflate dlls to process data, processes all streams but slow
rzlib = xtool blends zlib and reflate dlls to process data, processes all streams but with better size and speed.

PS: Even the archive is zip, don't have 7-Zip installed for my normal 7z attachments :o
PPS: Nothing is stopping the development of this project XD... I hope cause you never know

FitGirl

26-07-2019 18:33

Shit, mate! You're lucky to have not loosing sources this time! Gotta use some repo soft next time for more security, no? :)

Anyways, it's nice to see the project is coming to release.

Harsh ojha

26-07-2019 21:01

@Razor12911
Download Windows 10 64 bit Archived file and install

https://youtu.be/zGQ9xXYgVJM
Since i also lost my data

Well we are all depends our pc and laptops

Razor12911

28-07-2019 15:14

First release

Notes

- No idea whether x64 works or not, I just somehow managed to compile. That's up to you to find out.
- Reflate currently is set to use level 6, if you are looking for better output, best to use the old xtool. The reason it's like this is because I'm still working on a way to improve the horrible slow code I used before.
- Skip verification option is currently unavailable.

What I can say, run a lot of tests on this release. I've written some dangerous memory saving code and it may cause errors but if all is well then I can move on to add other codecs such as oodle, lzo, lz4 and the rest. BTW, crilayla will NOT be added, that piece of crap is incredibly slow :D

Simorq

28-07-2019 16:11

Image003.png

xtool_1907_R1 64Bit

Code:

Creating archive: data.arc using zlib

Compressed 1 file, 668,113 => 668,225 bytes. Ratio 100.02%                 

Compression time: cpu 0.02 sec/real 0.18 sec = 9%. Speed 3.71 mB/s

All OK



Testing archive: data.arc

Tested 1 file, 668,225 => 668,113 bytes. Ratio 100.02%                     

Testing time: cpu 0.00 sec/real 0.06 sec = 0%. Speed 10.77 mB/s

All OK

Code:

Creating archive: data.arc using reflate

Compressed 1 file, 668,113 => 1,767,491 bytes. Ratio 264.55%               

Compression time: cpu 0.00 sec/real 0.19 sec = 0%. Speed 3.44 mB/s

All OK



Testing archive: data.arc

Tested 1 file, 1,767,491 => 668,113 bytes. Ratio 264.55%                   

Testing time: cpu 0.00 sec/real 0.16 sec = 0%. Speed 4.31 mB/s

All OK

Code:

Creating archive: data.arc using rzlib

Compressed 1 file, 668,113 => 1,767,491 bytes. Ratio 264.55%               

Compression time: cpu 0.00 sec/real 0.28 sec = 0%. Speed 2.40 mB/s

All OK



Testing archive: data.arc

Tested 1 file, 1,767,491 => 668,113 bytes. Ratio 264.55%                   

Testing time: cpu 0.00 sec/real 0.16 sec = 0%. Speed 4.25 mB/s

All OK

Simorq

28-07-2019 16:30

xtool_1907_R1 Test

Does not work with ba2/csg files.

Code:

Fallout4 - Textures7.ba2 / DLCCoast - Geometry.csg

32bit= ERROR: write error (disk full?) in compression algorithm zlib/reflate/rzlib
64bit= ERROR: write error (disk full?) in compression algorithm zlib/reflate/rzlib or stuck cpu
http://s8.picofile.com/file/8368123326/zlib.jpg

-------------------------------------------------------------------

Works with this setting.

Code:

[External compressor:rzlib,zlib,reflate]

header    = 0

packcmd   = xtool.exe precomp:rzlib:c32mb,t1 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

unpackcmd = xtool.exe decode:t50p - - <stdin> <stdout>

Code:

Fallout4 - Textures7.ba2 520 MB

ZLIB

Code:

Creating archive: data.arc using zlib

Compressed 1 file, 545,959,220 => 1,237,654,873 bytes. Ratio 226.69%       

Compression time: cpu 0.52 sec/real 84.11 sec = 1%. Speed 6.49 mB/s

All OK



zlib: decode:t100p

Testing archive: data.arc

Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           

Testing time: cpu 0.45 sec/real 9.53 sec = 5%. Speed 57.29 mB/s

All OK

REFLATE

Code:

Creating archive: data.arc using reflate

Compressed 1 file, 545,959,220 => 1,237,783,515 bytes. Ratio 226.72%       

Compression time: cpu 0.42 sec/real 113.43 sec = 0%. Speed 4.81 mB/s

All OK



reflate: decode:t100p

Testing archive: data.arc

Tested 1 file, 1,237,783,515 => 545,959,220 bytes. Ratio 226.72%           

Testing time: cpu 0.36 sec/real 12.23 sec = 3%. Speed 44.62 mB/s

All OK

RZLIB

Code:

Creating archive: data.arc using rzlib

Compressed 1 file, 545,959,220 => 1,237,654,873 bytes. Ratio 226.69%       

Compression time: cpu 0.53 sec/real 85.22 sec = 1%. Speed 6.41 mB/s

All OK



rzlib: decode:t2

Testing archive: data.arc

Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           

Testing time: cpu 0.98 sec/real 73.38 sec = 1%. Speed 7.44 mB/s

All OK



rzlib: decode:t100p

Testing archive: data.arc

Tested 1 file, 1,237,654,873 => 545,959,220 bytes. Ratio 226.69%           

Testing time: cpu 0.77 sec/real 73.54 sec = 1%. Speed 7.42 mB/s

All OK

Razor12911

28-07-2019 17:14

Can you run xtool outside freearc, like via console.

Code:

@echo off

xtool.exe precomp:rzlib:c16mb,t100p %1 %1.out

pause

I'm interested in the exception/error message

Simorq

28-07-2019 17:21

@Razor12911
good
http://s8.picofile.com/file/8368124442/rrrr.jpg

-------------------------
I think the main problem is - -.
- - <stdin> <stdout>:confused:

Razor12911

28-07-2019 17:31

try this then

Code:

@echo off

xtool.exe precomp:rzlib:c32mb,t100p - - < %1 > %1.out

xtool.exe decode:t100p - - < %1.out > %1.res

fc /b %1 %1.res

pause

Simorq

28-07-2019 17:38

@Razor12911
no work
stuck cpu
http://s9.picofile.com/file/8368124692/rrr.jpg

Razor12911

28-07-2019 17:42

Nice ghost bugs, I'll check what I missed in the source and thanks for the tests.

All times are GMT -7. The time now is 06:01.

Page 3 of 37

Last »

Show 40 post(s) from this thread on one page