#271
|
|||
|
|||
Exactly! But.. I doubt I am doing anything wrong, I mean:
Code:
-mcustom6/$gamelz=xtool:mcp2077+srp64+xlzma:lc8 So *.archive = FA -mxtool:mcp2077+srp64+xlzma:lc8 or xtool precomp -t100p -mcp2077 to be precise Code:
[External compressor:xtool] header = 0 packcmd = _EC\xtool\xtool precomp -t100p {options} - - <stdin> <stdout> unpackcmd = _EC\xtool\xtool decode -t100p - - <stdin> <stdout> Code:
srp64 = srep64:m3f:c256:mem8g:m64k:b16m:a2 procdefault = srp64+xlzma:lc8 Code:
-mcustom6/$gamelz=xtool:mcp2077+procdefault EDIT(restarted without t100p, but I don't think that's it: xtool.png How you guys getting ~10mb/s is beyond me, are you tarring or doing individual files? As it may matter..) EDIT2(speed down to 0.6mb/s, estimation ~22h+ and slowing down. Really, this ain't right but tool appear to be working correct not hanging! Can anyone try: latest xtool version with cp2077 plugin and on tarred archives through FA not individual files? I am sure there is something wrong in one of those variables.) PS(I forgot to note that game is gog with 1.06 update, that means 9gb 1.05 was also applied, maybe it changed oodle structure? I will postpone compression until solution/cause is clear.) Last edited by elit; 27-12-2020 at 11:19. |
Sponsored Links |
#272
|
|||
|
|||
So I just quickly tried oo2rect. That one seem to be working ok! I am going to use it on whole game and will see. I also tried xtool with -mkraken instead cp2077 plugin and also on individual archives. Speed issue remains. It does seem to have problem no matter what, while oo2rect seems fine.
EDIT: oo2rect is same slow, after ~1h about 20gb and 0.xxmb/s speed. At this point I am pretty sure problem is not xtool nor oo2rect or plugin, but a game. If this was not a case with original version 1.03, try to update to 1.06, I think they changed data enough to become problem. For reference, files of concern are: basegame_3_nightcity.archive basegame_3_nightcity_gi.archive Last edited by elit; 27-12-2020 at 14:17. |
#273
|
|||
|
|||
Alright so I left it overnight. I can confirm it to work correctly but it took 11h:40min to compress, from that about ~10h due to xtool oodle. Archive tested ok and decompression was much quicker, compression is that slow though.
|
#274
|
||||
|
||||
No Idea what's wrong, but for me it's working just fine.
I use GOG version 1.06, along with an Ryzen 5 2600 and 16GB RAM and latest xtool version posted a few days ago. I can test the whole game folder if you wish, but I can tell you the results will be just fine and nothing wrong. Xtool settings I use: Code:
[External compressor:xtool] header = 0 packcmd = xtool.exe precomp -mcp2077 -c128mb -t100p --dbase - - <stdin> <stdout> unpackcmd = xtool.exe decode -t100p - - <stdin> <stdout> Code:
basegame_3_nightcity.archive Compressed 1 file, 7,958,224,896 => 22,714,137,974 bytes. Ratio 285.42% Compression time: cpu 9.34 sec/real 704.44 sec = 1%. Speed 11.30 mB/s All OK Extracted 1 file, 22,714,137,974 => 7,958,224,896 bytes. Ratio 285.42% Extraction time: cpu 8.11 sec/real 333.60 sec = 2%. Speed 23.86 mB/s All OK
__________________
Haters gonna hate
Last edited by KaktoR; 28-12-2020 at 03:38. |
#275
|
|||
|
|||
Hm, you have 6 core with hyperthreading. Maybe -t100p for you utilize all 12 virtual cores. You may try -t4 and also no --dbase to see if it affect speed that much. Also no need to test all archives only 2 files I mentioned above is enough. I wonder if --dbase or 12 virtual cores make for such difference. BTW thanks for the info above, it's interesting to know.
|
#276
|
||||
|
||||
i5 4590, 4 Cores (4 Threads)
Code:
[External compressor:xtool] header = 0 packcmd = xtool.exe precomp -mcp2077 -c128mb -t100p - - <stdin> <stdout> unpackcmd = xtool.exe decode -t100p - - <stdin> <stdout> Code:
basegame_3_nightcity.archive Compressed 1 file, 7,958,224,896 => 22,919,265,672 bytes. Ratio 287.99% Compression time: cpu 9.91 sec/real 1538.09 sec = 1%. Speed 5.17 mB/s All OK Tested 1 file, 22,919,265,672 => 7,958,224,896 bytes. Ratio 287.99% Testing time: cpu 6.45 sec/real 528.38 sec = 1%. Speed 15.06 mB/s All OK Last edited by Razor12911; 29-12-2020 at 00:01. |
The Following User Says Thank You to Razor12911 For This Useful Post: | ||
elit (29-12-2020) |
#277
|
||||
|
||||
Here's an interesting one.
ZZ1.dat - 175MB from Steel Division 2. XTool 2020 (zlib) Code:
Compressed 1 file, 183,623,680 => 183,661,889 bytes. Ratio 100.02% Compression time: cpu 0.16 sec/real 26.08 sec = 1%. Speed 7.04 mB/s All OK Code:
Compressed 1 file, 183,623,680 => 226,689,420 bytes. Ratio 123.45% Compression time: cpu 0.27 sec/real 26.18 sec = 1%. Speed 7.01 mB/s All OK GFS Detects zlib streams, whereas Drop Scan does not: ![]() ![]() https://anonfiles.com/FcY1Q060p9/ZZ_1_dat |
#278
|
||||
|
||||
Make headerless+force detection
PS: The anonfiles is shit. Always slow connection here ![]()
__________________
Haters gonna hate
|
#279
|
||||
|
||||
@Masquerade
Use reflate codec Code:
Compressed 1 file, 183,623,680 => 227,188,297 bytes. Ratio 123.72% Compression time: cpu 0.09 sec/real 5.12 sec = 2%. Speed 35.87 mB/s All OK Extracted 1 file, 227,188,297 => 183,623,680 bytes. Ratio 123.72% Extraction time: cpu 0.13 sec/real 1.07 sec = 12%. Speed 171.41 mB/s All OK
__________________
Haters gonna hate
|
The Following 3 Users Say Thank You to KaktoR For This Useful Post: | ||
#280
|
||||
|
||||
When you use zlib method in Xtool v12 or older variants of xtool, they automatically used reflate when you placed the libraries each time zlib fails, the new xtool does not do any of this. The user is the one who needs to do this by themselves because what I have noticed is people putting reflate libraries near xtool and they never know when xtool uses them or when it doesn't. This way the user knows what the issues is when decompression fails each time when they try to figure out what the actual problem is.
I also did point out that the best method for precompressing anything that is zlib/deflate compressed, use -mzlib along with either reflate/preflate so you get the best results. |
The Following 2 Users Say Thank You to Razor12911 For This Useful Post: | ||
Ele (07-01-2021), Masquerade (08-01-2021) |
#281
|
||||
|
||||
Razor12911,
I did use zlib+preflate, I simply forgot about reflate. After using relate, everything works fine. |
#282
|
||||
|
||||
Update available
Changes - updated library support - updated command line parser - included x86 build - fixed depthing issues Notes I have added 32-bit build as requested but you'll have to use x86 libraries or find a matching pair of x86/x64 libraries. You can get these from github of a project under the releases section. The versioning is brought back to the format of 1.0.0 instead of the build number as requested. |
#283
|
||||
|
||||
@Razor12911
Thanks for this new method... incredible! Code:
Compressing 1 file, 17,530,888,054 bytes Compressing TSCGame-WindowsNoEditor.pak Compressed 1 file, 17,530,888,054 => 28,460,607,223 bytes. Ratio 162.35% Compression time: cpu 18.42 sec/real 624.55 sec = 3%. Speed 28.07 mB/s All OK |
#284
|
||||
|
||||
@Razor12911, thanks for the 32-bit version
![]() 1) Also like to inform you that it is being detected as a false positive by the KIS anti-virus (I don't know if there is anything that can be done). P.S: The 64-bit version has not been captured. ![]() 2) You could share libraries compatible with the 32-bit version to be fully compatible and avoid errors due to incompatibilities. 3) I wonder if I have 3 games (collection) and each game uses XTool with different methods to compress. Assuming that only the libraries needed for compression (nothing more than necessary) for each Data.bin file are together with XTool.exe at the time of compressing. To perform the decompression, can I have other libraries that were not used to compress the Data.bin file together with XTool (Like: Have 1 library next to XTool when compressing and 5 when decompressing)? Thanks! |
#285
|
||||
|
||||
1) some interesting stuff,
x86: https://www.virustotal.com/gui/file/...296d/detection x64: https://www.virustotal.com/gui/file/...7dfe/detection the code is actually the same so I don't know... ![]() 2) Yes the issue is also the plugins which also have to shipped with 32-bit version, who still uses 32-bit system anyways? Capture2.PNG 3) Yes, shouldn't be a problem. I'll ship x86 libraries but any incompatibility issues should fall on the end user. |
![]() |
Thread Tools | |
Display Modes | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
[Dev]XTool | Razor12911 | Conversion Tutorials | 180 | 23-10-2020 06:26 |
Project Cars Digital Edition (3xDVD5) (srep+lzma) | GTX590 | PC Games - CD/DVD Conversions | 10 | 28-08-2017 08:34 |
Project IGI Anthology 1xCD700 CIUV2 2039 | mausschieber | PC Games - CD/DVD Conversions | 0 | 24-07-2017 15:12 |
Space Channel 5 Part 2 Translation Project | Christuserloeser | DC Games | 0 | 21-06-2004 18:16 |