Go Back   FileForums > Game Backup > PC Games > PC Games - CD/DVD Conversions > Conversion Tutorials

Reply
 
Thread Tools Display Modes
  #511  
Old 04-07-2022, 05:20
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 255
Thanks: 180
Thanked 315 Times in 113 Posts
elit is on a distinguished road
And preflate is worse than zlib hmmm..:

ff13pref.png
Reply With Quote
Sponsored Links
  #512  
Old 07-07-2022, 01:53
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,646
Thanks: 2,026
Thanked 10,409 Times in 2,211 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added wav stream detector
- added flac codec
- added jpg stream detector
- added packjpg, brunsli, jojpeg codec
- added feature that allows input to be a directory
- added feature to extract detected streams
- updated database feature
- updated deduplication feature
- IO function decode updated

Notes

I have added wav audio compression as an alternative of msc for people who has issues with it or don't want to use Freearc's tta codec as it processes wav files individually. One thing to note, it's not as good as tta, tak or even frog as it is based on flac codec (It was the only open source codec I could work with...).

Also added jpg image compression codecs, you can pick between packjpg, brunsli or jojpeg. packjpg seems to have a memory leak, brunsli is fast but cannot deal with large jpg images and jojpeg can be used if you're after ratio as it is paq based (seems to be buggy so stick to packjpg or brunsli for now)

A new parameter is added --extract=[path], this will extract all detected streams to a directory... if you're interested in the streams themselves.

database and deduplication features have been updated and can now be used at all times without worrying about crc collisions.

Special thanks to KaktoR for running several tests and Shelwien for providing compiled libraries for brunsli and jojpeg.

@elit
These issues will be investigated in time.

Last edited by Razor12911; 07-07-2022 at 01:56.
Reply With Quote
The Following 8 Users Say Thank You to Razor12911 For This Useful Post:
BLACKFIRE69 (07-07-2022), Cesar82 (07-07-2022), dixen (07-07-2022), elit (07-07-2022), ffmla (07-07-2022), Gehrman (07-07-2022), KaktoR (07-07-2022), NERV (07-07-2022)
  #513  
Old 07-07-2022, 02:52
dixen dixen is offline
Registered User
 
Join Date: Sep 2014
Location: Russia
Posts: 309
Thanks: 367
Thanked 339 Times in 159 Posts
dixen is on a distinguished road
Find little uncritical bug

Quote:
Compressing 77,790,338 bytes with xtool.exe precomp -mFLAC -c32mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 1/1
Time: 00:00:02 (00:00:02)
Memory: 276 MB (276 MB)

100%
Errorlevel=0
Compressed 1 file, 77,790,338 => 53,378,764 bytes. Ratio 68.62%
Compression time: cpu 0.08 sec/real 2.78 sec = 3%. Speed 28.01 mB/s
All OK

Quote:
Compressing 77,790,338 bytes with xtool.exe precomp -mFLAC --verbose -c32mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

[0] Performing scan from block 0000000000000000 to 0000000001FFFFFF (33554432)
EAccessViolation: Access violation at address 0000000000409A77 in module 'xtool.exe'. Read of address 00007FF4FE6BFBA4

Errorlevel=1

ERROR: general (de)compression error in xtool
Anyway, thank Razor for update and new functions)

One question. Why TTA but not TAK?

Quote:
Original.wav 74.1 mb
ENCTTA.tta 50.3 mb
ENCTAK.tak 48.7 mb

Last edited by dixen; 07-07-2022 at 02:57.
Reply With Quote
The Following User Says Thank You to dixen For This Useful Post:
Gehrman (07-07-2022)
  #514  
Old 07-07-2022, 03:40
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 776
Thanks: 175
Thanked 849 Times in 414 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by dixen View Post
One question. Why TTA but not TAK?
TAK isn't Open Source.
Reply With Quote
  #515  
Old 07-07-2022, 03:55
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 255
Thanks: 180
Thanked 315 Times in 113 Posts
elit is on a distinguished road
Interesting, I should note that compression difference between different wav codecs is minimal. Also for me in the past, I found wavpack the best, It handle more formats(WAV/BWF/RF64, Sony Wave64, Apple CAF, Philips DSDIFF, and Sony DSF (all with no size limitations)), more modes(8, 16, 24, and 32-bit integer PCM; 32-bit float PCM; DSD audio; mono, stereo, and multichannel; sampling rates from 6 to 192 kHz (and non-standard rates)), RIFF chunks, is open source, does support STDIO which for me work unlike flac, which for some reason during decompression kept running past original size inflating to infinity(when used in FA with stdio mode).

However, flac is great too because is non-symetric so decompression is fast(although wavpack also seem to support asym. cmp.).

I wonder how this work, if xtool is able to use it on chunks when wav files are inside big game packs, that would be indeed very useful. In fact, I don't know what would be the point beyond that as we could then just use standalone codecs?

Last edited by elit; 07-07-2022 at 03:57.
Reply With Quote
  #516  
Old 07-07-2022, 04:01
Prince4 Prince4 is offline
Registered User
 
Join Date: Mar 2015
Location: India->Delhi->Rohini
Posts: 120
Thanks: 108
Thanked 19 Times in 15 Posts
Prince4 is on a distinguished road
Post

Could someone please remind me what's the difference between the 0.3.21 and 0.6.0 versions?
__________________
Who's Got Time To Play Games? Huh??
Reply With Quote
  #517  
Old 07-07-2022, 05:27
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,646
Thanks: 2,026
Thanked 10,409 Times in 2,211 Posts
Razor12911 is on a distinguished road
Quote:
Originally Posted by dixen View Post
Find little uncritical bug
your chunk size is too small for that input
Quote:
Originally Posted by Prince4 View Post
Could someone please remind me what's the difference between the 0.3.21 and 0.6.0 versions?
0.3.21 is an old version that supported old plugins and it is kept there until all plugins are updated to support 0.4+ versions, use 0.6.0 if you are not using old plugins.

Quote:
Originally Posted by elit View Post
Interesting, I should note that compression difference between different wav codecs is minimal.
https://wiki.hydrogenaud.io/index.ph...ess_comparison

I picked codecs based on the information provided here, it's not that I haven't tried wavpack. I did but it requires additional code as it generates lossy wav and a patch file in case you wanted to restore the original pcm data whereas flac does not. In addition, it doesn't seem to be thread safe which means it cannot work in xtool environment (maybe I'm mistaking it with tta).

Quote:
Originally Posted by elit View Post
I wonder how this work, if xtool is able to use it on chunks when wav files are inside big game packs, that would be indeed very useful. In fact, I don't know what would be the point beyond that as we could then just use standalone codecs?
That's pretty much how it is intended to work, wav files are scanned within files and then fed to codecs.

Last edited by Razor12911; 07-07-2022 at 05:42.
Reply With Quote
The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
Gehrman (07-07-2022), Prince4 (10-07-2022)
  #518  
Old 07-07-2022, 09:43
elit elit is offline
Registered User
 
Join Date: Jun 2017
Location: sun
Posts: 255
Thanks: 180
Thanked 315 Times in 113 Posts
elit is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
I did but it requires additional code as it generates lossy wav and a patch file in case you wanted to restore the original pcm data whereas flac does not. In addition, it doesn't seem to be thread safe which means it cannot work in xtool environment (maybe I'm mistaking it with tta).
That is optional feature with extra switches, by default it work same as flac - no lossy wav. As for thread safety, I believe its single threaded app so could be used chunk per thread but I don't know for sure.
I am also perfectly ok with flac as well so don't worry about it, I just though this one may have been better due to more formats supported.
https://www.wavpack.com
Reply With Quote
The Following 2 Users Say Thank You to elit For This Useful Post:
Gehrman (12-07-2022), Razor12911 (12-07-2022)
  #519  
Old 12-07-2022, 11:43
Razor12911's Avatar
Razor12911 Razor12911 is offline
Programmer
 
Join Date: Jul 2012
Location: South Africa
Posts: 3,646
Thanks: 2,026
Thanked 10,409 Times in 2,211 Posts
Razor12911 is on a distinguished road
Update available

Changes

- added fast lzma2 compression for portable mode
- fixed issues with wav stream detection
- fixed minor issue with stream deduplication feature

Notes

I have added fast lzma2 compression for users who would want to use xtool without FA but still want to perform compression immediately after precompressing.

Example
Code:
xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup --compress=l10,t100p - -
Reply With Quote
The Following 12 Users Say Thank You to Razor12911 For This Useful Post:
:( Sad8669 (13-07-2022), BKR-TN (12-07-2022), BLACKFIRE69 (12-07-2022), Cesar82 (13-07-2022), CR2032 (13-07-2022), dixen (12-07-2022), elit (14-07-2022), Gehrman (12-07-2022), Masquerade (12-07-2022), NERV (12-07-2022), ScOOt3r (12-07-2022), shazzla (12-07-2022)
  #520  
Old 13-07-2022, 06:30
CR2032's Avatar
CR2032 CR2032 is offline
Registered User
 
Join Date: Feb 2020
Location: Russian
Posts: 8
Thanks: 8
Thanked 3 Times in 2 Posts
CR2032 is on a distinguished road
Quote:
Originally Posted by Razor12911 View Post
Update available

Changes

- added fast lzma2 compression for portable mode
- fixed issues with wav stream detection
- fixed minor issue with stream deduplication feature

Notes

I have added fast lzma2 compression for users who would want to use xtool without FA but still want to perform compression immediately after precompressing.

Example
Code:
xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup --compress=l10,t100p - -
Thank You So Much For Your Hard Work, Razor12911!
Reply With Quote
The Following User Says Thank You to CR2032 For This Useful Post:
Razor12911 (15-07-2022)
  #521  
Old 13-07-2022, 07:01
L33THAK0R's Avatar
L33THAK0R L33THAK0R is offline
Registered User
 
Join Date: Feb 2021
Location: Saudi Arabia
Posts: 242
Thanks: 82
Thanked 23 Times in 19 Posts
L33THAK0R is on a distinguished road
Are there any current situations/titles that could benefit from the fast lzma2 compression option? Not fully awake, apologies if this is a tad silly to ask, with the answer being plainly obvious.
Reply With Quote
  #522  
Old 13-07-2022, 08:06
:( Sad8669 :( Sad8669 is offline
Registered User
 
Join Date: Oct 2020
Location: -
Posts: 203
Thanks: 97
Thanked 185 Times in 101 Posts
:( Sad8669 is on a distinguished road
As far as i know, it was added to increase the portability of xtool.

Code:
Notes:
I have added fast lzma2 compression for users who would want to use xtool without FA but still want to perform compression immediately after precompressing.
Reply With Quote
The Following User Says Thank You to :( Sad8669 For This Useful Post:
Razor12911 (15-07-2022)
  #523  
Old 13-07-2022, 11:29
Cesar82's Avatar
Cesar82 Cesar82 is offline
Registered User
 
Join Date: May 2011
Location: Brazil
Posts: 813
Thanks: 1,348
Thanked 1,749 Times in 591 Posts
Cesar82 is on a distinguished road
Lightbulb

Quote:
Originally Posted by Razor12911 View Post
Code:
xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup --compress=l10,t100p - -
XTool is going independent, abandoning its helpers, lol.
Maybe in the future it will get its own final compression with high compression gain and higher performance than lolz... It would be great!
Reply With Quote
  #524  
Old 13-07-2022, 12:14
Masquerade's Avatar
Masquerade Masquerade is offline
Registered User
 
Join Date: Jan 2020
Location: Monte d'Or
Posts: 776
Thanks: 175
Thanked 849 Times in 414 Posts
Masquerade is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Are there any current situations/titles that could benefit from the fast lzma2 compression option? Not fully awake, apologies if this is a tad silly to ask, with the answer being plainly obvious.
No, it's just a final compresison option. A bit like how precomp has built in LZMA compressor

@Razor thanks for the continued updates!
Reply With Quote
The Following 2 Users Say Thank You to Masquerade For This Useful Post:
CR2032 (14-07-2022), Razor12911 (15-07-2022)
  #525  
Old 14-07-2022, 14:45
CR2032's Avatar
CR2032 CR2032 is offline
Registered User
 
Join Date: Feb 2020
Location: Russian
Posts: 8
Thanks: 8
Thanked 3 Times in 2 Posts
CR2032 is on a distinguished road
Quote:
Originally Posted by L33THAK0R View Post
Are there any current situations/titles that could benefit from the fast lzma2 compression option? Not fully awake, apologies if this is a tad silly to ask, with the answer being plainly obvious.
Hello.
No. It's just a creative and really a gifted step, not more.

Last edited by CR2032; 14-07-2022 at 14:50.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Dev]XTool Razor12911 Conversion Tutorials 180 23-10-2020 06:26
Project Cars Digital Edition (3xDVD5) (srep+lzma) GTX590 PC Games - CD/DVD Conversions 10 28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039 mausschieber PC Games - CD/DVD Conversions 0 24-07-2017 15:12
Space Channel 5 Part 2 Translation Project Christuserloeser DC Games 0 21-06-2004 18:16



All times are GMT -7. The time now is 21:17.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2000-2020, FileForums @ https://fileforums.com