Oodle precompressor (Side project)

Razor12911 · #1 07-12-2018, 16:10

Hello, I decided to give oo2core_6_win64.dll another shot to see what was really causing the crashing issues I was facing and somehow it the coding software itself so I decided to rewrite the code.... again... but now using Lazarus instead of Delphi because of it causing too many problems, I spent an entire day writing this because it was the only time I had, been very busy lately and I think I got it working.

I have tested on the samples I got from WWE2K19, it handles it just fine, not sure about the entire game but that's for you to find out.

If you look at the attachment, there is a separate jc4 executable. Well that's just a recompile of the main executable just with a few changes to it supports the remaining stream that game has since two compression methods were used on it but to make it work, you would have to combine the main precompressor with the jc4 to fully precompress the game.... I think... not sure since I really had very little time to work on this plus only had 2 samples from the game.
Combining the methods means -moo2rec+oo2recj

Support for now is simply limited to kraken compressed games, just take the dll from game and place it near exe, use the scanner included as a guide, it will show you exactly what method was used in that particular game.

DLLs supported:
oo2core_4_win64.dll
oo2core_5_win64.dll
oo2core_6_win64.dll
oo2core_7_win64.dll

Some results:

Just Cause 4 (game0.arc)

Code:

lzma:ultra:d176m:
15,8 MB (16 584 704 bytes) >> 13,6 MB (14 346 879 bytes)
oo2rec+lzma:ultra:d176m:
15,8 MB (16 584 704 bytes) >> 20,2 MB (21 202 181 bytes) >> 13,3 MB (13 965 518 bytes)
oo2rec+oo2recj+lzma:ultra:d176m:
15,8 MB (16 584 704 bytes) >> 25,5 MB (26 774 015 bytes) >> 12,3 MB (12 968 978 bytes)

Just Cause 4 (dlc.arc)

Code:

lzma:ultra:d176m:
22,8 MB (23 957 504 bytes) >> 20,2 MB (21 189 235 bytes)
oo2rec+lzma:ultra:d176m:
22,8 MB (23 957 504 bytes) >> 35,5 MB (37 286 828 bytes) >> 18,7 MB (19 661 381 bytes)
oo2rec+oo2recj+lzma:ultra:d176m:
22,8 MB (23 957 504 bytes) >> 45,2 MB (47 453 198 bytes) >> 17,1 MB (17 986 419 bytes)

These results are already making me wonder if I made mistakes somewhere and left several streams behind.... XDD

See you guys in a while, enjoy and I hope it works lol

KaktoR · #2 07-12-2018, 16:16

I would say, better than nothing

Thanks for this. Will test soon.

See ya

Simorq · #3 07-12-2018, 20:31

Thanks Razor12911

Code:

game0.arc 846 MB

oo2recj+oo2rec:
Compressed 1 file, 887,402,496 => 1,032,904,579 bytes. Ratio 116.40%
Compression time: cpu 1.69 sec/real 296.11 sec = 1%. Speed 3.00 mB/s
All OK

Tested 1 file, 1,032,904,579 => 887,402,496 bytes. Ratio 116.40%
Testing time: cpu 1.27 sec/real 5.13 sec = 25%. Speed 172.84 mB/s
All OK

------------------------------------------------
oo2rec:
Compressed 1 file, 887,402,496 => 1,029,825,704 bytes. Ratio 116.05%
Compression time: cpu 0.91 sec/real 16.80 sec = 5%. Speed 52.81 mB/s
All OK

Tested 1 file, 1,029,825,704 => 887,402,496 bytes. Ratio 116.05%
Testing time: cpu 1.02 sec/real 4.77 sec = 21%. Speed 185.88 mB/s
All OK

Code:

archives_win64\main\hires\game4.arc 791 MB
oo2rec+oo2recj:
ERROR: general (de)compression error in oo2recj

oo2rec:
Compressed 7 files, 835,010,548 => 835,010,824 bytes. Ratio 100.00%
Compression time: cpu 0.97 sec/real 8.87 sec = 11%. Speed 94.14 mB/s
All OK

And also does not work with the file set

masen485 · #4 07-12-2018, 21:06

Thanks Razor12911

I tried the file and folder and extension that address

Just Cause 4\archives_win64\main/game0.arc

oo2core_7_win64.dll Rename oo2core_4_win64.d

game0.arc/oo2rec:
-----------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,961,920 => 2,566,282,309 bytes. Ratio 288.04%
Compression time: cpu 0.75 sec/real 51.28 sec = 1%. Speed 17.37 mB/s
All OK

game0.arc/oo2recj:
-----------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,961,920 => 890,962,140 bytes. Ratio 100.00%
Compression time: cpu 0.77 sec/real 6.07 sec = 13%. Speed 146.81 mB/s
All OK

ShivShubh · #5 07-12-2018, 21:49

Quote:

Originally Posted by masen485

Thanks Razor12911

I tried the file and folder and extension that address

Just Cause 4\archives_win64\main/game0.arc

oo2core_7_win64.dll Rename oo2core_4_win64.d

game0.arc/oo2rec:
-----------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,961,920 => 2,566,282,309 bytes. Ratio 288.04%
Compression time: cpu 0.75 sec/real 51.28 sec = 1%. Speed 17.37 mB/s
All OK

game0.arc/oo2recj:
-----------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,961,920 => 890,962,140 bytes. Ratio 100.00%
Compression time: cpu 0.77 sec/real 6.07 sec = 13%. Speed 146.81 mB/s
All OK

Seems wrong to me, you aren't supposed to use oo2recj individually/separately on a file. You have to use oo2recj in combination with oo2rec. For example = arc a -moo2rec+oo2recj game0.test game0.arc

Razor12911 · #6 08-12-2018, 09:37

Update available

Changes

- Fixed crashing issues
- Improved scanner
- Added mermaid precompressor, tested on AC Origins, try it out on Odyssey (if it uses oodle in the first place) and report back

Info

oo2reck = kraken precompressor
oo2recm = mermaid precompressor
oo2recs = selkie precompressor
// reason for them being separated is, there is no stats handler being added in the source yet meaning if I combined all, precompression speed will be reduced drastically so they are left individually at the moment
// if you're wondering what happened to jc4, just use selkie instead, its the same thing. so you should have oo2reck+oo2recs for that game

Future changes

- Detect proper uncompressed sizes resulting in all streams being detected because currently, a few streams get left behind
- Combine all codecs

masen485 · #7 08-12-2018, 10:05

Just Cause 4 some .it doesn't work with ARC files
is there any solution ?

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,648 bytes. Ratio 100.00%
Compression time: cpu 0.89 sec/real 16.25 sec = 5%. Speed 54.82 mB/s
All OK

Razor12911 · #8 08-12-2018, 10:15

Quote:

Originally Posted by masen485

Just Cause 4 some .it doesn't work with ARC files
is there any solution ?

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,648 bytes. Ratio 100.00%
Compression time: cpu 0.89 sec/real 16.25 sec = 5%. Speed 54.82 mB/s
All OK

what does the scanner report on that file?

Simorq · #9 08-12-2018, 10:18

Code:

HD Textures JC4 4.08 GB >> 6.75 GB

Creating archive: data.arc using oo2recm:
Compressed 9 files, 4,388,958,444 => 7,249,249,802 bytes. Ratio 165.17%    
Compression time: cpu 5.08 sec/real 202.55 sec = 3%. Speed 21.67 mB/s
All OK

Extracting archive: Data.arc
Extracted 9 files, 7,249,249,802 => 4,388,958,444 bytes. Ratio 165.17%     
Extraction time: cpu 5.27 sec/real 172.72 sec = 3%. Speed 25.41 mB/s
All OK

masen485 · #10 08-12-2018, 10:18

Razor12911 · #11 08-12-2018, 10:21

Quote:

Originally Posted by masen485

Universal Oodle stream scanner

Code:

Created by Razor12911

[0] = Unknown/Invalid
[1] = Kraken/Hydra
[2] = Mermaid/Selkie/Hydra
[3] = Leviathan/Hydra

0001[0]| Pos: 00152FAD, Size: 72
0002[0]| Pos: 032C3880, Size: 245
0003[0]| Pos: 03701B74, Size: 2770
0004[0]| Pos: 038C8B5C, Size: 619
0005[0]| Pos: 043CA9F1, Size: 124
0006[0]| Pos: 046AC88B, Size: 3337
0007[0]| Pos: 06B91B94, Size: 3309
0008[0]| Pos: 07A9EC0F, Size: 997
0009[0]| Pos: 08014A4B, Size: 1845
0010[0]| Pos: 0885C8CE, Size: 4019
0011[0]| Pos: 096BBDD4, Size: 2393
0012[0]| Pos: 0C109512, Size: 1038
0013[0]| Pos: 0D585CA3, Size: 2258
0014[0]| Pos: 0EC8F2DB, Size: 1938
0015[0]| Pos: 0FFB679F, Size: 967
0016[0]| Pos: 107CA519, Size: 2362
0017[0]| Pos: 111B2EC9, Size: 709
0018[0]| Pos: 117B5F73, Size: 1423
0019[0]| Pos: 12C24BB4, Size: 2389
0020[0]| Pos: 149D742B, Size: 306
0021[0]| Pos: 14AEBC77, Size: 2512
0022[0]| Pos: 152FE676, Size: 1551
0023[0]| Pos: 160E4C0C, Size: 3829
0024[0]| Pos: 1629FCA3, Size: 1395
0025[0]| Pos: 166F24E7, Size: 840
0026[0]| Pos: 183960EC, Size: 70
0027[0]| Pos: 18B88BCA, Size: 3719
0028[0]| Pos: 198B1CFC, Size: 2212
0029[0]| Pos: 199B27A1, Size: 1864
0030[0]| Pos: 1A1EBCE1, Size: 3107
0031[0]| Pos: 1A3450BE, Size: 2669
0032[0]| Pos: 1C477707, Size: 1787
0033[0]| Pos: 1C729F37, Size: 453
0034[0]| Pos: 1DD43169, Size: 1777
0035[0]| Pos: 1E007F0E, Size: 124
0036[0]| Pos: 1E7B17C6, Size: 1189
0037[0]| Pos: 1EADBDAA, Size: 2176
0038[0]| Pos: 1F4DFBEA, Size: 3518
0039[0]| Pos: 1FB63B0A, Size: 3883
0040[0]| Pos: 1FB97B22, Size: 918
0041[0]| Pos: 1FD6DE7F, Size: 147
0042[0]| Pos: 2008869C, Size: 1099
0043[0]| Pos: 207CAEBD, Size: 613
0044[0]| Pos: 208603A4, Size: 745
0045[0]| Pos: 21954CE5, Size: 730
0046[0]| Pos: 21C40094, Size: 2759
0047[0]| Pos: 231C037C, Size: 1274
0048[0]| Pos: 23CC222A, Size: 273
0049[0]| Pos: 25C3DA65, Size: 1583
0050[0]| Pos: 26720D71, Size: 3208
0051[0]| Pos: 2796EF93, Size: 2358
0052[0]| Pos: 28C09C22, Size: 3577
0053[0]| Pos: 2901B338, Size: 2377
0054[0]| Pos: 2937140C, Size: 554
0055[0]| Pos: 29D6D202, Size: 3658
0056[0]| Pos: 2ACEEB8D, Size: 2176
0057[0]| Pos: 2B0035E5, Size: 473
0058[0]| Pos: 2B9FC064, Size: 2274
0059[0]| Pos: 2C3C3A96, Size: 760
0060[0]| Pos: 2C4DC365, Size: 837
0061[0]| Pos: 2C652A4C, Size: 1418
0062[0]| Pos: 2CA91096, Size: 1234
0063[0]| Pos: 2E36FCA2, Size: 1134
0064[0]| Pos: 2EC756A5, Size: 3169
0065[0]| Pos: 2FC01316, Size: 2357
0066[0]| Pos: 305D0FBA, Size: 1812
0067[0]| Pos: 322B7348, Size: 2980
0068[0]| Pos: 325FDFB2, Size: 3798
0069[0]| Pos: 32869832, Size: 2721
0070[0]| Pos: 3309B82C, Size: 2073
0071[0]| Pos: 3325B12C, Size: 73

Done!!!

Probably no streams are on that file or the scanner doesn't know how to find them.

Quote:

Originally Posted by Simorq

Code:

HD Textures JC4 4.08 GB >> 6.75 GB

Creating archive: data.arc using oo2recm:
Compressed 9 files, 4,388,958,444 => 7,249,249,802 bytes. Ratio 165.17%    
Compression time: cpu 5.08 sec/real 202.55 sec = 3%. Speed 21.67 mB/s
All OK

Extracting archive: Data.arc
Extracted 9 files, 7,249,249,802 => 4,388,958,444 bytes. Ratio 165.17%     
Extraction time: cpu 5.27 sec/real 172.72 sec = 3%. Speed 25.41 mB/s
All OK

hmm, so mermaid exists too, mind officially blown.
I guess blend all 3 codecs then for that game. I give up really.
The samples I received only had two codecs used on them.

masen485 · #12 08-12-2018, 10:29

C:\Users\RAMAZAN\Desktop\Yeni klasör (2)>arc.exe a -ep1 -r -ed -s; -w.\temp -moo2recm data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,428 bytes. Ratio 100.00%
Compression time: cpu 0.72 sec/real 14.98 sec = 5%. Speed 59.46 mB/s
All OK

masen485 · #13 08-12-2018, 10:37

Just Cause 4\archives_win64\main_patch/game4.arc

C:\Users\RAMAZAN\Desktop\Yeni klasör (2)>arc.exe a -ep1 -r -ed -s; -w.\temp -moo2recm data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,428 bytes. Ratio 100.00%
Compression time: cpu 0.86 sec/real 18.27 sec = 5%. Speed 48.76 mB/s
All OK

L0v3craft · #14 08-12-2018, 10:52

Sometimes i get errors like this:

Where is the problem ? Translating "Impossible access to the file. The file is used from another process."

ShivShubh · #15 08-12-2018, 11:10

Quote:

Originally Posted by masen485

Just Cause 4 some .it doesn't work with ARC files
is there any solution ?

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,648 bytes. Ratio 100.00%
Compression time: cpu 0.89 sec/real 16.25 sec = 5%. Speed 54.82 mB/s
All OK

For Just Cause 4, to remove confusion/crashes make sure you only have oo2core_7_win64.dll (remove other/older oo2core dlls) present alongside oo2reck,oo2recm,oo2recs. Now keep in mind that Just Cause 4 mostly uses Kraken so most precompression will come from oo2reck then whatever's left behind will be catched up by Selkie (oo2recs). Mermaid is a stronger oodle codec in terms of compression and there are rarely (perhaps none) any streams of it in the game except for hi-res textures (hires folder) where stronger compression is required.

So my suggestion is use: oo2reck+oo2recs on game files and use oo2recm on hires folder.

Result: oo2reck+oo2recs+srep+lolz = 17.2 GB without hi-res textures, credits (first bink video stream inside game37.arc), useless videos (2.66 GB, embedded inside game files). Best thing ? installation goes really fast, finishes in 30-35 mins on 4 cores+hdd if I remember correctly.

Now, inside "main" folder = game37,38,39,40,41.arc files only contains bink video files so precompression is useless on them. Also inside "main_patch" folder = game6,7,8,9,10.arc only contains bink video files so precompression is useless on them as well. So you may wanna separate them in a different folder.

Quote:

Originally Posted by masen485

C:\Users\RAMAZAN\Desktop\Yeni klasör (2)>arc.exe a -ep1 -r -ed -s; -w.\temp -moo2recm data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,428 bytes. Ratio 100.00%
Compression time: cpu 0.72 sec/real 14.98 sec = 5%. Speed 59.46 mB/s
All OK

Read above reply, don't use oo2recm on game files except for hires folder.

Quote:

Originally Posted by masen485

Just Cause 4\archives_win64\main_patch/game4.arc

C:\Users\RAMAZAN\Desktop\Yeni klasör (2)>arc.exe a -ep1 -r -ed -s; -w.\temp -moo2recm data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 890,974,208 => 890,974,428 bytes. Ratio 100.00%
Compression time: cpu 0.86 sec/real 18.27 sec = 5%. Speed 48.76 mB/s
All OK

Again why do you keep using oo2recm ?

Quote:

Originally Posted by Simorq

oo2core_7_win64.dll Rename oo2core_4_win64.dll

Why rename

Just Cause 4 only has oo2core_7_win64.dll which is whats required by razor's tool. I would recommend not to put any other dlls for this game.

Quote:

Originally Posted by L0v3craft

Sometimes i get errors like this:

Where is the problem ? Translating "Impossible access to the file. The file is used from another process."

Make sure only oo2core_7_win64.dll (remove other oo2core dlls) is present in your compression folder when compressing Just Cause 4.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Test: Universal Oodle stream scanner	Razor12911	Conversion Tutorials	23	09-11-2018 18:45
Project Cars Digital Edition (3xDVD5) (srep+lzma)	GTX590	PC Games - CD/DVD Conversions	10	28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039	mausschieber	PC Games - CD/DVD Conversions	0	24-07-2017 15:12
Space Channel 5 Part 2 Translation Project	Christuserloeser	DC Games	0	21-06-2004 18:16
PROJECT I.G.I ????? how to copy PROJECT I.G.I????	orion	PC Games	14	04-02-2001 15:01

The Following 30 Users Say Thank You to Razor12911 For This Useful Post:
Abbat (13-05-2023), Andu21 (16-08-2020), arkantos7 (22-05-2019), chum (03-08-2019), danswano (08-12-2018), DiCaPrIo (08-12-2018), elit (14-07-2020), felice2011 (08-12-2018), FoRMaT-2007 (16-04-2023), Harsh ojha (18-05-2019), hmou1 (29-03-2020), JustFun (14-02-2019), K4miKaZe (07-12-2018), KaktoR (07-12-2018), L0v3craft (08-12-2018), masen485 (07-12-2018), Mini (07-12-2018), newfolder (08-10-2022), NexusIV (08-12-2018), oltjon (07-12-2018), ScOOt3r (09-12-2021), shazzla (08-12-2018), ShivShubh (07-12-2018), Simorq (07-12-2018), Titeuf (11-12-2018), Toximan69 (09-12-2018), vint56 (07-12-2018), Vladislav989 (09-08-2022), ZAZA4EVER (08-12-2018), zirkhaki (09-12-2018)

The Following User Says Thank You to KaktoR For This Useful Post:
Razor12911 (08-12-2018)

The Following User Says Thank You to Simorq For This Useful Post:
Razor12911 (08-12-2018)

The Following User Says Thank You to masen485 For This Useful Post:
Razor12911 (08-12-2018)

The Following 11 Users Say Thank You to Razor12911 For This Useful Post:
danswano (08-12-2018), DiCaPrIo (08-12-2018), KaktoR (08-12-2018), L0v3craft (08-12-2018), oltjon (08-12-2018), ScOOt3r (22-12-2019), ShivShubh (08-12-2018), Simorq (08-12-2018), Stor31 (09-12-2018), ZAZA4EVER (09-12-2018), zirkhaki (09-12-2018)

The Following User Says Thank You to Razor12911 For This Useful Post:
ShivShubh (08-12-2018)

The Following 2 Users Say Thank You to Razor12911 For This Useful Post:
ShivShubh (08-12-2018), Simorq (08-12-2018)

The Following 3 Users Say Thank You to ShivShubh For This Useful Post:
Razor12911 (08-12-2018), Simorq (08-12-2018), zirkhaki (09-12-2018)