Archived

This topic is now archived and is closed to further replies.

Please be aware that the content of this thread may be outdated and no longer applicable.

Recommended Posts

Arc    7

Here is the tool we use, Get it here builder.zip

 

This isn't an official tool release, we won't be supporting it via bug fixes. We don't have any internal tools for unpacking a kwad, you'll have to make one yourselves. With all the source content available there wasn't any demand while making the game.

  • Like 2

Share this post


Link to post
Share on other sites

So, I just read over the readme. And I am deeply confused. Hopefully someone smarter than I can figure out how to decode this in a better way than I did. I do have one last request. This thread was originally started because someone wanted the Incognita sounds. I've managed to extract the fsb files, but they appear to be encrypted. Would you mind providing the password? If you don't want to, I understand, I imagine they were encrypted for a reason.

Share this post


Link to post
Share on other sites
Shirsh    129

Thank you for example stuff! Especially for doordecoder1.anim.

For science:

 243970_2015-05-26_00005.png
Now if an bright mind understands how to reverse process there could be a lot of mods with green dots everywhere  :grin:

Share this post


Link to post
Share on other sites
x43x61x69    4

Here is the tool we use, Get it here attachicon.gifbuilder.zip

 

This isn't an official tool release, we won't be supporting it via bug fixes. We don't have any internal tools for unpacking a kwad, you'll have to make one yourselves. With all the source content available there wasn't any demand while making the game.

 

So guess that my unpacker is not a time waste after all.

 

In case anyone wants to improve the unpacker according to the official code (which was written before the official builder was released): https://github.com/x43x61x69/KWAD-Tool

 

Thanks.

Share this post


Link to post
Share on other sites
HumanKirby    55

Ok, can someone explain me both, especally the Extractor.

I open the Python Command line, put this in and it doesnt seem to work. (E is my secondary HDD)

E:\Programms\Steam\...\InvisibleInc\kwad-tool.py  PATH_TO_KWAD_FILE E:\Programms\Steam\...\InvisibleInc\sound.kwad

Share this post


Link to post
Share on other sites
Shirsh    129

Okay, time to learn Python)

 

02.png

 

Ok, I get it! I supposed to use Python 2.7 not a Python 3.0 or 3.4

but:

 

03.png

 

and after that I delete brackets "[ ]" from extraction path and everything works!

 

with absolutely same result as with quickbms:

how to change tony_matte.png(48B)+tony_matte.tex(1.626MB) back into tony_matte.png(1.31MB) now, damn

 

Share this post


Link to post
Share on other sites
Shirsh    129

Since I just want to mix parts of characters skins (and for this I don't really need to open pics in editor), I start to think, how to make game use folders with extracted resources instead of .kwad

 

Or, extract an .kwad, mess with it and pack it back into an .kwad, hmmm

 

aaand I was able to do it with sound! 

but to make it work I manually delete Klei signatures before rebuilding kwad, it's ok for 10-11 files, but promised to be quite painful to animation folder with 777 files in Shalem's one >_<

 

another thing: kwad builded with example doordecoder.anim generates .adef and .abld files (ie, I have it after unpacking with kwad_tool), but after unpacking anim.kwad and characters.kwad with kwad_tool  I didn't had them >_<

it looks like an serious extra wall on the way to custom skins, in addition to 777 files: ie even if I create custom.kwad with mixed parts, it probably will not work without those additional .adef and .abld

 

Share this post


Link to post
Share on other sites
x43x61x69    4

Okay, time to learn Python)

 

02.png

 

Ok, I get it! I supposed to use Python 2.7 not a Python 3.0 or 3.4

but:

 

03.png

 

and after that I delete brackets "[ ]" from extraction path and everything works!

 

with absolutely same result as with quickbms:

how to change tony_matte.png(48B)+tony_matte.tex(1.626MB) back into tony_matte.png(1.31MB) now, damn

 

No idea what it will end up with on Windows, should work on OS X and Linux.

 

And yes it's for Python 2.7, not 3.x.

Share this post


Link to post
Share on other sites
Strelock    10

Here is a Template for 010 Editor tool: https://github.com/Psimage/KWADTemplate

 

Maybe it can help somebody to visualize the structure of a KWAD file. Just run in on any KWAD file and you will get something like this:

post-445693-0-95104900-1432652399_thumb.

 

Currently it only displays Surfaces and Textures.

 

To convert TEX back to original PNG:

  1. Locate surface struct
  2. Uncompress data with ZLib
  3. if isDXTCompressed - uncompress DXT5 (i used squish library)
  4. if not isDXTCompressed - its an RGBA array
  5. Convert raw data to PNG. The result is Texture Atlas
  6. "Cut" sprites (TEX) from Atlas using Affine2D struct data (every TEX has an index field pointing to SRF it resides in)
  7. ???
  8. Profit!

And thank you Arc!

  • Like 1

Share this post


Link to post
Share on other sites
Shirsh    129

at least some unimplemented assets detected:

 

1. Sankaku Grenadier 

Sankaku have much more units than everyone else, so it's not surpising, that he didn't make it into game

sg.png

 

2. Femme enforcer!!11

She doesn't look finished (when you see her head...), and, maybe, she also doesn't fit to concept or something...

fe.png

 

but I had some picture edit: 

WARNING! NOT AN ACTUAL FOOTAGE AHEAD!! 

fe2.png

 

Share this post


Link to post
Share on other sites
PistonMiner    1

Alright, I will join this discussion here. I tried and decompress the SRF data as well, and I got as far as trying to decompress the embedded ZLib blob, at which point all decompression methods told me that the data was corrupt. May I ask what decompression means you guys used to decompress the data?

Share this post


Link to post
Share on other sites
x43x61x69    4

Alright, I will join this discussion here. I tried and decompress the SRF data as well, and I got as far as trying to decompress the embedded ZLib blob, at which point all decompression methods told me that the data was corrupt. May I ask what decompression means you guys used to decompress the data?

 

If you can write Python, you can load the raw dump then use zlib to uncompress it.

 

https://docs.python.org/2/library/zlib.html

Share this post


Link to post
Share on other sites
Strelock    10

Alright, I will join this discussion here. I tried and decompress the SRF data as well, and I got as far as trying to decompress the embedded ZLib blob, at which point all decompression methods told me that the data was corrupt. May I ask what decompression means you guys used to decompress the data?

 

So ZLib 1.2.8 inflate() doesn't work?

Googled for stand-alone tool (7-zip 9.20 is not strong enough) and found ZDrop (http://forums.majorgeeks.com/showthread.php?t=223640). In 1 example case it worked.

Share this post


Link to post
Share on other sites
HumanKirby    55

Ok, I might just wait until this part develpoes much furture and its better documented, its just to complicated for me (atm). Also, builder only supports 64 bit, sad but in the future there might be an 32 bit and maybe even an offical release of the tool. If any could provide me a list with the assets and sound this would allow me to have more options for moding with out even toching the .kwad.

Share this post


Link to post
Share on other sites
PistonMiner    1

First of all, thanks for the tip with Python, I already tried that, returns Error -3: invalid distance too far back on blob1.tex, so maybe I am doing something wrong, this is the code I use in case you spot an immediate error, has been a while since I worked with Python binary reading:

[codesyntax]import zlib
import sys

filename_in = sys.argv[1]

input = open(filename_in, "rb")
input_data = input.read()
input.close()

decompressed = zlib.decompress(input_data)

filename_out = sys.argv[2]
output = open(filename , "wb")
output.write(decompressed)
output.close()[/codesyntax]

 

Secondly, I would like to contribute something, and even though its small, I think it should be noted:

 

The structure of a BLOB type is:

    4: u32 : FOURCC('K','L','E','I')
    4: u32 : FOURCC('B','L','O','B')

    4: u32 : size of data

    variable : data

 

So in order to decode a BLOB file, simply remove the first 12 bytes.

Just wanted to throw that out in case it helps.

Share this post


Link to post
Share on other sites

I've made a C# program for extracting everything out, and it's on github at:

https://github.com/MadDoctor5813/KWADExtractorV2/releases/tag/v1.0

It's not much yet, and basically duplicates the work of x43x61x69, but there will be more coming soon, and I hope to make it into sort of an integrated tool, so there can be a one click solution as opposed to the more manual way. In the meantime, feel free to test my work and suggest improvements.

  • Like 2

Share this post


Link to post
Share on other sites
HumanKirby    55

Thanks MadDoctor, the first Tool I was able to use. I didnt unterstand the first and the released doesnt work for me, since its exclusiv to 64 currently. Hope someone makes a tool to pack the files back in .kwad.

Share this post


Link to post
Share on other sites

If you know C#, and have some ideas for improvement, feel free to make a pull request. There's probably a more efficient way to extract it, but I can't seem to wrap my head around it just yet. I could have sworn the builder.exe was supposed to handle packing the files back, and I was planning to use it in the program later. Does it not work?

Share this post


Link to post
Share on other sites
Shirsh    129

When I experimented with unpack/repack of sound.kwad I had problem with builder: it add Klei signature at start of every file in .kwad, so when it pack previously unpacked files (where signature not deleted in an Hex editor) it adds second signature and it became not readable for game. What even worse, for textures it transorm pngs to png+tex (I can't clearly get which of this surface one etc), so if I want to mix different parts (pants of courier with Shalem's jacket) it trying to transform those pseudo-png into new png+tex. It's not enough to extract kwads, for modding it required to reverse those builder's magics.

 

Maybe I had some Windows problem though, do you have .adld and .adefs files from anim.kwad or characters.kwad on osX or Linux?  Wow, I get them with this awesome KWAD Extractor v2!

Hmm, I see all "home-maded" extractors create different output, on Win at least >_<

 

Or there need in creation of another .kwad packer that not transform files but simply put them in.

(or maybe there could work some tuning of build.lua?)

  • Like 1

Share this post


Link to post
Share on other sites
PistonMiner    1

Alright, found my problem. Using Notepad++ as a hex editor is not a good thing. Don't do that. Replaced 0x00 with 0x20, leading to corrupting my data. Well anyway, now I can extract all textures which is nice.

Next, I looked at the KLEIMDL1 format. The first few bytes are pretty much equal to the other formats, its KLEIMDL1 and then a byte for the total file size. So far so nice. For the back part of the file, it might be an index buffer, since theres quite a few low values which are easily parted by 3, and then maybe a vertex buffer for the rest of the data. Just guessing though, haven't looked into it too hard yet.

--PistonMiner

Share this post


Link to post
Share on other sites

Anyone have any idea how exactly the SRFs and TEXs are linked together? I'm working on pulling the extracted files into more readable formats now. I've got removal of those 12 byte signatures going, but the linkage between SRFs and TEXs is confusing to say the least. Also, I've managed to extract files, but some seem missing compared to the dump resources.txt gives. Can anyone download my tool and verify that I've managed to get everything?

Share this post


Link to post
Share on other sites
PistonMiner    1

Anyone have any idea how exactly the SRFs and TEXs are linked together? I'm working on pulling the extracted files into more readable formats now. I've got removal of those 12 byte signatures going, but the linkage between SRFs and TEXs is confusing to say the least. Also, I've managed to extract files, but some seem missing compared to the dump resources.txt gives. Can anyone download my tool and verify that I've managed to get everything?

 

The first 12 bytes of a TEX1 are the same as normal, signature and resource size. After that is the ID of the SRF1 this TEX1 relates to. That ID is related to the ID specified in the resource_info array. (If you want filenames, you can simply use the alias_info array.) After that follow width and height of this material, and then the appropriate UV transform off of the SRF1 specified.

 

So basically, a TEX1 is a cutout of a SRF1, meaning that SRF1's can be used as Atlas's like Strelock already said. -> Only one SRF1 per character required, the rest is TEX1's specifying the appropriate locations of the sub-sprites. So really, in order to extract a TEX1, you need the alias_info table dumped. Not sure if there's already a tool that does that, if not I will probably write one up as I could use that table too for my automatic extractor.

 

-- PistonMiner

Share this post


Link to post
Share on other sites

So sort of like a sprite sheet? Hmmm, it might be a pain to connect them together, because the way I extract the files means I may have to reload the .kwad to grab the aliases out. Hopefully, I can figure it out from just the file names.

Share this post


Link to post
Share on other sites
Shirsh    129

And yes, we can address this questions to Arc, but it's little by little starts looking like we ignored all "we can't do modding support" messages and kinda asking Klei-guys to help us with mods instead of working on more content for this game or another =\

Share this post


Link to post
Share on other sites