Archived

This topic is now archived and is closed to further replies.

Please be aware that the content of this thread may be outdated and no longer applicable.

Asphyxiator

.kwad?

154 posts in this topic

Please do not consider me as impolite, but in my opinion it is very unproductive to do all this kwad researchment week by week instead of just asking someone of developer team to clear all things out. It could take a couple of hours to get to know everything you need.

Share this post


Link to post
Share on other sites

I think we'll be fine. We basically have a complete file spec included with the tool. The hard part is translating it into code. The only thing I'm really missing is the .fsb password.

Share this post


Link to post
Share on other sites

If there are any specific questions you have, i'll see about answering them.

1 person likes this

Share this post


Link to post
Share on other sites

So, I'm working on getting .pngs out of the .tex files, and I've managed to get the textures decompressed, albeit with an out of memory error sometimes (which is probably my fault). However, I can't seem to get the raw data that comes from the DXT decompression to save into a readable image. Has anyone managed to do that, and if so, how? I'm working in C#, though if you have code in other languages, I might be able to translate it.

Share this post


Link to post
Share on other sites

MadDoctor5813, Shirsh, see guys, Arc came here to help you with your questions. The only thing you should do, is to ask and then be kind and patient.

Klei has surprisingly good, responsive teammates.

Share this post


Link to post
Share on other sites

I've managed to extract all the .texs out to .pngs. Also, it appears you can just straight edit them, repack the one file with the correct folder structure, and load the generated .kwad into game, replacing the stock file. I had to think of the best way to test it, and this is the only way I could think of:

(Forgive my horrendous art). Because you can directly edit the spritesheets, we may not need to extract the KLEISRFs after all, as they seem to be just a location.

 

post-650222-0-33112500-1432860684_thumb.

1 person likes this

Share this post


Link to post
Share on other sites

MadDoctor5813, Shirsh, see guys, Arc came here to help you with your questions. The only thing you should do, is to ask and then be kind and patient.

Klei has surprisingly good, responsive teammates.

 

But where is the fun in just knowing everything? It isn't just about the knowledge gained. It is about the journey too, not just the destination. A lot of the fun lies in writing your own tools to decipher stuff, getting that lil' AHA! moment when understanding a format, advancing your research, etc..

Share this post


Link to post
Share on other sites

PistonMiner, I agree with you, if you decided, that your goal is to do research for the sake of research process itself. I was thinking, that people here want to do mods, and from this point of view it looks rather sensless to put so much effort into researches instead of doing more important things...

Well, let's just leave this conversation out of scope :-)

Share this post


Link to post
Share on other sites

I've managed to extract all the .texs out to .pngs. Also, it appears you can just straight edit them, repack the one file with the correct folder structure, and load the generated .kwad into game, replacing the stock file. I had to think of the best way to test it, and this is the only way I could think of:

(Forgive my horrendous art). Because you can directly edit the spritesheets, we may not need to extract the KLEISRFs after all, as they seem to be just a location.

 

It's amazing seeing how much work you guys have put into this. I have to applaud the dedication! I too would love to be able to get access to the audio clips in this game. So I'm hoping you guys can work your magic.

Share this post


Link to post
Share on other sites

It's amazing seeing how much work you guys have put into this. I have to applaud the dedication! I too would love to be able to get access to the audio clips in this game. So I'm hoping you guys can work your magic.

 

 

That should be pretty easy, as the sound format employed is a third party one, FMOD. First of all, you need to extract sound.kwad, which That contains the sound files. Then, open each one of the files in the extracted file up with a Hex-Editor, and remove the first 12 bytes of each of them. That will leave you with raw FMOD files, which you should then be able to open using the FMOD tools which you can find here. My first attempts at opening those files using the tools they have there however failed, I am trying to write up something custom right now. However, the FMOD tools are commonly used in game development, so if you really want something right now, you can probably find a tool for the job.

 

--PistonMiner

Share this post


Link to post
Share on other sites

All .fsb files start with a signature: FSB5 in text. However, the fsbs extracted don't seem to have this signature. This leads to believe they are encrypted, which is sort of common in released games. This would explain why the tools couldn't open them. We could try to crack the password, which can be done with several programs on the web, or we can ask Arc very nicely for the password.

Share this post


Link to post
Share on other sites

All .fsb files start with a signature: FSB5 in text. However, the fsbs extracted don't seem to have this signature. This leads to believe they are encrypted, which is sort of common in released games. This would explain why the tools couldn't open them. We could try to crack the password, which can be done with several programs on the web, or we can ask Arc very nicely for the password.

 

I looked at the files and the signature, and came to the same conclusion as you. Theres also an additional possibility; We could ask Arc if we may try and find out the encryption routine from analysing invisibleinc.exe, which is where all the resource loading is stored I believe, but due to the contracts between KLEI and the user we may not just do that, we need permission by KLEI, and Arc may even not be able to allow us to. If that's the case, we may have to simply ask for the password, as brute-forcing may very well be not an option due to password strength etc.. Also, its only half the fun (imo) Sooo...

 

@Arc, may we try and analyse/reverse engineer invisibleinc.exe to find out the password?

 

It is reasonable to assume though that KLEI won't/can't/doesn't to want to allow this, which would pretty much be the standard reaction for a company.

 

--PistonMiner

Share this post


Link to post
Share on other sites

So, I'm now working on extracting the smaller pngs, and I'm having trouble loading in the affinetransform part of the files. Currently, I just read in 6 floats and stick them into a c# matrix using the standard constructor, but this gives me the wrong images. Does anyone know the proper format for this?

Share this post


Link to post
Share on other sites

So, I'm now working on extracting the smaller pngs, and I'm having trouble loading in the affinetransform part of the files. Currently, I just read in 6 floats and stick them into a c# matrix using the standard constructor, but this gives me the wrong images. Does anyone know the proper format for this?

 

see http://de.mathworks.com/help/images/ref/affine2d-class.html

The thing special about the ones used in here is that the transforms are floats from 0 to 1 relative to the surface size, like OpenGL coordinates.

Share this post


Link to post
Share on other sites

This is probably what I missed. So, I just multiply all the fields by the width or the height, right? If this is so, which fields are multiplied by which dimension?

 

UPDATE: Ok, now I'm running into a different problem. The extracted spritesheets are rather large, and having to load them into memory is throwing out of memory exceptions. Do you know of a way to get a subimage out of a large image without loading the whole thing? I've tried a lot of methods, but none of them really work.

Share this post


Link to post
Share on other sites

This is probably what I missed. So, I just multiply all the fields by the width or the height, right? If this is so, which fields are multiplied by which dimension?

 

UPDATE: Ok, now I'm running into a different problem. The extracted spritesheets are rather large, and having to load them into memory is throwing out of memory exceptions. Do you know of a way to get a subimage out of a large image without loading the whole thing? I've tried a lot of methods, but none of them really work.

 

Probably just leave the SRF1's in the raw format and access each row seperately.

Share this post


Link to post
Share on other sites

Almost finished with KWAD format spec: https://github.com/Psimage/KWADTemplate/blob/master/KWAD2Template.bt

 

Can i get a cookie?

 

I need only 4 clarifications from you, @Arc:

  1. MDL1._q_signature Is this a signature for the following polygon mesh format? The value is constant "KMe1" (4B 4D 65 31)
  2. MDL1._q_polygonCount Is this actually a polygon count (number of triangles)?
  3. ANM1.Instance.CMIdx What does CM stands for? Color Modifier?
  4. ANM1.Instance.CAIdx The value is always 1. Have no clue what it is used for.

And after this questions answered i'm probably done with KWAD. BibleThump

1 person likes this

Share this post


Link to post
Share on other sites

Thanks for the help, this will be a great help if I ever manage to finish the extractor. Have an internet cookie, on the house.

 

EDIT: Speaking of finishing the extractor, I've having some trouble with loading in the transform from the SRF files. I'm loading in the data like this:

affineData[0] = reader.ReadSingle() * width; //denormalizing from OpenGL coordsaffineData[1] = reader.ReadSingle() * height;affineData[2] = reader.ReadSingle();affineData[3] = reader.ReadSingle();affineData[4] = reader.ReadSingle() * width;affineData[5] = reader.ReadSingle() * height;

where affineData is an array I use to populate the matrix, and where width and height are read in from the SRF file. However, this gives me some crazy dimensions, with values like 36000 for the size of the extracted surface, which C# refuses to instantiate. I've also tried using the height and width of the .tex files, but that alos gives me crazy dimensions, though different ones. Anything I'm missing?

Share this post


Link to post
Share on other sites

Thanks for the help, this will be a great help if I ever manage to finish the extractor. Have an internet cookie, on the house.

 

EDIT: Speaking of finishing the extractor, I've having some trouble with loading in the transform from the SRF files. I'm loading in the data like this:

affineData[0] = reader.ReadSingle() * width; //denormalizing from OpenGL coordsaffineData[1] = reader.ReadSingle() * height;affineData[2] = reader.ReadSingle();affineData[3] = reader.ReadSingle();affineData[4] = reader.ReadSingle() * width;affineData[5] = reader.ReadSingle() * height;

where affineData is an array I use to populate the matrix, and where width and height are read in from the SRF file. However, this gives me some crazy dimensions, with values like 36000 for the size of the extracted surface, which C# refuses to instantiate. I've also tried using the height and width of the .tex files, but that alos gives me crazy dimensions, though different ones. Anything I'm missing?

 

You are missing the way affine2d works. Sent you a link a couple of posts back, I recommend you look at it. Its an actual matrix, not just a bunch of values stacked. You are 'denormalizing' the wrong values.

 

Also: I did my magic. The password for the FSB files is:

 

mint78run52

 

Can I get a cookie too? :)

 

--The magical PistonMiner

1 person likes this

Share this post


Link to post
Share on other sites

I did read the link, though I think the math just flustered me. I will take a more detailed look at it later, though. On the topic of the fsb password, woo! Now we can finally accomplish the mission given to us at the beginning of this thread, so many days ago. I hope I don't have to install all of FMOD to decrypt it.

Share this post


Link to post
Share on other sites

I did read the link, though I think the math just flustered me. I will take a more detailed look at it later, though. On the topic of the fsb password, woo! Now we can finally accomplish the mission given to us at the beginning of this thread, so many days ago. I hope I don't have to install all of FMOD to decrypt it.

 

Just use decfsb on the files.

For your matrix stuff:

[0] = XScale // normalize[1] = plusY_X[2] = plusX_Y[3] = YScale // normalize[4] = XOffset // normalize[5] = YOffset // normalize

You can probably ignore [1] and [2], I doubt they are used, they basically add n * x to y and n * y to x.

Also, sad that in the eandavour of getting the password I missed out on the format "competition" :(

Share this post


Link to post
Share on other sites

Almost finished with KWAD format spec: https://github.com/Psimage/KWADTemplate/blob/master/KWAD2Template.bt

 

Can i get a cookie?

 

I need only 4 clarifications from you, @Arc:

  1. MDL1._q_signature Is this a signature for the following polygon mesh format? The value is constant "KMe1" (4B 4D 65 31)
  2. MDL1._q_polygonCount Is this actually a polygon count (number of triangles)?
  3. ANM1.Instance.CMIdx What does CM stands for? Color Modifier?
  4. ANM1.Instance.CAIdx The value is always 1. Have no clue what it is used for.

And after this questions answered i'm probably done with KWAD. BibleThump

 

Hi Strelock, this info should help you out.

 

m_CMIdx is a index into m_Colours, its used as a color modulate

m_CAIdx is used as a color add

 

MDL1

    u32 FOURCC KLEI

    u32 FOURCC MDL1

    u32 resourcesize

    u32 texture idx

    u32 FOURCC KMe1

    u32 pcount

    u32 icount

    u32 vcount

    s32[icount] indices

    vtx[vcount] vertices

 

vertex

    f32 x

    f32 y
    f32 u
    f32 v

Share this post


Link to post
Share on other sites

Woo, I got the SRFs extracted! It was more fumbling around than math, but it still works, albeit over a long time. It turns out bitmaps in C# have a global lock on them, so multithreading is a no no for those, meaning it could take a while for extraction. However, I have never been more happy to see tables and filing cabinets in my life.

Share this post


Link to post
Share on other sites

Thank you, Arc.

 

Corrections applied. The next step would be to "paint" KWAD structure. That is a real challenge. Feel free to help :-)

 

And thank you MadDoctor5813 for the cookie. It was delicious.

Share this post


Link to post
Share on other sites

No problem. This gives me an idea for a medgel retexture: a chocolate chip cookie. Now that we have unlocked Klei's mysteries, the possibilities are endless.

Share this post


Link to post
Share on other sites